mirror of
https://git.asonix.dog/asonix/pict-rs
synced 2024-12-22 19:31:35 +00:00
Start work on release announcement
This commit is contained in:
parent
686be1d93b
commit
6bc7e86099
1 changed files with 194 additions and 0 deletions
194
releases/0.4.0.md
Normal file
194
releases/0.4.0.md
Normal file
|
|
@ -0,0 +1,194 @@
|
||||||
|
# pict-rs 0.4.0
|
||||||
|
|
||||||
|
I am excited to announce a new stable version of pict-rs, version 0.4.0. There's a few big things in
|
||||||
|
0.4 that might be exciting to developers and admins. Below you'll find a list of changes from 0.3 to
|
||||||
|
0.4.
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
### Headline Features:
|
||||||
|
|
||||||
|
- [Reworked Configuration](#reworked-configuration)
|
||||||
|
- [Reworked Commandline](#reworked-commandline)
|
||||||
|
- [Object Storage](#object-storage)
|
||||||
|
- [Backgrounded Uploads](#backgrounded-uploads)
|
||||||
|
- [Full Video](#full-video)
|
||||||
|
|
||||||
|
|
||||||
|
### Other Features:
|
||||||
|
|
||||||
|
- [Published Dependencies](#published-dependencies)
|
||||||
|
- [HEAD Endpoints](#head-endpoints)
|
||||||
|
- [Healthcheck Endpoints](#healthcheck-endpoints)
|
||||||
|
- [Image Preprocessing](#image-preprocessing)
|
||||||
|
- [GIF configuration](#gif-configuration)
|
||||||
|
- [JXL and AVIF](#jxl-and-avif)
|
||||||
|
- [H265, VP8, VP9, and AV1](#h265-vp8-vp9-and-av1)
|
||||||
|
- [404 Image](#404-image)
|
||||||
|
- [DB Export](#db-export)
|
||||||
|
- [Variant cleanup endpoint](#variant-cleanup-endpoint)
|
||||||
|
- [Client pool configuration](#client-pool-configuration)
|
||||||
|
|
||||||
|
|
||||||
|
### Fixes
|
||||||
|
|
||||||
|
- [GIF/webm Transparency](#gif-webm-transparency)
|
||||||
|
- [Image Orientation](#image-orientation)
|
||||||
|
- [io_uring file write](#io_uring-file-write)
|
||||||
|
- [Blur Sigma](#blur-sigma)
|
||||||
|
|
||||||
|
|
||||||
|
### Removals
|
||||||
|
|
||||||
|
- [Purge by disk name](#purge-by-disk-name)
|
||||||
|
- [Aliases by disk name](#aliases-by-disk-name)
|
||||||
|
|
||||||
|
|
||||||
|
## Upgrade Notes
|
||||||
|
|
||||||
|
TODO
|
||||||
|
|
||||||
|
|
||||||
|
## Descriptions
|
||||||
|
|
||||||
|
### Reworked Configuration
|
||||||
|
|
||||||
|
Starting off with the most important information for server admins: The configuration format has
|
||||||
|
changed. The `pict-rs.toml` is now far better organized, and includes more configuration options
|
||||||
|
than before. Every field has been moved, so please take a look at the [example
|
||||||
|
pict-rs.toml](https://git.asonix.dog/asonix/pict-rs/src/branch/main/pict-rs.toml) file for
|
||||||
|
information on how configuration works in 0.4.
|
||||||
|
|
||||||
|
Notable changes:
|
||||||
|
- `RUST_LOG` no longer has an effect, see `PICTRS__TRACING__LOGGING__TARGETS` and
|
||||||
|
`PICTRS__TRACING__OPENTELEMETRY__TARGETS` for configuring log levels.
|
||||||
|
- `PICTRS__ADDRESS` has become `PICTRS__SERVER__ADDRESS`
|
||||||
|
- `PICTRS__PATH` has become `PICTRS__OLD_DB__PATH`, `PICTRS__REPO__PATH` and
|
||||||
|
`PICTRS__STORE__PATH`
|
||||||
|
|
||||||
|
|
||||||
|
### Reworked Commandline
|
||||||
|
|
||||||
|
Also notable for server admins: the commandline interface has changed. All configuration that can be
|
||||||
|
expressed via the pict-rs.toml file or related environment variables can be set via the commandline
|
||||||
|
as well.
|
||||||
|
|
||||||
|
Running the pict-rs server now requires invoking the `run` subcommand. Here are some examples:
|
||||||
|
```bash
|
||||||
|
$ pict-rs run
|
||||||
|
$ pict-rs -c /etc/pict-rs.toml run
|
||||||
|
$ pict-rs run filesystem -p /path/to/files sled -p /path/to/metadata
|
||||||
|
```
|
||||||
|
|
||||||
|
There is one flag that I will call out here: `--save-to <path>`. This can be incredibly helpful when
|
||||||
|
debugging pict-rs. It dumps the configuration that pict-rs is currently using to a file of your
|
||||||
|
choosing. Example:
|
||||||
|
```bash
|
||||||
|
$ PICTRS__SERVER__ADDRESS=127.0.0.1:1234 pict-rs --save-to debug.toml run
|
||||||
|
CTRL^C
|
||||||
|
$ cat debug.toml
|
||||||
|
```
|
||||||
|
It can also be useful to save your existing commandline arguments into a reusable configuration:
|
||||||
|
```bash
|
||||||
|
$ pict-rs --save-to config.toml \
|
||||||
|
--log-targets warn \
|
||||||
|
--log-format json \
|
||||||
|
--console-address '127.0.0.1:6669' \
|
||||||
|
--opentelemetry-url 'http://localhost:4317' \
|
||||||
|
run \
|
||||||
|
-a '127.0.0.1:1234' \
|
||||||
|
--api-key "$API_KEY" \
|
||||||
|
--client-pool-size 10 \
|
||||||
|
--media-max-width 200 \
|
||||||
|
--media-max-height 200 \
|
||||||
|
--media-max-file-size 1 \
|
||||||
|
--media-enable-full-video true \
|
||||||
|
--media-video-codec av1 \
|
||||||
|
--media-format webp \
|
||||||
|
filesystem \
|
||||||
|
-p /opt/pict-rs/files \
|
||||||
|
sled \
|
||||||
|
-p /opt/pict-rs/metadata \
|
||||||
|
-e /opt/pict-rs/exports
|
||||||
|
CTRL^C
|
||||||
|
$ cat config.toml
|
||||||
|
$ pict-rs -c config.toml run
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### Object Storage
|
||||||
|
|
||||||
|
This is the biggest feature for pict-rs 0.4, but it isn't new. pict-rs 0.3 had initial support for
|
||||||
|
uploading to object storage hidden behind the `object-storage` compile flag. My docker images and
|
||||||
|
provided binaries did not enable this feature, so only folks compiling from source could even have
|
||||||
|
tried using it. In 0.4 I am now much more confident in the object storage support, and use it myself
|
||||||
|
for my personal setup.
|
||||||
|
|
||||||
|
Although pict-rs now supports object storage as a backend for storing uploaded media, it doesn't
|
||||||
|
support generating URLs that bypass the pict-rs server for serving images. I've seen this
|
||||||
|
misconception a couple times during the release candidate process and I would like to put these
|
||||||
|
rumors to rest.
|
||||||
|
|
||||||
|
Using object storage when you're in a cloud environment is sensible. The cost for adding object
|
||||||
|
storage is far less than for adding block storage, and although there are egress fees, they are
|
||||||
|
typically very low. Admins currently running pict-rs on a VPS should consider moving to object
|
||||||
|
storage if they find their disk usage is quickly growing.
|
||||||
|
|
||||||
|
Using object storage does not fully remove pict-rs' dependence on a local disk. pict-rs stores its
|
||||||
|
metadata in an embedded key/value store called [sled](https://github.com/spacejam/sled/), which
|
||||||
|
means it still requires disk access to function.
|
||||||
|
|
||||||
|
For new admins, using object storage from the start might be worth considering, but is not
|
||||||
|
required. pict-rs provides a built-in migration path for moving from filesystem storage to object
|
||||||
|
storage, which is documented in the [pict-rs
|
||||||
|
readme](https://git.asonix.dog/asonix/pict-rs/src/branch/main#user-content-filesystem-to-object-storage-migration).
|
||||||
|
|
||||||
|
|
||||||
|
### Backgrounded Uploads
|
||||||
|
|
||||||
|
This feature is important for keeping pict-rs fast. Since media needs to be processed on upload,
|
||||||
|
upload times may grow long. This is especially true when considering pict-rs' other new features
|
||||||
|
like [full video](#full-video) and [image preprocessing](#image-preprocessing).
|
||||||
|
|
||||||
|
Backgrounded uploads work by deferring the image processing work to a background task in pict-rs,
|
||||||
|
and responding to the upload HTTP request as soon as the provided file has been stored
|
||||||
|
successfully. Instead of returning a file alias like the inline upload endpoint, it instead returns
|
||||||
|
a UUID that represents the current upload process. That UUID can be given to a new `claim` endpoint
|
||||||
|
to retrieve the uploaded media's alias, or an error in the event the media was rejected or failed to
|
||||||
|
process. The claim endpoint uses a technique called "long polling" in order to notify clients on
|
||||||
|
completion of the background processing. Clients can request to claim uploaded media as soon as the
|
||||||
|
backgrounded endpoint reponds with an upload ID, and the server will hold the connection open for at
|
||||||
|
most 10 seconds while it waits for processing to complete. If processing did not complete in time,
|
||||||
|
the client can request again to claim the upload.
|
||||||
|
|
||||||
|
|
||||||
|
### Full Video
|
||||||
|
|
||||||
|
pict-rs now supports uploading video with audio. In 0.3, pict-rs would always strip audio from
|
||||||
|
uploaded videos. This was intended as a gif-like feature, since pict-rs' primary use is for storing
|
||||||
|
pictures and not videos. By default, full video is disabled in pict-rs 0.4, but server admins can
|
||||||
|
opt into full video uploads by setting `PICTRS__MEDIA__ENABLE_FULL_VIDEO=true`.
|
||||||
|
|
||||||
|
Along with supporting full video, an additional configuration option has been added to help keep
|
||||||
|
down on file sizes and processing time: `PICTRS__MEDIA__MAX_FRAME_COUNT`. This option limits how
|
||||||
|
many frames an uploaded video is allowed to have, rejecting the upload if it exceeds the provided
|
||||||
|
value.
|
||||||
|
|
||||||
|
It is important to note that while pict-rs is capable of storing videos, it's processing
|
||||||
|
functionality is limited to just the video's thumbnail.
|
||||||
|
|
||||||
|
|
||||||
|
### Published Dependencies
|
||||||
|
|
||||||
|
pict-rs 0.3's experimental object storage support relied on a custom fork of the `rust-s3` library,
|
||||||
|
which did useful things like enable re-use of an HTTP Client between connections, and making the
|
||||||
|
Client a `trait`, so it could be implemented for any arbitrary HTTP Client. Since then, the PR that
|
||||||
|
was meant to upstream those changes was closed, and so pict-rs needed to find a replacement that
|
||||||
|
suited it's needs.
|
||||||
|
|
||||||
|
pict-rs now relies on the [`rusty-s3`](https://github.com/paolobarbolini/rusty-s3) library for
|
||||||
|
enabling object storage. This library takes a sans-io approach, and was simple to integrate with
|
||||||
|
`awc`, my HTTP Client of choice.
|
||||||
|
|
||||||
|
|
||||||
|
### HEAD Endpoints
|
||||||
Loading…
Reference in a new issue