mirror of
https://git.asonix.dog/asonix/pict-rs
synced 2024-11-10 06:25:00 +00:00
106 lines
4.2 KiB
Markdown
106 lines
4.2 KiB
Markdown
# pict-rs 0.4.1
|
|
|
|
## Overview
|
|
|
|
Today I am releasing pict-rs 0.4.1. This includes a couple bugfixes and a few requested features.
|
|
Most notably is the object-storage migration now uploads files concurrently. It should be much
|
|
faster than before.
|
|
|
|
### Features
|
|
|
|
- [Add Concurrency to Migration](#add-concurrency-to-migration)
|
|
- [Improved Error Messages](#improved-error-messages)
|
|
- [Configurable Object Storage Timeouts](#configurable-object-storage-timeouts)
|
|
- [Set Content-Type in Object Storage](#set-content-type-in-object-storage)
|
|
|
|
|
|
### Bugfix
|
|
|
|
- [Alias Cleanup](#alias-cleanup)
|
|
- [ImageMagick Security Policy](#imagemagick-security-policy)
|
|
- [Broken Pipe on Upload Error](#broken-pipe-on-upload-error)
|
|
|
|
|
|
## Upgrade Notes
|
|
|
|
There's no significant changes from 0.4, so upgrading should be as simple as pulling a newer version
|
|
of pict-rs.
|
|
|
|
|
|
## Descriptions
|
|
|
|
### Add Concurrency to Migration
|
|
|
|
Adding concurrency means changing how pict-rs kept track of migration progress. Previously it was
|
|
setting a single flag with the "newest" hash that it had migrated, so a resuming migration would
|
|
iterate until it found that hash before continuing linearly. Now, pict-rs keeps track of each
|
|
individual file. That way it does not matter the order in which files are migrated.
|
|
|
|
The migration is hardcoded to try moving 32 files at a time, so ideally this would mean a 32x
|
|
speedup over the previous linear version. Whether it actually ends up being so fast is yet to be
|
|
seen.
|
|
|
|
|
|
### Improved Error Messages
|
|
|
|
Previously errors encountered when failing to run a process were pretty opaque. pict-rs relies on
|
|
running the `ffmpeg`, `ffprobe`, `magick`, and `exiftool` commands, and if it could not run them it
|
|
would print something vague like "File not found" or "Permission Denied", without much context for
|
|
what wasn't found or what was denied.
|
|
|
|
Now, pict-rs properly informs the admin that "Required command magick not found" or "Cannot run
|
|
command ffmpeg due to invalid permissions on binary".
|
|
|
|
Additionally, the error messages returned in the API were not very useful either. "Error in ffmpeg"
|
|
or "Error in imagemagick" are not super useful. Now pict-rs will return the "root cause" of errors,
|
|
which is generally more helpful. There's still more work to be done here, so this topic will be
|
|
mentioned in future releases.
|
|
|
|
|
|
### Configurable Object Storage Timeouts
|
|
|
|
pict-rs now provides the ability to increase the HTTP timeout and the Signature Expiration for
|
|
requests to object storage. This may not be super helpful, but if your object storage is
|
|
particularly slow you can now increase the values.
|
|
|
|
The relevant variables are `PICTRS__SToRE__SIGNATURE_EXPIRATION` and
|
|
`PICTRS__STORE__CLIENT_TIMEOUT`.
|
|
|
|
In toml, that would be
|
|
```toml
|
|
[store]
|
|
signature_expiration = 15
|
|
client_timeout = 30
|
|
```
|
|
|
|
|
|
### Set Content-Type in Object Storage
|
|
|
|
This is not immediately useful, but it means that the object storage will begin knowing about the
|
|
content-type of uploaded files, so in the future when pict-rs supports serving directly from
|
|
object-storage it will provide a meaningful content-type header to browsers.
|
|
|
|
|
|
### Alias Cleanup
|
|
|
|
There was a bug introduced in 0.4 that made using the "delete" endpoint not actually remove the file
|
|
from storage. It would un-link the alias so it couldn't be accessed, but full deletion never
|
|
happened. This was a result of me reordering a couple lines in the alias cleanup. 0.4.1 fixes this
|
|
issue by un-reordering them.
|
|
|
|
|
|
### ImageMagick Security Policy
|
|
|
|
The security policy used in the pict-rs docker container has been updated to allow for more file
|
|
types. This makes the new JSON-based imagemagick and ffmpeg output parsers work, and might make one
|
|
or two rejected files now no longer be rejected. I don't know the full extent to which this was a
|
|
real problem in 0.4, but regardless it's better now.
|
|
|
|
|
|
### Broken Pipe on Upload Error
|
|
|
|
Previously if a file was uploaded that was too large, it would be rejected immediately with a file
|
|
size error. This sometimes resulted in clients receiving a Broken Pipe error on the connection
|
|
instead of properly receiving an HTTP Response. This has been fixed in the `actix-form-data` library
|
|
(that I also maintain) by ensuring that we read the entire upload (dropping the bytes immediately)
|
|
before replying.
|