Pull over 0.4.1 release md

releases/0.4.1.md

@@ -0,0 +1,106 @@
+ # pict-rs 0.4.1
+
+## Overview
+
+Today I am releasing pict-rs 0.4.1. This includes a couple bugfixes and a few requested features.
+Most notably is the object-storage migration now uploads files concurrently. It should be much
+faster than before.
+
+### Features
+
+- [Add Concurrency to Migration](#add-concurrency-to-migration)
+- [Improved Error Messages](#improved-error-messages)
+- [Configurable Object Storage Timeouts](#configurable-object-storage-timeouts)
+- [Set Content-Type in Object Storage](#set-content-type-in-object-storage)
+
+
+### Bugfix
+
+- [Alias Cleanup](#alias-cleanup)
+- [ImageMagick Security Policy](#imagemagick-security-policy)
+- [Broken Pipe on Upload Error](#broken-pipe-on-upload-error)
+
+
+## Upgrade Notes
+
+There's no significant changes from 0.4, so upgrading should be as simple as pulling a newer version
+of pict-rs.
+
+
+## Descriptions
+
+### Add Concurrency to Migration
+
+Adding concurrency means changing how pict-rs kept track of migration progress. Previously it was
+setting a single flag with the "newest" hash that it had migrated, so a resuming migration would
+iterate until it found that hash before continuing linearly. Now, pict-rs keeps track of each
+individual file. That way it does not matter the order in which files are migrated.
+
+The migration is hardcoded to try moving 32 files at a time, so ideally this would mean a 32x
+speedup over the previous linear version. Whether it actually ends up being so fast is yet to be
+seen.
+
+
+### Improved Error Messages
+
+Previously errors encountered when failing to run a process were pretty opaque. pict-rs relies on
+running the `ffmpeg`, `ffprobe`, `magick`, and `exiftool` commands, and if it could not run them it
+would print something vague like "File not found" or "Permission Denied", without much context for
+what wasn't found or what was denied.
+
+Now, pict-rs properly informs the admin that "Required command magick not found" or "Cannot run
+command ffmpeg due to invalid permissions on binary".
+
+Additionally, the error messages returned in the API were not very useful either. "Error in ffmpeg"
+or "Error in imagemagick" are not super useful. Now pict-rs will return the "root cause" of errors,
+which is generally more helpful. There's still more work to be done here, so this topic will be
+mentioned in future releases.
+
+
+### Configurable Object Storage Timeouts
+
+pict-rs now provides the ability to increase the HTTP timeout and the Signature Expiration for
+requests to object storage. This may not be super helpful, but if your object storage is
+particularly slow you can now increase the values.
+
+The relevant variables are `PICTRS__SToRE__SIGNATURE_EXPIRATION` and
+`PICTRS__STORE__CLIENT_TIMEOUT`.
+
+In toml, that would be
+```toml
+[store]
+signature_expiration = 15
+client_timeout = 30
+```
+
+
+### Set Content-Type in Object Storage
+
+This is not immediately useful, but it means that the object storage will begin knowing about the
+content-type of uploaded files, so in the future when pict-rs supports serving directly from
+object-storage it will provide a meaningful content-type header to browsers.
+
+
+### Alias Cleanup
+
+There was a bug introduced in 0.4 that made using the "delete" endpoint not actually remove the file
+from storage. It would un-link the alias so it couldn't be accessed, but full deletion never
+happened. This was a result of me reordering a couple lines in the alias cleanup. 0.4.1 fixes this
+issue by un-reordering them.
+
+
+### ImageMagick Security Policy
+
+The security policy used in the pict-rs docker container has been updated to allow for more file
+types. This makes the new JSON-based imagemagick and ffmpeg output parsers work, and might make one
+or two rejected files now no longer be rejected. I don't know the full extent to which this was a
+real problem in 0.4, but regardless it's better now.
+
+
+### Broken Pipe on Upload Error
+
+Previously if a file was uploaded that was too large, it would be rejected immediately with a file
+size error. This sometimes resulted in clients receiving a Broken Pipe error on the connection
+instead of properly receiving an HTTP Response. This has been fixed in the `actix-form-data` library
+(that I also maintain) by ensuring that we read the entire upload (dropping the bytes immediately)
+before replying.