From 4826dbb3af2d61d78a44ca533b464bbc72a42d8c Mon Sep 17 00:00:00 2001 From: asonix Date: Wed, 24 Jan 2024 17:36:33 -0600 Subject: [PATCH] Update release document --- releases/0.5.2.md | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/releases/0.5.2.md b/releases/0.5.2.md index 6ed81aa..974ed12 100644 --- a/releases/0.5.2.md +++ b/releases/0.5.2.md @@ -2,7 +2,14 @@ ## Overview -A quick release to improve postgres support. +A quick release to avoid a bug leading to incredibly large cleanup queue sizes and resulting in high +CPU use. Upgrading to 0.5.2 is recommended for any deployment that recently upgraded to 0.5.0 or +0.5.1. + +### Bugfixes + +- [At-Most Once Cleanup Jobs](#at-most-once-cleanup-jobs) + ### Changes @@ -18,6 +25,27 @@ of pict-rs. ## Descriptions +### At-Most Once Cleanup Jobs + +pict-rs 0.5.2 now is smarter about queueing certain cleanup jobs. Previously, it might queue an +arbitrary number of the same cleanup job while under significant load, resulting in many redundant +jobs consuming CPU. The reason for this is pict-rs' strategy for handling variants and proxied media +that hadn't been accessed within the configured timeframe was to queue a job to check every 30 +seconds. This is fine in normal conditions, but can be a problem after upgrading to 0.5 from 0.4. + +During the upgrade, pict-rs marks all variants as having been accessed at the time of the upgrade, +since this information was not tracked in 0.4. This results in many variants being queued for +cleanup exactly 1 week (or otherwise configured cache duration) after upgrading. When the cleanup +queue fills with these cleanup jobs, it takes longer to process them all than the 30 second window +between queues of the general cleanup jobs, resulting in redundant cleanup jobs that spawn the same +redundant removal jobs, increasing the queue size by the number of outdated variants each time. + +By adding an option to the queueing system to mark certain jobs as unique, pict-rs can now ensure +that only one copy of these general cleanup jobs can be queued or running at any given time, +ensuring that all individual variant cleanup jobs will be queued after a general cleanup job runs +before the next general cleanup job is queued. + + ### Support TLS in Postgres Connections The postgres repo now supports some new options to enable connecting via TLS. If the postgres @@ -56,6 +84,7 @@ PICTRS__REPO__USE_TLS=true PICTRS__REPO__CERTIFICATE_FILE=/path/to/ca/certificate.crt ``` + ### Compiler Version Requirements pict-rs 0.5.2 now takes advantage of rust's native support for AFIT (Async Fn In Trait) for the