2
0
Fork 0
mirror of https://git.asonix.dog/asonix/pict-rs synced 2024-12-22 19:31:35 +00:00
Commit graph

590 commits

Author SHA1 Message Date
ee00fafee4 Yield more often in queue, simplify traces, log job_id in higher span 2024-01-30 14:18:07 -06:00
dc12f6f79d replace remaining web::block with spawn_blocking 2024-01-30 14:16:05 -06:00
7282ee9312 Give background jobs more opportunity to yield to runtime 2024-01-25 16:59:46 -06:00
819b83bab7 Use a far smaller pool to listen for db notifications 2024-01-25 16:50:36 -06:00
c65b49339f Take at most 2048 variants and proxies during each cleanup job 2024-01-25 16:26:29 -06:00
574b12b23f Fix postgres commandline, add default for use_tls 2024-01-25 08:55:51 -06:00
2f588aa613 Ignore tokio-uring test in docs 2024-01-24 17:41:42 -06:00
fe1132aec1 Enable at-most-once queueing for some cleanup jobs 2024-01-24 17:14:31 -06:00
fee2294901 clippy 2024-01-15 18:21:36 -05:00
19147e2035 postgres: allow connecting to TLS-enabled databases 2024-01-15 18:11:08 -05:00
f6087d65be Use async fn in trait for Store 2024-01-15 15:51:20 -05:00
9ed90efed4 Add release document, use .extend rather than .join 2024-01-11 16:48:18 -06:00
708c25f20a Join identifier onto public_endpoint, rather than overwriting the existing path 2024-01-11 16:06:44 -06:00
9ceb26bb5c Clippy 2024-01-07 10:09:06 -06:00
65e165af49 Set default audio codecs if none are specified 2024-01-07 10:03:40 -06:00
ca34a15cd3 Debug-log command arguments 2024-01-07 10:01:02 -06:00
004a54c4ae Improve cooperation from background jobs 2024-01-06 17:47:44 -06:00
bcf73eb4e4 Improve cooperation from from_iterator to prevent task starvation 2024-01-05 19:50:10 -06:00
4145637a33 Add a few more metrics, increase repo disconnect sleep 2024-01-05 19:35:52 -06:00
688c797082 Remove LIMIT 10, single_value is already in use 2024-01-04 11:13:29 -06:00
6514487ea9 Index on queue_time for job queue, limit to 10 jobs in ID query 2024-01-04 11:01:25 -06:00
0fd6c459d0 Clippy 2024-01-04 10:42:34 -06:00
7d4f44a540 Set worker-id to None when marking a job as runnable after expired heartbeats 2024-01-04 10:29:37 -06:00
7e215c691b Support RUST_LOG 2023-12-28 12:40:59 -06:00
bfc2410552 Add trace-level logs to every loop (paranoid) 2023-12-28 11:58:38 -06:00
dc90e00324 Update metrics to 0.22 (plus mechanical changes) 2023-12-26 18:06:38 -06:00
bd3cdc925e Remove special-case for animated avif 2023-12-23 21:44:11 -06:00
6d259a0eea Don't kill already-exited processes, don't log tmp_dir when fetching details 2023-12-23 13:29:30 -06:00
c2e53766c2 Don't -ping when extracting file details (workaround for imagemagick json bug) 2023-12-23 12:48:32 -06:00
df8fc00ad5 Try to clean up properly even if command errors 2023-12-23 12:11:15 -06:00
89a4e3995f Add warn logs when performing blocking removes due to dropped files 2023-12-23 12:10:25 -06:00
6fa79b9188 Do more cleanup inline 2023-12-23 11:58:20 -06:00
e8380c31c1 Fix remaining warnings 2023-12-22 21:03:51 -06:00
c2b1b3db5d Apply cargo fix 2023-12-22 21:01:19 -06:00
c544f167fb Remove unused code 2023-12-22 21:00:37 -06:00
b938b09901 Apply cargo fix 2023-12-22 20:54:02 -06:00
5624671cbf Rework ProcessRead to require a closure for using the internal AsyncRead 2023-12-22 20:52:58 -06:00
df88660b9e Lower priority of some object storage spans 2023-12-22 13:58:05 -06:00
4369283534 More tracing cleaning 2023-12-22 13:30:17 -06:00
aa8fe7f3e8 Clean unused imports 2023-12-22 13:20:36 -06:00
0dbe722524 Don't poll handle if closed, prevents panic if child wait finishes before all bytes are read 2023-12-22 13:20:21 -06:00
b94ba5fcfc Inline process background future, clean tracing a bit 2023-12-22 13:12:19 -06:00
db43392a3b Simplify process future by removing explicit channel, don't lock mutex as often 2023-12-22 12:03:05 -06:00
c9155f7ce7 Include not_found key in repo migration, rework ProcessRead, add timeout to 0.4 migration 2023-12-17 23:15:27 -06:00
bd0b89b165 Clean spans a bit 2023-12-15 22:34:45 -06:00
2544cb478f Add static bound to appease 1.74.1 2023-12-12 17:17:16 -06:00
f1c5a56353 Port prune_missing from 0.4.x 2023-12-12 16:54:41 -06:00
3ef4097ceb Enable building without tokio_unstable 2023-12-12 16:23:02 -06:00
6daa745446 Add optional ability to log spans 2023-12-11 14:58:53 -06:00
cd80ab16c3 Downgrade retry warnings to debug 2023-12-11 13:08:10 -06:00
b452a577a7 Add object ID to not found error 2023-12-11 13:05:29 -06:00
7293628148 put timeout on collecting payloads 2023-12-10 19:26:16 -06:00
961de20a00 Add timeout and metrics to media processing 2023-12-10 19:11:36 -06:00
2523b6644b Add AggregateTimeout, don't aggregate with permit 2023-12-04 15:39:34 -06:00
6b2ccbf975 Allow configuring pict-rs' temporary directory 2023-11-24 11:48:45 -06:00
a30044278c Improve documentation for using pict-rs as a library 2023-11-24 11:38:14 -06:00
71f8f52b58 Tweak error reporting 2023-11-13 17:24:56 -06:00
1538389caf Clippy 2023-11-11 14:28:05 -06:00
210af5d7d9 Add danger_dummy_mode 2023-11-11 14:22:12 -06:00
5fd59fc5b4 Change commandline flag to allow disabling video, since it is enabled by default 2023-11-11 13:22:12 -06:00
3a0546694f Update opentelemetry dependencies 2023-11-10 15:19:54 -06:00
15189d293b clippy 2023-11-09 18:26:57 -06:00
f61dac8187 Attempt to clean stray magick files
This change sets a unique temp directory for each invocation of imagemagick and spawns a task to remove that directory after the command terminates
2023-11-09 18:20:59 -06:00
20d07ea29b Remove direct dependency on quick-xml, update dependencies (minor & point) 2023-11-04 13:18:22 -05:00
c795c1edfa Replace most of actix-rt with tokio, give names to tasks 2023-10-20 19:08:11 -05:00
0bf8b6a2a8 Update rusty-s3 to 0.5.0 2023-10-08 11:29:59 -05:00
71fbc4c55b Just have different mains 2023-10-07 12:31:05 -05:00
760b9d8c59 Install tracing & metrics from within runtime 2023-10-07 12:29:01 -05:00
3f5bc1d311 Build same pict-rs config for either runtime option 2023-10-07 12:22:17 -05:00
f97f76c01e Fix io-uring initialization 2023-10-07 12:12:40 -05:00
4a45aa889c Enable running on bare tokio 2023-10-07 11:48:58 -05:00
7f5cbc4571 Attempt clean-drop of tmp_dir 2023-10-07 11:36:49 -05:00
7b5a3020fa Don't rely on global state for tmp_dir 2023-10-07 11:32:36 -05:00
fff4afe105 Start working on per-server temporary directory 2023-10-06 19:42:24 -05:00
c202484e9d clippy 2023-10-04 12:46:38 -05:00
914e21c043 Implement constant-time equality for delete tokens, inline alias cleanup 2023-10-04 12:11:29 -05:00
d28c1ac628 Upgrade Concurrency: Update docs, configs, limit maxiumum details operations 2023-10-03 17:04:40 -05:00
f223d477c7 Enable specifying concurrency for 0.4 to 0.5 migration 2023-10-03 16:27:19 -05:00
75551dc3a1 Increase deadpool wait timeout, helps with extreme concurrency 2023-10-03 16:15:49 -05:00
247902e600 Enable specifying migrate-store concurrency 2023-10-03 15:50:53 -05:00
8568ae3364 Clippy 2023-09-30 22:47:49 -05:00
4c7067d4ca Allow running without old_repo specified 2023-09-30 22:46:15 -05:00
81c6e73b5f Simplify old_repo, remove client_pool_size 2023-09-30 22:46:15 -05:00
47751f3875 Downgrade some WARN logs to INFO, update docs for public methods 2023-09-30 17:52:58 -05:00
01cbe34ac3 Remove unneeded annotation 2023-09-30 17:34:05 -05:00
4bb9d81f51 Move internal middleware into own file 2023-09-30 17:33:01 -05:00
e3462f6664 payload middleware: switch to Rc, always inject if Payload isn't None 2023-09-30 17:24:48 -05:00
3267fb8301 Fail requests for malformed deadlines 2023-09-30 17:20:32 -05:00
65ea518542 Default log level to info 2023-09-30 16:28:04 -05:00
66e1711723 Enable proper draining of dropped request payloads
Doing this as the outermost middleware ensures all endpoints are covered.

Update request deadline to turn negative deadlines into immediate failures
2023-09-30 16:26:43 -05:00
6f44b05c0a Remove references to Arbiter 2023-09-29 11:54:43 -05:00
735fcd1b49 Remove dependency on actix-web-codegen 2023-09-24 15:52:31 -05:00
ecdbaa9c92 Use diesel-async re-export of Hook 2023-09-24 15:48:19 -05:00
745e990fa0 Remove direct dependency on actix-rt and actix-server 2023-09-24 15:45:10 -05:00
0926739d3c Remove once_cell 2023-09-24 15:32:00 -05:00
e0968dcce3 Remove serde-cbor, num_cpus 2023-09-24 15:22:57 -05:00
fd965bb1a5 Allow processing still images into animation formats 2023-09-24 13:20:33 -05:00
f3ef2d870b Generate details during ingest 2023-09-24 12:58:16 -05:00
8f62b0c325 clippy 2023-09-24 12:42:23 -05:00
9ffdd0432f Infer video thumbnail format from image format 2023-09-24 12:37:43 -05:00