2
0
Fork 0
mirror of https://git.asonix.dog/asonix/pict-rs synced 2024-11-01 01:59:58 +00:00
Commit graph

572 commits

Author SHA1 Message Date
4145637a33 Add a few more metrics, increase repo disconnect sleep 2024-01-05 19:35:52 -06:00
688c797082 Remove LIMIT 10, single_value is already in use 2024-01-04 11:13:29 -06:00
6514487ea9 Index on queue_time for job queue, limit to 10 jobs in ID query 2024-01-04 11:01:25 -06:00
0fd6c459d0 Clippy 2024-01-04 10:42:34 -06:00
7d4f44a540 Set worker-id to None when marking a job as runnable after expired heartbeats 2024-01-04 10:29:37 -06:00
7e215c691b Support RUST_LOG 2023-12-28 12:40:59 -06:00
bfc2410552 Add trace-level logs to every loop (paranoid) 2023-12-28 11:58:38 -06:00
dc90e00324 Update metrics to 0.22 (plus mechanical changes) 2023-12-26 18:06:38 -06:00
bd3cdc925e Remove special-case for animated avif 2023-12-23 21:44:11 -06:00
6d259a0eea Don't kill already-exited processes, don't log tmp_dir when fetching details 2023-12-23 13:29:30 -06:00
c2e53766c2 Don't -ping when extracting file details (workaround for imagemagick json bug) 2023-12-23 12:48:32 -06:00
df8fc00ad5 Try to clean up properly even if command errors 2023-12-23 12:11:15 -06:00
89a4e3995f Add warn logs when performing blocking removes due to dropped files 2023-12-23 12:10:25 -06:00
6fa79b9188 Do more cleanup inline 2023-12-23 11:58:20 -06:00
e8380c31c1 Fix remaining warnings 2023-12-22 21:03:51 -06:00
c2b1b3db5d Apply cargo fix 2023-12-22 21:01:19 -06:00
c544f167fb Remove unused code 2023-12-22 21:00:37 -06:00
b938b09901 Apply cargo fix 2023-12-22 20:54:02 -06:00
5624671cbf Rework ProcessRead to require a closure for using the internal AsyncRead 2023-12-22 20:52:58 -06:00
df88660b9e Lower priority of some object storage spans 2023-12-22 13:58:05 -06:00
4369283534 More tracing cleaning 2023-12-22 13:30:17 -06:00
aa8fe7f3e8 Clean unused imports 2023-12-22 13:20:36 -06:00
0dbe722524 Don't poll handle if closed, prevents panic if child wait finishes before all bytes are read 2023-12-22 13:20:21 -06:00
b94ba5fcfc Inline process background future, clean tracing a bit 2023-12-22 13:12:19 -06:00
db43392a3b Simplify process future by removing explicit channel, don't lock mutex as often 2023-12-22 12:03:05 -06:00
c9155f7ce7 Include not_found key in repo migration, rework ProcessRead, add timeout to 0.4 migration 2023-12-17 23:15:27 -06:00
bd0b89b165 Clean spans a bit 2023-12-15 22:34:45 -06:00
2544cb478f Add static bound to appease 1.74.1 2023-12-12 17:17:16 -06:00
f1c5a56353 Port prune_missing from 0.4.x 2023-12-12 16:54:41 -06:00
3ef4097ceb Enable building without tokio_unstable 2023-12-12 16:23:02 -06:00
6daa745446 Add optional ability to log spans 2023-12-11 14:58:53 -06:00
cd80ab16c3 Downgrade retry warnings to debug 2023-12-11 13:08:10 -06:00
b452a577a7 Add object ID to not found error 2023-12-11 13:05:29 -06:00
7293628148 put timeout on collecting payloads 2023-12-10 19:26:16 -06:00
961de20a00 Add timeout and metrics to media processing 2023-12-10 19:11:36 -06:00
2523b6644b Add AggregateTimeout, don't aggregate with permit 2023-12-04 15:39:34 -06:00
6b2ccbf975 Allow configuring pict-rs' temporary directory 2023-11-24 11:48:45 -06:00
a30044278c Improve documentation for using pict-rs as a library 2023-11-24 11:38:14 -06:00
71f8f52b58 Tweak error reporting 2023-11-13 17:24:56 -06:00
1538389caf Clippy 2023-11-11 14:28:05 -06:00
210af5d7d9 Add danger_dummy_mode 2023-11-11 14:22:12 -06:00
5fd59fc5b4 Change commandline flag to allow disabling video, since it is enabled by default 2023-11-11 13:22:12 -06:00
3a0546694f Update opentelemetry dependencies 2023-11-10 15:19:54 -06:00
15189d293b clippy 2023-11-09 18:26:57 -06:00
f61dac8187 Attempt to clean stray magick files
This change sets a unique temp directory for each invocation of imagemagick and spawns a task to remove that directory after the command terminates
2023-11-09 18:20:59 -06:00
20d07ea29b Remove direct dependency on quick-xml, update dependencies (minor & point) 2023-11-04 13:18:22 -05:00
c795c1edfa Replace most of actix-rt with tokio, give names to tasks 2023-10-20 19:08:11 -05:00
0bf8b6a2a8 Update rusty-s3 to 0.5.0 2023-10-08 11:29:59 -05:00
71fbc4c55b Just have different mains 2023-10-07 12:31:05 -05:00
760b9d8c59 Install tracing & metrics from within runtime 2023-10-07 12:29:01 -05:00
3f5bc1d311 Build same pict-rs config for either runtime option 2023-10-07 12:22:17 -05:00
f97f76c01e Fix io-uring initialization 2023-10-07 12:12:40 -05:00
4a45aa889c Enable running on bare tokio 2023-10-07 11:48:58 -05:00
7f5cbc4571 Attempt clean-drop of tmp_dir 2023-10-07 11:36:49 -05:00
7b5a3020fa Don't rely on global state for tmp_dir 2023-10-07 11:32:36 -05:00
fff4afe105 Start working on per-server temporary directory 2023-10-06 19:42:24 -05:00
c202484e9d clippy 2023-10-04 12:46:38 -05:00
914e21c043 Implement constant-time equality for delete tokens, inline alias cleanup 2023-10-04 12:11:29 -05:00
d28c1ac628 Upgrade Concurrency: Update docs, configs, limit maxiumum details operations 2023-10-03 17:04:40 -05:00
f223d477c7 Enable specifying concurrency for 0.4 to 0.5 migration 2023-10-03 16:27:19 -05:00
75551dc3a1 Increase deadpool wait timeout, helps with extreme concurrency 2023-10-03 16:15:49 -05:00
247902e600 Enable specifying migrate-store concurrency 2023-10-03 15:50:53 -05:00
8568ae3364 Clippy 2023-09-30 22:47:49 -05:00
4c7067d4ca Allow running without old_repo specified 2023-09-30 22:46:15 -05:00
81c6e73b5f Simplify old_repo, remove client_pool_size 2023-09-30 22:46:15 -05:00
47751f3875 Downgrade some WARN logs to INFO, update docs for public methods 2023-09-30 17:52:58 -05:00
01cbe34ac3 Remove unneeded annotation 2023-09-30 17:34:05 -05:00
4bb9d81f51 Move internal middleware into own file 2023-09-30 17:33:01 -05:00
e3462f6664 payload middleware: switch to Rc, always inject if Payload isn't None 2023-09-30 17:24:48 -05:00
3267fb8301 Fail requests for malformed deadlines 2023-09-30 17:20:32 -05:00
65ea518542 Default log level to info 2023-09-30 16:28:04 -05:00
66e1711723 Enable proper draining of dropped request payloads
Doing this as the outermost middleware ensures all endpoints are covered.

Update request deadline to turn negative deadlines into immediate failures
2023-09-30 16:26:43 -05:00
6f44b05c0a Remove references to Arbiter 2023-09-29 11:54:43 -05:00
735fcd1b49 Remove dependency on actix-web-codegen 2023-09-24 15:52:31 -05:00
ecdbaa9c92 Use diesel-async re-export of Hook 2023-09-24 15:48:19 -05:00
745e990fa0 Remove direct dependency on actix-rt and actix-server 2023-09-24 15:45:10 -05:00
0926739d3c Remove once_cell 2023-09-24 15:32:00 -05:00
e0968dcce3 Remove serde-cbor, num_cpus 2023-09-24 15:22:57 -05:00
fd965bb1a5 Allow processing still images into animation formats 2023-09-24 13:20:33 -05:00
f3ef2d870b Generate details during ingest 2023-09-24 12:58:16 -05:00
8f62b0c325 clippy 2023-09-24 12:42:23 -05:00
9ffdd0432f Infer video thumbnail format from image format 2023-09-24 12:37:43 -05:00
6c921817e1 Enable thumbnailing animations, use dynamic dispatch for a number of async readers 2023-09-24 11:54:16 -05:00
cde4a72203 Add metrics to object storage body streams 2023-09-20 19:46:10 -05:00
49d3037358 Add metrics to object storage requests 2023-09-20 19:39:03 -05:00
7df6f3906e Add timeout, metrics to external validation request 2023-09-20 19:29:19 -05:00
d0646dee25 Add config option for external validation timeout 2023-09-20 19:23:55 -05:00
4a5b9712aa Combine map_ok and map_err underlying impl 2023-09-10 23:51:07 -04:00
2b10c48619 Update io-uring to use streem for file bytes 2023-09-10 23:45:24 -04:00
ea75ca24b5 Add track_caller to sync methods, update streem 2023-09-10 20:43:51 -04:00
b2674f06d0 More streme 2023-09-10 20:08:01 -04:00
1b97ac1c5a Start replacing manual stream implementations with streem 2023-09-10 18:55:13 -04:00
858899b943 Add metrics & tracing to dropped uploads, turn pop into a single query 2023-09-07 19:20:41 -05:00
783df60f81 Fix on_conflict for postgres store migration path 2023-09-06 20:49:00 -05:00
a761daca34 clippy 2023-09-05 20:55:11 -05:00
e15a82c0c7 Add external validation check 2023-09-05 20:45:07 -05:00
bc49f8ca37 Migrate with timestamp 2023-09-04 22:25:11 -05:00
9c50bbe23a match -> matches! 2023-09-04 22:06:28 -05:00
492b99922b 10 second worker delay after disconnect error 2023-09-04 22:04:15 -05:00
62ebc1c60a Move timeout after metrics - timeouts count as incomplete 2023-09-04 22:02:59 -05:00