6f95c72070
Add new feature for testing with errors, test & fix job retries
2024-03-10 22:02:27 -05:00
286279cdf5
Control warn level for long polls via cargo feature
2024-03-10 20:07:59 -05:00
dff588aafd
Switch from deadpool to bb8
2024-03-10 15:59:08 -05:00
4976fcb2eb
Remove unneeded code
2024-03-09 22:53:46 -06:00
aa4582a3f8
Remove unused store method
2024-03-09 15:38:39 -06:00
e302df7e39
Add more poll timers, spawn process from background threads
2024-03-09 15:19:13 -06:00
9fe586b9dd
Implement retries for jobs, start warning on long polls
2024-03-09 12:15:23 -06:00
40bb58d603
clippy
2024-03-03 11:13:51 -06:00
4897c90ed7
Add blurhash cleanup for sled
2024-03-03 11:11:43 -06:00
3ecefcb64e
Save blurhashes in repo, improve some error responses, simplify extracting aliases
2024-03-02 13:27:58 -06:00
348f4ce0a3
Use same UUID for directory and filename
2024-03-02 12:20:50 -06:00
25ef3861f1
Remove more unneeded 'static
2024-02-27 20:46:04 -06:00
6ee7c5c4a0
Remove unneeded 'static
2024-02-27 20:41:39 -06:00
04dcc9a0c8
Include file extension in identifiers
2024-02-27 20:41:25 -06:00
d13f7fe969
Include a few random bytes in the directory structure
2024-02-27 20:18:35 -06:00
16890eaa45
Just keep timestamp component of v7 for directory
2024-02-26 15:58:06 -06:00
c17a8722c6
Simplify object and file path generation
2024-02-26 15:43:30 -06:00
7c6112e631
ffprobe: read from file, simplify input-file access
2024-02-25 13:05:47 -06:00
277b47af46
Fix imagemagick ffmpeg delegate, strip metadata while converting videos
2024-02-25 11:40:03 -06:00
b139190663
clippy
2024-02-24 19:36:29 -06:00
03bd3cbe2f
Even more piping
2024-02-24 19:27:34 -06:00
13fc0df31a
More piping
2024-02-24 19:07:48 -06:00
2074334131
Switch more commands to be driven via STDIN
2024-02-24 14:02:41 -06:00
eabd7ea228
Delete unused functions
2024-02-24 13:21:35 -06:00
ad51e6cd9f
Avoid writing blurhashed images to tmp
2024-02-24 13:16:25 -06:00
0fd19a5682
Start work on blurhash
2024-02-23 22:12:19 -06:00
6a6c61058a
Split save_stream for object storage to early-drop stack-pinned stream
2024-02-22 18:55:12 -06:00
d73e683d48
Remove unused From
2024-02-22 18:23:39 -06:00
16bf18bda4
Remove some Unpin and 'static bounds
2024-02-22 18:05:04 -06:00
227e9cc3a7
BytesReader: use built-in state tracking
2024-02-22 17:49:16 -06:00
59b03d548d
Fix is_empty
2024-02-22 17:09:03 -06:00
f3e455a1c3
Increase buffer size for AsyncRead -> Stream conversion
2024-02-22 16:25:03 -06:00
00a08a8bc9
Improve AyncRead impl
2024-02-22 16:21:31 -06:00
de356c1f12
Remove MessageBody impl
2024-02-22 16:15:32 -06:00
3a7d5b7bfb
Re-use try_from_stream where possible
2024-02-22 16:10:34 -06:00
0ebee2a07c
cargo fix
2024-02-22 16:03:49 -06:00
c1e651c01a
Use BytesStream in more places
2024-02-22 16:03:04 -06:00
9db5cc82f8
Add special handling for FileSize errors
2024-02-19 12:55:44 -06:00
4a7a51d147
Add more imagemagick security configuration
...
Remove default security policy, since it is overridden anyway
Update default value for max_area
Inherit animation frame count configuration for list-length
Add release document
2024-02-05 14:30:01 -06:00
71d81681d5
It compiles
2024-02-04 15:45:47 -06:00
d8c4a22ba9
Start describing metrics
2024-02-04 14:26:18 -06:00
8869b82065
Update rustls for actix-web
2024-02-03 21:58:08 -06:00
85f6356025
Fix images with trailing bytes failing to upload
2024-02-03 19:50:00 -06:00
835647d290
Unite launch-with-store fns
2024-02-03 18:42:34 -06:00
c176e4c686
clippy
2024-02-03 18:30:52 -06:00
880dfc20ee
Remove unneeded struct, import
2024-02-03 18:20:17 -06:00
7eba45f37e
cargo fix & cargo fmt
2024-02-03 18:19:14 -06:00
50e31f96da
It compiles again
2024-02-03 18:18:13 -06:00
f2410a9283
use State
2024-02-03 13:32:20 -06:00
cf7c753e65
Pass a State value around rather than a bunch of arguments
2024-02-03 13:31:54 -06:00
c3e322f2c9
typo
2024-02-02 14:00:54 -06:00
3a6faef5ef
Add logs for pict-rs binds
2024-02-01 21:55:31 -06:00
5805eb0aed
Enable configuring imagemagick security policy from pictrs
2024-01-31 21:32:01 -06:00
50fd3e6182
Enable serving over TLS
2024-01-31 17:47:42 -06:00
08472eabeb
Log console address, prometheus address with tracing
2024-01-30 15:21:15 -06:00
e7357cbdaf
Count spawned tasks in cleanup, debug log queue length
2024-01-30 14:57:48 -06:00
34cadb86b8
Don't propogate pop span into blocking closure
2024-01-30 14:30:24 -06:00
ee00fafee4
Yield more often in queue, simplify traces, log job_id in higher span
2024-01-30 14:18:07 -06:00
dc12f6f79d
replace remaining web::block with spawn_blocking
2024-01-30 14:16:05 -06:00
7282ee9312
Give background jobs more opportunity to yield to runtime
2024-01-25 16:59:46 -06:00
819b83bab7
Use a far smaller pool to listen for db notifications
2024-01-25 16:50:36 -06:00
c65b49339f
Take at most 2048 variants and proxies during each cleanup job
2024-01-25 16:26:29 -06:00
574b12b23f
Fix postgres commandline, add default for use_tls
2024-01-25 08:55:51 -06:00
2f588aa613
Ignore tokio-uring test in docs
2024-01-24 17:41:42 -06:00
fe1132aec1
Enable at-most-once queueing for some cleanup jobs
2024-01-24 17:14:31 -06:00
fee2294901
clippy
2024-01-15 18:21:36 -05:00
19147e2035
postgres: allow connecting to TLS-enabled databases
2024-01-15 18:11:08 -05:00
f6087d65be
Use async fn in trait for Store
2024-01-15 15:51:20 -05:00
9ed90efed4
Add release document, use .extend rather than .join
2024-01-11 16:48:18 -06:00
708c25f20a
Join identifier onto public_endpoint, rather than overwriting the existing path
2024-01-11 16:06:44 -06:00
9ceb26bb5c
Clippy
2024-01-07 10:09:06 -06:00
65e165af49
Set default audio codecs if none are specified
2024-01-07 10:03:40 -06:00
ca34a15cd3
Debug-log command arguments
2024-01-07 10:01:02 -06:00
004a54c4ae
Improve cooperation from background jobs
2024-01-06 17:47:44 -06:00
bcf73eb4e4
Improve cooperation from from_iterator to prevent task starvation
2024-01-05 19:50:10 -06:00
4145637a33
Add a few more metrics, increase repo disconnect sleep
2024-01-05 19:35:52 -06:00
688c797082
Remove LIMIT 10, single_value is already in use
2024-01-04 11:13:29 -06:00
6514487ea9
Index on queue_time for job queue, limit to 10 jobs in ID query
2024-01-04 11:01:25 -06:00
0fd6c459d0
Clippy
2024-01-04 10:42:34 -06:00
7d4f44a540
Set worker-id to None when marking a job as runnable after expired heartbeats
2024-01-04 10:29:37 -06:00
7e215c691b
Support RUST_LOG
2023-12-28 12:40:59 -06:00
bfc2410552
Add trace-level logs to every loop (paranoid)
2023-12-28 11:58:38 -06:00
dc90e00324
Update metrics to 0.22 (plus mechanical changes)
2023-12-26 18:06:38 -06:00
bd3cdc925e
Remove special-case for animated avif
2023-12-23 21:44:11 -06:00
6d259a0eea
Don't kill already-exited processes, don't log tmp_dir when fetching details
2023-12-23 13:29:30 -06:00
c2e53766c2
Don't -ping when extracting file details (workaround for imagemagick json bug)
2023-12-23 12:48:32 -06:00
df8fc00ad5
Try to clean up properly even if command errors
2023-12-23 12:11:15 -06:00
89a4e3995f
Add warn logs when performing blocking removes due to dropped files
2023-12-23 12:10:25 -06:00
6fa79b9188
Do more cleanup inline
2023-12-23 11:58:20 -06:00
e8380c31c1
Fix remaining warnings
2023-12-22 21:03:51 -06:00
c2b1b3db5d
Apply cargo fix
2023-12-22 21:01:19 -06:00
c544f167fb
Remove unused code
2023-12-22 21:00:37 -06:00
b938b09901
Apply cargo fix
2023-12-22 20:54:02 -06:00
5624671cbf
Rework ProcessRead to require a closure for using the internal AsyncRead
2023-12-22 20:52:58 -06:00
df88660b9e
Lower priority of some object storage spans
2023-12-22 13:58:05 -06:00
4369283534
More tracing cleaning
2023-12-22 13:30:17 -06:00
aa8fe7f3e8
Clean unused imports
2023-12-22 13:20:36 -06:00
0dbe722524
Don't poll handle if closed, prevents panic if child wait finishes before all bytes are read
2023-12-22 13:20:21 -06:00
b94ba5fcfc
Inline process background future, clean tracing a bit
2023-12-22 13:12:19 -06:00
db43392a3b
Simplify process future by removing explicit channel, don't lock mutex as often
2023-12-22 12:03:05 -06:00
