|
|
f2410a9283
|
use State
|
2024-02-03 13:32:20 -06:00 |
|
|
|
cf7c753e65
|
Pass a State value around rather than a bunch of arguments
|
2024-02-03 13:31:54 -06:00 |
|
|
|
c3e322f2c9
|
typo
|
2024-02-02 14:00:54 -06:00 |
|
|
|
3a6faef5ef
|
Add logs for pict-rs binds
|
2024-02-01 21:55:31 -06:00 |
|
|
|
5805eb0aed
|
Enable configuring imagemagick security policy from pictrs
|
2024-01-31 21:32:01 -06:00 |
|
|
|
50fd3e6182
|
Enable serving over TLS
|
2024-01-31 17:47:42 -06:00 |
|
|
|
08472eabeb
|
Log console address, prometheus address with tracing
|
2024-01-30 15:21:15 -06:00 |
|
|
|
e7357cbdaf
|
Count spawned tasks in cleanup, debug log queue length
|
2024-01-30 14:57:48 -06:00 |
|
|
|
34cadb86b8
|
Don't propogate pop span into blocking closure
|
2024-01-30 14:30:24 -06:00 |
|
|
|
ee00fafee4
|
Yield more often in queue, simplify traces, log job_id in higher span
|
2024-01-30 14:18:07 -06:00 |
|
|
|
dc12f6f79d
|
replace remaining web::block with spawn_blocking
|
2024-01-30 14:16:05 -06:00 |
|
|
|
7282ee9312
|
Give background jobs more opportunity to yield to runtime
|
2024-01-25 16:59:46 -06:00 |
|
|
|
819b83bab7
|
Use a far smaller pool to listen for db notifications
|
2024-01-25 16:50:36 -06:00 |
|
|
|
c65b49339f
|
Take at most 2048 variants and proxies during each cleanup job
|
2024-01-25 16:26:29 -06:00 |
|
|
|
574b12b23f
|
Fix postgres commandline, add default for use_tls
|
2024-01-25 08:55:51 -06:00 |
|
|
|
2f588aa613
|
Ignore tokio-uring test in docs
|
2024-01-24 17:41:42 -06:00 |
|
|
|
fe1132aec1
|
Enable at-most-once queueing for some cleanup jobs
|
2024-01-24 17:14:31 -06:00 |
|
|
|
fee2294901
|
clippy
|
2024-01-15 18:21:36 -05:00 |
|
|
|
19147e2035
|
postgres: allow connecting to TLS-enabled databases
|
2024-01-15 18:11:08 -05:00 |
|
|
|
f6087d65be
|
Use async fn in trait for Store
|
2024-01-15 15:51:20 -05:00 |
|
|
|
9ed90efed4
|
Add release document, use .extend rather than .join
|
2024-01-11 16:48:18 -06:00 |
|
|
|
708c25f20a
|
Join identifier onto public_endpoint, rather than overwriting the existing path
|
2024-01-11 16:06:44 -06:00 |
|
|
|
9ceb26bb5c
|
Clippy
|
2024-01-07 10:09:06 -06:00 |
|
|
|
65e165af49
|
Set default audio codecs if none are specified
|
2024-01-07 10:03:40 -06:00 |
|
|
|
ca34a15cd3
|
Debug-log command arguments
|
2024-01-07 10:01:02 -06:00 |
|
|
|
004a54c4ae
|
Improve cooperation from background jobs
|
2024-01-06 17:47:44 -06:00 |
|
|
|
bcf73eb4e4
|
Improve cooperation from from_iterator to prevent task starvation
|
2024-01-05 19:50:10 -06:00 |
|
|
|
4145637a33
|
Add a few more metrics, increase repo disconnect sleep
|
2024-01-05 19:35:52 -06:00 |
|
|
|
688c797082
|
Remove LIMIT 10, single_value is already in use
|
2024-01-04 11:13:29 -06:00 |
|
|
|
6514487ea9
|
Index on queue_time for job queue, limit to 10 jobs in ID query
|
2024-01-04 11:01:25 -06:00 |
|
|
|
0fd6c459d0
|
Clippy
|
2024-01-04 10:42:34 -06:00 |
|
|
|
7d4f44a540
|
Set worker-id to None when marking a job as runnable after expired heartbeats
|
2024-01-04 10:29:37 -06:00 |
|
|
|
7e215c691b
|
Support RUST_LOG
|
2023-12-28 12:40:59 -06:00 |
|
|
|
bfc2410552
|
Add trace-level logs to every loop (paranoid)
|
2023-12-28 11:58:38 -06:00 |
|
|
|
dc90e00324
|
Update metrics to 0.22 (plus mechanical changes)
|
2023-12-26 18:06:38 -06:00 |
|
|
|
bd3cdc925e
|
Remove special-case for animated avif
|
2023-12-23 21:44:11 -06:00 |
|
|
|
6d259a0eea
|
Don't kill already-exited processes, don't log tmp_dir when fetching details
|
2023-12-23 13:29:30 -06:00 |
|
|
|
c2e53766c2
|
Don't -ping when extracting file details (workaround for imagemagick json bug)
|
2023-12-23 12:48:32 -06:00 |
|
|
|
df8fc00ad5
|
Try to clean up properly even if command errors
|
2023-12-23 12:11:15 -06:00 |
|
|
|
89a4e3995f
|
Add warn logs when performing blocking removes due to dropped files
|
2023-12-23 12:10:25 -06:00 |
|
|
|
6fa79b9188
|
Do more cleanup inline
|
2023-12-23 11:58:20 -06:00 |
|
|
|
e8380c31c1
|
Fix remaining warnings
|
2023-12-22 21:03:51 -06:00 |
|
|
|
c2b1b3db5d
|
Apply cargo fix
|
2023-12-22 21:01:19 -06:00 |
|
|
|
c544f167fb
|
Remove unused code
|
2023-12-22 21:00:37 -06:00 |
|
|
|
b938b09901
|
Apply cargo fix
|
2023-12-22 20:54:02 -06:00 |
|
|
|
5624671cbf
|
Rework ProcessRead to require a closure for using the internal AsyncRead
|
2023-12-22 20:52:58 -06:00 |
|
|
|
df88660b9e
|
Lower priority of some object storage spans
|
2023-12-22 13:58:05 -06:00 |
|
|
|
4369283534
|
More tracing cleaning
|
2023-12-22 13:30:17 -06:00 |
|
|
|
aa8fe7f3e8
|
Clean unused imports
|
2023-12-22 13:20:36 -06:00 |
|
|
|
0dbe722524
|
Don't poll handle if closed, prevents panic if child wait finishes before all bytes are read
|
2023-12-22 13:20:21 -06:00 |
|
|
|
b94ba5fcfc
|
Inline process background future, clean tracing a bit
|
2023-12-22 13:12:19 -06:00 |
|
|
|
db43392a3b
|
Simplify process future by removing explicit channel, don't lock mutex as often
|
2023-12-22 12:03:05 -06:00 |
|
|
|
c9155f7ce7
|
Include not_found key in repo migration, rework ProcessRead, add timeout to 0.4 migration
|
2023-12-17 23:15:27 -06:00 |
|
|
|
bd0b89b165
|
Clean spans a bit
|
2023-12-15 22:34:45 -06:00 |
|
|
|
2544cb478f
|
Add static bound to appease 1.74.1
|
2023-12-12 17:17:16 -06:00 |
|
|
|
f1c5a56353
|
Port prune_missing from 0.4.x
|
2023-12-12 16:54:41 -06:00 |
|
|
|
3ef4097ceb
|
Enable building without tokio_unstable
|
2023-12-12 16:23:02 -06:00 |
|
|
|
6daa745446
|
Add optional ability to log spans
|
2023-12-11 14:58:53 -06:00 |
|
|
|
cd80ab16c3
|
Downgrade retry warnings to debug
|
2023-12-11 13:08:10 -06:00 |
|
|
|
b452a577a7
|
Add object ID to not found error
|
2023-12-11 13:05:29 -06:00 |
|
|
|
7293628148
|
put timeout on collecting payloads
|
2023-12-10 19:26:16 -06:00 |
|
|
|
961de20a00
|
Add timeout and metrics to media processing
|
2023-12-10 19:11:36 -06:00 |
|
|
|
2523b6644b
|
Add AggregateTimeout, don't aggregate with permit
|
2023-12-04 15:39:34 -06:00 |
|
|
|
6b2ccbf975
|
Allow configuring pict-rs' temporary directory
|
2023-11-24 11:48:45 -06:00 |
|
|
|
a30044278c
|
Improve documentation for using pict-rs as a library
|
2023-11-24 11:38:14 -06:00 |
|
|
|
71f8f52b58
|
Tweak error reporting
|
2023-11-13 17:24:56 -06:00 |
|
|
|
1538389caf
|
Clippy
|
2023-11-11 14:28:05 -06:00 |
|
|
|
210af5d7d9
|
Add danger_dummy_mode
|
2023-11-11 14:22:12 -06:00 |
|
|
|
5fd59fc5b4
|
Change commandline flag to allow disabling video, since it is enabled by default
|
2023-11-11 13:22:12 -06:00 |
|
|
|
3a0546694f
|
Update opentelemetry dependencies
|
2023-11-10 15:19:54 -06:00 |
|
|
|
15189d293b
|
clippy
|
2023-11-09 18:26:57 -06:00 |
|
|
|
f61dac8187
|
Attempt to clean stray magick files
This change sets a unique temp directory for each invocation of imagemagick and spawns a task to remove that directory after the command terminates
|
2023-11-09 18:20:59 -06:00 |
|
|
|
20d07ea29b
|
Remove direct dependency on quick-xml, update dependencies (minor & point)
|
2023-11-04 13:18:22 -05:00 |
|
|
|
c795c1edfa
|
Replace most of actix-rt with tokio, give names to tasks
|
2023-10-20 19:08:11 -05:00 |
|
|
|
0bf8b6a2a8
|
Update rusty-s3 to 0.5.0
|
2023-10-08 11:29:59 -05:00 |
|
|
|
71fbc4c55b
|
Just have different mains
|
2023-10-07 12:31:05 -05:00 |
|
|
|
760b9d8c59
|
Install tracing & metrics from within runtime
|
2023-10-07 12:29:01 -05:00 |
|
|
|
3f5bc1d311
|
Build same pict-rs config for either runtime option
|
2023-10-07 12:22:17 -05:00 |
|
|
|
f97f76c01e
|
Fix io-uring initialization
|
2023-10-07 12:12:40 -05:00 |
|
|
|
4a45aa889c
|
Enable running on bare tokio
|
2023-10-07 11:48:58 -05:00 |
|
|
|
7f5cbc4571
|
Attempt clean-drop of tmp_dir
|
2023-10-07 11:36:49 -05:00 |
|
|
|
7b5a3020fa
|
Don't rely on global state for tmp_dir
|
2023-10-07 11:32:36 -05:00 |
|
|
|
fff4afe105
|
Start working on per-server temporary directory
|
2023-10-06 19:42:24 -05:00 |
|
|
|
c202484e9d
|
clippy
|
2023-10-04 12:46:38 -05:00 |
|
|
|
914e21c043
|
Implement constant-time equality for delete tokens, inline alias cleanup
|
2023-10-04 12:11:29 -05:00 |
|
|
|
d28c1ac628
|
Upgrade Concurrency: Update docs, configs, limit maxiumum details operations
|
2023-10-03 17:04:40 -05:00 |
|
|
|
f223d477c7
|
Enable specifying concurrency for 0.4 to 0.5 migration
|
2023-10-03 16:27:19 -05:00 |
|
|
|
75551dc3a1
|
Increase deadpool wait timeout, helps with extreme concurrency
|
2023-10-03 16:15:49 -05:00 |
|
|
|
247902e600
|
Enable specifying migrate-store concurrency
|
2023-10-03 15:50:53 -05:00 |
|
|
|
8568ae3364
|
Clippy
|
2023-09-30 22:47:49 -05:00 |
|
|
|
4c7067d4ca
|
Allow running without old_repo specified
|
2023-09-30 22:46:15 -05:00 |
|
|
|
81c6e73b5f
|
Simplify old_repo, remove client_pool_size
|
2023-09-30 22:46:15 -05:00 |
|
|
|
47751f3875
|
Downgrade some WARN logs to INFO, update docs for public methods
|
2023-09-30 17:52:58 -05:00 |
|
|
|
01cbe34ac3
|
Remove unneeded annotation
|
2023-09-30 17:34:05 -05:00 |
|
|
|
4bb9d81f51
|
Move internal middleware into own file
|
2023-09-30 17:33:01 -05:00 |
|
|
|
e3462f6664
|
payload middleware: switch to Rc, always inject if Payload isn't None
|
2023-09-30 17:24:48 -05:00 |
|
|
|
3267fb8301
|
Fail requests for malformed deadlines
|
2023-09-30 17:20:32 -05:00 |
|
|
|
65ea518542
|
Default log level to info
|
2023-09-30 16:28:04 -05:00 |
|
|
|
66e1711723
|
Enable proper draining of dropped request payloads
Doing this as the outermost middleware ensures all endpoints are covered.
Update request deadline to turn negative deadlines into immediate failures
|
2023-09-30 16:26:43 -05:00 |
|
|
|
6f44b05c0a
|
Remove references to Arbiter
|
2023-09-29 11:54:43 -05:00 |
|
|
|
735fcd1b49
|
Remove dependency on actix-web-codegen
|
2023-09-24 15:52:31 -05:00 |
|
|
|
ecdbaa9c92
|
Use diesel-async re-export of Hook
|
2023-09-24 15:48:19 -05:00 |
|
|
|
745e990fa0
|
Remove direct dependency on actix-rt and actix-server
|
2023-09-24 15:45:10 -05:00 |
|
|
|
0926739d3c
|
Remove once_cell
|
2023-09-24 15:32:00 -05:00 |
|
|
|
e0968dcce3
|
Remove serde-cbor, num_cpus
|
2023-09-24 15:22:57 -05:00 |
|
|
|
fd965bb1a5
|
Allow processing still images into animation formats
|
2023-09-24 13:20:33 -05:00 |
|
|
|
f3ef2d870b
|
Generate details during ingest
|
2023-09-24 12:58:16 -05:00 |
|
|
|
8f62b0c325
|
clippy
|
2023-09-24 12:42:23 -05:00 |
|
|
|
9ffdd0432f
|
Infer video thumbnail format from image format
|
2023-09-24 12:37:43 -05:00 |
|
|
|
6c921817e1
|
Enable thumbnailing animations, use dynamic dispatch for a number of async readers
|
2023-09-24 11:54:16 -05:00 |
|
|
|
cde4a72203
|
Add metrics to object storage body streams
|
2023-09-20 19:46:10 -05:00 |
|
|
|
49d3037358
|
Add metrics to object storage requests
|
2023-09-20 19:39:03 -05:00 |
|
|
|
7df6f3906e
|
Add timeout, metrics to external validation request
|
2023-09-20 19:29:19 -05:00 |
|
|
|
d0646dee25
|
Add config option for external validation timeout
|
2023-09-20 19:23:55 -05:00 |
|
|
|
4a5b9712aa
|
Combine map_ok and map_err underlying impl
|
2023-09-10 23:51:07 -04:00 |
|
|
|
2b10c48619
|
Update io-uring to use streem for file bytes
|
2023-09-10 23:45:24 -04:00 |
|
|
|
ea75ca24b5
|
Add track_caller to sync methods, update streem
|
2023-09-10 20:43:51 -04:00 |
|
|
|
b2674f06d0
|
More streme
|
2023-09-10 20:08:01 -04:00 |
|
|
|
1b97ac1c5a
|
Start replacing manual stream implementations with streem
|
2023-09-10 18:55:13 -04:00 |
|
|
|
858899b943
|
Add metrics & tracing to dropped uploads, turn pop into a single query
|
2023-09-07 19:20:41 -05:00 |
|
|
|
783df60f81
|
Fix on_conflict for postgres store migration path
|
2023-09-06 20:49:00 -05:00 |
|
|
|
a761daca34
|
clippy
|
2023-09-05 20:55:11 -05:00 |
|
|
|
e15a82c0c7
|
Add external validation check
|
2023-09-05 20:45:07 -05:00 |
|
|
|
bc49f8ca37
|
Migrate with timestamp
|
2023-09-04 22:25:11 -05:00 |
|
|
|
9c50bbe23a
|
match -> matches!
|
2023-09-04 22:06:28 -05:00 |
|
|
|
492b99922b
|
10 second worker delay after disconnect error
|
2023-09-04 22:04:15 -05:00 |
|
|
|
62ebc1c60a
|
Move timeout after metrics - timeouts count as incomplete
|
2023-09-04 22:02:59 -05:00 |
|
|
|
3bd0f78e75
|
Use with_timeout for all timeouts
|
2023-09-04 21:58:57 -05:00 |
|
|
|
a4b1ab7dfb
|
Instrument postgres db calls
|
2023-09-04 21:51:27 -05:00 |
|
|
|
37448722de
|
postgres: Rework job & upload notifications (more)
postgres: Add metrics to job push & pop, upload wait
sled: add upload wait metrics
|
2023-09-04 16:20:26 -05:00 |
|
|
|
00aa00c55c
|
postgres: Enable 'previous page' when less than full limit fit on previous page
|
2023-09-03 21:58:31 -05:00 |
|
|
|
a43de122f9
|
postgres: add already-claimed case, general: tracing paranoia
|
2023-09-03 21:30:47 -05:00 |
|
|
|
31caea438e
|
Fix slow connection pool access
|
2023-09-03 20:05:29 -05:00 |
|
|
|
8a892ba622
|
it might work
|
2023-09-03 18:21:46 -05:00 |
|
|
|
a2933dbebc
|
Implement all the todos
|
2023-09-03 17:11:34 -05:00 |
|
|
|
94cb2a9ef3
|
Pass clippy
|
2023-09-03 16:59:41 -05:00 |
|
|
|
443d327edf
|
Implement a couple more repo traits
|
2023-09-03 12:47:06 -05:00 |
|
|
|
eac4cd54a4
|
Initial work for pg notifications
|
2023-09-02 20:13:32 -05:00 |
|
|
|
e580e7701e
|
Finish implementing HashRepo
|
2023-09-02 19:13:00 -05:00 |
|
|
|
8921f57a21
|
Remove Identifier trait, Replace with Arc<str>
|
2023-09-02 18:38:02 -05:00 |
|
|
|
8eb4cda256
|
Implement first query
|
2023-09-02 13:52:15 -05:00 |
|
|
|
d475723087
|
Finish writing migrations & generate diesel schema
|
2023-09-02 13:35:30 -05:00 |
|
|
|
8c532c97e6
|
Initial postgres work
|
2023-09-02 11:52:55 -05:00 |
|
|
|
ba3a23ed43
|
Add error codes
|
2023-09-01 20:51:01 -05:00 |
|
|
|
fd74161c61
|
Rename some repo methods, generate UploadId in repo
|
2023-09-01 18:41:04 -05:00 |
|
|
|
cbb66f1b75
|
Add internal endpoint for deleting alias without a delete token
|
2023-09-01 17:25:13 -05:00 |
|
|
|
8fb90a6f69
|
Give a meaningful distinction for format in details
|
2023-08-31 16:26:45 -05:00 |
|
|
|
dce0827099
|
Increase default animation bounds, remove references to converting animations to videos
|
2023-08-30 21:55:53 -05:00 |
|
|
|
0b70ca49e4
|
clippy
|
2023-08-30 21:00:15 -05:00 |
|
|
|
721d9e1706
|
Don't try parsing an internal format for OldDetails
|
2023-08-30 20:55:38 -05:00 |
|
