|
|
c9155f7ce7
|
Include not_found key in repo migration, rework ProcessRead, add timeout to 0.4 migration
|
2023-12-17 23:15:27 -06:00 |
|
|
|
bd0b89b165
|
Clean spans a bit
|
2023-12-15 22:34:45 -06:00 |
|
|
|
2544cb478f
|
Add static bound to appease 1.74.1
|
2023-12-12 17:17:16 -06:00 |
|
|
|
f1c5a56353
|
Port prune_missing from 0.4.x
|
2023-12-12 16:54:41 -06:00 |
|
|
|
3ef4097ceb
|
Enable building without tokio_unstable
|
2023-12-12 16:23:02 -06:00 |
|
|
|
6daa745446
|
Add optional ability to log spans
|
2023-12-11 14:58:53 -06:00 |
|
|
|
cd80ab16c3
|
Downgrade retry warnings to debug
|
2023-12-11 13:08:10 -06:00 |
|
|
|
b452a577a7
|
Add object ID to not found error
|
2023-12-11 13:05:29 -06:00 |
|
|
|
7293628148
|
put timeout on collecting payloads
|
2023-12-10 19:26:16 -06:00 |
|
|
|
961de20a00
|
Add timeout and metrics to media processing
|
2023-12-10 19:11:36 -06:00 |
|
|
|
2523b6644b
|
Add AggregateTimeout, don't aggregate with permit
|
2023-12-04 15:39:34 -06:00 |
|
|
|
6b2ccbf975
|
Allow configuring pict-rs' temporary directory
|
2023-11-24 11:48:45 -06:00 |
|
|
|
a30044278c
|
Improve documentation for using pict-rs as a library
|
2023-11-24 11:38:14 -06:00 |
|
|
|
71f8f52b58
|
Tweak error reporting
|
2023-11-13 17:24:56 -06:00 |
|
|
|
1538389caf
|
Clippy
|
2023-11-11 14:28:05 -06:00 |
|
|
|
210af5d7d9
|
Add danger_dummy_mode
|
2023-11-11 14:22:12 -06:00 |
|
|
|
5fd59fc5b4
|
Change commandline flag to allow disabling video, since it is enabled by default
|
2023-11-11 13:22:12 -06:00 |
|
|
|
3a0546694f
|
Update opentelemetry dependencies
|
2023-11-10 15:19:54 -06:00 |
|
|
|
15189d293b
|
clippy
|
2023-11-09 18:26:57 -06:00 |
|
|
|
f61dac8187
|
Attempt to clean stray magick files
This change sets a unique temp directory for each invocation of imagemagick and spawns a task to remove that directory after the command terminates
|
2023-11-09 18:20:59 -06:00 |
|
|
|
20d07ea29b
|
Remove direct dependency on quick-xml, update dependencies (minor & point)
|
2023-11-04 13:18:22 -05:00 |
|
|
|
c795c1edfa
|
Replace most of actix-rt with tokio, give names to tasks
|
2023-10-20 19:08:11 -05:00 |
|
|
|
0bf8b6a2a8
|
Update rusty-s3 to 0.5.0
|
2023-10-08 11:29:59 -05:00 |
|
|
|
71fbc4c55b
|
Just have different mains
|
2023-10-07 12:31:05 -05:00 |
|
|
|
760b9d8c59
|
Install tracing & metrics from within runtime
|
2023-10-07 12:29:01 -05:00 |
|
|
|
3f5bc1d311
|
Build same pict-rs config for either runtime option
|
2023-10-07 12:22:17 -05:00 |
|
|
|
f97f76c01e
|
Fix io-uring initialization
|
2023-10-07 12:12:40 -05:00 |
|
|
|
4a45aa889c
|
Enable running on bare tokio
|
2023-10-07 11:48:58 -05:00 |
|
|
|
7f5cbc4571
|
Attempt clean-drop of tmp_dir
|
2023-10-07 11:36:49 -05:00 |
|
|
|
7b5a3020fa
|
Don't rely on global state for tmp_dir
|
2023-10-07 11:32:36 -05:00 |
|
|
|
fff4afe105
|
Start working on per-server temporary directory
|
2023-10-06 19:42:24 -05:00 |
|
|
|
c202484e9d
|
clippy
|
2023-10-04 12:46:38 -05:00 |
|
|
|
914e21c043
|
Implement constant-time equality for delete tokens, inline alias cleanup
|
2023-10-04 12:11:29 -05:00 |
|
|
|
d28c1ac628
|
Upgrade Concurrency: Update docs, configs, limit maxiumum details operations
|
2023-10-03 17:04:40 -05:00 |
|
|
|
f223d477c7
|
Enable specifying concurrency for 0.4 to 0.5 migration
|
2023-10-03 16:27:19 -05:00 |
|
|
|
75551dc3a1
|
Increase deadpool wait timeout, helps with extreme concurrency
|
2023-10-03 16:15:49 -05:00 |
|
|
|
247902e600
|
Enable specifying migrate-store concurrency
|
2023-10-03 15:50:53 -05:00 |
|
|
|
8568ae3364
|
Clippy
|
2023-09-30 22:47:49 -05:00 |
|
|
|
4c7067d4ca
|
Allow running without old_repo specified
|
2023-09-30 22:46:15 -05:00 |
|
|
|
81c6e73b5f
|
Simplify old_repo, remove client_pool_size
|
2023-09-30 22:46:15 -05:00 |
|
|
|
47751f3875
|
Downgrade some WARN logs to INFO, update docs for public methods
|
2023-09-30 17:52:58 -05:00 |
|
|
|
01cbe34ac3
|
Remove unneeded annotation
|
2023-09-30 17:34:05 -05:00 |
|
|
|
4bb9d81f51
|
Move internal middleware into own file
|
2023-09-30 17:33:01 -05:00 |
|
|
|
e3462f6664
|
payload middleware: switch to Rc, always inject if Payload isn't None
|
2023-09-30 17:24:48 -05:00 |
|
|
|
3267fb8301
|
Fail requests for malformed deadlines
|
2023-09-30 17:20:32 -05:00 |
|
|
|
65ea518542
|
Default log level to info
|
2023-09-30 16:28:04 -05:00 |
|
|
|
66e1711723
|
Enable proper draining of dropped request payloads
Doing this as the outermost middleware ensures all endpoints are covered.
Update request deadline to turn negative deadlines into immediate failures
|
2023-09-30 16:26:43 -05:00 |
|
|
|
6f44b05c0a
|
Remove references to Arbiter
|
2023-09-29 11:54:43 -05:00 |
|
|
|
735fcd1b49
|
Remove dependency on actix-web-codegen
|
2023-09-24 15:52:31 -05:00 |
|
|
|
ecdbaa9c92
|
Use diesel-async re-export of Hook
|
2023-09-24 15:48:19 -05:00 |
|
|
|
745e990fa0
|
Remove direct dependency on actix-rt and actix-server
|
2023-09-24 15:45:10 -05:00 |
|
|
|
0926739d3c
|
Remove once_cell
|
2023-09-24 15:32:00 -05:00 |
|
|
|
e0968dcce3
|
Remove serde-cbor, num_cpus
|
2023-09-24 15:22:57 -05:00 |
|
|
|
fd965bb1a5
|
Allow processing still images into animation formats
|
2023-09-24 13:20:33 -05:00 |
|
|
|
f3ef2d870b
|
Generate details during ingest
|
2023-09-24 12:58:16 -05:00 |
|
|
|
8f62b0c325
|
clippy
|
2023-09-24 12:42:23 -05:00 |
|
|
|
9ffdd0432f
|
Infer video thumbnail format from image format
|
2023-09-24 12:37:43 -05:00 |
|
|
|
6c921817e1
|
Enable thumbnailing animations, use dynamic dispatch for a number of async readers
|
2023-09-24 11:54:16 -05:00 |
|
|
|
cde4a72203
|
Add metrics to object storage body streams
|
2023-09-20 19:46:10 -05:00 |
|
|
|
49d3037358
|
Add metrics to object storage requests
|
2023-09-20 19:39:03 -05:00 |
|
|
|
7df6f3906e
|
Add timeout, metrics to external validation request
|
2023-09-20 19:29:19 -05:00 |
|
|
|
d0646dee25
|
Add config option for external validation timeout
|
2023-09-20 19:23:55 -05:00 |
|
|
|
4a5b9712aa
|
Combine map_ok and map_err underlying impl
|
2023-09-10 23:51:07 -04:00 |
|
|
|
2b10c48619
|
Update io-uring to use streem for file bytes
|
2023-09-10 23:45:24 -04:00 |
|
|
|
ea75ca24b5
|
Add track_caller to sync methods, update streem
|
2023-09-10 20:43:51 -04:00 |
|
|
|
b2674f06d0
|
More streme
|
2023-09-10 20:08:01 -04:00 |
|
|
|
1b97ac1c5a
|
Start replacing manual stream implementations with streem
|
2023-09-10 18:55:13 -04:00 |
|
|
|
858899b943
|
Add metrics & tracing to dropped uploads, turn pop into a single query
|
2023-09-07 19:20:41 -05:00 |
|
|
|
783df60f81
|
Fix on_conflict for postgres store migration path
|
2023-09-06 20:49:00 -05:00 |
|
|
|
a761daca34
|
clippy
|
2023-09-05 20:55:11 -05:00 |
|
|
|
e15a82c0c7
|
Add external validation check
|
2023-09-05 20:45:07 -05:00 |
|
|
|
bc49f8ca37
|
Migrate with timestamp
|
2023-09-04 22:25:11 -05:00 |
|
|
|
9c50bbe23a
|
match -> matches!
|
2023-09-04 22:06:28 -05:00 |
|
|
|
492b99922b
|
10 second worker delay after disconnect error
|
2023-09-04 22:04:15 -05:00 |
|
|
|
62ebc1c60a
|
Move timeout after metrics - timeouts count as incomplete
|
2023-09-04 22:02:59 -05:00 |
|
|
|
3bd0f78e75
|
Use with_timeout for all timeouts
|
2023-09-04 21:58:57 -05:00 |
|
|
|
a4b1ab7dfb
|
Instrument postgres db calls
|
2023-09-04 21:51:27 -05:00 |
|
|
|
37448722de
|
postgres: Rework job & upload notifications (more)
postgres: Add metrics to job push & pop, upload wait
sled: add upload wait metrics
|
2023-09-04 16:20:26 -05:00 |
|
|
|
00aa00c55c
|
postgres: Enable 'previous page' when less than full limit fit on previous page
|
2023-09-03 21:58:31 -05:00 |
|
|
|
a43de122f9
|
postgres: add already-claimed case, general: tracing paranoia
|
2023-09-03 21:30:47 -05:00 |
|
|
|
31caea438e
|
Fix slow connection pool access
|
2023-09-03 20:05:29 -05:00 |
|
|
|
8a892ba622
|
it might work
|
2023-09-03 18:21:46 -05:00 |
|
|
|
a2933dbebc
|
Implement all the todos
|
2023-09-03 17:11:34 -05:00 |
|
|
|
94cb2a9ef3
|
Pass clippy
|
2023-09-03 16:59:41 -05:00 |
|
|
|
443d327edf
|
Implement a couple more repo traits
|
2023-09-03 12:47:06 -05:00 |
|
|
|
eac4cd54a4
|
Initial work for pg notifications
|
2023-09-02 20:13:32 -05:00 |
|
|
|
e580e7701e
|
Finish implementing HashRepo
|
2023-09-02 19:13:00 -05:00 |
|
|
|
8921f57a21
|
Remove Identifier trait, Replace with Arc<str>
|
2023-09-02 18:38:02 -05:00 |
|
|
|
8eb4cda256
|
Implement first query
|
2023-09-02 13:52:15 -05:00 |
|
|
|
d475723087
|
Finish writing migrations & generate diesel schema
|
2023-09-02 13:35:30 -05:00 |
|
|
|
8c532c97e6
|
Initial postgres work
|
2023-09-02 11:52:55 -05:00 |
|
|
|
ba3a23ed43
|
Add error codes
|
2023-09-01 20:51:01 -05:00 |
|
|
|
fd74161c61
|
Rename some repo methods, generate UploadId in repo
|
2023-09-01 18:41:04 -05:00 |
|
|
|
cbb66f1b75
|
Add internal endpoint for deleting alias without a delete token
|
2023-09-01 17:25:13 -05:00 |
|
|
|
8fb90a6f69
|
Give a meaningful distinction for format in details
|
2023-08-31 16:26:45 -05:00 |
|
|
|
dce0827099
|
Increase default animation bounds, remove references to converting animations to videos
|
2023-08-30 21:55:53 -05:00 |
|
|
|
0b70ca49e4
|
clippy
|
2023-08-30 21:00:15 -05:00 |
|
|
|
721d9e1706
|
Don't try parsing an internal format for OldDetails
|
2023-08-30 20:55:38 -05:00 |
|
|
|
92ee60549f
|
Enable copying input av1 stream from mp4 to webm
|
2023-08-30 20:40:08 -05:00 |
|
|
|
b48a9233b2
|
Remove transcode from animation to video, make video transcoding 'optional'
Video transcoding still happens, but in many cases the video stream is able to be copied verbatim rather than being decoded & encoded
|
2023-08-30 20:37:54 -05:00 |
|
|
|
08fd96c2f7
|
clippy nit
|
2023-08-29 13:29:40 -05:00 |
|
|
|
c0f6b1e988
|
Enable searching hashes by date
|
2023-08-29 13:27:18 -05:00 |
|
|
|
0146202236
|
Fully commit to HumanDate
|
2023-08-29 12:59:36 -05:00 |
|
|
|
64950bfe0e
|
Shorten pagination slugs
|
2023-08-29 12:53:29 -05:00 |
|
|
|
1271ff2cc7
|
Clippy
|
2023-08-28 18:49:30 -05:00 |
|
|
|
2b6ea97773
|
Enable setting timestamp on hash creation, improve paging api
|
2023-08-28 18:43:24 -05:00 |
|
|
|
24812afeba
|
Add paged hash access
|
2023-08-28 16:02:11 -05:00 |
|
|
|
e302ab1f3d
|
Migrate not-found alias
|
2023-08-28 13:40:19 -05:00 |
|
|
|
b28cc38c38
|
Simplify Streamer
|
2023-08-23 19:12:59 -05:00 |
|
|
|
95637fdfe5
|
Remove direct dep on futures-util
|
2023-08-23 19:10:10 -05:00 |
|
|
|
8f50a15b25
|
Reduce reliance on futures-util
|
2023-08-23 11:59:42 -05:00 |
|
|
|
0352ada9d8
|
Don't doulbe-open repo
|
2023-08-16 16:39:41 -05:00 |
|
|
|
88d4cc1a6a
|
Exit after repo migration
|
2023-08-16 16:34:42 -05:00 |
|
|
|
f95e61e402
|
Expose repo migrations
|
2023-08-16 16:32:19 -05:00 |
|
|
|
ba27a1a223
|
Build out repo-repo migration
|
2023-08-16 16:09:40 -05:00 |
|
|
|
1559d57f0a
|
Don't overwrite existing variants
|
2023-08-16 15:12:16 -05:00 |
|
|
|
22cfbe979d
|
Note how many failures have occurred for a given hash
|
2023-08-16 13:29:22 -05:00 |
|
|
|
ee88823d41
|
clippy
|
2023-08-16 12:43:12 -05:00 |
|
|
|
521cdd5b9d
|
Enforce format on new details
|
2023-08-16 12:36:18 -05:00 |
|
|
|
f3c6239979
|
Allow skipping details generation if already generated
|
2023-08-16 11:57:09 -05:00 |
|
|
|
4d909ba32a
|
Make migration concurrent
|
2023-08-16 11:47:36 -05:00 |
|
|
|
ac996ff832
|
Remove unused error
|
2023-08-15 21:25:04 -05:00 |
|
|
|
1de257bb07
|
Ensure access values are unique
|
2023-08-15 21:18:25 -05:00 |
|
|
|
bd3b668d81
|
Don't error on duplicate - happens on retried migration
|
2023-08-15 20:28:42 -05:00 |
|
|
|
9fe7410dd6
|
Add logging to migration
|
2023-08-15 19:33:19 -05:00 |
|
|
|
1ee77b19d8
|
Make Repo object safe. Create migration
|
2023-08-15 19:19:03 -05:00 |
|
|
|
09236d731d
|
Fix poll after completion, misused compare_and_swap
|
2023-08-14 21:17:57 -05:00 |
|
|
|
26ca3a7195
|
Remove associated Bytes type
|
2023-08-14 20:00:00 -05:00 |
|
|
|
da876fd553
|
Make it compile
|
2023-08-14 14:25:19 -05:00 |
|
|
|
3129f7844e
|
BROKEN: start work on hash discriminant
|
2023-08-13 22:06:42 -05:00 |
|
|
|
e4e93cddf9
|
Return JobId from push
|
2023-08-13 19:52:27 -05:00 |
|
|
|
5d4486fdf4
|
Finish queue implementation update for sled
|
2023-08-13 19:47:20 -05:00 |
|
|
|
35d0c065e9
|
Start work on 0.5 db format - initial queuerepo changes
|
2023-08-13 14:12:38 -05:00 |
|
|
|
0c4d632dda
|
Consider timeouts a client error
|
2023-08-05 16:18:06 -05:00 |
|
|
|
fee4ed1e3e
|
Add process timeout
|
2023-08-05 12:41:06 -05:00 |
|
|
|
5f12be0c6a
|
Clippy
|
2023-07-26 22:53:41 -05:00 |
|
|
|
2a59a6492a
|
Bring back hash_alias_key
|
2023-07-25 20:30:22 -05:00 |
|
|
|
bd3975f455
|
Simplify some repo functions, remove 0.3 migration
|
2023-07-25 20:08:18 -05:00 |
|
|
|
f221887599
|
Clippy
|
2023-07-23 16:04:17 -05:00 |
|
|
|
aac1bb7bc4
|
Finish media proxy implementation
|
2023-07-23 15:45:52 -05:00 |
|
|
|
4cca7d0f86
|
Prepare for proxying media
|
2023-07-23 12:10:36 -05:00 |
|
|
|
75bf425c6e
|
Update alias query, process query to accept proxy url
|
2023-07-23 11:45:10 -05:00 |
|
|
|
e59483c12c
|
Add ProxyRepo
|
2023-07-23 10:23:46 -05:00 |
|
|
|
d0c619bdf7
|
Clippy
|
2023-07-22 21:14:24 -05:00 |
|
|
|
77cdeab57e
|
Improve metrics, add job metrics
|
2023-07-22 21:11:28 -05:00 |
|
|
|
7dc8ede8eb
|
Gate cleanup behind read-only config
|
2023-07-22 20:26:28 -05:00 |
|
|
|
13d6e10886
|
Reduce log level on variants access
|
2023-07-22 20:20:58 -05:00 |
|
|
|
1f3298363f
|
Attempt to mark unmarked variants as accessed on launch
|
2023-07-22 19:55:50 -05:00 |
|
|
|
fe1f7c869f
|
Enable cleanup of variants
Still TODO: doing a first pass to mark variants as accessed on launch
|
2023-07-22 19:41:50 -05:00 |
|
|
|
47e13ec04e
|
Mark variants as accessed on fetch
|
2023-07-22 18:50:04 -05:00 |
|
