175b0f962a
Use off-the-shelf object storage library
...
This library bundles its own reqwest client unfortunately
2024-06-24 14:19:36 -05:00
58529a2eb2
Update reqwest to 0.12
...
This pulls in hyper 1 and http 1, but removes rustls 0.21
2024-05-01 14:46:29 -05:00
6f95c72070
Add new feature for testing with errors, test & fix job retries
2024-03-10 22:02:27 -05:00
4976fcb2eb
Remove unneeded code
2024-03-09 22:53:46 -06:00
aa4582a3f8
Remove unused store method
2024-03-09 15:38:39 -06:00
348f4ce0a3
Use same UUID for directory and filename
2024-03-02 12:20:50 -06:00
04dcc9a0c8
Include file extension in identifiers
2024-02-27 20:41:25 -06:00
c17a8722c6
Simplify object and file path generation
2024-02-26 15:43:30 -06:00
7c6112e631
ffprobe: read from file, simplify input-file access
2024-02-25 13:05:47 -06:00
6a6c61058a
Split save_stream for object storage to early-drop stack-pinned stream
2024-02-22 18:55:12 -06:00
16bf18bda4
Remove some Unpin and 'static bounds
2024-02-22 18:05:04 -06:00
227e9cc3a7
BytesReader: use built-in state tracking
2024-02-22 17:49:16 -06:00
f3e455a1c3
Increase buffer size for AsyncRead -> Stream conversion
2024-02-22 16:25:03 -06:00
d8c4a22ba9
Start describing metrics
2024-02-04 14:26:18 -06:00
dc12f6f79d
replace remaining web::block with spawn_blocking
2024-01-30 14:16:05 -06:00
f6087d65be
Use async fn in trait for Store
2024-01-15 15:51:20 -05:00
9ed90efed4
Add release document, use .extend rather than .join
2024-01-11 16:48:18 -06:00
708c25f20a
Join identifier onto public_endpoint, rather than overwriting the existing path
2024-01-11 16:06:44 -06:00
bfc2410552
Add trace-level logs to every loop (paranoid)
2023-12-28 11:58:38 -06:00
df88660b9e
Lower priority of some object storage spans
2023-12-22 13:58:05 -06:00
b94ba5fcfc
Inline process background future, clean tracing a bit
2023-12-22 13:12:19 -06:00
bd0b89b165
Clean spans a bit
2023-12-15 22:34:45 -06:00
b452a577a7
Add object ID to not found error
2023-12-11 13:05:29 -06:00
20d07ea29b
Remove direct dependency on quick-xml, update dependencies (minor & point)
2023-11-04 13:18:22 -05:00
c795c1edfa
Replace most of actix-rt with tokio, give names to tasks
2023-10-20 19:08:11 -05:00
0bf8b6a2a8
Update rusty-s3 to 0.5.0
2023-10-08 11:29:59 -05:00
745e990fa0
Remove direct dependency on actix-rt and actix-server
2023-09-24 15:45:10 -05:00
cde4a72203
Add metrics to object storage body streams
2023-09-20 19:46:10 -05:00
49d3037358
Add metrics to object storage requests
2023-09-20 19:39:03 -05:00
1b97ac1c5a
Start replacing manual stream implementations with streem
2023-09-10 18:55:13 -04:00
a43de122f9
postgres: add already-claimed case, general: tracing paranoia
2023-09-03 21:30:47 -05:00
8a892ba622
it might work
2023-09-03 18:21:46 -05:00
8921f57a21
Remove Identifier trait, Replace with Arc<str>
2023-09-02 18:38:02 -05:00
8c532c97e6
Initial postgres work
2023-09-02 11:52:55 -05:00
ba3a23ed43
Add error codes
2023-09-01 20:51:01 -05:00
95637fdfe5
Remove direct dep on futures-util
2023-08-23 19:10:10 -05:00
8f50a15b25
Reduce reliance on futures-util
2023-08-23 11:59:42 -05:00
1ee77b19d8
Make Repo object safe. Create migration
2023-08-15 19:19:03 -05:00
bd3975f455
Simplify some repo functions, remove 0.3 migration
2023-07-25 20:08:18 -05:00
c1127b4117
Set content-length in PutObject, CompleteMultipart
2023-07-21 19:45:22 -05:00
2f0a3618d8
Replace awc with reqwest
2023-07-21 16:58:31 -05:00
4d78ce7689
Fix store delegation for public_url
2023-07-14 15:23:07 -05:00
786f583d98
Add public_url to Store for optional redirects to public object store urls
...
Add details generation to file migration to set the content-type for uploads
2023-07-14 14:53:37 -05:00
691bca286c
Make object storage timeouts configurable
2023-07-11 13:01:58 -05:00
4be2eb67d1
Increase object storage signature expiry to 15 seconds
2023-07-10 16:28:51 -05:00
4680f9c3aa
Instrument repo and store build methods
2023-07-08 10:42:03 -05:00
7274538b70
Try adding store healthcheck
2023-07-07 12:05:42 -05:00
a1d48cbb42
Share client between object store and request extractor
...
Add config to limit connection pool size
2023-06-23 11:20:20 -05:00
071d788349
Include original error in object storage sendrequest error
2023-06-21 17:07:14 -05:00
554d852e68
Create unique errors for Repo and Store, separate from UploadError
...
Add .is_not_found() for Store errors, wire up Object and File storage to properly set NotFound
Allow skipping files that are not found during store migration
2023-06-20 15:59:08 -05:00
