443d327edf
Implement a couple more repo traits
2023-09-03 12:47:06 -05:00
eac4cd54a4
Initial work for pg notifications
2023-09-02 20:13:32 -05:00
e580e7701e
Finish implementing HashRepo
2023-09-02 19:13:00 -05:00
8921f57a21
Remove Identifier trait, Replace with Arc<str>
2023-09-02 18:38:02 -05:00
8eb4cda256
Implement first query
2023-09-02 13:52:15 -05:00
d475723087
Finish writing migrations & generate diesel schema
2023-09-02 13:35:30 -05:00
8c532c97e6
Initial postgres work
2023-09-02 11:52:55 -05:00
ba3a23ed43
Add error codes
2023-09-01 20:51:01 -05:00
fd74161c61
Rename some repo methods, generate UploadId in repo
2023-09-01 18:41:04 -05:00
cbb66f1b75
Add internal endpoint for deleting alias without a delete token
2023-09-01 17:25:13 -05:00
8fb90a6f69
Give a meaningful distinction for format in details
2023-08-31 16:26:45 -05:00
dce0827099
Increase default animation bounds, remove references to converting animations to videos
2023-08-30 21:55:53 -05:00
0b70ca49e4
clippy
2023-08-30 21:00:15 -05:00
721d9e1706
Don't try parsing an internal format for OldDetails
2023-08-30 20:55:38 -05:00
92ee60549f
Enable copying input av1 stream from mp4 to webm
2023-08-30 20:40:08 -05:00
b48a9233b2
Remove transcode from animation to video, make video transcoding 'optional'
...
Video transcoding still happens, but in many cases the video stream is able to be copied verbatim rather than being decoded & encoded
2023-08-30 20:37:54 -05:00
08fd96c2f7
clippy nit
2023-08-29 13:29:40 -05:00
c0f6b1e988
Enable searching hashes by date
2023-08-29 13:27:18 -05:00
0146202236
Fully commit to HumanDate
2023-08-29 12:59:36 -05:00
64950bfe0e
Shorten pagination slugs
2023-08-29 12:53:29 -05:00
1271ff2cc7
Clippy
2023-08-28 18:49:30 -05:00
2b6ea97773
Enable setting timestamp on hash creation, improve paging api
2023-08-28 18:43:24 -05:00
24812afeba
Add paged hash access
2023-08-28 16:02:11 -05:00
e302ab1f3d
Migrate not-found alias
2023-08-28 13:40:19 -05:00
b28cc38c38
Simplify Streamer
2023-08-23 19:12:59 -05:00
95637fdfe5
Remove direct dep on futures-util
2023-08-23 19:10:10 -05:00
8f50a15b25
Reduce reliance on futures-util
2023-08-23 11:59:42 -05:00
0352ada9d8
Don't doulbe-open repo
2023-08-16 16:39:41 -05:00
88d4cc1a6a
Exit after repo migration
2023-08-16 16:34:42 -05:00
f95e61e402
Expose repo migrations
2023-08-16 16:32:19 -05:00
ba27a1a223
Build out repo-repo migration
2023-08-16 16:09:40 -05:00
1559d57f0a
Don't overwrite existing variants
2023-08-16 15:12:16 -05:00
22cfbe979d
Note how many failures have occurred for a given hash
2023-08-16 13:29:22 -05:00
ee88823d41
clippy
2023-08-16 12:43:12 -05:00
521cdd5b9d
Enforce format on new details
2023-08-16 12:36:18 -05:00
f3c6239979
Allow skipping details generation if already generated
2023-08-16 11:57:09 -05:00
4d909ba32a
Make migration concurrent
2023-08-16 11:47:36 -05:00
ac996ff832
Remove unused error
2023-08-15 21:25:04 -05:00
1de257bb07
Ensure access values are unique
2023-08-15 21:18:25 -05:00
bd3b668d81
Don't error on duplicate - happens on retried migration
2023-08-15 20:28:42 -05:00
9fe7410dd6
Add logging to migration
2023-08-15 19:33:19 -05:00
1ee77b19d8
Make Repo object safe. Create migration
2023-08-15 19:19:03 -05:00
09236d731d
Fix poll after completion, misused compare_and_swap
2023-08-14 21:17:57 -05:00
26ca3a7195
Remove associated Bytes type
2023-08-14 20:00:00 -05:00
da876fd553
Make it compile
2023-08-14 14:25:19 -05:00
3129f7844e
BROKEN: start work on hash discriminant
2023-08-13 22:06:42 -05:00
e4e93cddf9
Return JobId from push
2023-08-13 19:52:27 -05:00
5d4486fdf4
Finish queue implementation update for sled
2023-08-13 19:47:20 -05:00
35d0c065e9
Start work on 0.5 db format - initial queuerepo changes
2023-08-13 14:12:38 -05:00
0c4d632dda
Consider timeouts a client error
2023-08-05 16:18:06 -05:00
fee4ed1e3e
Add process timeout
2023-08-05 12:41:06 -05:00
5f12be0c6a
Clippy
2023-07-26 22:53:41 -05:00
2a59a6492a
Bring back hash_alias_key
2023-07-25 20:30:22 -05:00
bd3975f455
Simplify some repo functions, remove 0.3 migration
2023-07-25 20:08:18 -05:00
f221887599
Clippy
2023-07-23 16:04:17 -05:00
aac1bb7bc4
Finish media proxy implementation
2023-07-23 15:45:52 -05:00
4cca7d0f86
Prepare for proxying media
2023-07-23 12:10:36 -05:00
75bf425c6e
Update alias query, process query to accept proxy url
2023-07-23 11:45:10 -05:00
e59483c12c
Add ProxyRepo
2023-07-23 10:23:46 -05:00
d0c619bdf7
Clippy
2023-07-22 21:14:24 -05:00
77cdeab57e
Improve metrics, add job metrics
2023-07-22 21:11:28 -05:00
7dc8ede8eb
Gate cleanup behind read-only config
2023-07-22 20:26:28 -05:00
13d6e10886
Reduce log level on variants access
2023-07-22 20:20:58 -05:00
1f3298363f
Attempt to mark unmarked variants as accessed on launch
2023-07-22 19:55:50 -05:00
fe1f7c869f
Enable cleanup of variants
...
Still TODO: doing a first pass to mark variants as accessed on launch
2023-07-22 19:41:50 -05:00
47e13ec04e
Mark variants as accessed on fetch
2023-07-22 18:50:04 -05:00
b9e6d67d15
Add retention configuration
...
This doesn't do anything yet, but is part of the Media Proxy and Variant Cleanup tickets
2023-07-22 18:37:25 -05:00
13387dec43
Fix specs
2023-07-22 17:58:03 -05:00
b786406ad0
Change IdentifierAccess to VariantAccess
2023-07-22 17:57:52 -05:00
ce0df080f4
Add prometheus metrics
2023-07-22 16:47:59 -05:00
d661732fb4
Tidy imports
2023-07-22 12:47:32 -05:00
a390b21df1
Make running pict-rs fluent
2023-07-22 12:39:39 -05:00
3ca994ee31
Replace global config with passed-around config
...
This will enable spinning up pict-rs multiple times in the same process with different configurations
2023-07-22 12:31:01 -05:00
e1262a5fda
Extract preprocess steps as urlencoded in config deserialization
2023-07-22 11:38:09 -05:00
ac48003f45
Extract ProcessMap out of static
2023-07-22 11:16:19 -05:00
8d35c2449d
Simplify the cancel-safe processor
2023-07-22 10:46:16 -05:00
c1127b4117
Set content-length in PutObject, CompleteMultipart
2023-07-21 19:45:22 -05:00
2f0a3618d8
Replace awc with reqwest
2023-07-21 16:58:31 -05:00
fc14ac2038
Clippy
2023-07-18 21:59:27 -05:00
3d93300ba9
Add access repos for alias & identifier
2023-07-18 21:56:13 -05:00
127dd6cdad
Make max file count configurable
2023-07-18 20:32:17 -05:00
eeac900d7e
Add quality value to process method, move -coalesce earlier in pipelines
2023-07-18 16:18:01 -05:00
558605381d
Fix dangling unprocessed uploads
...
Adds error boundary around backgrounded ingest
2023-07-17 22:32:08 -05:00
496bcc5dc1
Only include -b:v 0 for VP9
...
also simplify building arguments
2023-07-17 18:13:50 -05:00
3aecef3a9e
Remove println
2023-07-17 17:47:48 -05:00
b1bbc6b159
Attempt adding quality settings to pict-rs
2023-07-17 17:45:26 -05:00
c4c920191f
Add read-only mode
2023-07-17 14:24:49 -05:00
dd1d509bb1
Enable configuring download client timeout
2023-07-17 13:44:31 -05:00
5e8ab7856d
Extract Status errors into command-specific errors
2023-07-17 13:30:08 -05:00
2961aae6e3
Port migration changes from 0.4.1
2023-07-16 22:07:42 -05:00
a751d92436
Port error changes to 0.5
2023-07-16 21:51:14 -05:00
9871ad2ee0
Clippy
2023-07-14 23:16:57 -05:00
4d78ce7689
Fix store delegation for public_url
2023-07-14 15:23:07 -05:00
786f583d98
Add public_url to Store for optional redirects to public object store urls
...
Add details generation to file migration to set the content-type for uploads
2023-07-14 14:53:37 -05:00
4e58388b41
Clean alias before checking hash option
2023-07-13 19:58:31 -05:00
afeac8294a
Fix Alias cleanup ordering
...
Update dev.toml
2023-07-13 19:57:09 -05:00
ec6c9aac1e
Better discovery validation, proper invoking of imagemagick
2023-07-13 19:21:57 -05:00
9b1dd1f8c3
Copy process from 0.4.x branch
2023-07-13 19:21:28 -05:00
2c22f2ee3a
It compiles and runs, but doesn't work
2023-07-13 17:42:21 -05:00
27451971a6
Fix tests, add animated_webp test
2023-07-13 14:34:40 -05:00
33b83f97f2
It compiles
...
Need to reinstate media limits
2023-07-13 13:48:59 -05:00
ad1837f9dd
VERY BROKEN: start replacing parts of pict-rs
2023-07-12 22:12:21 -05:00
58d9765594
WIP: Rework format detection and conversions
...
- Add a bunch of types & methods that aren't used yet. The idea is
creating the output from the known input parameters is pure, and works
for any uploaded media
- Introduce a new kind of media "animation" which is neither an image
nor a video. It is likely that animations can be processed properly at
the /image/process.{ext} endpoint with a little massaging
TODO:
- Integrate this into exiftool, ffmpeg, magick commands
- Detect apng with ffmpeg (imagemagick treats it as a still png unless it's given a
hint)
- Infallible conversion from Details to OutputFile - this might be
impossible given that we don't currently store more than a mime type
and a frame count to give any sort of indication of real type
- Try not to break API
2023-07-11 23:11:25 -05:00
691bca286c
Make object storage timeouts configurable
2023-07-11 13:01:58 -05:00
f5c39f9be5
Typo, error formatting
2023-07-10 17:15:43 -05:00
4be2eb67d1
Increase object storage signature expiry to 15 seconds
2023-07-10 16:28:51 -05:00
9004ecaadf
Better classify process related errors
2023-07-10 15:29:41 -05:00
e10cb1401d
Bump version
2023-07-09 15:16:04 -05:00
7dadbcf3ed
Replace UnsupportedFormat error with more helpful errors
2023-07-09 15:07:49 -05:00
1694f49436
Use json for ffmpeg, imagemagick details parsing
2023-07-09 14:50:58 -05:00
686be1d93b
Turn export_path into flag for sled
2023-07-08 19:56:26 -05:00
7768d4e58e
Add database exports
...
This should help admins running pict-rs on traditional filesystems to
keep valid backups of pict-rs' sled repo
2023-07-08 17:36:01 -05:00
4680f9c3aa
Instrument repo and store build methods
2023-07-08 10:42:03 -05:00
d51c348a3f
Run health checks on old and new stores before migrating
2023-07-07 14:05:13 -05:00
3ccb8ecd8c
Validate alias represents real file before setting not found
2023-07-07 13:33:27 -05:00
9e7376d411
Remove is_missing error, make Identifier and DeleteToken return Option on fetch
2023-07-07 13:17:26 -05:00
7274538b70
Try adding store healthcheck
2023-07-07 12:05:42 -05:00
c150903980
Expand store differentiation
2023-07-07 11:19:39 -05:00
75be876d3c
Differentiate missing files in old vs new store for migration
2023-07-07 11:03:10 -05:00
508dd4340b
Fix panic when migrating < 100 files, improve resumed migration messaging
2023-07-06 16:06:37 -05:00
3487cb0e30
Add ability to set 404 image
...
Fix imagemagick blur arguments
2023-07-05 19:07:03 -05:00
e7e4876908
Explicitly handle cases of invalid aliases
2023-07-05 16:46:44 -05:00
23e67b9697
Enhance migration handling of missing identifiers
...
Queue a cleanup task for hashes that don't have original file identifiers
Be more specific about what field is missing when a field is missing
2023-07-05 09:52:19 -05:00
2274dfecb4
Fix io-uring - read_to_end was polling Stream mapped through StreamReader after completion
2023-07-05 00:35:07 -05:00
c8e84b8914
Greatly improve error information during migration
2023-07-02 21:07:15 -05:00
e6f17cbac6
Log on successful migration
2023-06-29 11:45:06 -05:00
01d3610f46
Add basic progress indication for store migration
2023-06-29 11:39:47 -05:00
0d1df5ebc2
Avoid failed identifier cleanup on failed duplicate upload
2023-06-23 11:43:15 -05:00
7ba2fc9b26
Fix duplicate uploads potentially purging existing uploads on failure
2023-06-23 11:39:43 -05:00
58f0c328d9
Fix typo in error message
2023-06-23 11:20:55 -05:00
a1d48cbb42
Share client between object store and request extractor
...
Add config to limit connection pool size
2023-06-23 11:20:20 -05:00
071d788349
Include original error in object storage sendrequest error
2023-06-21 17:07:14 -05:00
a3ff072350
Remove unused cache duration parameter
2023-06-21 17:05:53 -05:00
6d2aef8cc0
Add jpegxl and avif support
2023-06-21 17:05:35 -05:00
554d852e68
Create unique errors for Repo and Store, separate from UploadError
...
Add .is_not_found() for Store errors, wire up Object and File storage to properly set NotFound
Allow skipping files that are not found during store migration
2023-06-20 15:59:08 -05:00
1214b51ad7
Make use of FilesystemDefaults when constructing migrate commands
2023-06-19 15:04:36 -05:00
a3a986638d
Improve gif handling.
...
- Allow a fast-path exif cleaning if we'd re-encode a gif to a gif
- Use single-frame palettes to better map transparency from frame to frame
Unrelated:
- decrease ffmpeg logging when creating thumbnails
2023-06-19 14:25:31 -05:00
22e526fd2a
Allow type underscore (probably in instrument)
2023-06-01 17:33:43 -05:00
cc314c37a6
Use libopus for ffmpeg
2023-03-09 21:29:56 -06:00
dc27200a35
Add transparency to gifs, rustfmt
2023-02-25 14:31:57 -06:00
97ba4e806b
Clippy
2023-02-25 13:01:55 -06:00
9a2f846cd1
Detect input transparency for gif/webm
2023-02-25 12:59:47 -06:00
281ac43dff
Remove cache functionality
2023-02-25 11:34:48 -06:00
123488f617
clippy
2023-02-12 23:18:27 -06:00
788b87eceb
Remove obsolute pre-0.3 migrate check
2023-02-12 23:13:47 -06:00
138499d47f
Use transpose in a couple places
2023-02-05 20:50:59 -06:00
cc75d2d882
Clippy
2023-02-04 17:54:56 -06:00
4cc810d372
Add control over gif frame count
2023-02-04 17:52:23 -06:00
88ca9793e8
s/vrames/frames:v
2023-02-04 17:47:03 -06:00
40f57be0c7
Allow uploading small gifs
2023-02-04 17:32:36 -06:00
0aa3f574a5
Clippy
2023-01-29 11:57:59 -06:00
f6d6d54b88
Update deps
2023-01-29 11:47:28 -06:00
f0f40db8c3
Add healthcheck for db
2023-01-29 11:36:09 -06:00
5e5dd27a05
Retry migrate_file, remove variant before relating new one
2023-01-04 18:58:05 -06:00
28f7a139a0
Retry in-process, and rate-limit self
2023-01-04 17:26:05 -06:00
c55b1115d1
Increase client timeout, attempt to keep track of migration progress for resuming
2023-01-04 15:58:32 -06:00
dabcc24f98
Parse environment variables - main app
2023-01-04 15:05:34 -06:00
a489411c6c
Parse environment variables
2023-01-04 14:42:04 -06:00
3c844d86c5
Allow configuring pict-rs through serializable types
2022-12-26 16:35:25 -06:00
2c1a37fb62
Extract pict-rs as library
2022-12-23 12:56:15 -06:00
74cd3392fa
Borrow parent spans when creating children
2022-12-07 22:43:12 -06:00
3325b222a2
clippy
2022-11-26 11:18:34 -06:00
7037ee5791
Support auto-orienting photos
2022-10-15 11:13:24 -05:00
669b3fb86f
Fix webp metadata stripping, more tracing cleanup
2022-10-01 22:47:52 -05:00
718f09c43a
Clean tracing, simplify validation, rename InputFormat -> VideoFormat
2022-10-01 21:17:18 -05:00
e7cf21f862
Allow concurrency in small object uploads
2022-10-01 13:06:19 -05:00
fe2a499110
Fix sending upload parts (set size), optimize small file uploads
2022-10-01 13:00:07 -05:00
0307dc5a3b
Introduce BytesStream to simplify aggregating, forwarding bytes
2022-10-01 12:06:33 -05:00
c767804430
Default to vp9 for uploaded video
2022-10-01 10:17:32 -05:00
85c5ce955d
Clippy
2022-09-30 20:02:46 -05:00
d86d9a3228
Put proper file extension on videos
2022-09-30 20:00:14 -05:00
8eb2293808
Enable selecting video and audio codecs for uploaded media
2022-09-30 19:38:11 -05:00
f4542efcc1
Share ffprobe details code
2022-09-30 17:43:40 -05:00
3ceaaa391c
Update clap
2022-09-28 18:23:41 -05:00
656b0225af
Clippy
2022-09-27 23:23:06 -05:00
29cab025c6
Improve 0.3 migration code, repo traces
2022-09-27 23:19:52 -05:00
85f20bbe0e
Add new resize options, update README with current APIs
2022-09-26 18:14:25 -05:00
9db057fdc5
Add admin endpoint to fetch identifier from an alias
2022-09-25 20:59:57 -05:00
17dab63662
Clippy
2022-09-25 20:44:24 -05:00
80c83eb491
Add ffprobe for details inspection - vastly improve video detection speed
2022-09-25 20:39:09 -05:00
5449bb82f1
Better identify video uploads with ffprobe
2022-09-25 19:34:51 -05:00
21e3e63ac3
Instrument upload store methods
2022-09-25 18:46:50 -05:00
51d8621015
Defensively format errors in traces to avoid deadlocks
2022-09-25 18:46:26 -05:00
890478e794
Support audio in uploaded videos, allow webm uploads
2022-09-25 18:16:37 -05:00
c57a48db8a
Start introducing more video controls
2022-09-25 17:36:07 -05:00
c9a74a73ca
Reduce redundancy in process module
2022-09-25 17:35:52 -05:00
50d118a2a7
Implement image pre-processing on upload
2022-09-25 15:17:33 -05:00
04bc586a26
Fix hasher test
2022-09-25 09:09:05 -05:00
dc95417846
Improve concurrency for upload streams part 2
2022-09-25 08:27:11 -05:00
bf3c47e457
Improve concurrency for upload streams
2022-09-24 22:07:06 -05:00
a6adde874e
Implement s3 multipart uploads
2022-09-24 20:33:59 -05:00
25209e29c0
Make it compile
2022-09-24 17:18:53 -05:00
ff1771e016
More migration work for rusty-s3
2022-09-24 14:18:49 -05:00
ab7fd9aaf7
Start work on using rusty-s3 instead of rust-s3
2022-09-24 13:39:27 -05:00
c5b107eb4c
clippy
2022-09-11 10:04:37 -05:00
7f49af58fe
Update actix-form-data, impl Store & Repo for Data<T>
2022-09-10 19:29:22 -05:00
e0bc95f785
Remove unneeded reborrow
2022-08-15 15:00:12 -05:00
547934071a
Don't migrate from the old database if it doesn't exist
2022-08-14 11:49:20 -05:00
e0543b2c1c
Add HEAD endpoints for process, serve
2022-07-17 11:15:43 -05:00
cb9a594f24
Fix content type on already processed images
2022-06-21 11:27:22 -05:00
dc73bdbad4
Make already-claimed a 400
2022-06-05 23:24:58 -05:00
2db76ad8c5
Update dependencies
2022-04-29 15:54:54 -05:00
8ee26ccf4f
change some import paths
2022-04-27 20:13:49 -05:00
4ac20546ce
Improve migration from 0.3 formats
2022-04-24 17:10:15 -05:00
b3cbda1337
Add internal endpoint for cleaning variants
2022-04-11 16:56:39 -05:00
e50ccfb48c
Update deps
2022-04-11 15:17:54 -05:00
05533f7e3a
Use rfc3339 dates for details responses
2022-04-08 13:36:06 -05:00
7436f15267
Dry download client logic
2022-04-08 13:16:22 -05:00
5f19b63166
Extract details processing
2022-04-08 13:03:00 -05:00
31c5a36c77
Better instrument drops, jobs. Properly disarm backgrounded downloads
2022-04-08 12:51:33 -05:00
55d5e43cd2
Consolidate endpoints for downloading media, cached or backgrounded
2022-04-08 12:05:14 -05:00
07c61cbe87
Add download backgrounded
2022-04-07 19:07:30 -05:00
8781bc8f28
Add follows-from relation for background tasks
2022-04-07 13:28:28 -05:00
e493e90dd4
Defensive tracing
2022-04-07 12:56:40 -05:00
c80d207a87
Improve tracing
2022-04-06 21:40:49 -05:00
05e2cf5e08
Cache on Alias rather than Hash
...
Make sled repo blocking tasks share parent span
2022-04-06 12:14:12 -05:00
865f4ee09c
Caching works:
...
- Insert new bucket for current access time before removing
previous access time record.
- Add instrument to more methods for more precise tracing
2022-04-05 21:47:35 -05:00
6cdae7b318
Add 'cache' endpoint for ingesting ephemeral media
...
By default, cached media should only stick around for 7 days, however
The timeout is reset every time media is accessed, so only obscure
cached media will be flushed from the cache. This '7 days' number is
configurable through the commandline run options as --media-cache-duration
and in the pict-rs.toml file as [media] cache_duration
2022-04-05 20:29:30 -05:00
b0670c2f66
Move migrate to old
2022-04-03 15:07:31 -05:00
409b7d4c54
Remove unused conversions
2022-04-03 12:51:21 -05:00
29f0774331
Add ability to claim uploads
2022-04-02 21:15:39 -05:00
8734dfbdc7
Add backgrounding routes
...
- Accept backgrounded uploads
- Allow backgrounded processing
Still TODO:
- Endpoint for waiting on/claiming an upload
2022-04-02 20:56:29 -05:00
c4d014597e
Better job recovery
2022-04-02 18:53:03 -05:00
77a400c7ca
Implement UploadRepo
2022-04-02 17:41:00 -05:00
132e395e5c
Multiple items:
...
- Reduce duplicate work in generate job
- Use hash instead of identifier for unique processing
- Move motion ident generation behind concurrent processor lock
2022-04-02 17:40:08 -05:00
09f53b9ce6
Move away from UploadManager to direct repo & store actions
2022-04-02 16:44:03 -05:00
6ed592c432
In Progress: process jobs
...
- Is this a good idea? it definitely will make interacting with pict-rs harder. Maybe it's best not to do this
2022-04-01 16:51:12 -05:00
c0d8e0e8e3
Support multiple queues for job processor
2022-04-01 11:51:46 -05:00
09281d9ae8
Move stream timeout to all response bodies
2022-03-29 16:48:26 -05:00
63d66050c8
Enable buffering iterator items for iterstream
2022-03-29 16:18:00 -05:00
5adb3fde89
Move some Stream implementations into 'stream' module
2022-03-29 15:59:17 -05:00
0e490ff54a
Remove custom BytesFreezer stream
2022-03-29 14:28:08 -05:00
25f4480809
Remove custom stream error mapping types
2022-03-29 14:17:20 -05:00
8226a3571d
Enable multi-threaded job processing
2022-03-29 13:18:47 -05:00
602d1ea935
Begin implementing queue, put cleanups in it
2022-03-29 12:51:16 -05:00
e25a4781a8
Remove unneeded variable
2022-03-29 11:09:23 -05:00
99d042f36e
Allow disabling gif/mp4
2022-03-29 11:04:56 -05:00
eb5e39c634
Use color-eyre
2022-03-28 20:47:46 -05:00
1291bf8beb
Divorce reqwest from main application's spans
2022-03-28 18:42:22 -05:00
1662f153ce
Numerous changes:
...
- Improve error printing (display chain in addition to spantrace)
- Fix migration (read main identifier from identifier tree, not filename tree)
- Ensure uniqueness for processed images in ConcurrentProcessor (use source identifier in addition to thumbnail path, include extension in thumbnail path)
- Update default log levels (make pict-rs quieter)
- Add timeout for serving images from object storage (5 seconds)
2022-03-28 15:34:36 -05:00
3792a8923a
Finish redoing config probably
2022-03-27 23:27:07 -05:00
ca28f68ef5
Start redoing config again
2022-03-27 19:10:06 -05:00
37e6b21b55
Fix Range, consolidate errors, test object storage
2022-03-26 20:45:12 -05:00
15b52ba6ec
Get it compiling again
2022-03-26 16:49:23 -05:00
323016f994
Build out migration path
2022-03-25 18:47:50 -05:00
750ce4782e
Rework configuration
2022-03-24 22:06:29 -05:00
d6567fbbbd
Implement sled repo
2022-03-24 17:09:15 -05:00
a0c99d05eb
Create initial trait representation for database
2022-03-24 13:16:41 -05:00
2ad536db17
Promote console from compile flag to runtime flag
2022-03-21 22:41:51 -05:00
f13f870a92
Only use console config when console feature is enabled
2022-03-21 22:05:32 -05:00
62615be371
Allow configuring console buffer capacity
2022-03-21 21:56:19 -05:00
d30415bb49
Don't debug PathStore
2022-03-21 21:43:38 -05:00
9a8dc0e44f
Fix sled cache capacity help text
2022-03-21 21:18:51 -05:00
da159c69cb
Try setting a lower default cache capacity
2022-03-21 21:15:43 -05:00
21b6e1dc08
Use actix-web's Range header
2022-03-01 11:23:15 -06:00
73137ede21
Release candidate 1
2022-02-26 12:22:30 -06:00
0e948a2bf5
Add flag for console
2022-02-01 10:10:52 -06:00
445c99dbf1
Re-use existing BytesMut when streaming file
2022-01-13 18:11:46 -06:00
f79d38df8d
Update rust-runner image, improve tokio-console support
2021-12-19 12:59:51 -06:00
c48808c1e1
Update deps
2021-12-18 20:24:22 -06:00
44d171252b
console
2021-12-16 20:46:46 -06:00
c445615c47
Update to latest betas
2021-12-12 14:48:37 -06:00
e209a282d9
No more follows-from for process
2021-12-06 21:11:43 -06:00
11739cd42c
Remove fancy span following code
2021-12-06 20:35:22 -06:00
a47f2a3428
Log on new span, no pretty
2021-12-06 18:48:07 -06:00
920f1f154c
Improve follows-from and parent span relationships
2021-12-06 12:43:03 -06:00
42aed5165c
Update tracing-awc, add drone
2021-12-03 14:21:31 -06:00
7a3d1bef5a
Only log closing spans, not opening spans
2021-11-24 21:54:27 -06:00
5f5c1d3530
Merge branch 'main' of git.asonix.dog:asonix/pict-rs into main
2021-11-23 16:31:19 -06:00
5b44de2070
clippy
2021-11-23 16:31:15 -06:00
79afd0e8ea
Update deps
2021-11-23 12:29:42 -06:00
0d153ca239
Make stores take filename into account when saving files
2021-11-16 09:45:16 -06:00
c5d6610276
Properly handle out-of-bounds range requests
2021-11-03 12:04:59 -05:00
4c8da2b414
Keep a reqwest client around for sharing an object storage connection pool
2021-11-02 17:21:00 -05:00
ad7c323985
Update readme, cleanup migration key on completion
2021-11-01 10:44:49 -05:00
93c268fc58
Flush at every migrated file
2021-11-01 10:38:11 -05:00
ccb9f49b8b
Implement storage migration
2021-10-31 21:11:35 -05:00
cf50da30ec
Use rust-s3 fork with asyncwrite support
2021-10-31 12:35:11 -05:00
76e3d37009
Clean up ObjectStore's Debug impl
2021-10-28 21:07:31 -05:00
366db1e1fe
Make object storage with minio work
2021-10-28 20:59:11 -05:00
657152e1a8
Don't println
2021-10-28 00:18:18 -05:00
f596edc363
Improve configuration documentation
2021-10-28 00:17:37 -05:00
f9f4fc63d6
Optionally support s3-compatible storage (untested)
2021-10-27 23:06:03 -05:00
26a2401027
Use tmp files for mp4s
2021-10-23 14:14:12 -05:00
6f04595c3b
Remove unneeded mime conversions
2021-10-23 12:35:07 -05:00
36b83f114a
Merge branch 'main' into asonix/generic-pictrs-storage
2021-10-22 23:49:47 -05:00
48557bc2ea
Make pict-rs generic over file storage
2021-10-22 23:48:56 -05:00
2295837ea4
2021
2021-10-21 16:36:31 -05:00
f67aeb92fa
Keep better track of gif/mp4 thumbnail
2021-10-20 21:36:18 -05:00
1767942e31
Remove recursion
2021-10-20 20:36:10 -05:00
d69d261999
Make it actually compilable
2021-10-20 20:30:21 -05:00
4aefc0403a
Remove more boxes
2021-10-20 20:13:39 -05:00
877390878b
Remove more boxes
2021-10-20 19:28:40 -05:00
e7c3e2c96c
Rename stream module
2021-10-20 19:00:41 -05:00
e7b4e4d1cc
Remove a few boxes
2021-10-20 18:58:32 -05:00
922200673e
Update to latest betas
2021-10-20 18:16:53 -05:00
9559a8b202
Update cli flags
2021-10-18 23:51:04 -05:00
3fe66129be
Update opentelemetry url info
2021-10-18 23:44:56 -05:00
34038a0f16
Fix restructure, upload
2021-10-18 23:37:11 -05:00
195f614a4b
Restructure attempt 1
2021-10-18 20:29:06 -05:00
