Commit graph

191 commits

Author SHA1 Message Date
db88ba3cbb Allow bypassing image proxy for specific domains (#5223)
* Allow bypassing proxy for some domains with ProxyAllImages

* remove web::

* remove expect

* bypass imgur by default

* correct imgur domain

* restore processing, cleanup

---------

Co-authored-by: sunaurus <sander@saarend.com>
2024-12-03 12:41:01 +01:00
9ab039da3b Proxy pictrs healthz endpoint (#5224) 2024-12-03 12:41:01 +01:00
Sander Saarend
2eab4fa267 Image proxy fixes (#4871)
* Fix remote image urls

* Remove unnecessary URL decode

* Removing the proxy changes.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2024-10-01 10:02:11 +02:00
6840a5e367 Add category to RSS feeds (fixes #3446) (#5030) 2024-09-20 10:06:41 +02:00
SleeplessOne1917
1ef375e5c3
Replace 3rd party once_cell crate with recently stabilized standard library API (#4919)
* Replace 3rd party once_cell crate with recently stabilized standard library API

* Bump CI rust version
2024-07-30 10:11:39 -04:00
Dessalines
6ff128341c
Do pictrs transformations for proxied image urls. (#4895)
- Fixes #4893
2024-07-11 17:31:31 +02:00
Dessalines
6d8d23130d
Adding an image_details table to store image dimensions. (#4704)
* Adding an image_details table to store image dimensions.

- Adds an image_details table, which stores the height,
  width, and content_type for local and remote images.
- For LocalImages, this information already comes back with
  the upload.
- For RemoteImages, it calls the pictrs details endpoint.
- Fixed some issues with proxying non-image urls.
- Fixes #3328
- Also fixes #4703

* Running sql format.

* Running fmt.

* Don't fetch metadata in background for local API requests.

* Dont export remote_image table to typescript.

* Cleaning up validate.

* Dont proxy url.

* Fixing tests, fixing issue with federated thumbnails.

* Fix tests.

* Updating corepack, fixing issue.

* Refactoring image inserts to use transactions.

* Use select exists again.

* Fixing imports.

* Fix test.

* Removing pointless backgrounded metadata generation version.

* Removing public pictrs details route.

* Fixing clippy.

* Running prettier.

* A few more fixes.

* Moving diesel schema check back down.

* Addressing PR comments.

* Changing back request head to get.

* Fixing lockfile.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-06-20 12:44:06 +02:00
dullbananas
6497ec519e
Refactor LocalUser settings conditions in database views (#4746)
* Create viewer.rs

* Rename viewer.rs to viewer.rs

* Update viewer.rs

* Update post_view.rs

* Update distinguish.rs

* Update like.rs

* Update viewer.rs

* Update list_comment_likes.rs

* Update like.rs

* Update save.rs

* Update like.rs

* revert changes in api crate

* Update post_view.rs

* Update post_view.rs

* Update comment_view.rs

* Update post_view.rs

* Update community_view.rs

* Update comment_view.rs

* Update post_view.rs

* Update viewer.rs

* Update post_view.rs

* Update community_view.rs

* Update local_user_view.rs

* Update viewer.rs

* Update community_view.rs

* Update viewer.rs

* Update lib.rs

* Update comment_view.rs

* Update post_view.rs

* Update viewer.rs

* Update viewer.rs

* Update viewer.rs

* Update viewer.rs

* Update local_user_view.rs

* Update viewer.rs

* Update viewer.rs

* Update local_user_view.rs

* Update community_view.rs

* Update viewer.rs

* Update crates/db_schema/src/viewer.rs

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>

* Update viewer.rs

* Update viewer.rs

* Update viewer.rs

* Update viewer.rs

* Update post_view.rs

* Update community_view.rs

* Update comment_view.rs

* Update viewer.rs

* Update post_view.rs

* Update save.rs

* Update resolve_object.rs

* Update viewer.rs

* Update save.rs

* Update resolve_object.rs

* Update comment_view.rs

* Update post_view.rs

* Update community_view.rs

* Update local_user_view.rs

* Update post_view.rs

* Update viewer.rs

* Update comment_view.rs

* Update post_view.rs

* Update community_view.rs

* Update viewer.rs

* Update viewer.rs

* Update viewer.rs

* Update viewer.rs

* Update viewer.rs

* Some additions to localuser DB view helpers. (#39)

* Some additions to localuser DB view helpers.

- Getting rid of generics.
- Passing in only LocalUser to views.

* Formatting fixes.

* Getting rid of unecessary as_refs

* Fixing clippy.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2024-06-14 21:51:24 -04:00
Dessalines
aefb41b551
Remove .json from nodeinfo urls, according to spec. (#4773) 2024-06-03 17:30:16 -04:00
Dessalines
4195a9b5a1
Fetch nodeinfo href from .well-known/nodeinfo . Fixes #4757 (#4765)
* Fetch nodeinfo href from .well-known/nodeinfo . Fixes #4757

* Addressing PR comments.

* Fixing clippy.

* Adding tests.
2024-06-03 17:30:00 -04:00
Dessalines
6a6c915014
Changing NodeInfo metadata to HashMap from vector. Fixes #4762 (#4764) 2024-05-31 16:38:46 -04:00
b2c1a14234
Correct url for nodeinfo version (#4734)
* Correct url for nodeinfo version

* add compat redirect

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-05-23 10:59:56 -04:00
d8dc38eb06
Upgrade dependencies (#4740) 2024-05-23 10:55:20 -04:00
Dessalines
55f84dd38a
Fixing proxy images (#4722)
* Adding an image_details table to store image dimensions.

- Adds an image_details table, which stores the height,
  width, and content_type for local and remote images.
- For LocalImages, this information already comes back with
  the upload.
- For RemoteImages, it calls the pictrs details endpoint.
- Fixed some issues with proxying non-image urls.
- Fixes #3328
- Also fixes #4703

* Running sql format.

* Running fmt.

* Don't fetch metadata in background for local API requests.

* Dont export remote_image table to typescript.

* Cleaning up validate.

* Dont proxy url.

* Fixing tests, fixing issue with federated thumbnails.

* Fix tests.

* Updating corepack, fixing issue.

* Refactoring image inserts to use transactions.

* Use select exists again.

* Fixing imports.

* Fix test.

* Removing pointless backgrounded metadata generation version.

* Removing public pictrs details route.

* Fixing clippy.

* Fixing proxy image fetching. Fixes #4703

- This extracts only the proxy image fixes from #4704, leaving off
  thumbnails.

* Fix test.

* Addressing PR comments.

* Address PR comments 2.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-05-22 10:28:47 +02:00
8b6a4c060e
Make nodeinfo standard compliant, upgrade to nodeinfo 2.1 (fixes #4702) (#4706)
* Always set activitypub protocol in nodeinfo response (fixes #4702)

* Add mandatory fields
2024-05-13 22:53:20 -04:00
TechVest
b0370ae2fd
chore: fix some comments (#4637)
Signed-off-by: TechVest <techdashen@qq.com>
2024-04-17 14:35:54 +02:00
Dessalines
d075acce43
Make all single-fetch database calls return an Option. (#4617)
- Diesel ordinarily throws an error when no results are returned for a
  single fetch, which is a bit confusing. This PR ensures that the
  missing value cases are all caught, and wrapped with new LemmyErrors,
  rather than diesel errors.
- Fixes #4601
2024-04-16 14:48:15 +02:00
Dessalines
5dea21d531
Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613 (#4614)
* Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613

* Fixing clippy.
2024-04-10 10:14:11 -04:00
ae9f82b452
Read crate version from cargo.toml (fixes #4583) (#4584) 2024-04-02 11:19:51 -04:00
5859502a2a
Fix missing private key for signed fetch (#4516)
* Fix missing private key for signed fetch (fixes #4451)

* clippy

* instance actor name and webfinger

* better webfinger handling

* upgrade lib

* update test asset
2024-03-08 10:23:15 -05:00
00f7778485
Store thumbnails in db table local_image (#4512)
* Store thumbnails in db table local_image

* fmt
2024-03-08 10:17:26 -05:00
fed6b61eaf
Upgrade apub lib, correct webfinger content-type (#4498)
* Upgrade apub lib, correct webfinger content-type

* fmt

* fix test by avoiding network fetch
2024-03-06 11:21:46 -05:00
328a48c9f5
Remove error-type feature from lemmy-utils (#4474)
* Remove error-type feature from lemmy-utils

* fixes

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-02-27 11:13:52 -05:00
SleeplessOne1917
f42420809b
Expose LemmyErrorType in lemmy_api_common (#4439)
* Expose LemmyErrorType in lemmy_api_common

* Make conditional compilation gates for utils

* Make it so api_common doesn't pull in unnecessary deps

* Make error type non exhaustive

* Fix formatting

* Format toml

* Add some convenience derives to LemmyError

* Simplify features

* Fix CI compile error

---------

Co-authored-by: SleeplessOne1917 <insomnia-void@protonmail.com>
2024-02-24 19:54:27 -05:00
86b44c2a4d
Add site.content_warning, local_site.default_post_listing_mode (#4393)
* Include local_site.content_warning setting for showing nsfw by default

* Add community setting `only_followers_can_vote`

* clippy

* add auto_expand_images site setting

* cleanup

* add missing api params

* postquery/communityquery changes

* clippy

* change error

* replace auto_expand_images with default_site_post_listing_mode

* change post/community query params

* get rid of only_followers_can_vote

* machete

* fix

* clippy

* revert remaining vote changes

* remove dead code

* remove unused var

* fmt
2024-02-16 07:24:35 -05:00
Lcchy
8a6a86c1bb
Add support for RSS media enclosures in feeds (#4442)
* Add support for RSS media enclosures in feeds

* Use post.url_content_type
2024-02-13 10:46:46 +01:00
Elara
328d48ef7e
Remove invalid XML characters from RSS feeds (#4416)
* Remove all characters that are disallowed by XML

* Combine contiguous unicode ranges into one range
2024-01-30 15:55:45 +01:00
Elara
0e9924a2b3
Add media:content thumbnail to RSS feed (#4413)
* Add media:content thumbnail to RSS feed

* Run formatter

* Add media namespace definition

* Add comment linking to media-rss documentation
2024-01-29 08:56:35 -05:00
0f414a95d5
Local only community (#4350)
* Add support for local only community (fixes #1576)

* add filters and tests to db views

* dont federate local only community

* test get apub community http

* tests

* more checks

* wip

* api test

* fix tests

* change community.local_only column to visibility enum
(for private communities)

* sql fmt

* rename vars

* clippy

* fix tests

* update lib

* review

* fix js client version

* update client
2024-01-25 11:04:25 -05:00
e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00
Dessalines
1782aafd10
Upgrading deps. (#4401) 2024-01-25 10:24:07 +01:00
Dessalines
20fd4b5869
Clippy fixes for 1.77.0 nightly (#4395)
* A few 1.77.0-nightly clippy fixes.

* Dead code warnings.

* More fixes.
2024-01-24 10:34:09 +01:00
3d6f7ff911
Revert "Dont ignore errors during login (fixes #4319) (#4321)" (#4380)
This reverts commit 4163e0465e.

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-19 11:21:43 -05:00
4163e0465e
Dont ignore errors during login (fixes #4319) (#4321)
* Dont ignore errors during login (fixes #4319)

* fix test

* fmt
2024-01-03 10:34:03 -05:00
SleeplessOne1917
4ef00e068f
Prevent crates that shouldn't be published from being published (#4309)
* Prevent crates that shouldn't be published from being published

* Make dependent crates publishable

* Make dependent crates publishable

* fix toml formatting

* Removing publish=true

* Removing versioned deps.

---------

Co-authored-by: SleeplessOne1917 <insomnia-void@protonmail.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-12-20 09:11:58 -05:00
c7ee53026f
Upgrade dependencies (#4248)
* Upgrade dependencies

* clippy
2023-12-12 08:56:33 -05:00
Dessalines
a986db1a00
Fixing rust-analyzer suggestions on rss feeds. (#4210)
* Fixing rust-analyzer suggestions on rss feeds.

* Get rid of other pointless builders.

* More cleanup.
2023-11-29 11:00:06 +01:00
dullbananas
30f7135f32
Use rust 1.74.0 in CI and move clippy lints to Cargo.toml (#4170)
* ci rust 1.74.0

* do it

* Update .woodpecker.yml

* Update Cargo.toml

* Update lint.sh

* Update lib.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-11-21 08:51:22 -05:00
İsmail Karslı
2070381e81
Add upload_timeout to PictrsConfig (#4159)
* Add upload timeout to PictrsConfig

* Bad space 🤔

* Update PictrsConfig upload timeout to include units.
2023-11-17 04:01:32 -05:00
9007d49324
Remove unused deps (#4111)
* Remove unused deps

* Adding cargo-machete to woodpecker.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-31 11:11:12 +01:00
Dessalines
bd9739b5b7
Upgrading deps. (#4103) 2023-10-25 05:32:21 -04:00
Dessalines
c0aee244e7
Hide public rss items for private instance. Fixes #3785 (#4097) 2023-10-25 10:41:52 +02:00
Dessalines
1596aee724
Adding /version route. Fixes #2914 (#4059)
* Adding /version route. Fixes #2914

* Using a simple version string.

* Use nginx rewriting to solve version.

* Forgot to remove version.

* Using an actix redirect.
2023-10-24 10:49:52 +02:00
9e099726e6
Cleanup checks for community actions (fixes #2858, fixes #2868) (#4028)
* Cleanup checks for community actions (fixes #2858, fixes #2868)

* allow restoring deleted community

* review changes

* remove unneeded sql

* remove joins

* change mod log check
2023-10-13 09:48:18 -04:00
291ff19718
Only sanitize strings when generating RSS feeds and emails (fixes #4003) (#4024)
* Only sanitize strings when generating RSS feeds and emails (fixes #4003)

* clippy

* fix test
2023-10-11 16:48:19 +02:00
dc327652a5
Add db table for login tokens which allows for invalidation (#3818)
* wip

* stuff

* fmt

* fmt 2

* fmt 3

* fix default feature

* use Authorization header

* store ip and user agent for each login

* add list_logins endpoint

* serde(skip) for token

* fix api tests

* A few suggestions for login_token (#3991)

* A few suggestions.

* Fixing SQL format.

* review

* review

* rename cookie

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-09 12:46:12 +02:00
d45a2a6441
Increase timeout for image upload to 30s (fixes #3920) (#3998) 2023-09-29 11:20:14 -04:00
f858d8cbce
Remove explicit auth params (#3946)
* Remove explicit auth params (ref #3725)

Only take auth via header or cookie. This requires a new version
of lemmy-js-client for api tests to pass.

* rework api_crud

* remove remaining auth params, move logic to session middleware

* fmt, fix test

* update js client

* remove auth param from api tests

* Pass auth as header

* add !

* url vars, setHeader

* cleanup

* fmt

* update

* Updating for new lemmy-js-client.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-09-21 06:42:28 -04:00
Anon
fe3ebea95a
Add logging for pictrs uploads (#3927)
* Add logging for pictrs uploads

* cleanup
2023-09-06 09:13:30 -04:00
fed6542055
Implement remote follow (#3738)
* Add remote follow url to webfinger response

* update apub lib
2023-08-31 09:00:41 -04:00