Commit graph

173 commits

Author SHA1 Message Date
Dessalines
d476d32200
Removing a few more Result<bool> . ()
* Removing a few more Result<bool> .

* Running taplo fmt.

* Running fmt.

* Adding email taken test.

* Fixing tests.

* Adding back in missing admin check.

* Rename check_has_local_followers function.
2024-09-23 20:55:35 -04:00
a8843335a6
Simplify handling of NotFound SQL errors (fixes ) ()
* Simplify handling of NotFound SQL errors (fixes )

* fmt

* wip

* compiling

* clippy

* api tests

* fix
2024-09-23 11:26:50 -04:00
Dessalines
2b3fd70afd
Adding ability to restore content on user unban. ()
* Adding ability to restore content on user unban.

- Fixes 

* Fixing api tests.

* Fix package.json

* Fixing lemmy-js-client dep.

* Adding API test for restoring content.
2024-09-18 09:11:42 -04:00
flamingos-cant
bd1b7aa6ef
Let federation modify groups ()
* Don't check if group is local

* Add API test for mods modifying communities
2024-08-20 16:16:14 +02:00
Dessalines
f8c7375731
Upgrading deps. () 2024-08-01 17:04:28 -04:00
phiresky
a08642f813
federation: parallel sending per instance ()
* federation: parallel sending

* federation: some comments

* lint and set force_write true when a request fails

* inbox_urls return vec

* split inbox functions into separate file

* cleanup

* extract sending task code to separate file

* move federation concurrent config to config file

* off by one issue

* improve msg

* fix both permanent stopping of federation queues and multiple creation of the same federation queues

* fix after merge

* lint fix

* Update crates/federate/src/send.rs

Co-authored-by: dullbananas <dull.bananas0@gmail.com>

* comment about reverse ordering

* remove crashable, comment

* comment

* move comment

* run federation tests twice

* fix test run

* prettier

* fix config default

* upgrade rust to 1.78 to fix diesel cli

* fix clippy

* delay

* add debug to make localhost urls not valid in ap crate, add some debug logs

* federation tests: ensure server stop after test and random activity id

* ci fix

* add test to federate 100 events

* fix send 100 test

* different data every time so activities are distinguishable

* allow out of order receives in test

* lint

* comment about https://github.com/LemmyNet/lemmy/pull/4623#discussion_r1565437391

* move sender for clarity, add comment

* move more things to members

* update test todo comment, use same env var as worker test but default to 1

* remove else below continue

* some more cleanup

* handle todo about smooth exit

* add federate inboxes collector tests

* lint

* actor max length

* don't reset fail count if activity skipped

* fix some comments

* reuse vars

* format

* Update .woodpecker.yml

* fix recheck time

* fix inboxes tests under fast mode

* format

* make i32 and ugly casts

* clippy

---------

Co-authored-by: dullbananas <dull.bananas0@gmail.com>
2024-07-21 11:50:50 -04:00
Dessalines
6d8d23130d
Adding an image_details table to store image dimensions. ()
* Adding an image_details table to store image dimensions.

- Adds an image_details table, which stores the height,
  width, and content_type for local and remote images.
- For LocalImages, this information already comes back with
  the upload.
- For RemoteImages, it calls the pictrs details endpoint.
- Fixed some issues with proxying non-image urls.
- Fixes 
- Also fixes 

* Running sql format.

* Running fmt.

* Don't fetch metadata in background for local API requests.

* Dont export remote_image table to typescript.

* Cleaning up validate.

* Dont proxy url.

* Fixing tests, fixing issue with federated thumbnails.

* Fix tests.

* Updating corepack, fixing issue.

* Refactoring image inserts to use transactions.

* Use select exists again.

* Fixing imports.

* Fix test.

* Removing pointless backgrounded metadata generation version.

* Removing public pictrs details route.

* Fixing clippy.

* Running prettier.

* A few more fixes.

* Moving diesel schema check back down.

* Addressing PR comments.

* Changing back request head to get.

* Fixing lockfile.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-06-20 12:44:06 +02:00
Dessalines
16a82862b8
Allow empty string to clear URL-type DB fields. ()
* Allow empty string to clear URL-type DB fields.

- To address difficulties with clearing URL-type fields like
  avatars, banners, site icons, this PR turns the URL type form
  fields into strings.
- This allows an empty string to be used as a "clear data", as
  in the case with the regular text form fields.
- Also includes various cleanups.
- Fixes 
- Context: 

* Fixing comment.

* Use Option<&str> and deref.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-06-06 09:55:08 -04:00
Dessalines
8bf17946bd
Fix issue with avatar / icon deletion when saving settings. ()
* Fix issue with avatar / icon deletion when saving settings.

- Fixes 

* Update crates/api_common/src/request.rs

Co-authored-by: dullbananas <dull.bananas0@gmail.com>

* Fixing an existing test, and adding another for replace images.

---------

Co-authored-by: dullbananas <dull.bananas0@gmail.com>
2024-06-04 08:28:22 -04:00
Richard Schwab
91e57ff954
Prevent bot replies from increasing unread reply count when bot accounts are not shown ()
* Prevent bot replies from increasing unread reply count when bot accounts are not shown

* Pass LocalUser for unread replies count query

* Prevent bot mentions from increasing unread reply count when bot accounts are not shown
2024-05-29 17:55:15 -04:00
Dessalines
6a6108ac55
Fixing proxied images for federated posts. ()
* Fixing proxied images for federated posts.

- Also added test.
- Fixes 

* Address PR comments.
2024-05-23 11:11:25 -04:00
Dessalines
55f84dd38a
Fixing proxy images ()
* Adding an image_details table to store image dimensions.

- Adds an image_details table, which stores the height,
  width, and content_type for local and remote images.
- For LocalImages, this information already comes back with
  the upload.
- For RemoteImages, it calls the pictrs details endpoint.
- Fixed some issues with proxying non-image urls.
- Fixes 
- Also fixes 

* Running sql format.

* Running fmt.

* Don't fetch metadata in background for local API requests.

* Dont export remote_image table to typescript.

* Cleaning up validate.

* Dont proxy url.

* Fixing tests, fixing issue with federated thumbnails.

* Fix tests.

* Updating corepack, fixing issue.

* Refactoring image inserts to use transactions.

* Use select exists again.

* Fixing imports.

* Fix test.

* Removing pointless backgrounded metadata generation version.

* Removing public pictrs details route.

* Fixing clippy.

* Fixing proxy image fetching. Fixes 

- This extracts only the proxy image fixes from , leaving off
  thumbnails.

* Fix test.

* Addressing PR comments.

* Address PR comments 2.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-05-22 10:28:47 +02:00
7fb03c502e
Add test to ensure reports are sent to user's home instance (ref ) ()
* Add test to ensure reports are sent to user's home instance (ref )

* enable all tests

* set package-manager-strict=false
2024-05-14 22:48:24 -04:00
31829b6c05
Untangle thumbnail generation logic (ref ) ()
* Untangle thumbnail generation logic (ref )

* prettier

* test cleanup

* fix tests

* also consider opengraph image for local thumbnail generation
2024-04-17 10:36:45 -04:00
TechVest
b0370ae2fd
chore: fix some comments ()
Signed-off-by: TechVest <techdashen@qq.com>
2024-04-17 14:35:54 +02:00
Dessalines
6efab9aab1
Creating a LocalImageView, so that front ends have the Person struct. ()
* Creating a LocalImageView, so that front ends have the Person struct.

* Removing local_user from LocalImageView.

* Add uploader check.
2024-04-16 19:20:44 -04:00
Dessalines
555f789269
Fixing custom_thumbnail updates. ()
* Fixing custom_thumbnail updates.

* Fixing issue with image posts.

* Fixing upgrade deps script.

* Adding API tests for custom thumbnails.

* Remove pointless todo.

* Address PR comments.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-04-10 10:59:46 -04:00
9059de8569
Allow fetching from local url, add fetch redirect test (fixes ) ()
* Allow fetching from local url, at fetch redirect test (fixes )

* prettier

* update lib

* update apub lib
2024-04-10 10:04:57 -04:00
a4b79ca610
Generate post thumbnail/metadata in background (ref ) ()
* Generate post thumbnail/metadata in background (ref )

* fix api test

* Apply suggestions from code review

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>

* fix test

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-03-27 10:54:42 -04:00
85ee89f4e8
When uploading new icon/avatar/banner, delete old one () 2024-03-27 09:00:52 -04:00
Dessalines
6bfbb9332d
Adding listMedia endpoint, to view all your local image uploads. ()
* Adding listMedia endpoint, to view all your local image uploads.

- Fixes 

* Fix ts import.

* Forgot to order by published desc

* Adding an endpoint to list all images, for admins only.

* Forgot to add file.

* Add additional test.

* Use better logic for no-limit version.

* Better call sites.

* Adding another test.

* Fix tests.

* Moving list_media to /account action.

* Addressing PR comments.

* Removing pointless comment.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-03-26 12:06:11 -04:00
846848c4f6
On registration, automatically set content languages from accept-language header ()
* On registration, automatically set content languages from accept header

* no need to set site language or default language for new user anymore

* fix test

* fix langs

* avoid duplicate writing of new user languages
2024-03-25 16:02:12 -04:00
Dessalines
2fd81067c7
Temporarily comment sticky test. () 2024-03-18 12:54:45 +01:00
9d4299aaac
Dont require leading ! or @ for webfinger resolve ()
* Dont require leading ! or @ for webfinger resolve

* fmt

* clippy
2024-03-15 08:42:09 -04:00
flamingos-cant
19a1a077c5
Add a blocklist for URLs. ()
* Add a blocklist for URLs.

* Fix SQL format

* Make clippy happy.

* Use regex for URL matching.

* Escape regex chars in URLs.

* Use post for modification.

* Make URL block regex static and remove API routes.

* Add date fields to table and use transaction.

* Use Cache for blocklist.

* Rename check_links + move list to parameters of process_markdown.

* SQL format.

* Format, again.

* Remove println.

* Add API test.

* Set a shorter lifetime for regex in debug mode.

* Add missing macro.

* Update lemmy-js-client

* Update api_test/pnpm-lock.yaml

* Don't break other tests

* Use different URL for test

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2024-03-15 07:03:29 -04:00
157378b4c9
Clear text of deleted/removed comments () 2024-03-05 08:52:35 -05:00
Dessalines
7f9950fe85
Add alt_text for posts. Fixes ()
* Add alt_text for posts. Fixes 

* Moving alt_text to attachment name.

* Cleaning up mod action line.

* Addressing PR comments

* Addressing PR comments.

* Fixing clones.
2024-03-05 11:34:57 +01:00
3c358e5b0b
Mods and admins can comment in locked posts (fixes ) ()
* Mods and admins can comment in locked posts (fixes )

* fmt

* fix

* fix test
2024-03-04 09:15:21 -05:00
Dessalines
7eec8714d7
When site banning a federated user, also remove their content from our local communities. ()
* When banning a federated user, also remove their content from our local
communities.

- This works by:
  - Before a site ban, find all posts and comments to local communities
  - Send a federated community ban action for each local comm.
  - This also removes their content in the apub receive code.
- Adding back in federated community ban api tests.
- Adding in two more api tests for site bans.
- Fixes 

* Add local community ban, and nonlocal person check.

* Ignoring errors.

* Move local check into function.

* Addressing PR comments 2
2024-03-01 13:45:06 -05:00
ab4deaa49a
Add api test for synchronizing featured posts (ref ) ()
* Correctly synchronize collection of community featured posts (fixes )

* Add api test for synchronizing featured posts (ref )

* prettier
2024-02-27 09:11:41 -05:00
0f414a95d5
Local only community ()
* Add support for local only community (fixes )

* add filters and tests to db views

* dont federate local only community

* test get apub community http

* tests

* more checks

* wip

* api test

* fix tests

* change community.local_only column to visibility enum
(for private communities)

* sql fmt

* rename vars

* clippy

* fix tests

* update lib

* review

* fix js client version

* update client
2024-01-25 11:04:25 -05:00
e8a52d3a5c
Rewrite images to use local proxy ()
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes )

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes )

* include url content type when sending post over apub (fixes )

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess ()

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00
İsmail Karslı
8670403a67
Add local_subscribers field to CommunityAggregates. Fixes ()
* Add upload timeout to PictrsConfig

* Bad space 🤔

* Update PictrsConfig upload timeout to include units.

* Add local_subscribers field to CommunityAggregates
struct and schema

* sql format

* local_subscribers test

* fix local_subscribers test

* Revert "fix local_subscribers test"

This reverts commit 4bbac5ce4a.

* Revert "local_subscribers test"

This reverts commit 735107e1f7.

* Create trigger for local_subscribers

* Rename variable

* re-trigger ci

* re-trigger ci

* Add local_subscribers count to follow.spec.ts

* Rename local_subscribers to subscribers_local

* Add subscribers_local to community_aggregates

* added subscribers_local to the aggregate tests

* Check if person exists on community_follower trigger

* Delete community follows before deleting person

* Update lemmy-js-client in api_tests

* Refactor local_subscriber migration

* fix format

* Move migration files date to now

* Fix test to wait for aggregates to federate

* re-trigger ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-24 10:22:05 -05:00
3d6f7ff911
Revert "Dont ignore errors during login (fixes ) ()" ()
This reverts commit 4163e0465e.

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-19 11:21:43 -05:00
1be7dbde33
Count chars, not bytes for max title length (fixes ) ()
* Count chars, not bytes for max title length (fixes )

* fix api test
2024-01-15 09:33:39 -05:00
023c9f4fcd
Fix fetching of community posts (fixes ) ()
* Fix fetching of community posts (fixes )

Also use spawn_try_task to fetch community outbox, mods etc to avoid
delay/timeout when fetching new community.

* prettier

* fix test

* fix api test

* prettier

* add delay

* Update run-federation-test.sh

* fix test
2024-01-04 11:42:18 -05:00
4163e0465e
Dont ignore errors during login (fixes ) ()
* Dont ignore errors during login (fixes )

* fix test

* fmt
2024-01-03 10:34:03 -05:00
Dessalines
bc32b408b5
Fixing private message reports. () 2023-12-18 10:25:05 +01:00
Dessalines
ac209d58b4
Adding a retry on a few fetches. () 2023-12-14 16:26:42 +01:00
01aa17f38e
Reenable API tests that were accidentally skipped ()
* Reenable API tests that were accidentally skipped

* log get comment parent id failure
2023-12-13 09:29:10 -05:00
e0e74e50ae
Dont use test.png image for api tests () 2023-12-12 10:58:22 -05:00
b2ae69f96c
Set explicit limit for getPosts/getComments in tests ()
* Set explicit limit for getPosts/getComments in tests

* pretier
2023-12-12 08:54:54 -05:00
Dessalines
2b5a31b411
Upgrading deps. ()
* Upgrading deps.

* Try to use native node-fetch function.

* Fixing image upload tests.
2023-11-24 10:29:41 +01:00
cafeb14f1c
Add API tests for purge user and purge post ()
* Add API tests for purge user and purge post

* prettier

* fix test

* ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-11-22 10:15:06 -05:00
e58a167d64
Fix remote community posts appearing in local listing (fixes ) ()
* Fix remote community posts appearing in local listing (fixes )

* fmt

* fix typo
2023-11-22 09:41:52 -05:00
25450ea090
Fix following local communities stuck on pending (fixes ) ()
* Fix following local communities stuck on pending (fixes )

* fmt

* remove import

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-11-17 04:55:26 -05:00
525359f7c5
Add api tests for image endpoints ()
* Add api tests for image endpoints (fixes )

* curl instead of wget

* add missing files

* revert cargo update

* simplify setup

* use const

* rename to image.spec.ts

* adjust to client changes

* update client lib

* remove todos, move import

* try to fix ci

---------

Co-authored-by: SleeplessOne1917 <insomnia_void@protonmail.com>
2023-11-16 23:43:40 -05:00
a00313e680
Merge /site_inbox into /inbox, remove unique constraint for inboxes ()
* Merge /site_inbox into /inbox (fixes )

Get rid of different inboxes, only use /inbox

Remove shared_inbox_url db columns

add code migration

move to db migration, fixes

machete

fix sql

drop inbox url unique constraints

Dont create auth cookie in backend ()

dont change individual inboxes to shared inbox

Dont send comment reply to user who has community blocked. Fixes  ()

* Dont send comment reply to user who has community blocked. Fixes 

* Adding source instance block check.

* Adding api test.

* Addressing PR comments.

* move site inbox rewrite to db

* fix test

* clippy

* clippy 2

* fix test
2023-11-16 14:22:40 +01:00
Dessalines
e00b0c9da2
Dont send comment reply to user who has community blocked. Fixes ()
* Dont send comment reply to user who has community blocked. Fixes 

* Adding source instance block check.

* Adding api test.

* Addressing PR comments.
2023-11-09 12:03:25 +01:00
568233b062
Resolve federated objects from other instances via redirect (fixes ) ()
* Resolve federated objects from other instances via redirect (fixes )

* restore domain check using library change

* add test case, update apub lib

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-25 07:14:59 -04:00