* On registration, automatically set content languages from accept header
* no need to set site language or default language for new user anymore
* fix test
* fix langs
* avoid duplicate writing of new user languages
* Add a blocklist for URLs.
* Fix SQL format
* Make clippy happy.
* Use regex for URL matching.
* Escape regex chars in URLs.
* Use post for modification.
* Make URL block regex static and remove API routes.
* Add date fields to table and use transaction.
* Use Cache for blocklist.
* Rename check_links + move list to parameters of process_markdown.
* SQL format.
* Format, again.
* Remove println.
* Add API test.
* Set a shorter lifetime for regex in debug mode.
* Add missing macro.
* Update lemmy-js-client
* Update api_test/pnpm-lock.yaml
* Don't break other tests
* Use different URL for test
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
* When banning a federated user, also remove their content from our local
communities.
- This works by:
- Before a site ban, find all posts and comments to local communities
- Send a federated community ban action for each local comm.
- This also removes their content in the apub receive code.
- Adding back in federated community ban api tests.
- Adding in two more api tests for site bans.
- Fixes#4118
* Add local community ban, and nonlocal person check.
* Ignoring errors.
* Move local check into function.
* Addressing PR comments 2
* Adding ability to hide posts.
- Adds an post/hide API route.
- Adds a `show_hidden` (default false) to `GetPosts`.
- Adds a `hidden` field to `PostView`.
- Removes the single `post_id` from MarkPostAsRead.
- Fixes#1403
* Add a check to make sure hidden field is true.
* Fixing test.
* Add back semicolon
* Add support for local only community (fixes#1576)
* add filters and tests to db views
* dont federate local only community
* test get apub community http
* tests
* more checks
* wip
* api test
* fix tests
* change community.local_only column to visibility enum
(for private communities)
* sql fmt
* rename vars
* clippy
* fix tests
* update lib
* review
* fix js client version
* update client
* Add markdown rule to add rel=nofollow for all links
* Add markdown image rule to add local image proxy (fixes#1036)
* comments
* rewrite markdown image links working
* add comment
* perform markdown image processing in api/apub receivers
* clippy
* add db table to validate proxied links
* rewrite link fields for avatar, banner etc
* sql fmt
* proxy links received over federation
* add config option
* undo post.url rewriting, move http route definition
* add tests
* proxy images through pictrs
* testing
* cleanup request.rs file
* more cleanup (fixes#2611)
* include url content type when sending post over apub (fixes#2611)
* store post url content type in db
* should be media_type
* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.
* fix tests
* add setting disable_external_link_previews
* federate post url as image depending on mime type
* change setting again
* machete
* invert
* support custom emoji
* clippy
* update defaults
* add image proxy test, fix test
* fix test
* clippy
* revert accidental changes
* address review
* clippy
* Markdown link rule-dess (#4356)
* Extracting opengraph_data to its own type.
* A few additions for markdown-link-rule.
---------
Co-authored-by: Nutomic <me@nutomic.com>
* fix setting
* use enum for image proxy setting
* fix test configs
* add config backwards compat
* clippy
* machete
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Dont send comment reply to user who has community blocked. Fixes#3684
* Adding source instance block check.
* Adding api test.
* Addressing PR comments.
* Handle invalid ban expires values (fixes#4045)
* Adding a few missing expire time checks. Fixing up time conversions. (#4051)
* Adding a few missing expire time checks. Fixing up time conversions.
* Increase settings export wait time.
* get rid of RemoveCommunity.expires
* fmt
* tests
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Ignore activities in remote communities without local followers (fixes#3568)
* x
* comments
* prettier
* fix api test
* fix test
* cleanup
* fix remaining test
* clippy
* decrease delay
* wip
* stuff
* fmt
* fmt 2
* fmt 3
* fix default feature
* use Authorization header
* store ip and user agent for each login
* add list_logins endpoint
* serde(skip) for token
* fix api tests
* A few suggestions for login_token (#3991)
* A few suggestions.
* Fixing SQL format.
* review
* review
* rename cookie
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* User can block instances (fixes#2397)
* update comments
* review comments
* use route
* update
* add api test
* update tests
* fix
* fix test
* ci
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* persistent activity queue
* fixes
* fixes
* make federation workers function callable from outside
* log federation instances
* dead instance detection not needed here
* taplo fmt
* split federate bin/lib
* minor fix
* better logging
* log
* create struct to hold cancellable task for readability
* use boxfuture for readability
* reset submodule
* fix
* fix lint
* swap
* remove json column, use separate array columns instead
* some review comments
* make worker a struct for readability
* minor readability
* add local filter to community follower view
* remove separate lemmy_federate entry point
* fix remaining duration
* address review comments mostly
* fix lint
* upgrade actitypub-fed to simpler interface
* fix sql format
* increase delays a bit
* fixes after merge
* remove selectable
* fix instance selectable
* add comment
* start federation based on latest id at the time
* rename federate process args
* dead instances in one query
* filter follow+report activities by local
* remove synchronous federation
remove activity sender queue
* lint
* fix federation tests by waiting for results to change
* fix fed test
* fix comment report
* wait some more
* Apply suggestions from code review
Co-authored-by: SorteKanin <sortekanin@gmail.com>
* fix most remaining tests
* wait until private messages
* fix community tests
* fix community tests
* move arg parse
* use instance_id instead of domain in federation_queue_state table
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
* Revert "Automatically resolve report when post/comment is removed (#3850)"
This reverts commit f7f6766650.
* Automatically resolve reports using db trigger
* lint
* use mod log tables
* fix migration
* fix ci
* fix clippy
* Move admin flag from person to local_user (fixes#3060)
The person table is for federated data, but admin flag can only
apply to local users. Thats why it really belongs in the local_user
table. This will also prevent the federation code from accidentally
overwriting the admin flag
* fmt
* try to fix api tests
* lint
* fix person view
* ci
* ci
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Lowercase domain on db query filters (#3849)
* Add test to get a community on different cased domain (#3849)
* Lowercase the identity for webfinger (#3849)
* Lowercase both sides of the domain comparison (#3849)
* Format api_tests (#3849)
* Lowercase domain lookup on Instance and Person (#3849)
---------
Co-authored-by: Freek van Zee <freek.van.zee@mediamonks.com>
Co-authored-by: Freakazoid182 <>
* generate sitemap.xml file
* set up endpoint for sitemap
* Update sitemap generation
- remove sitemap generation from scheduled tasks
- add posts query for sitemap
- create sitemap module in API crate
* remove priority and change freq from sitemap
* add configuration option for number of posts for sitemap
* fix default config
* rate limit sitemap endpoint
* update sitemap query
* update sitemap generation
- remove config value for query limit
- adjust sitemap generation to query changes
- tidy up error handling
* refactor sitemap generation loop
* remove `limit` argument
* refactor `generate_urlset` and add unit test
* change query to only fetch local posts of past 24h
* fix outdated comment and log
* cargo fmt
* Try using drone cache plugin
* Try another path
* Include volume
* Fix formatting
* Include fmt
* Exclude cargo dir from prettier
* Don't override cargo
* Just do check
* Add cache key
* Use different cache plugin
* Add clippy
* Try minio
* Add quotes
* Try adding secrets
* Try again
* Again
* Use correct secret formation
* Add back clippy
* Use secret for the root bucket name
* Try drone cache instead
* Add region
* Add path-style option
* Include cargo clippy
* Include everything again
* Fix formatting
* Don't run clippy twice
* Add `allow` statements for tests to pass
* Adjust endpoint to be a secret
* Fix prettier
* Merge & fix tests
* Try to restart the woodpecker test
* Change the ENV var name
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Split activity table into sent and received parts (fixes#3103)
The received activities are only stored in order to avoid processing
the same incoming activity multiple times. For this purpose it is
completely unnecessary to store the data. So we can split the
table into sent_activity and received_activity parts, where
only sent_activity table needs to store activity data. This should
reduce storage use significantly.
Also reduces activity storage duration to three months, we can reduce
this further if necessary.
Additionally the id columns of activity tables are removed because
they are completely unused and risk overflowing (fixes#3560).
* address review
* move insert_received_activity() methods to verify handlers
* remove unnecessary conflict line
* clippy
* use on conflict, add tests
