Commit graph

196 commits

Author SHA1 Message Date
Dessalines
7f9950fe85
Add alt_text for posts. Fixes #1086 (#4477)
* Add alt_text for posts. Fixes #1086

* Moving alt_text to attachment name.

* Cleaning up mod action line.

* Addressing PR comments

* Addressing PR comments.

* Fixing clones.
2024-03-05 11:34:57 +01:00
3c358e5b0b
Mods and admins can comment in locked posts (fixes #4116) (#4488)
* Mods and admins can comment in locked posts (fixes #4116)

* fmt

* fix

* fix test
2024-03-04 09:15:21 -05:00
Dessalines
f3d48f2c2c
Adding some recommended fixes from nightly clippy. (#4473) 2024-02-26 09:47:10 -05:00
7316dd281a
Correctly synchronize collection of community featured posts (fixes #3867) (#4475) 2024-02-26 09:45:23 -05:00
86b44c2a4d
Add site.content_warning, local_site.default_post_listing_mode (#4393)
* Include local_site.content_warning setting for showing nsfw by default

* Add community setting `only_followers_can_vote`

* clippy

* add auto_expand_images site setting

* cleanup

* add missing api params

* postquery/communityquery changes

* clippy

* change error

* replace auto_expand_images with default_site_post_listing_mode

* change post/community query params

* get rid of only_followers_can_vote

* machete

* fix

* clippy

* revert remaining vote changes

* remove dead code

* remove unused var

* fmt
2024-02-16 07:24:35 -05:00
0f414a95d5
Local only community (#4350)
* Add support for local only community (fixes #1576)

* add filters and tests to db views

* dont federate local only community

* test get apub community http

* tests

* more checks

* wip

* api test

* fix tests

* change community.local_only column to visibility enum
(for private communities)

* sql fmt

* rename vars

* clippy

* fix tests

* update lib

* review

* fix js client version

* update client
2024-01-25 11:04:25 -05:00
e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00
023c9f4fcd
Fix fetching of community posts (fixes #4283) (#4293)
* Fix fetching of community posts (fixes #4283)

Also use spawn_try_task to fetch community outbox, mods etc to avoid
delay/timeout when fetching new community.

* prettier

* fix test

* fix api test

* prettier

* add delay

* Update run-federation-test.sh

* fix test
2024-01-04 11:42:18 -05:00
ALEX11BR
a5289dd4cf
Use pretty_assertions for assertions (#4347)
* Use `pretty_assertions` for assertions

* fixed ordering of `use`s

* ci

---------

Co-authored-by: Felix Ableitner <me@nutomic.com>
2024-01-04 04:47:18 -05:00
fc56d0aa05
Use result instead of unwrap in apub tests (#4168) 2023-11-16 22:51:33 -05:00
ec0a707110
Avoid using proxy for pictrs requests (fixes #3489) (#4072)
* Avoid using proxy for pictrs requests (fixes #3489)

* fmt
2023-10-20 11:09:34 -04:00
dullbananas
a14657d124
Refactor rate limiter and improve rate limit bucket cleanup (#3937)
* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update scheduled_tasks.rs

* Shrink `RateLimitBucket`

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* rerun ci

* Update rate_limiter.rs

* Undo changes to  fields

* Manually undo changes to RateLimitBucket fields

* fmt

* Bucket cleanup loop in rate_limit/mod.rs

* Remove rate limit bucket cleanup from scheduled_tasks.rs

* Remove ;

* Remove UNINITIALIZED_TOKEN_AMOUNT

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* stuff

* MapLevel trait

* fix merge

* Prevent negative numbers in buckets

* Clean up MapLevel::check

* MapLevel::remove_full_buckets

* stuff

* Use remove_full_buckets to avoid allocations

* stuff

* remove tx

* Remove RateLimitConfig

* Rename settings_updated_channel to rate_limit_cell

* Remove global rate limit cell

* impl Default for RateLimitCell

* bucket_configs doc comment to explain EnumMap

* improve test_rate_limiter

* rename default to with_test_config

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2023-10-19 15:31:51 +02:00
6d27bfed08
Handle invalid ban expires values (fixes #4045) (#4046)
* Handle invalid ban expires values (fixes #4045)

* Adding a few missing expire time checks. Fixing up time conversions. (#4051)

* Adding a few missing expire time checks. Fixing up time conversions.

* Increase settings export wait time.

* get rid of RemoveCommunity.expires

* fmt

* tests

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-17 13:25:35 -04:00
56322c75f0
Read community follower count from home instance (fixes #1440) (#4013)
* Read community follower count from home instance (fixes #1440)

* fmt

* prettier

* fix tests

* fmt

* rename fn

* fmt

* Run prettier

* increase timeout

* ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-17 10:34:38 -04:00
9e099726e6
Cleanup checks for community actions (fixes #2858, fixes #2868) (#4028)
* Cleanup checks for community actions (fixes #2858, fixes #2868)

* allow restoring deleted community

* review changes

* remove unneeded sql

* remove joins

* change mod log check
2023-10-13 09:48:18 -04:00
608bb6b1b4
Convert titles for posts from Mastodon to plaintext (fixes #3828) (#4033)
* Convert titles for posts from Mastodon to plaintext (fixes #3828)

* Fix prettier.

* Trigger build

* Convert titles for posts from Mastodon to plaintext (fixes #3828)

* Fix prettier.

* Fix sanizize.

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-12 20:36:02 -04:00
291ff19718
Only sanitize strings when generating RSS feeds and emails (fixes #4003) (#4024)
* Only sanitize strings when generating RSS feeds and emails (fixes #4003)

* clippy

* fix test
2023-10-11 16:48:19 +02:00
Apple Sheeple
b431c9bdf9 Sanitize apub post body
Signed-off-by: Apple Sheeple <AppleSheeple@github>
2023-09-18 16:00:29 +02:00
phiresky
375d9a2a3c
Persistent, performant, reliable federation queue (#3605)
* persistent activity queue

* fixes

* fixes

* make federation workers function callable from outside

* log federation instances

* dead instance detection not needed here

* taplo fmt

* split federate bin/lib

* minor fix

* better logging

* log

* create struct to hold cancellable task for readability

* use boxfuture for readability

* reset submodule

* fix

* fix lint

* swap

* remove json column, use separate array columns instead

* some review comments

* make worker a struct for readability

* minor readability

* add local filter to community follower view

* remove separate lemmy_federate entry point

* fix remaining duration

* address review comments mostly

* fix lint

* upgrade actitypub-fed to simpler interface

* fix sql format

* increase delays a bit

* fixes after merge

* remove selectable

* fix instance selectable

* add comment

* start federation based on latest id at the time

* rename federate process args

* dead instances in one query

* filter follow+report activities by local

* remove synchronous federation

remove activity sender queue

* lint

* fix federation tests by waiting for results to change

* fix fed test

* fix comment report

* wait some more

* Apply suggestions from code review

Co-authored-by: SorteKanin <sortekanin@gmail.com>

* fix most remaining tests

* wait until private messages

* fix community tests

* fix community tests

* move arg parse

* use instance_id instead of domain in federation_queue_state table

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
2023-09-09 12:25:03 -04:00
71d61138bc
Replace ammonia lib with manual html escaping (fixes #3774) (#3938)
* Replace ammonia lib with manual html escaping (fixes #3774)

* prettier

* clippy

* remove sanitize unit test

* fix tests

* fix schema
2023-09-06 10:56:26 -04:00
Dessalines
56e26fc3d4
Adding a post_view mode. Fixes #3730 (#3731)
* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Fixing column order.

* Fix default Ok returns.

* Removing return Err(... where feasible.
2023-08-31 09:01:08 -04:00
phiresky
514f2222e0
Fix time zone handling (#3496)
* convert naive time to utc time

* compounding fixes

* cargo fmt

* fix the rest

* fix down migration

* fix migrations

* fix after merge

* clippy fix

* ap-fed 0.5.0

---------

Co-authored-by: Nutomic <me@nutomic.com>
2023-08-24 11:27:00 -04:00
6047257bfc
Move admin flag from person to local_user (fixes #3060) (#3403)
* Move admin flag from person to local_user (fixes #3060)

The person table is for federated data, but admin flag can only
apply to local users. Thats why it really belongs in the local_user
table. This will also prevent the federation code from accidentally
overwriting the admin flag

* fmt

* try to fix api tests

* lint

* fix person view

* ci

* ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-08-24 05:40:08 -04:00
Freakazoid182
ee7b35a04a
Add person name to PersonIsBannedFromSite error (#3786) (#3855)
* Add person name to PersonIsBannedFromSite error (#3786)

* Log PersonId instead of Person name (#3850)

* Log actor_id for PersonIsBannedFromSite (#3850)

---------

Co-authored-by: Freek van Zee <freek.van.zee@mediamonks.com>
2023-08-16 11:36:58 +02:00
dullbananas
969f8b2ce9
Replace TypedBuilder with Default in update forms (#3814)
* Update comment.rs

* Update community.rs

* Update local_site.rs

* Update local_site_rate_limit.rs

* Update local_user.rs

* Update person.rs

* Update comment.rs

* Update community.rs

* Update local_site.rs

* Update local_site_rate_limit.rs

* Update local_user.rs

* Update post.rs

* Update private_message.rs

* Update site.rs

* Update post.rs

* Update person.rs

* Update private_message.rs

* Update comment.rs

* Update create.rs

* Update leave_admin.rs

* Update update.rs

* Update remove.rs

* Update add_admin.rs

* Update verify_email.rs

* Update mod.rs

* Update mod.rs

* Update undo_delete.rs

* Update undo_delete.rs

* Update utils.rs

* Update feature.rs

* Update delete.rs

* Update lock.rs

* Update create.rs

* Update approve.rs

* Update update.rs

* Update lock_page.rs

* Update block_user.rs

* Update delete.rs

* Update undo_block_user.rs

* Update collection_remove.rs

* Update post.rs

* Update hide.rs

* Update person.rs

* Update remove.rs

* Update post_view.rs

* Update create.rs

* Update remove.rs

* Update collection_add.rs

* Update community.rs

* Update update.rs

* Update post_aggregates.rs

* Update update.rs

* Update comment.rs

* Update code_migrations.rs

* Update registration_application_view.rs

* Update update.rs

* Update ban_person.rs

* Update community.rs

* Update delete.rs

* Update delete.rs

* Update delete.rs

* Update person_aggregates.rs

* Update save_settings.rs

* Update distinguish.rs

* Update mark_read.rs

* Update site_aggregates.rs

* Update create.rs

* Fix

* rerun ci

* Update comment.rs

* rerun ci

* Update create.rs

* Update create.rs

* Update post_view.rs

* rerun ci

* Update undo_delete.rs

* rerun ci
2023-08-08 11:41:41 +02:00
3471f3533c
Sanitize html (#3708)
* HTML sanitization in apub code

* Sanitize API inputs

* fmt

* Dont allow html a, img tags

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-26 14:01:15 -04:00
cetra3
9256895635
Cache & Optimize Woodpecker CI (#3450)
* Try using drone cache plugin

* Try another path

* Include volume

* Fix formatting

* Include fmt

* Exclude cargo dir from prettier

* Don't override cargo

* Just do check

* Add cache key

* Use different cache plugin

* Add clippy

* Try minio

* Add quotes

* Try adding secrets

* Try again

* Again

* Use correct secret formation

* Add back clippy

* Use secret for the root bucket name

* Try drone cache instead

* Add region

* Add path-style option

* Include cargo clippy

* Include everything again

* Fix formatting

* Don't run clippy twice

* Add `allow` statements for tests to pass

* Adjust endpoint to be a secret

* Fix prettier

* Merge & fix tests

* Try to restart the woodpecker test

* Change the ENV var name

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-17 11:04:14 -04:00
dullbananas
1d38aad9d3
Make functions work with both connection and pool (#3420)
* a lot

* merge

* Fix stuff broken by merge

* Get rid of repetitive `&mut *context.conn().await?`

* Add blank lines under each line with `conn =`

* Fix style mistakes (partial)

* Revert "Fix style mistakes (partial)"

This reverts commit 48a033b87f.

* Revert "Add blank lines under each line with `conn =`"

This reverts commit 773a6d3beb.

* Revert "Get rid of repetitive `&mut *context.conn().await?`"

This reverts commit d2c6263ea1.

* Use DbConn for CaptchaAnswer methods

* DbConn trait

* Remove more `&mut *`

* Fix stuff

* Re-run CI

* try to make ci start

* fix

* fix

* Fix api_common::utils

* Fix apub::activities::block

* Fix apub::api::resolve_object

* Fix some things

* Revert "Fix some things"

This reverts commit 2bf8574bc8.

* Revert "Fix apub::api::resolve_object"

This reverts commit 3e4059aabb.

* Revert "Fix apub::activities::block"

This reverts commit 3b02389abd.

* Revert "Fix api_common::utils"

This reverts commit 7dc73de613.

* Revert "Revert "Fix api_common::utils""

This reverts commit f740f115e5.

* Revert "Revert "Fix apub::activities::block""

This reverts commit 2ee206af7c.

* Revert "Revert "Fix apub::api::resolve_object""

This reverts commit 96ed8bf2e9.

* Fix fetch_local_site_data

* Fix get_comment_parent_creator

* Remove unused perma deleted text

* Fix routes::feeds

* Fix lib.rs

* Update lib.rs

* rerun ci

* Attempt to create custom GetConn and RunQueryDsl traits

* Start over

* Add GetConn trait

* aaaa

* Revert "aaaa"

This reverts commit acc9ca1aed.

* Revert "Revert "aaaa""

This reverts commit 443a2a00a5.

* still aaaaaaaaaaaaa

* Return to earlier thing

Revert "Add GetConn trait"

This reverts commit ab4e94aea5.

* Try to use DbPool enum

* Revert "Try to use DbPool enum"

This reverts commit e4d1712646.

* DbConn and DbPool enums (db_schema only fails to compile for tests)

* fmt

* Make functions take `&mut DbPool<'_>` and make db_schema tests compile

* Add try_join_with_pool macro and run fix-clippy on more crates

* Fix some errors

* I did it

* Remove function variants that take connection

* rerun ci

* rerun ci

* rerun ci
2023-07-11 09:09:59 -04:00
93225e5ddf
Error enum fixed (#3487)
* Create error type enum

* Replace magic string slices with LemmyErrorTypes

* Remove unused enum

* Add rename snake case to error enum

* Rename functions

* clippy

* Fix merge errors

* Serialize in PascalCase instead of snake_case

* Revert src/lib

* Add serialization tests

* Update translations

* Fix compilation error in test

* Fix another compilation error

* Add code for generating typescript types

* Various fixes to avoid breaking api

* impl From<LemmyErrorType> for LemmyError

* with_lemmy_type

* trigger ci

---------

Co-authored-by: SleeplessOne1917 <abias1122@gmail.com>
2023-07-10 16:50:07 +02:00
00f9f79a44
Only allow http(s) scheme for urls (ref #3505) (#3508)
With this change only http(s) schemes are allowed for post.url
field. This is checked for incoming api and federation requests.
Existing posts in database which are sent to clients are not
checked. Neither does it check urls in markdown.
2023-07-06 08:29:51 -04:00
phiresky
45b1a0d4fb
improve performance of community followers inbox query (#3482)
* improve performance of community followers inbox query

* nightly format

* force woodpecker to retry

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-05 11:50:26 -04:00
ebaf69bd70
Cache federation blocklist (#3486)
* Cache federation blocklist

* revert submodule change
2023-07-05 11:08:02 -04:00
Lemmus.org
810762762f
Update federated posts to not cache sensitive images if not allow by local site (#3253)
* Update federated posts to not cache sensitive images if not allow by local site

* Refactor thumbnail match to simplify logic
2023-06-30 12:42:42 +02:00
50efb1d519
Fetch community outbox and moderators in parallel (#3360)
This will speedup first time fetching of a remote community
2023-06-26 12:14:50 -04:00
cetra3
d7da911a48
Remove actix_rt & use standard tokio spawn (#3158)
* Remove `actix_rt` & use standard tokio spawn

* Adjust rust log back down

* Format correctly

* Update cargo lock

* Add DB settings

* Change name and update to latest rev

* Clean up formatting changes

* Move `worker_count` and `worker_retry_count` to settings

* Update defaults

* Use `0.4.4` instead of git branch
2023-06-26 10:24:11 +02:00
ffce66abb8
Update apub library to 0.4.4 (#3258)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-06-22 08:35:12 -04:00
e3eda68147
Correct error messages if user registers with taken user/email (#3093)
* Correct error messages if user registers with taken user/email (fixes #2955)

* exists
2023-06-21 05:26:07 -04:00
2423b89ced
Activitypub breaking changes for Lemmy 0.18 (#2705)
* Remove deprecated field `group.moderators`

* Drop column local_site.federation_debug

* Remove backwards compatibility code for feature/lock post

https://github.com/LemmyNet/lemmy/pull/2716

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-06-07 15:18:17 -04:00
3565ad984a
Remove chatserver (#2919)
* Remove chatserver

* fix clippy

* Remove captchas (fixes #2922)

* fix prettier

* fix api_common build

* cargo fmt
2023-06-06 12:27:22 -04:00
7248c4a6f4
Filter empty username strings from Mastodon (fixes #2886) (#2887)
* Filter empty username strings from Mastodon (fixes #2886)

* add issue link
2023-06-05 13:56:53 +02:00
436a293f38 Dont refetch post url metadata when post is received again 2023-05-18 16:34:52 +02:00
Dessalines
eb40aeb89b
Remove last Option<Vec.. from API. Fixes #2820 (#2822)
* Remove last Option<Vec.. from API. Fixes #2820

* Add empty allowed_instances check.

* Adding comment for allowed_instances.
2023-04-21 23:41:03 +02:00
Dessalines
63f54a3103
Making the chat server an actor. (#2793)
* Making the chat server an actor.

- Fixes #2778
- #2787

* Forgot to add handlers folder.

* Some cleanup.

* Forgot to remove a comment.

* Address PR comments.

* Using ToString for enum operations.
2023-04-13 06:53:55 -04:00
165b19e75c
Optimize federated language updates to avoid unnecessary db writes (#2786)
* Optimize federated language updates to avoid unnecessary db writes (fixes #2772)

* fix tests

* fix test, rename functions

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-03-30 11:03:13 -04:00
6f513793cb
Activitypub crate rewrite (#2782)
* update activitypub-federation crate to 0.4.0

* fixes

* apub compiles!

* everything compiling!

* almost done, federated follow failing

* some test fixes

* use release

* add code back in
2023-03-21 16:03:05 +01:00
d9e7f0100a
Dont upsert Instance row every apub fetch (#2771)
This is not necessary because the domain cant change, so we only
need to insert if no row exists for this domain.

Also fetch instance actor when parsing person, not only community

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-02-28 21:36:57 -05:00
62663a9f2e
Implement separate mod activities for feature, lock post (#2716)
* Implement separate mod activities for feature, lock post

Also includes collection for featured posts. Later we also need
to do the same for Comment.distinguished

* some changes

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-02-18 09:50:28 -05:00
Dessalines
36aaa50644
Organize utils into separate files. Fixes #2295 (#2736)
* Organize utils into separate files. Fixes #2295

* Moving tests.

* Fix test.

* Fix test 2
2023-02-16 05:05:14 +01:00
3bb98fcc64
Fix post titles being cut off (fixes #2718) (#2720)
I didnt notice that this limit would also apply to Lemmy and other
software which has the title field. So gonna change it to be
identical to db limit
2023-02-10 13:35:23 -05:00
Kradyz
ab9b60e86d
Convert HTML from titles into markdown (#2709)
Co-authored-by: Kradyz <k@radiz.nl>
2023-02-07 03:27:17 +01:00