Commit graph

296 commits

Author SHA1 Message Date
af034f3b5e
Unit tests and cleanup for outgoing federation code (#4733)
* test setup

* code cleanup

* cleanup

* move stats to own file

* basic test working

* cleanup

* processes test

* more test cases

* fmt

* add file

* add assert

* error handling

* fmt

* use instance id instead of domain for stats channel
2024-05-27 09:34:58 -04:00
Dessalines
6a6108ac55
Fixing proxied images for federated posts. (#4737)
* Fixing proxied images for federated posts.

- Also added test.
- Fixes #4736

* Address PR comments.
2024-05-23 11:11:25 -04:00
c96017c009
Configure max comment width in clippy (#4738)
* Configure max comment width in clippy

* update default config
2024-05-23 08:46:26 -04:00
943c31cc72
Allow passing command line params via environment (fixes #4603) (#4729)
* Allow passing command line params via environment (fixes #4603)

* add prefix

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-05-22 08:39:01 -04:00
4ffaa93431
Dont allow reusing password reset token, use normal rate limit (#4719)
* Dont allow reusing password reset token, use normal rate limit

* fix
2024-05-21 14:46:49 -04:00
49bb17b583
Stricter rate limit for login (#4718) 2024-05-14 22:43:43 -04:00
b4f9ef24a5
Dont exit early when running only scheduled tasks (#4707)
* Dont exit early when running only scheduled tasks (fixes #4709)

* fix
2024-05-08 14:56:44 +02:00
dullbananas
4ba6221e04
Move SQL triggers from migrations into reusable sql file (#4333)
* stuff

* stuff including batch_upsert function

* stuff

* do things

* stuff

* different timestamps

* stuff

* Revert changes to comment.rs

* Update comment.rs

* Update comment.rs

* Update post_view.rs

* Update utils.rs

* Update up.sql

* Update up.sql

* Update down.sql

* Update up.sql

* Update main.rs

* use anyhow macro

* Create down.sql

* Create up.sql

* Create replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update utils.rs

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Create dump_schema.sh

* Update start_dev_db.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* stuff

* Update replaceable_schema.sql

* Update .pg_format

* fmt

* stuff

* stuff (#21)

* Update replaceable_schema.sql

* Update up.sql

* Update replaceable_schema.sql

* fmt

* update cargo.lock

* stuff

* Update replaceable_schema.sql

* Remove truncate trigger because truncate is already restricted by foreign keys

* Update replaceable_schema.sql

* fix some things

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update .woodpecker.yml

* stuff

* fix TG_OP

* Psql env vars

* try to fix combine_transition_tables parse error

* Revert "try to fix combine_transition_tables parse error"

This reverts commit 75d00a4626.

* refactor combine_transition_tables

* try to fix create_triggers

* fix some things

* try to fix combined_transition_tables

* fix sql errors

* update comment count in post trigger

* fmt

* Revert "fmt"

This reverts commit a5bcd0834b.

* Revert "update comment count in post trigger"

This reverts commit 0066a4b42b.

* fix everything

* Update replaceable_schema.sql

* actually fix everything

* refactor create_triggers

* fix

* add semicolons

* add is_counted function and fix incorrect bool operator in update_comment_count_from_post

* refactor comment trigger

* refactor post trigger

* fix

* Delete crates/db_schema/src/utils/series.rs

* subscribers_local

* edit migrations

* move migrations

* remove utils::series module declaration

* fix everything

* stuff

* Move sql to schema_setup dir

* utils.sql

* delete .pg_format

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update .woodpecker.yml

* Merge remote-tracking branch 'upstream/main' into bliss

* fmt

* Create main.rs

* Update lib.rs

* Update main.rs

* Update .woodpecker.yml

* Update main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* YAY

* Update mod.rs

* Update Cargo.toml

* a

* Update Cargo.toml

* Update Cargo.toml

* Delete crates/db_schema/src/main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update utils.sql

* Update utils.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update down.sql

* Update up.sql

* Update triggers.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* Update down.sql

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update mod.rs

* Update Cargo.toml

* Update mod.rs

* make dump_schema.sh executable

* fix dump_schema.sh

* defer

* diff dumps

* fmt

* Update utils.sql

* Update .woodpecker.yml

* use correct version for pg_dump

* Update .woodpecker.yml

* Update .woodpecker.yml

* change migration date

* atomic site_aggregates insert

* temporarily repeat tests in CI

* drop r schema in CI migration check

* show ReceivedActivity::create error

* move check_diesel_migration CI step

* Update .woodpecker.yml

* Update scheduled_tasks.rs

* Update scheduled_tasks.rs

* update cargo.lock

* move sql files

* move rank functions

* filter post_aggregates update

* fmt

* cargo fmt

* replace post_id with id

* update cargo.lock

* avoid locking rows that need no change in up.sql

* only run replaceable_schema if migrations were run

* debug ci test failure

* make replaceable_schema work in CI

* Update .woodpecker.yml

* remove println

* Use migration revert and git checkout

* Update schema_setup.rs

* Fix

* Update schema_setup.rs

* Update schema_setup.rs

* Update .woodpecker.yml

---------

Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-04-17 20:58:44 -04:00
TechVest
b0370ae2fd
chore: fix some comments (#4637)
Signed-off-by: TechVest <techdashen@qq.com>
2024-04-17 14:35:54 +02:00
Dessalines
d075acce43
Make all single-fetch database calls return an Option. (#4617)
- Diesel ordinarily throws an error when no results are returned for a
  single fetch, which is a bit confusing. This PR ensures that the
  missing value cases are all caught, and wrapped with new LemmyErrors,
  rather than diesel errors.
- Fixes #4601
2024-04-16 14:48:15 +02:00
Dessalines
5dea21d531
Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613 (#4614)
* Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613

* Fixing clippy.
2024-04-10 10:14:11 -04:00
Dessalines
705e86eb4c
Fixing docker release (#4592)
* Re-add notif on tag failure.

* Upping rust version.

* Version 0.19.4-beta.1a

* Try again.

* Version 0.19.4-beta.1b

* Removing unstable inspect.

* Version 0.19.4-beta.1c

* Remove use release cache.

* Trying to fix cargo publish 1.

* Version 0.19.4-beta.1d

* Re-adding publish release

* Version 0.19.4-beta.2

* Fixing workspace for lemmy_federate
2024-04-04 16:14:59 +02:00
ae9f82b452
Read crate version from cargo.toml (fixes #4583) (#4584) 2024-04-02 11:19:51 -04:00
Dessalines
6bfbb9332d
Adding listMedia endpoint, to view all your local image uploads. (#4509)
* Adding listMedia endpoint, to view all your local image uploads.

- Fixes #4445

* Fix ts import.

* Forgot to order by published desc

* Adding an endpoint to list all images, for admins only.

* Forgot to add file.

* Add additional test.

* Use better logic for no-limit version.

* Better call sites.

* Adding another test.

* Fix tests.

* Moving list_media to /account action.

* Addressing PR comments.

* Removing pointless comment.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-03-26 12:06:11 -04:00
Dessalines
95069d7648
Fixing some clippy and woodpecker lints. (#4565)
* Fixing some clippy and woodpecker lints.

* Try fixing woodpecker 1.

* Revert "Try fixing woodpecker 1."

This reverts commit 7c2020a08d.
2024-03-26 10:17:42 +01:00
846848c4f6
On registration, automatically set content languages from accept-language header (#4550)
* On registration, automatically set content languages from accept header

* no need to set site language or default language for new user anymore

* fix test

* fix langs

* avoid duplicate writing of new user languages
2024-03-25 16:02:12 -04:00
Dessalines
87b577467b
Adding ability to hide posts. (#4480)
* Adding ability to hide posts.

- Adds an post/hide API route.
- Adds a `show_hidden` (default false) to `GetPosts`.
- Adds a `hidden` field to `PostView`.
- Removes the single `post_id` from MarkPostAsRead.
- Fixes #1403

* Add a check to make sure hidden field is true.

* Fixing test.

* Add back semicolon
2024-02-29 10:42:34 -05:00
Dessalines
890565ca14
Deleting denied local_users older than a week. Fixes #4434 (#4448)
* Deleting denied local_users older than a week. Fixes #4434

* Addressing PR comments.

* Upping rust to 1.76

* Delete the person rows also.
2024-02-15 07:50:53 -05:00
e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00
3d6f7ff911
Revert "Dont ignore errors during login (fixes #4319) (#4321)" (#4380)
This reverts commit 4163e0465e.

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-19 11:21:43 -05:00
b58da11fb7
Mark instance as alive after successful activity send (fixes #4039) (#4377)
* Mark instance as alive after successful activity send (fixes #4039)

* clippy

* Instance::update

* domain

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-19 09:40:12 -05:00
3cad3b2119
Dont overwrite cache-control header in session middleware (#4337) 2024-01-04 11:44:36 -05:00
ALEX11BR
a5289dd4cf
Use pretty_assertions for assertions (#4347)
* Use `pretty_assertions` for assertions

* fixed ordering of `use`s

* ci

---------

Co-authored-by: Felix Ableitner <me@nutomic.com>
2024-01-04 04:47:18 -05:00
Dessalines
009a45dffb
Adding /post/like/list and /comment/like/list for admins. (#4332)
- Allows admins to view likes, sorted by downvotes first,
  for a given comment or post.
- Fixes #4088
2024-01-03 13:39:21 -05:00
4163e0465e
Dont ignore errors during login (fixes #4319) (#4321)
* Dont ignore errors during login (fixes #4319)

* fix test

* fmt
2024-01-03 10:34:03 -05:00
6626d35b98
Second attempt to make command line options more consistent (#4249)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-12-13 09:14:59 -05:00
cef3f220a2
Make command line options consistent (#4243) 2023-12-11 09:39:18 -05:00
Kroese
203ca9d617
Print version to log (#4226)
* Print version to log

* Keep cargofmt happy

* Keep cargo_fmt happy

* Keep Clippy happy
2023-12-05 11:35:59 -05:00
a790a24c4d
Revert debug auth (#4232)
* Revert "Some changes to help debug auth problems on lemmy.ml (#4220)"

This reverts commit 16ac893e15.

* Rename auth cookie back to jwt
2023-12-05 11:22:08 -05:00
a5386187e3
Enable missing code for prometheus actix-web stats (#4230)
* Enable missing code for prometheus actix-web stats

* enable middleware conditionally
2023-12-04 09:53:53 -05:00
16ac893e15
Some changes to help debug auth problems on lemmy.ml (#4220)
* Some changes to help debug auth problems on lemmy.ml

* fix

* clippy
2023-12-01 09:18:29 -05:00
d7376d9541
Fix cors_origin wildcard (fixes #4214) (#4221) 2023-12-01 09:16:55 -05:00
934fe7d1cb
Remove pictrs config section old_db (#4199)
* Remove pictrs config section `old_db` (ref #4194)

* ci

* remove cargo config comment

* ci

* Revert "remove cargo config comment"

This reverts commit 9389e93649.

* rename .cargo to .cargo_home
2023-11-24 11:26:21 -05:00
7d489f1e3f
Upgrade pictrs to 0.5.0-beta.2 (#4194)
* Upgrade pictrs to 0.5.0-beta.2

* Update src/main.rs

Co-authored-by: asonix <asonix@asonix.dog>

* add comment

---------

Co-authored-by: asonix <asonix@asonix.dog>
2023-11-24 07:44:17 -05:00
a00313e680
Merge /site_inbox into /inbox, remove unique constraint for inboxes (#4138)
* Merge /site_inbox into /inbox (fixes #4137)

Get rid of different inboxes, only use /inbox

Remove shared_inbox_url db columns

add code migration

move to db migration, fixes

machete

fix sql

drop inbox url unique constraints

Dont create auth cookie in backend (#4136)

dont change individual inboxes to shared inbox

Dont send comment reply to user who has community blocked. Fixes #3684 (#4096)

* Dont send comment reply to user who has community blocked. Fixes #3684

* Adding source instance block check.

* Adding api test.

* Addressing PR comments.

* move site inbox rewrite to db

* fix test

* clippy

* clippy 2

* fix test
2023-11-16 14:22:40 +01:00
9a94a86363
Fix cors wildcard (ref #4095) (#4156)
* Fix cors wildcard (ref #4095)

* cleanup

* clippy
2023-11-14 09:39:13 -05:00
dullbananas
8e2cbc9a0f
Remove id column and use different primary key on some tables (#4093)
* post_saved

* fmt

* remove unique and not null

* put person_id first in primary key and remove index

* use post_saved.find

* change captcha_answer

* remove removal of not null

* comment_aggregates

* comment_like

* comment_saved

* aggregates

* remove "\"

* deduplicate site_aggregates

* person_post_aggregates

* community_moderator

* community_block

* community_person_ban

* custom_emoji_keyword

* federation allow/block list

* federation_queue_state

* instance_block

* local_site_rate_limit, local_user_language, login_token

* person_ban, person_block, person_follower, post_like, post_read, received_activity

* community_follower, community_language, site_language

* fmt

* image_upload

* remove unused newtypes

* remove more indexes

* use .find

* merge

* fix site_aggregates_site function

* fmt

* Primary keys dess (#17)

* Also order reports by oldest first (ref #4123) (#4129)

* Support signed fetch for federation (fixes #868) (#4125)

* Support signed fetch for federation (fixes #868)

* taplo

* add federation queue state to get_federated_instances api (#4104)

* add federation queue state to get_federated_instances api

* feature gate

* move retry sleep function

* move stuff around

* Add UI setting for collapsing bot comments. Fixes #3838 (#4098)

* Add UI setting for collapsing bot comments. Fixes #3838

* Fixing clippy check.

* Only keep sent and received activities for 7 days (fixes #4113, fixes #4110) (#4131)

* Only check auth secure on release mode. (#4127)

* Only check auth secure on release mode.

* Fixing wrong js-client.

* Adding is_debug_mode var.

* Fixing the desktop image on the README. (#4135)

* Delete dupes and add possibly missing unique constraint on person_aggregates.

* Fixing clippy lints.

---------

Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: phiresky <phireskyde+git@gmail.com>

* fmt

* Update community_block.rs

* Update instance_block.rs

* Update person_block.rs

* Update person_block.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: phiresky <phireskyde+git@gmail.com>
2023-11-13 08:14:07 -05:00
SleeplessOne1917
1dc6c60760
Make import and export settings endpoints reachable (#4143)
* Make import and export settings endpoints reachable

* Preserve original routes
2023-11-13 08:08:07 -05:00
cb01427dcf
Only keep sent and received activities for 7 days (fixes #4113, fixes #4110) (#4131) 2023-11-06 16:17:53 -05:00
8c85f35b19
Support signed fetch for federation (fixes #868) (#4125)
* Support signed fetch for federation (fixes #868)

* taplo
2023-11-06 16:02:01 -05:00
08739e2925
Move usage of env::var to lemmy_utils, simplify db init (ref #4095) (#4108) 2023-10-25 11:34:38 -04:00
Dessalines
1b751a8cac
Adding cors_origin to settings. Fixes #3665 (#4095)
* Adding cors_origin to settings. Fixes #3665

* Fix result to option.

* Forgot to update config defaults.

* Setting a cors origin doku default.

* Adding comments for CORS.
2023-10-25 16:46:34 +02:00
45bed71c36
Include prometheus in default build, remove build feature (fixes #3558) (#4071)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-25 06:54:58 -04:00
Dessalines
c38dfdcd64
Disable ansi coloring in logs. Fixes #3975 (#4101) 2023-10-25 10:58:55 +02:00
ec0a707110
Avoid using proxy for pictrs requests (fixes #3489) (#4072)
* Avoid using proxy for pictrs requests (fixes #3489)

* fmt
2023-10-20 11:09:34 -04:00
Dessalines
cae25486e4
Adding ts-rs directives for login_token. (#4063)
* Adding ts-rs directives for login_token.

* Fixing ts-rs derive.
2023-10-19 20:16:12 -04:00
dullbananas
a14657d124
Refactor rate limiter and improve rate limit bucket cleanup (#3937)
* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update scheduled_tasks.rs

* Shrink `RateLimitBucket`

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* rerun ci

* Update rate_limiter.rs

* Undo changes to  fields

* Manually undo changes to RateLimitBucket fields

* fmt

* Bucket cleanup loop in rate_limit/mod.rs

* Remove rate limit bucket cleanup from scheduled_tasks.rs

* Remove ;

* Remove UNINITIALIZED_TOKEN_AMOUNT

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* stuff

* MapLevel trait

* fix merge

* Prevent negative numbers in buckets

* Clean up MapLevel::check

* MapLevel::remove_full_buckets

* stuff

* Use remove_full_buckets to avoid allocations

* stuff

* remove tx

* Remove RateLimitConfig

* Rename settings_updated_channel to rate_limit_cell

* Remove global rate limit cell

* impl Default for RateLimitCell

* bucket_configs doc comment to explain EnumMap

* improve test_rate_limiter

* rename default to with_test_config

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2023-10-19 15:31:51 +02:00
3f62135083
Add validate_auth api endpoint (fixes #3702) (#4049)
* Add validate_auth api endpoint (fixes #3702)

* clippy

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-17 11:25:48 -04:00
9e099726e6
Cleanup checks for community actions (fixes #2858, fixes #2868) (#4028)
* Cleanup checks for community actions (fixes #2858, fixes #2868)

* allow restoring deleted community

* review changes

* remove unneeded sql

* remove joins

* change mod log check
2023-10-13 09:48:18 -04:00
6d7b38f4de
Implement user data import/export (#3976)
* Implement endpoints for user data import/export

* add test

* exclude avatar/banner

* increase import url count, add rate limit

* also export/import saved posts

* rate limit

* rename

* saved posts also exist

* rename routes

* fix test

* error handling

* clippy

* limit parallelism

* clippy

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-11 16:47:22 +02:00