Commit graph

398 commits

Author SHA1 Message Date
beec080274
Testing for federation with NodeBB, make community.followers_url optional (#4629)
* Testing for federation with NodeBB, make community.followers_url optional

* clippy
2024-04-29 12:34:11 +02:00
Dessalines
492d8f1b01
Fix communities with broken outboxes, and use PostView. Fixes #4658 (#4668)
* Fix communities with broken outboxes, and use PostView. Fixes #4658

* Fixing tests.

* Dont pass ref and clone.
2024-04-29 12:22:00 +02:00
dullbananas
d3737d4453
Optimize actor_language.rs (#4612)
* Remove useless transaction in actor_language.rs

* Update actor_language.rs

* site

* community

* Update actor_language.rs

* Update actor_language.rs

* Update actor_language.rs

* Update actor_language.rs

* Update actor_language.rs
2024-04-27 10:59:58 -04:00
dullbananas
4ba6221e04
Move SQL triggers from migrations into reusable sql file (#4333)
* stuff

* stuff including batch_upsert function

* stuff

* do things

* stuff

* different timestamps

* stuff

* Revert changes to comment.rs

* Update comment.rs

* Update comment.rs

* Update post_view.rs

* Update utils.rs

* Update up.sql

* Update up.sql

* Update down.sql

* Update up.sql

* Update main.rs

* use anyhow macro

* Create down.sql

* Create up.sql

* Create replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update utils.rs

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Create dump_schema.sh

* Update start_dev_db.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* stuff

* Update replaceable_schema.sql

* Update .pg_format

* fmt

* stuff

* stuff (#21)

* Update replaceable_schema.sql

* Update up.sql

* Update replaceable_schema.sql

* fmt

* update cargo.lock

* stuff

* Update replaceable_schema.sql

* Remove truncate trigger because truncate is already restricted by foreign keys

* Update replaceable_schema.sql

* fix some things

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update .woodpecker.yml

* stuff

* fix TG_OP

* Psql env vars

* try to fix combine_transition_tables parse error

* Revert "try to fix combine_transition_tables parse error"

This reverts commit 75d00a4626.

* refactor combine_transition_tables

* try to fix create_triggers

* fix some things

* try to fix combined_transition_tables

* fix sql errors

* update comment count in post trigger

* fmt

* Revert "fmt"

This reverts commit a5bcd0834b.

* Revert "update comment count in post trigger"

This reverts commit 0066a4b42b.

* fix everything

* Update replaceable_schema.sql

* actually fix everything

* refactor create_triggers

* fix

* add semicolons

* add is_counted function and fix incorrect bool operator in update_comment_count_from_post

* refactor comment trigger

* refactor post trigger

* fix

* Delete crates/db_schema/src/utils/series.rs

* subscribers_local

* edit migrations

* move migrations

* remove utils::series module declaration

* fix everything

* stuff

* Move sql to schema_setup dir

* utils.sql

* delete .pg_format

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update .woodpecker.yml

* Merge remote-tracking branch 'upstream/main' into bliss

* fmt

* Create main.rs

* Update lib.rs

* Update main.rs

* Update .woodpecker.yml

* Update main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* YAY

* Update mod.rs

* Update Cargo.toml

* a

* Update Cargo.toml

* Update Cargo.toml

* Delete crates/db_schema/src/main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update utils.sql

* Update utils.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update down.sql

* Update up.sql

* Update triggers.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* Update down.sql

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update mod.rs

* Update Cargo.toml

* Update mod.rs

* make dump_schema.sh executable

* fix dump_schema.sh

* defer

* diff dumps

* fmt

* Update utils.sql

* Update .woodpecker.yml

* use correct version for pg_dump

* Update .woodpecker.yml

* Update .woodpecker.yml

* change migration date

* atomic site_aggregates insert

* temporarily repeat tests in CI

* drop r schema in CI migration check

* show ReceivedActivity::create error

* move check_diesel_migration CI step

* Update .woodpecker.yml

* Update scheduled_tasks.rs

* Update scheduled_tasks.rs

* update cargo.lock

* move sql files

* move rank functions

* filter post_aggregates update

* fmt

* cargo fmt

* replace post_id with id

* update cargo.lock

* avoid locking rows that need no change in up.sql

* only run replaceable_schema if migrations were run

* debug ci test failure

* make replaceable_schema work in CI

* Update .woodpecker.yml

* remove println

* Use migration revert and git checkout

* Update schema_setup.rs

* Fix

* Update schema_setup.rs

* Update schema_setup.rs

* Update .woodpecker.yml

---------

Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-04-17 20:58:44 -04:00
TechVest
b0370ae2fd
chore: fix some comments (#4637)
Signed-off-by: TechVest <techdashen@qq.com>
2024-04-17 14:35:54 +02:00
Dessalines
6efab9aab1
Creating a LocalImageView, so that front ends have the Person struct. (#4631)
* Creating a LocalImageView, so that front ends have the Person struct.

* Removing local_user from LocalImageView.

* Add uploader check.
2024-04-16 19:20:44 -04:00
Dessalines
d075acce43
Make all single-fetch database calls return an Option. (#4617)
- Diesel ordinarily throws an error when no results are returned for a
  single fetch, which is a bit confusing. This PR ensures that the
  missing value cases are all caught, and wrapped with new LemmyErrors,
  rather than diesel errors.
- Fixes #4601
2024-04-16 14:48:15 +02:00
Dessalines
5dea21d531
Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613 (#4614)
* Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613

* Fixing clippy.
2024-04-10 10:14:11 -04:00
0203b62a6d
Ignore old federated post edits (ref #4529) (#4586)
* Ignore old federated post edits (ref #4529)

* use filter on insert

* coalesce(updated, published)

* avoid comment conflict clause

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-04-10 10:03:51 -04:00
1d0a6ac08f
Avoid breaking api change, reduce api cache duration (#4610)
* Dont mark site.public_key as `serde(skip)` to avoid breaking change (fixes #4605)

* Reduce cache duration for api
2024-04-09 10:10:20 -04:00
087684658a
Cache result of LocalSite::read to avoid unnecessary db calls (#4585)
* Cache result of LocalSite::read to avoid unnecessary db calls

* single const for cache duration

* clippy

* revert apub send changes

* clippy

* fmt
2024-04-03 17:38:31 -04:00
dullbananas
007e9b7aab
Optimize Community::set_featured_posts (#4579)
* Don't lock excess rows in Community::set_featured_posts

* Update community.rs

* Update community.rs

* Update community.rs

* Update community.rs
2024-04-02 11:19:04 -04:00
85ee89f4e8
When uploading new icon/avatar/banner, delete old one (#4573) 2024-03-27 09:00:52 -04:00
Dessalines
6bfbb9332d
Adding listMedia endpoint, to view all your local image uploads. (#4509)
* Adding listMedia endpoint, to view all your local image uploads.

- Fixes #4445

* Fix ts import.

* Forgot to order by published desc

* Adding an endpoint to list all images, for admins only.

* Forgot to add file.

* Add additional test.

* Use better logic for no-limit version.

* Better call sites.

* Adding another test.

* Fix tests.

* Moving list_media to /account action.

* Addressing PR comments.

* Removing pointless comment.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-03-26 12:06:11 -04:00
Dessalines
7929e77602
Fixing issue with comment replies wrongly marked as read. (#4567)
* Fixing issue with comment replies wrongly marked as read.

- Fixes #4566

* Elaborating on a comment.
2024-03-26 10:46:37 -04:00
Dessalines
95069d7648
Fixing some clippy and woodpecker lints. (#4565)
* Fixing some clippy and woodpecker lints.

* Try fixing woodpecker 1.

* Revert "Try fixing woodpecker 1."

This reverts commit 7c2020a08d.
2024-03-26 10:17:42 +01:00
846848c4f6
On registration, automatically set content languages from accept-language header (#4550)
* On registration, automatically set content languages from accept header

* no need to set site language or default language for new user anymore

* fix test

* fix langs

* avoid duplicate writing of new user languages
2024-03-25 16:02:12 -04:00
Dessalines
0f77951e05
Upgrading deps. (#4537)
* Upgrading deps.

* Addressing PR comments
2024-03-18 10:36:49 +01:00
flamingos-cant
19a1a077c5
Add a blocklist for URLs. (#4515)
* Add a blocklist for URLs.

* Fix SQL format

* Make clippy happy.

* Use regex for URL matching.

* Escape regex chars in URLs.

* Use post for modification.

* Make URL block regex static and remove API routes.

* Add date fields to table and use transaction.

* Use Cache for blocklist.

* Rename check_links + move list to parameters of process_markdown.

* SQL format.

* Format, again.

* Remove println.

* Add API test.

* Set a shorter lifetime for regex in debug mode.

* Add missing macro.

* Update lemmy-js-client

* Update api_test/pnpm-lock.yaml

* Don't break other tests

* Use different URL for test

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2024-03-15 07:03:29 -04:00
Dessalines
15f02f00a9
Add a vote_display_mode local_user setting. (#4450)
* Add a vote_display_mode local_user setting.

- Fixes #4449

* Changing HideDownvotes to Score.

* Adding ScoreAndDownvote display mode.

* Adding upvote and downvote mode.

* Extracting vote_display_mode to another table.

* Fixing fmt.

* Remove published and updated columns.

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-03-13 12:10:58 -04:00
5859502a2a
Fix missing private key for signed fetch (#4516)
* Fix missing private key for signed fetch (fixes #4451)

* clippy

* instance actor name and webfinger

* better webfinger handling

* upgrade lib

* update test asset
2024-03-08 10:23:15 -05:00
00f7778485
Store thumbnails in db table local_image (#4512)
* Store thumbnails in db table local_image

* fmt
2024-03-08 10:17:26 -05:00
battmdpkq
f228f9d7a9
fix some typos (#4519)
Signed-off-by: battmdpkq <cmaker@163.com>
2024-03-08 10:38:20 +01:00
Dessalines
7f9950fe85
Add alt_text for posts. Fixes #1086 (#4477)
* Add alt_text for posts. Fixes #1086

* Moving alt_text to attachment name.

* Cleaning up mod action line.

* Addressing PR comments

* Addressing PR comments.

* Fixing clones.
2024-03-05 11:34:57 +01:00
SleeplessOne1917
36ad1868b3
Make SiteAggregates derive Copy and Hash (#4501)
Co-authored-by: SleeplessOne1917 <insomnia-void@protonmail.com>
2024-03-05 11:14:12 +01:00
Dessalines
7eec8714d7
When site banning a federated user, also remove their content from our local communities. (#4464)
* When banning a federated user, also remove their content from our local
communities.

- This works by:
  - Before a site ban, find all posts and comments to local communities
  - Send a federated community ban action for each local comm.
  - This also removes their content in the apub receive code.
- Adding back in federated community ban api tests.
- Adding in two more api tests for site bans.
- Fixes #4118

* Add local community ban, and nonlocal person check.

* Ignoring errors.

* Move local check into function.

* Addressing PR comments 2
2024-03-01 13:45:06 -05:00
Dessalines
87b577467b
Adding ability to hide posts. (#4480)
* Adding ability to hide posts.

- Adds an post/hide API route.
- Adds a `show_hidden` (default false) to `GetPosts`.
- Adds a `hidden` field to `PostView`.
- Removes the single `post_id` from MarkPostAsRead.
- Fixes #1403

* Add a check to make sure hidden field is true.

* Fixing test.

* Add back semicolon
2024-02-29 10:42:34 -05:00
328a48c9f5
Remove error-type feature from lemmy-utils (#4474)
* Remove error-type feature from lemmy-utils

* fixes

---------

Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
2024-02-27 11:13:52 -05:00
Dessalines
f3d48f2c2c
Adding some recommended fixes from nightly clippy. (#4473) 2024-02-26 09:47:10 -05:00
7316dd281a
Correctly synchronize collection of community featured posts (fixes #3867) (#4475) 2024-02-26 09:45:23 -05:00
SleeplessOne1917
f42420809b
Expose LemmyErrorType in lemmy_api_common (#4439)
* Expose LemmyErrorType in lemmy_api_common

* Make conditional compilation gates for utils

* Make it so api_common doesn't pull in unnecessary deps

* Make error type non exhaustive

* Fix formatting

* Format toml

* Add some convenience derives to LemmyError

* Simplify features

* Fix CI compile error

---------

Co-authored-by: SleeplessOne1917 <insomnia-void@protonmail.com>
2024-02-24 19:54:27 -05:00
dullbananas
f56b84615c
Move DbUrl trait impls to newtypes.rs (#4463)
* Move DbUrl trait impls to newtypes.rs

* Update utils.rs
2024-02-19 12:41:28 -05:00
dullbananas
d79502dff3
Escape backslashes in fuzzy_search (#4462)
* Escape backslashes in fuzzy_search

* Update utils.rs
2024-02-18 09:12:56 -05:00
Dessalines
5d551e6da5
Adding an instance-level default_sort_type (#4454)
* Adding an instance-level default_sort_type

- Fixes #3796

* Fixing comment.

* Put user sort before site sort.
2024-02-16 09:36:46 -05:00
ffcf415cac
Dont log db url on connection error (fixes #4453) (#4456)
* Dont log db url on connection error (fixes #4453)

* remove format
2024-02-16 08:50:06 -05:00
86b44c2a4d
Add site.content_warning, local_site.default_post_listing_mode (#4393)
* Include local_site.content_warning setting for showing nsfw by default

* Add community setting `only_followers_can_vote`

* clippy

* add auto_expand_images site setting

* cleanup

* add missing api params

* postquery/communityquery changes

* clippy

* change error

* replace auto_expand_images with default_site_post_listing_mode

* change post/community query params

* get rid of only_followers_can_vote

* machete

* fix

* clippy

* revert remaining vote changes

* remove dead code

* remove unused var

* fmt
2024-02-16 07:24:35 -05:00
Dessalines
a3bf2f1cf1
Auto resolve reports on removing a comment or post. Fixes #4390 (#4402)
* Automatically resolve report when post/comment is removed (#3850)

* Automatically resolve report when post/comment is removed

* also handle apub removes

* Removing auto-resolve report triggers.

* Dont allow creating reports for deleted / removed items.

* Running pgformat.

* Fixing test.

* Addressing PR comments.

* Forgot comment report.

---------

Co-authored-by: Nutomic <me@nutomic.com>
2024-02-15 08:52:04 -05:00
Dessalines
890565ca14
Deleting denied local_users older than a week. Fixes #4434 (#4448)
* Deleting denied local_users older than a week. Fixes #4434

* Addressing PR comments.

* Upping rust to 1.76

* Delete the person rows also.
2024-02-15 07:50:53 -05:00
Dessalines
33989f5518
Blocking an instance also hides private messages from their users. (#4447)
* Blocking an instance also hides private messages from their users.

- Fixes #4444

* Separating private message tests.
2024-02-14 10:49:55 +01:00
dullbananas
677d54ae57
Allow better query plans (#4424)
* Update utils.rs

* Create bind_if_some.rs

* limit connection age

* Delete crates/db_schema/src/utils/bind_if_some.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs

* Update utils.rs
2024-02-12 16:44:29 +01:00
SleeplessOne1917
300869d397
Make request models derive PartialEq, Eq, and Hash (#4443)
* Make request models derive PartialEq, Eq, and Hash

* Fix clippy error

---------

Co-authored-by: SleeplessOne1917 <insomnia-void@protonmail.com>
2024-02-11 00:32:14 -05:00
0f414a95d5
Local only community (#4350)
* Add support for local only community (fixes #1576)

* add filters and tests to db views

* dont federate local only community

* test get apub community http

* tests

* more checks

* wip

* api test

* fix tests

* change community.local_only column to visibility enum
(for private communities)

* sql fmt

* rename vars

* clippy

* fix tests

* update lib

* review

* fix js client version

* update client
2024-01-25 11:04:25 -05:00
e8a52d3a5c
Rewrite images to use local proxy (#4035)
* Add markdown rule to add rel=nofollow for all links

* Add markdown image rule to add local image proxy (fixes #1036)

* comments

* rewrite markdown image links working

* add comment

* perform markdown image processing in api/apub receivers

* clippy

* add db table to validate proxied links

* rewrite link fields for avatar, banner etc

* sql fmt

* proxy links received over federation

* add config option

* undo post.url rewriting, move http route definition

* add tests

* proxy images through pictrs

* testing

* cleanup request.rs file

* more cleanup (fixes #2611)

* include url content type when sending post over apub (fixes #2611)

* store post url content type in db

* should be media_type

* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.

* fix tests

* add setting disable_external_link_previews

* federate post url as image depending on mime type

* change setting again

* machete

* invert

* support custom emoji

* clippy

* update defaults

* add image proxy test, fix test

* fix test

* clippy

* revert accidental changes

* address review

* clippy

* Markdown link rule-dess (#4356)

* Extracting opengraph_data to its own type.

* A few additions for markdown-link-rule.

---------

Co-authored-by: Nutomic <me@nutomic.com>

* fix setting

* use enum for image proxy setting

* fix test configs

* add config backwards compat

* clippy

* machete

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-25 09:22:11 -05:00
dullbananas
d8f9e8a64c
Post view: move cursor pagination to separate library, add backward pagination to PostQuery (#4320)
* stuff

* stuff

* crates.io

* Update up.sql

* Rerun federation tests

* Update post_view.rs

* Update post_view.rs

* Update up.sql

* Update utils.rs

* Fix precision loss

* Update up.sql

* Update down.sql

* remove unwrap

* Update post_view.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-24 10:50:11 -05:00
dullbananas
759f6d8a9a
Better query plan viewing experience (#4285)
* stuff

* stuff including batch_upsert function

* stuff

* do things

* stuff

* different timestamps

* stuff

* Revert changes to comment.rs

* Update comment.rs

* Update comment.rs

* Update post_view.rs

* Update utils.rs

* Update up.sql

* Update up.sql

* Update down.sql

* Update up.sql

* Update main.rs

* use anyhow macro

* replace get(0) with first()

* as_slice

* Update series.rs

* Update db_perf.sh

* Update and rename crates/db_schema/src/utils/series.rs to crates/db_perf/src/series.rs

* Update utils.rs

* Update main.rs

* Update main.rs

* Update .woodpecker.yml

* fmt main.rs

* Update .woodpecker.yml

* Instance::delete at end

* Update main.rs

* Update Cargo.toml

---------

Co-authored-by: Nutomic <me@nutomic.com>
2024-01-24 10:22:33 -05:00
İsmail Karslı
8670403a67
Add local_subscribers field to CommunityAggregates. Fixes #4144 (#4166)
* Add upload timeout to PictrsConfig

* Bad space 🤔

* Update PictrsConfig upload timeout to include units.

* Add local_subscribers field to CommunityAggregates
struct and schema

* sql format

* local_subscribers test

* fix local_subscribers test

* Revert "fix local_subscribers test"

This reverts commit 4bbac5ce4a.

* Revert "local_subscribers test"

This reverts commit 735107e1f7.

* Create trigger for local_subscribers

* Rename variable

* re-trigger ci

* re-trigger ci

* Add local_subscribers count to follow.spec.ts

* Rename local_subscribers to subscribers_local

* Add subscribers_local to community_aggregates

* added subscribers_local to the aggregate tests

* Check if person exists on community_follower trigger

* Delete community follows before deleting person

* Update lemmy-js-client in api_tests

* Refactor local_subscriber migration

* fix format

* Move migration files date to now

* Fix test to wait for aggregates to federate

* re-trigger ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-24 10:22:05 -05:00
b58da11fb7
Mark instance as alive after successful activity send (fixes #4039) (#4377)
* Mark instance as alive after successful activity send (fixes #4039)

* clippy

* Instance::update

* domain

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-19 09:40:12 -05:00
Dessalines
3b717cfc88
Removing serde_skip from newest_comment_time. Fixes #4351 (#4360) 2024-01-10 10:44:27 +01:00
7d9b59c467
Reduce default db pool size to 30, remove db timeout (ref #4282) (#4301)
* Reduce default db pool size to 30 (ref #4282)

* remove db timeout
2024-01-04 12:16:51 -05:00
38e64825e6
Add macro assert_length!() for tests (#4348)
* Add macro assert_length!() for tests

* fix

* number
2024-01-04 11:51:55 -05:00
ALEX11BR
a5289dd4cf
Use pretty_assertions for assertions (#4347)
* Use `pretty_assertions` for assertions

* fixed ordering of `use`s

* ci

---------

Co-authored-by: Felix Ableitner <me@nutomic.com>
2024-01-04 04:47:18 -05:00
dullbananas
dcb89f52d6
Don't update comment_aggregates if updating path fails (#4281) 2023-12-18 10:31:39 +01:00
Dessalines
719b76a6e7
Create comment in transaction. Fixes #3741 (#4265)
* Create comment in transaction. Fixes #3741

* Removing if let on comment create.
2023-12-15 11:36:58 +01:00
Dessalines
246e38a45b
Making login case-insensitive. Fixes #4272 (#4273)
* Making login case-insensitive. Fixes #4272

* Removing redundant lowercase.
2023-12-15 11:34:17 +01:00
c7ee53026f
Upgrade dependencies (#4248)
* Upgrade dependencies

* clippy
2023-12-12 08:56:33 -05:00
Kroese
2d4037ba61
Extend sitemap span (#4231)
* Extend sitemap span

* Keep cargo_fmt happy

* Add FETCH_LIMIT_SITEMAP

* Use FETCH_LIMIT_SITEMAP

* Keep cargo_fmt happy

* Update utils.rs

* Use SITEMAP_DAYS

* Keep cargo_fmt happy

* Sitemap

* Keep cargo_fmt happy

* Sitemap

* Sitemap

* Increase to 31 days
2023-12-11 11:24:51 +01:00
dullbananas
d95df3a46f
Update diesel-async (#4203)
* Update diesel-async

* Fix
2023-11-27 10:31:19 +01:00
7d489f1e3f
Upgrade pictrs to 0.5.0-beta.2 (#4194)
* Upgrade pictrs to 0.5.0-beta.2

* Update src/main.rs

Co-authored-by: asonix <asonix@asonix.dog>

* add comment

---------

Co-authored-by: asonix <asonix@asonix.dog>
2023-11-24 07:44:17 -05:00
Dessalines
6e4278b1c9
Setting rank column defaults to a small number. Fixes #4178 (#4191)
* Setting rank column defaults to a small number. Fixes #4178

* Fixing tests.
2023-11-23 20:39:46 -05:00
dullbananas
30f7135f32
Use rust 1.74.0 in CI and move clippy lints to Cargo.toml (#4170)
* ci rust 1.74.0

* do it

* Update .woodpecker.yml

* Update Cargo.toml

* Update lint.sh

* Update lib.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-11-21 08:51:22 -05:00
Dessalines
82b30b5467
Adding Selectable and diesel postgres backend check. (#4175) 2023-11-21 08:42:28 -05:00
f786df151f
Include instance blocks in settings export (#4173) 2023-11-17 21:59:15 -05:00
Dessalines
28c30cc502
Adding skip_serializing_none to federation state. (#4152) 2023-11-14 08:20:44 -05:00
dullbananas
8e2cbc9a0f
Remove id column and use different primary key on some tables (#4093)
* post_saved

* fmt

* remove unique and not null

* put person_id first in primary key and remove index

* use post_saved.find

* change captcha_answer

* remove removal of not null

* comment_aggregates

* comment_like

* comment_saved

* aggregates

* remove "\"

* deduplicate site_aggregates

* person_post_aggregates

* community_moderator

* community_block

* community_person_ban

* custom_emoji_keyword

* federation allow/block list

* federation_queue_state

* instance_block

* local_site_rate_limit, local_user_language, login_token

* person_ban, person_block, person_follower, post_like, post_read, received_activity

* community_follower, community_language, site_language

* fmt

* image_upload

* remove unused newtypes

* remove more indexes

* use .find

* merge

* fix site_aggregates_site function

* fmt

* Primary keys dess (#17)

* Also order reports by oldest first (ref #4123) (#4129)

* Support signed fetch for federation (fixes #868) (#4125)

* Support signed fetch for federation (fixes #868)

* taplo

* add federation queue state to get_federated_instances api (#4104)

* add federation queue state to get_federated_instances api

* feature gate

* move retry sleep function

* move stuff around

* Add UI setting for collapsing bot comments. Fixes #3838 (#4098)

* Add UI setting for collapsing bot comments. Fixes #3838

* Fixing clippy check.

* Only keep sent and received activities for 7 days (fixes #4113, fixes #4110) (#4131)

* Only check auth secure on release mode. (#4127)

* Only check auth secure on release mode.

* Fixing wrong js-client.

* Adding is_debug_mode var.

* Fixing the desktop image on the README. (#4135)

* Delete dupes and add possibly missing unique constraint on person_aggregates.

* Fixing clippy lints.

---------

Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: phiresky <phireskyde+git@gmail.com>

* fmt

* Update community_block.rs

* Update instance_block.rs

* Update person_block.rs

* Update person_block.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: phiresky <phireskyde+git@gmail.com>
2023-11-13 08:14:07 -05:00
Dessalines
e00b0c9da2
Dont send comment reply to user who has community blocked. Fixes #3684 (#4096)
* Dont send comment reply to user who has community blocked. Fixes #3684

* Adding source instance block check.

* Adding api test.

* Addressing PR comments.
2023-11-09 12:03:25 +01:00
Dessalines
97a4fb9a72
Add UI setting for collapsing bot comments. Fixes #3838 (#4098)
* Add UI setting for collapsing bot comments. Fixes #3838

* Fixing clippy check.
2023-11-06 16:09:12 -05:00
phiresky
b9b65c9c18
add federation queue state to get_federated_instances api (#4104)
* add federation queue state to get_federated_instances api

* feature gate

* move retry sleep function

* move stuff around
2023-11-06 16:07:04 -05:00
8c85f35b19
Support signed fetch for federation (fixes #868) (#4125)
* Support signed fetch for federation (fixes #868)

* taplo
2023-11-06 16:02:01 -05:00
9007d49324
Remove unused deps (#4111)
* Remove unused deps

* Adding cargo-machete to woodpecker.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-31 11:11:12 +01:00
08739e2925
Move usage of env::var to lemmy_utils, simplify db init (ref #4095) (#4108) 2023-10-25 11:34:38 -04:00
Dessalines
2df21b9d83
Dont serialize karma. #3393 (#4089) 2023-10-24 17:26:09 -04:00
Dessalines
5d48ee3dc8
Add creator_is_moderator to Comment and PostViews. Fixes #3347 (#4087)
* Add creator_is_moderator to Comment and PostViews. Fixes #3347

* Fixing community_moderator join.

* Addressing PR comments.
2023-10-24 08:37:03 -04:00
Dessalines
cae25486e4
Adding ts-rs directives for login_token. (#4063)
* Adding ts-rs directives for login_token.

* Fixing ts-rs derive.
2023-10-19 20:16:12 -04:00
6d27bfed08
Handle invalid ban expires values (fixes #4045) (#4046)
* Handle invalid ban expires values (fixes #4045)

* Adding a few missing expire time checks. Fixing up time conversions. (#4051)

* Adding a few missing expire time checks. Fixing up time conversions.

* Increase settings export wait time.

* get rid of RemoveCommunity.expires

* fmt

* tests

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-17 13:25:35 -04:00
3a19af5215
Allow marking multiple posts as read in single api call (fixes #3963) (#4048)
* Allow marking multiple posts as read in single api call (fixes #3963)

* cleanup

* limit array length

* fix test

* review

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-17 12:35:51 -04:00
SleeplessOne1917
cf1c32d2ab
Enable animated avatars setting (#4040)
* feat: Add user setting for toggling avatar gif animations

* Add forgotten post listing mode option to settings form

* Fix compilation error

* Fix formatting

* Change name of column to be more general purpose

* Fix sql format.

* Fix comments

* Running format.

---------

Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-17 10:52:34 -04:00
56322c75f0
Read community follower count from home instance (fixes #1440) (#4013)
* Read community follower count from home instance (fixes #1440)

* fmt

* prettier

* fix tests

* fmt

* rename fn

* fmt

* Run prettier

* increase timeout

* ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-17 10:34:38 -04:00
332e698336
Cleanup public api (#4047)
* Convert PersonSortType to purely internal

* Remove hot rank and other db optimizations from public api
2023-10-16 19:37:28 -04:00
256ee61908
Ignore activities in remote communities without local followers (#4006)
* Ignore activities in remote communities without local followers (fixes #3568)

* x

* comments

* prettier

* fix api test

* fix test

* cleanup

* fix remaining test

* clippy

* decrease delay
2023-10-16 06:03:49 -04:00
9e099726e6
Cleanup checks for community actions (fixes #2858, fixes #2868) (#4028)
* Cleanup checks for community actions (fixes #2858, fixes #2868)

* allow restoring deleted community

* review changes

* remove unneeded sql

* remove joins

* change mod log check
2023-10-13 09:48:18 -04:00
6d7b38f4de
Implement user data import/export (#3976)
* Implement endpoints for user data import/export

* add test

* exclude avatar/banner

* increase import url count, add rate limit

* also export/import saved posts

* rate limit

* rename

* saved posts also exist

* rename routes

* fix test

* error handling

* clippy

* limit parallelism

* clippy

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-11 16:47:22 +02:00
Hamzah Mansour
6470ea81a7
add keyboard navigation setting (#3569)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-10-10 12:17:27 +02:00
dc327652a5
Add db table for login tokens which allows for invalidation (#3818)
* wip

* stuff

* fmt

* fmt 2

* fmt 3

* fix default feature

* use Authorization header

* store ip and user agent for each login

* add list_logins endpoint

* serde(skip) for token

* fix api tests

* A few suggestions for login_token (#3991)

* A few suggestions.

* Fixing SQL format.

* review

* review

* rename cookie

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-10-09 12:46:12 +02:00
22608ae983
Rework the way 2FA is enabled/disabled (fixes #3309) (#3959)
* Rework the way 2FA is enabled/disabled (fixes #3309)

* postgres format

* change algo to sha1 for better compat

* review comments

* review

* clippy

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-09-20 10:49:54 -04:00
50f81cf157
User can block instances (fixes #2397) (#3869)
* User can block instances (fixes #2397)

* update comments

* review comments

* use route

* update

* add api test

* update tests

* fix

* fix test

* ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-09-20 11:56:13 +02:00
phiresky
375d9a2a3c
Persistent, performant, reliable federation queue (#3605)
* persistent activity queue

* fixes

* fixes

* make federation workers function callable from outside

* log federation instances

* dead instance detection not needed here

* taplo fmt

* split federate bin/lib

* minor fix

* better logging

* log

* create struct to hold cancellable task for readability

* use boxfuture for readability

* reset submodule

* fix

* fix lint

* swap

* remove json column, use separate array columns instead

* some review comments

* make worker a struct for readability

* minor readability

* add local filter to community follower view

* remove separate lemmy_federate entry point

* fix remaining duration

* address review comments mostly

* fix lint

* upgrade actitypub-fed to simpler interface

* fix sql format

* increase delays a bit

* fixes after merge

* remove selectable

* fix instance selectable

* add comment

* start federation based on latest id at the time

* rename federate process args

* dead instances in one query

* filter follow+report activities by local

* remove synchronous federation

remove activity sender queue

* lint

* fix federation tests by waiting for results to change

* fix fed test

* fix comment report

* wait some more

* Apply suggestions from code review

Co-authored-by: SorteKanin <sortekanin@gmail.com>

* fix most remaining tests

* wait until private messages

* fix community tests

* fix community tests

* move arg parse

* use instance_id instead of domain in federation_queue_state table

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
2023-09-09 12:25:03 -04:00
3b67642ec2
Add ts(skip) attributes to fix js client code generation (#3952)
* Add ts(skip) attributes to fix js client code generation

* use feature flag
2023-09-08 08:35:58 -04:00
Dessalines
9785b20843
Adding a scaled sort, to boost smaller communities. (#3907)
* Adding a scaled sort, to boost smaller communities.

- Previously referred to as *best* .
- Fixes #3622

* Fixing scheduled task update.

* Converting hot_rank integers to floats.

* Altering hot_rank psql function to default to zero after a week.

* Setting scaled_rank to zero, where hot_rank is zero.

* Adding image_upload table.
2023-09-06 13:43:27 -04:00
71d61138bc
Replace ammonia lib with manual html escaping (fixes #3774) (#3938)
* Replace ammonia lib with manual html escaping (fixes #3774)

* prettier

* clippy

* remove sanitize unit test

* fix tests

* fix schema
2023-09-06 10:56:26 -04:00
Anon
fe3ebea95a
Add logging for pictrs uploads (#3927)
* Add logging for pictrs uploads

* cleanup
2023-09-06 09:13:30 -04:00
a0ea8dbc00
Auto resolve reports trigger (#3871)
* Revert "Automatically resolve report when post/comment is removed (#3850)"

This reverts commit f7f6766650.

* Automatically resolve reports using db trigger

* lint

* use mod log tables

* fix migration

* fix ci

* fix clippy
2023-09-05 05:35:10 -04:00
dullbananas
15930cbf4d
Use Queryable instead of JoinView (#3917)
* Update utils.rs

* Update traits.rs

* Update comment_report_view.rs

* Update comment_view.rs

* Update local_user_view.rs

* Update post_report_view.rs

* Update post_view.rs

* Update private_message_report_view.rs

* Update private_message_view.rs

* Update registration_application_view.rs

* Update site_view.rs

* Update structs.rs

* Update comment_reply_view.rs

* Update community_block_view.rs

* Update community_follower_view.rs

* Update community_moderator_view.rs

* Update community_person_ban_view.rs

* Update community_person_ban_view.rs

* Update community_view.rs

* Update person_block_view.rs

* Update person_mention_view.rs

* Update person_view.rs

* Update structs.rs

* Update admin_purge_comment_view.rs

* Update admin_purge_community_view.rs

* Update admin_purge_person_view.rs

* Update admin_purge_post_view.rs

* Update mod_add_community_view.rs

* Update mod_add_view.rs

* Update mod_ban_from_community_view.rs

* Update mod_ban_view.rs

* Update mod_feature_post_view.rs

* Update mod_hide_community_view.rs

* Update mod_lock_post_view.rs

* Update mod_remove_comment_view.rs

* Update mod_remove_community_view.rs

* Update mod_remove_post_view.rs

* Update mod_transfer_community_view.rs

* Update structs.rs

* Update utils.rs

* Update private_message_view.rs

* Update comment_report_view.rs

* Update registration_application_view.rs

* Update utils.rs

* fix

* fix db_views

* fix

* Update comment_view.rs
2023-08-31 15:26:10 +02:00
Dessalines
56e26fc3d4
Adding a post_view mode. Fixes #3730 (#3731)
* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Adding a post_view mode. Fixes #3730

* Fix test.

* Addressing PR comments.

* Fixing column order.

* Fix default Ok returns.

* Removing return Err(... where feasible.
2023-08-31 09:01:08 -04:00
biosfood
384e55f0e4
update moderator view (#3820)
* update api tests for new moderator view

* chage moderator view to be a listing type in get posts

Note: Internally, the listing type is called ListingType.ModeratorView,
but it's called "Moderator View" in the api endpoint

* fix formatting

* add support for moderator view to list comments

* add api test for moderator view when listing comments

* fix api test formatting

* retry tests

* don't filter out blocked users and communities when using moderator view

* fix cargo tests failing

* fix formatting

* fix previous merge

* Adding ModeratorView to listing_type_enums

* Fixing fmt.

* Adding a default to ListingType.

* Upgrading to use new lemmy-js-client.

---------

Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
2023-08-31 13:07:45 +02:00
Lukas Trombach
26d125cc63
Sitemap - use UTC time (#3914)
* change local filter to use `eq` function

* use Utc time for sitemap
2023-08-28 11:09:20 +02:00
phiresky
514f2222e0
Fix time zone handling (#3496)
* convert naive time to utc time

* compounding fixes

* cargo fmt

* fix the rest

* fix down migration

* fix migrations

* fix after merge

* clippy fix

* ap-fed 0.5.0

---------

Co-authored-by: Nutomic <me@nutomic.com>
2023-08-24 11:27:00 -04:00
6047257bfc
Move admin flag from person to local_user (fixes #3060) (#3403)
* Move admin flag from person to local_user (fixes #3060)

The person table is for federated data, but admin flag can only
apply to local users. Thats why it really belongs in the local_user
table. This will also prevent the federation code from accidentally
overwriting the admin flag

* fmt

* try to fix api tests

* lint

* fix person view

* ci

* ci

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-08-24 05:40:08 -04:00
Freakazoid182
51ccf318e8
Lowercase domain on db query filters (#3849) (#3873)
* Lowercase domain on db query filters (#3849)

* Add test to get a community on different cased domain (#3849)

* Lowercase the identity for webfinger (#3849)

* Lowercase both sides of the domain comparison (#3849)

* Format api_tests (#3849)

* Lowercase domain lookup on Instance and Person (#3849)

---------

Co-authored-by: Freek van Zee <freek.van.zee@mediamonks.com>
Co-authored-by: Freakazoid182 <>
2023-08-22 11:10:21 -04:00
Lukas Trombach
28324ad2c8
Sitemap (#3808)
* generate sitemap.xml file

* set up endpoint for sitemap

* Update sitemap generation

- remove sitemap generation from scheduled tasks
- add posts query for sitemap
- create sitemap module in API crate

* remove priority and change freq from sitemap

* add configuration option for number of posts for sitemap

* fix default config

* rate limit sitemap endpoint

* update sitemap query

* update sitemap generation

- remove config value for query limit
- adjust sitemap generation to query changes
- tidy up error handling

* refactor sitemap generation loop

* remove `limit` argument

* refactor `generate_urlset` and add unit test

* change query to only fetch local posts of past 24h

* fix outdated comment and log

* cargo fmt
2023-08-22 10:30:15 -04:00
f7f6766650
Automatically resolve report when post/comment is removed (#3850)
* Automatically resolve report when post/comment is removed

* also handle apub removes
2023-08-08 11:19:55 -04:00