Commit graph

65 commits

Author SHA1 Message Date
Dessalines
fa4825b524
Removing a few expects from production and test code. (#5193)
* Removing a few expects from production and test code.

- Fixes #5192

* Using if let filter for admin emails.

* Fixing unused error.

* Adding expect_used = deny to clippy lints.

* Update src/lib.rs

Co-authored-by: Nutomic <me@nutomic.com>

* Update crates/utils/src/settings/structs.rs

Co-authored-by: Nutomic <me@nutomic.com>

* Update crates/utils/src/settings/mod.rs

Co-authored-by: Nutomic <me@nutomic.com>

* Some more cleanup.

* Fix clippy

---------

Co-authored-by: Nutomic <me@nutomic.com>
2024-11-15 08:18:52 -05:00
Dessalines
231cce9350
Cleanup post action forms (#5197)
* Removing a few SuccessResponses for PostHide and MarkPostAsRead.

- This also removes the pointless multiple post_ids. These can be done
  as individual calls on the front end anyway.
- Fixes #4755

* Fixing federation tests.

* Upgrading lemmy-js-client deps.

* Add ability to mark several posts as read.

Context:

- https://github.com/LemmyNet/lemmy/pull/5043
- https://github.com/LemmyNet/lemmy/issues/4755
- https://github.com/LemmyNet/lemmy/pull/5160

* Simplifying forms.

* Fixing forms.

* Cleanup post action forms by using derive_new defaults.

- Fixes #5195

* Fix ntfy to notify on success builds also.

* Removing pointless naive_now function.

* Running taplo fmt.
2024-11-15 11:21:08 +01:00
dce6c6bbf0
Move aggregates to replaceable_schema, fix error (fixes #5186) (#5190)
* Move aggregates to replaceable_schema, fix error (fixes #5186)

* fmt

* r prefix
2024-11-12 12:03:30 -05:00
dullbananas
2e8687e203
Combine action tables (#4459)
* Update comment_report_view.rs

* Update comment_report_view.rs

* Update post_view.rs

* Update utils.rs

* Update schema.rs

* stuff

* stuff

* fix actions

* PostLike

* fmt

* more post stuff (partial)

* remove uplete

* returning

* rename read_comments field

* PersonPostAggregates

* a

* fix usage of read_comments_amount

* comment

* community

* community_block

* instance_block

* LocalUser::export_backup

* person_block

* person

* stuff (partial)

* update cargo.lock

* fix lemmy_db_schema

* post_view

* comment_report_view

* comment_view

* post_report_view

* find and replace some selected values

* private_message_view

* vote_view

* comment_reply_view

* some action views

* action_query, find_action

* community_view

* block views

* person_mention_view

* remove struct update

* refactor actions function

* actions_alias

* clean up return types and trait bounds

* fix all rust code

* fmt

* clippy fix

* Migrate tables

* migrate indexes and contraints, and add statistics

* fix what appears to be a messed up merge

* commented thing

* Create uplete.rs

* Update uplete.rs

* Update uplete.rs

* Update up.sql

* Update comment.rs

* Update Cargo.toml

* Update comment.rs

* Update post.rs

* Update comment_view.rs

* Update post_view.rs

* Update comment_reply_view.rs

* Update person_mention_view.rs

* Update Cargo.toml

* Update utils.rs

* Update comment.rs

* Update utils.rs

* Update uplete.rs

* Update uplete.rs

* Update uplete.rs

* Update comment.rs

* Update uplete.rs

* Update uplete.rs

* Update uplete.rs

* Update uplete.rs

* Update uplete.rs

* Update uplete.rs

* Update comment_view.rs

* Update post_view.rs

* Update triggers.sql

* Update triggers.sql

* Update triggers.sql

* Update comment_reply_view.rs

* Update person_mention_view.rs

* Update person_mention_view.rs

* Update comment_reply_view.rs

* Update uplete.rs

* start removing post_id column

* Update down.sql

* Update schema.rs

* Update comment.rs

* Update comment.rs

* Update comment.rs

* Update schema.rs

* Update comment.rs

* Update like.rs

* Update comment.rs

* Update up.sql

* Update down.sql

* Update down.sql

* Update up.sql

* Update up.sql

* Update down.sql

* Update comment.rs

* Update vote_view.rs

* Update vote_view.rs

* Update comment_aggregates.rs

* Update person_aggregates.rs

* Update comment_view.rs

* Update vote_view.rs

* Update mod.rs

* Update create.rs

* Update comment.rs

* Update community.rs

* Update community.rs

* Update up.sql

* Update uplete.rs

* Update uplete.rs

* revert to UpleteCount

* Update comment.rs

* Update traits.rs

* Update comment.rs

* Update community.rs

* Update community_block.rs

* Update community.rs

* Update instance_block.rs

* Update instance.rs

* Update community_block.rs

* Update person.rs

* Update person_block.rs

* Update person.rs

* Update person_block.rs

* Update person.rs

* Update instance.rs

* Update instance_block.rs

* Update instance.rs

* Update person.rs

* Update post.rs

* Update comment.rs

* Update community.rs

* Update person.rs

* Update post_view.rs

* Update comment.rs

* reduce diff

* revert some changes in views

* Update post_view.rs

* Update comment.rs

* Update post.rs

* fix missing cfg_attr

* rewrite uplete

* Update Cargo.toml

* Update Cargo.toml

* Update uplete.rs

* add `pub` to structs that appear in trait bounds

* optional = true

* Update uplete.rs

* Update community.rs

* Update comment.rs

* Update uplete.rs

* Update uplete.rs

* Update uplete.rs

* Update uplete.rs

* something is wrong with me

* use new uplete function everywhere

* fmt

* fmt

* Keep post_id when comment_actions::liked is not null

* Update up.sql

* Update up.sql

* clean up up.sql

* clean up down.sql

* fix

* Update person_aggregates.rs

* fmt

* Update uplete.rs

* fmt

* Update uplete.rs

* Update community.rs

* Update uplete.rs

* Update local_user.rs

* fmt

* fix

* fix

* fmt

* improve uplete api

* Update uplete.rs

* fix

* fix

* Update uplete.rs

* Update uplete.rs

* Update uplete.rs

* Update uplete.rs

* fix

* fix test

* fix db_views_actor imports

* fix uplete import in post_view test

* rerun ci

* fix AllNull syntax error

* fix DynColumn

* Fix rust syntax

* fmt

* fix iter

* pain

* Update community_moderators.rs

* Update community_moderator_view.rs

* Update uplete.rs

* Fix mistake found by chatgpt

* revert debugging stuff, change migration date, refactor constraint

* Update down.sql

* Update down.sql

* fmt

* make things added to db_schema::utils more understandable

* update rust version for woodpecker

* finish merge

* Fix index that checked read_comments twice instead of also checking read_comments_amount

* fix

* uplete: test_count, test_generated_sql_setting_one_column_null, test_count_methods

* refactor uplete sql test

* test setting both columns to null in uplete

* make AllNull generic

* test AllNull

* Merge remote-tracking branch 'upstream/main' into smoosh-tables-together

---------

Co-authored-by: phiresky <phireskyde+git@gmail.com>
2024-11-11 11:34:10 +01:00
ad90cd77f9
Implement private communities (#5076)
* add private visibility

* filter private communities in post_view.rs

* also filter in comment_view

* community follower state

* remove unused method

* sql fmt

* add CommunityFollower.approved_by

* implement api endpoints

* api changes

* only admins can create private community for now

* add local api tests

* fix api tests

* follow remote private community

* use authorized fetch for content in private community

* federate community visibility

* dont mark content in private community as public

* expose ApprovalRequired in api

* also check content fetchable for outbox/featured

* address private community content to followers

* implement reject activity

* fix tests

* add files

* remove local api tests

* dont use delay

* is_new_instance

* single query for is_new_instance

* return subscribed type for pending follow

* working

* need to catch errors in waitUntil

* clippy

* fix query

* lint for unused async

* diesel.toml comment

* add comment

* avoid db reads

* rename approved_by to approver_id

* add helper

* form init

* list pending follows should return items for all communities

* clippy

* ci

* fix down migration

* fix api tests

* references

* rename

* run git diff

* ci

* fix schema check

* fix joins

* ci

* ci

* skip_serializing_none

* fix test

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-11-07 05:49:05 -05:00
Dessalines
18bf9843bc
Fixing LemmyError imports. (#5157) 2024-11-04 10:44:58 +01:00
Integral
8f88dda28f
refactor: destructure tuples to enhance readability (#5151) 2024-10-31 08:12:24 -04:00
f6a24e133a
Replace clippy allow annotation with expect (fixes #5012) (#5048) 2024-09-24 13:29:02 -04:00
9eee61dd06
Post scheduling (fixes #234) (#5025)
* Post scheduling (fixes #234)

* clippy

* replace map_err with inspect_err

* ignore unpublished posts in read queries

* add api test

* fmt

* add some checks

* address some review comments

* allow updating schedule time

* rewrite scheduled task

* fmt

* machete

* compare date in sql, more filters

* check for community ban in sql

* remove api test (scheduled task only runs every 10 mins)

* remove mut

* add index

* remove Post::read impl

* fmt

* fix

* correctly handle changes to schedule time

* normal users can only schedule up to 10 posts
2024-09-24 05:39:40 -04:00
a8843335a6
Simplify handling of NotFound SQL errors (fixes #4633) (#5031)
* Simplify handling of NotFound SQL errors (fixes #4633)

* fmt

* wip

* compiling

* clippy

* api tests

* fix
2024-09-23 11:26:50 -04:00
Dessalines
458bb60144
Get rid of a lot of pointless mut form initializations. (#5037)
* Get rid of a lot of pointless mut form initializations.

- Fixes #5036

* Fix clippy.
2024-09-23 12:05:18 +02:00
ad75192bae
Remove TypedBuilder in favor of derive_new (fixes #4863) (#5020)
* Remove TypedBuilder in favor of derive_new (fixes #4863)

* fix

* fix
2024-09-20 08:15:25 -04:00
Dessalines
f229f09f92
Changing nodeinfo test from voyager.lemmy.ml to lemmy.ml (#4891)
- Fixes #4870
2024-07-07 19:01:03 -04:00
dullbananas
9ceb5b6386
Clean up build_update_instance_form in scheduled_tasks.rs (#4775)
* Clean up build_update_instance_form in scheduled_tasks.rs

* remove unused import
2024-06-04 08:04:16 -04:00
Dessalines
4195a9b5a1
Fetch nodeinfo href from .well-known/nodeinfo . Fixes #4757 (#4765)
* Fetch nodeinfo href from .well-known/nodeinfo . Fixes #4757

* Addressing PR comments.

* Fixing clippy.

* Adding tests.
2024-06-03 17:30:00 -04:00
c96017c009
Configure max comment width in clippy (#4738)
* Configure max comment width in clippy

* update default config
2024-05-23 08:46:26 -04:00
dullbananas
4ba6221e04
Move SQL triggers from migrations into reusable sql file (#4333)
* stuff

* stuff including batch_upsert function

* stuff

* do things

* stuff

* different timestamps

* stuff

* Revert changes to comment.rs

* Update comment.rs

* Update comment.rs

* Update post_view.rs

* Update utils.rs

* Update up.sql

* Update up.sql

* Update down.sql

* Update up.sql

* Update main.rs

* use anyhow macro

* Create down.sql

* Create up.sql

* Create replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update utils.rs

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Create dump_schema.sh

* Update start_dev_db.sh

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* stuff

* Update replaceable_schema.sql

* Update .pg_format

* fmt

* stuff

* stuff (#21)

* Update replaceable_schema.sql

* Update up.sql

* Update replaceable_schema.sql

* fmt

* update cargo.lock

* stuff

* Update replaceable_schema.sql

* Remove truncate trigger because truncate is already restricted by foreign keys

* Update replaceable_schema.sql

* fix some things

* Update replaceable_schema.sql

* Update replaceable_schema.sql

* Update .woodpecker.yml

* stuff

* fix TG_OP

* Psql env vars

* try to fix combine_transition_tables parse error

* Revert "try to fix combine_transition_tables parse error"

This reverts commit 75d00a4626.

* refactor combine_transition_tables

* try to fix create_triggers

* fix some things

* try to fix combined_transition_tables

* fix sql errors

* update comment count in post trigger

* fmt

* Revert "fmt"

This reverts commit a5bcd0834b.

* Revert "update comment count in post trigger"

This reverts commit 0066a4b42b.

* fix everything

* Update replaceable_schema.sql

* actually fix everything

* refactor create_triggers

* fix

* add semicolons

* add is_counted function and fix incorrect bool operator in update_comment_count_from_post

* refactor comment trigger

* refactor post trigger

* fix

* Delete crates/db_schema/src/utils/series.rs

* subscribers_local

* edit migrations

* move migrations

* remove utils::series module declaration

* fix everything

* stuff

* Move sql to schema_setup dir

* utils.sql

* delete .pg_format

* Update .woodpecker.yml

* Update sql_format_check.sh

* Update .woodpecker.yml

* Merge remote-tracking branch 'upstream/main' into bliss

* fmt

* Create main.rs

* Update lib.rs

* Update main.rs

* Update .woodpecker.yml

* Update main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* YAY

* Update mod.rs

* Update Cargo.toml

* a

* Update Cargo.toml

* Update Cargo.toml

* Delete crates/db_schema/src/main.rs

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update utils.sql

* Update utils.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update down.sql

* Update up.sql

* Update triggers.sql

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update triggers.sql

* Update down.sql

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update Cargo.toml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update .woodpecker.yml

* Update mod.rs

* Update Cargo.toml

* Update mod.rs

* make dump_schema.sh executable

* fix dump_schema.sh

* defer

* diff dumps

* fmt

* Update utils.sql

* Update .woodpecker.yml

* use correct version for pg_dump

* Update .woodpecker.yml

* Update .woodpecker.yml

* change migration date

* atomic site_aggregates insert

* temporarily repeat tests in CI

* drop r schema in CI migration check

* show ReceivedActivity::create error

* move check_diesel_migration CI step

* Update .woodpecker.yml

* Update scheduled_tasks.rs

* Update scheduled_tasks.rs

* update cargo.lock

* move sql files

* move rank functions

* filter post_aggregates update

* fmt

* cargo fmt

* replace post_id with id

* update cargo.lock

* avoid locking rows that need no change in up.sql

* only run replaceable_schema if migrations were run

* debug ci test failure

* make replaceable_schema work in CI

* Update .woodpecker.yml

* remove println

* Use migration revert and git checkout

* Update schema_setup.rs

* Fix

* Update schema_setup.rs

* Update schema_setup.rs

* Update .woodpecker.yml

---------

Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-04-17 20:58:44 -04:00
TechVest
b0370ae2fd
chore: fix some comments (#4637)
Signed-off-by: TechVest <techdashen@qq.com>
2024-04-17 14:35:54 +02:00
Dessalines
5dea21d531
Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613 (#4614)
* Convert all Result<..., LemmyError> into LemmyResult<...> Fixes #4613

* Fixing clippy.
2024-04-10 10:14:11 -04:00
Dessalines
705e86eb4c
Fixing docker release (#4592)
* Re-add notif on tag failure.

* Upping rust version.

* Version 0.19.4-beta.1a

* Try again.

* Version 0.19.4-beta.1b

* Removing unstable inspect.

* Version 0.19.4-beta.1c

* Remove use release cache.

* Trying to fix cargo publish 1.

* Version 0.19.4-beta.1d

* Re-adding publish release

* Version 0.19.4-beta.2

* Fixing workspace for lemmy_federate
2024-04-04 16:14:59 +02:00
Dessalines
95069d7648
Fixing some clippy and woodpecker lints. (#4565)
* Fixing some clippy and woodpecker lints.

* Try fixing woodpecker 1.

* Revert "Try fixing woodpecker 1."

This reverts commit 7c2020a08d.
2024-03-26 10:17:42 +01:00
Dessalines
890565ca14
Deleting denied local_users older than a week. Fixes #4434 (#4448)
* Deleting denied local_users older than a week. Fixes #4434

* Addressing PR comments.

* Upping rust to 1.76

* Delete the person rows also.
2024-02-15 07:50:53 -05:00
b58da11fb7
Mark instance as alive after successful activity send (fixes #4039) (#4377)
* Mark instance as alive after successful activity send (fixes #4039)

* clippy

* Instance::update

* domain

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2024-01-19 09:40:12 -05:00
ALEX11BR
a5289dd4cf
Use pretty_assertions for assertions (#4347)
* Use `pretty_assertions` for assertions

* fixed ordering of `use`s

* ci

---------

Co-authored-by: Felix Ableitner <me@nutomic.com>
2024-01-04 04:47:18 -05:00
dullbananas
8e2cbc9a0f
Remove id column and use different primary key on some tables (#4093)
* post_saved

* fmt

* remove unique and not null

* put person_id first in primary key and remove index

* use post_saved.find

* change captcha_answer

* remove removal of not null

* comment_aggregates

* comment_like

* comment_saved

* aggregates

* remove "\"

* deduplicate site_aggregates

* person_post_aggregates

* community_moderator

* community_block

* community_person_ban

* custom_emoji_keyword

* federation allow/block list

* federation_queue_state

* instance_block

* local_site_rate_limit, local_user_language, login_token

* person_ban, person_block, person_follower, post_like, post_read, received_activity

* community_follower, community_language, site_language

* fmt

* image_upload

* remove unused newtypes

* remove more indexes

* use .find

* merge

* fix site_aggregates_site function

* fmt

* Primary keys dess (#17)

* Also order reports by oldest first (ref #4123) (#4129)

* Support signed fetch for federation (fixes #868) (#4125)

* Support signed fetch for federation (fixes #868)

* taplo

* add federation queue state to get_federated_instances api (#4104)

* add federation queue state to get_federated_instances api

* feature gate

* move retry sleep function

* move stuff around

* Add UI setting for collapsing bot comments. Fixes #3838 (#4098)

* Add UI setting for collapsing bot comments. Fixes #3838

* Fixing clippy check.

* Only keep sent and received activities for 7 days (fixes #4113, fixes #4110) (#4131)

* Only check auth secure on release mode. (#4127)

* Only check auth secure on release mode.

* Fixing wrong js-client.

* Adding is_debug_mode var.

* Fixing the desktop image on the README. (#4135)

* Delete dupes and add possibly missing unique constraint on person_aggregates.

* Fixing clippy lints.

---------

Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: phiresky <phireskyde+git@gmail.com>

* fmt

* Update community_block.rs

* Update instance_block.rs

* Update person_block.rs

* Update person_block.rs

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: phiresky <phireskyde+git@gmail.com>
2023-11-13 08:14:07 -05:00
cb01427dcf
Only keep sent and received activities for 7 days (fixes #4113, fixes #4110) (#4131) 2023-11-06 16:17:53 -05:00
dullbananas
a14657d124
Refactor rate limiter and improve rate limit bucket cleanup (#3937)
* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update scheduled_tasks.rs

* Shrink `RateLimitBucket`

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update mod.rs

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* Update rate_limiter.rs

* rerun ci

* Update rate_limiter.rs

* Undo changes to  fields

* Manually undo changes to RateLimitBucket fields

* fmt

* Bucket cleanup loop in rate_limit/mod.rs

* Remove rate limit bucket cleanup from scheduled_tasks.rs

* Remove ;

* Remove UNINITIALIZED_TOKEN_AMOUNT

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* fmt

* Update rate_limiter.rs

* stuff

* MapLevel trait

* fix merge

* Prevent negative numbers in buckets

* Clean up MapLevel::check

* MapLevel::remove_full_buckets

* stuff

* Use remove_full_buckets to avoid allocations

* stuff

* remove tx

* Remove RateLimitConfig

* Rename settings_updated_channel to rate_limit_cell

* Remove global rate limit cell

* impl Default for RateLimitCell

* bucket_configs doc comment to explain EnumMap

* improve test_rate_limiter

* rename default to with_test_config

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
2023-10-19 15:31:51 +02:00
Alex Roman
6735a98d35
Async scheduler (#3949)
* fix: switch to async scheduler

* fix: pass context to scheduled tasks

* Merge remote-tracking branch 'upstream/main' into async-scheduler

* retrigger ci

* retrigger ci
2023-09-11 11:12:16 +02:00
Dessalines
9785b20843
Adding a scaled sort, to boost smaller communities. (#3907)
* Adding a scaled sort, to boost smaller communities.

- Previously referred to as *best* .
- Fixes #3622

* Fixing scheduled task update.

* Converting hot_rank integers to floats.

* Altering hot_rank psql function to default to zero after a week.

* Setting scaled_rank to zero, where hot_rank is zero.

* Adding image_upload table.
2023-09-06 13:43:27 -04:00
phiresky
514f2222e0
Fix time zone handling (#3496)
* convert naive time to utc time

* compounding fixes

* cargo fmt

* fix the rest

* fix down migration

* fix migrations

* fix after merge

* clippy fix

* ap-fed 0.5.0

---------

Co-authored-by: Nutomic <me@nutomic.com>
2023-08-24 11:27:00 -04:00
Alex Roman
ab828b81e4
fix: add client connect timeout to scheduled task (#3883) 2023-08-21 10:53:35 +02:00
Denis Dzyubenko
55e383ae38
Fix fetch instance software version from nodeinfo (#3772)
Fixes #3771
2023-08-01 10:55:28 +02:00
Sander Saarend
77a8e3b897
Don't panic when scheduled tasks can't connect to database (#3634)
* Don't panic when scheduled tasks can't connect to database

* Use map & map_err instead of match

* Empty commit for Woodpecker re-run
2023-07-19 15:00:44 +02:00
cetra3
9256895635
Cache & Optimize Woodpecker CI (#3450)
* Try using drone cache plugin

* Try another path

* Include volume

* Fix formatting

* Include fmt

* Exclude cargo dir from prettier

* Don't override cargo

* Just do check

* Add cache key

* Use different cache plugin

* Add clippy

* Try minio

* Add quotes

* Try adding secrets

* Try again

* Again

* Use correct secret formation

* Add back clippy

* Use secret for the root bucket name

* Try drone cache instead

* Add region

* Add path-style option

* Include cargo clippy

* Include everything again

* Fix formatting

* Don't run clippy twice

* Add `allow` statements for tests to pass

* Adjust endpoint to be a secret

* Fix prettier

* Merge & fix tests

* Try to restart the woodpecker test

* Change the ENV var name

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-07-17 11:04:14 -04:00
Sander Saarend
6688a8a5d4
Optimize hot rank updates (#3617) 2023-07-17 11:05:55 +02:00
e9e76549a8
Split activity table into sent and received parts (fixes #3103) (#3583)
* Split activity table into sent and received parts (fixes #3103)

The received activities are only stored in order to avoid processing
the same incoming activity multiple times. For this purpose it is
completely unnecessary to store the data. So we can split the
table into sent_activity and received_activity parts, where
only sent_activity table needs to store activity data. This should
reduce storage use significantly.

Also reduces activity storage duration to three months, we can reduce
this further if necessary.

Additionally the id columns of activity tables are removed because
they are completely unused and risk overflowing (fixes #3560).

* address review

* move insert_received_activity() methods to verify handlers

* remove unnecessary conflict line

* clippy

* use on conflict, add tests
2023-07-14 11:17:06 -04:00
7d8cb93b53
Check for dead federated instances (fixes #2221) (#3427)
* Check for dead federated instances (fixes #2221)

* move to apub crate, use timestamp

* make it compile

* clippy

* use moka to cache blocklists, dead instances, restore orig scheduled tasks

* remove leftover last_alive var

* error handling

* wip

* fix alive check for instances without nodeinfo, add coalesce

* clippy

* move federation blocklist cache to #3486

* unused deps
2023-07-13 16:12:01 +02:00
Sander Saarend
651f2747ee
Only update site_aggregates for local site (#3516) 2023-07-10 11:20:39 -04:00
TKilFree
2aef6a5a33
feat: re-added captcha checks (#3289) 2023-06-27 06:38:53 -04:00
Sander Saarend
211e76dc27
Batch hot rank updates (#3175) 2023-06-27 10:13:51 +02:00
Vijay Ramesh
4db65c191c after 30 days post deletion, replace comment.content and post.body with 'Permanently Deleted' 2023-06-21 12:46:56 -07:00
dullbananas
45818fb4c5
Reduce memory usage of rate limiting (#3111)
* Reduce Vec allocations

* Optimize stuff

* Move embedded migrations to separate crate

* Revert "Move embedded migrations to separate crate"

This reverts commit 44b1049970.

* clippy, fmt

* Shrink rate limit allowance to f32

* Initialize rate limit allowance directly

* Add removal of old rate limit buckets

* Improve readability

* Remove usage of is_okay_and for Rust 1.67 compatibility

* Add dhat-heap feature

* Fix api_benchmark.sh and add run_and_benchmark.sh

* Revert "Fix api_benchmark.sh and add run_and_benchmark.sh"

This reverts commit b4528e5b85.

* Revert "Add dhat-heap feature"

This reverts commit 08e835d487.

* Manually revert remaining stuff

* Use Ipv6Addr in RateLimitStorage

* Shrink last_checked in RateLimitBucket to 32 bits

* Fix rate_limit::get_ip

* Stuff (#1)

* Update rate_limiter.rs

* Update mod.rs

* Update scheduled_tasks.rs

* Fix rate_limiter.rs

* Dullbananas patch 1 (#2)

* Update rate_limiter.rs

* Update mod.rs

* Update scheduled_tasks.rs

* Fix rate_limiter.rs

* Rate limit IPv6 addresses in groups

* Fmt lib.rs

* woodpicker trigger

* Refactor and comment `check_rate_limit_full`

* Add `test_split_ipv6`

* Replace -2.0 with UNINITIALIZED_TOKEN_AMOUNT

* Add `test_rate_limiter`

---------

Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
2023-06-21 10:28:20 +02:00
Dessalines
981c111567
Move connection creation into scheduler. (#3120)
* Move connection creation into scheduler.

- #3076

* Fix clippy.
2023-06-20 11:33:03 +02:00
Sander Saarend
68d814b9b1
Handle scheduled task errors (#3090)
* Add --disable-scheduled-tasks CLI flag

* Add error handling for scheduled tasks
2023-06-15 11:29:12 +02:00
Dessalines
8cb5939f50
Adding hot_rank columns in place of function sorting. (#2952)
* Adding hot_rank columns in place of function sorting.

- Creates hot_rank columns for post, comment, and community.
- Fixes #2932

* Updating all hot ranks on startup.

* Fixing post.url migration.

* Removing update_instance_software from startup.

* Adding post_rank query
2023-06-08 16:15:15 -04:00
Dessalines
1b5437cbe3
Addressing slow profile queries. #2777 (#2830)
* Addressing slow profile queries. #2777

* Addressing PR comments.
2023-04-25 19:28:06 -04:00
Dessalines
209c8a9185
Switch from bb8 to deadpool. Fixes #2765 (#2768)
* Switch from bb8 to deadpool. Fixes #2765

* Remove unecessary deadpool dependency.

* Ignoring nodeinfo test.
2023-02-28 22:45:37 +01:00
Dessalines
3735c6fabf
Adding instance software and version. Fixes #2222 (#2733)
* Adding instance software and version. Fixes #2222

* Fix clippy.

* Fix clippy 2

* Fixing some more issues.
2023-02-18 09:36:12 -05:00
Dessalines
a610211557
Fixing .drone.yml (#2677)
* Try to fix docker/drone plugin

* Trying to use one rust image.

* Fixing drone 1.

* Fixing drone 2.

* Add drone notify.

* Fixing drone 3.

* Fixing drone 4.

* Fix clippy.

* Remove uninlined format lint.

* Combine all cargo tasks

* Fixing drone 5.

* Fixing drone 6.

* Fixing drone 7.

* Fixing drone 8.

* Fixing drone 9.

* Fixing drone 10.

* Fixing drone 12.

* Fixing drone 13.

* Fixing drone 14.

* Fixing drone 15.

* Fixing drone 16.

* Fixing drone 17.

* Fixing drone 18.

* Fixing drone 19.

* Fixing drone 20.

* Fixing drone 21.

* Fixing drone 22.

* Fixing drone 23.

* Fixing drone 24.

* Fixing drone 25.

* Fixing drone 26.

* Fixing drone 27.

* Fixing drone 28.

* Fixing drone 29.

* Fixing drone 30.

* Fixing drone 31.

* Fixing drone 32.

* Fixing drone 33.

* Fixing drone 34.

* Fixing drone 35.

* Fixing drone 36.

* Fixing drone 37.

* Fixing drone 38.

* Fixing drone 39.

* Fixing drone 40.

* Fixing drone 41.

* Fixing drone 43.

* Fixing drone 44.

* Fixing drone 45.

* Last cleanup.

* Fixing drone 46.

* Separate ci steps (#2679)

* separate ci steps

* fix 1

* add comments

* dont add rustfmt explicitly

* Revert "dont add rustfmt explicitly"

This reverts commit 358ce3302a134b7ac88d90a854079356995e9725.

* dont use all features for tests

---------

Co-authored-by: Nutomic <me@nutomic.com>
2023-01-30 14:17:24 -05:00
6f3bf4634b
Various pedantic clippy fixes (#2568)
* Various pedantic clippy fixes

* more clippy pedantic fixes

* try to fix ci

* add fix clippy script, use rust 1.65

* fix clippy
2022-11-19 04:33:54 +00:00