* Check for dead federated instances (fixes#2221)
* move to apub crate, use timestamp
* make it compile
* clippy
* use moka to cache blocklists, dead instances, restore orig scheduled tasks
* remove leftover last_alive var
* error handling
* wip
* fix alive check for instances without nodeinfo, add coalesce
* clippy
* move federation blocklist cache to #3486
* unused deps
* a lot
* merge
* Fix stuff broken by merge
* Get rid of repetitive `&mut *context.conn().await?`
* Add blank lines under each line with `conn =`
* Fix style mistakes (partial)
* Revert "Fix style mistakes (partial)"
This reverts commit 48a033b87f4fdc1ce14ff86cc019e1c703cd2741.
* Revert "Add blank lines under each line with `conn =`"
This reverts commit 773a6d3beba2cf89eac75913078b40c4f5190dd4.
* Revert "Get rid of repetitive `&mut *context.conn().await?`"
This reverts commit d2c6263ea13710177d49b2791278db5ad115fca5.
* Use DbConn for CaptchaAnswer methods
* DbConn trait
* Remove more `&mut *`
* Fix stuff
* Re-run CI
* try to make ci start
* fix
* fix
* Fix api_common::utils
* Fix apub::activities::block
* Fix apub::api::resolve_object
* Fix some things
* Revert "Fix some things"
This reverts commit 2bf8574bc8333d8d34ca542d61a0a5b50039c24d.
* Revert "Fix apub::api::resolve_object"
This reverts commit 3e4059aabbe485b2ff060bdeced8ef958ff62832.
* Revert "Fix apub::activities::block"
This reverts commit 3b02389abd780a7b1b8a2c89e26febdaa6a12159.
* Revert "Fix api_common::utils"
This reverts commit 7dc73de613a5618fa57eb06450f3699bbcb41254.
* Revert "Revert "Fix api_common::utils""
This reverts commit f740f115e5457e83e53cc223e48196a2c47a9975.
* Revert "Revert "Fix apub::activities::block""
This reverts commit 2ee206af7c885c10092cf209bf4a5b1d60327866.
* Revert "Revert "Fix apub::api::resolve_object""
This reverts commit 96ed8bf2e9dcadae760743929498312334e23d2e.
* Fix fetch_local_site_data
* Fix get_comment_parent_creator
* Remove unused perma deleted text
* Fix routes::feeds
* Fix lib.rs
* Update lib.rs
* rerun ci
* Attempt to create custom GetConn and RunQueryDsl traits
* Start over
* Add GetConn trait
* aaaa
* Revert "aaaa"
This reverts commit acc9ca1aed10c39efdd91cefece066e035a1fe80.
* Revert "Revert "aaaa""
This reverts commit 443a2a00a56d152bb7eb429efd0d29a78e21b163.
* still aaaaaaaaaaaaa
* Return to earlier thing
Revert "Add GetConn trait"
This reverts commit ab4e94aea5bd9d34cbcddf017339131047e75344.
* Try to use DbPool enum
* Revert "Try to use DbPool enum"
This reverts commit e4d1712646a52006b865a1fbe0dcf79976fdb027.
* DbConn and DbPool enums (db_schema only fails to compile for tests)
* fmt
* Make functions take `&mut DbPool<'_>` and make db_schema tests compile
* Add try_join_with_pool macro and run fix-clippy on more crates
* Fix some errors
* I did it
* Remove function variants that take connection
* rerun ci
* rerun ci
* rerun ci
Two bugs were found and fixed:
- previously elements removal and deletion were counted as two separate disappearances
- removing comments did not affect post aggregations
* Use serde(skip) instead of skip_serializing
The latter breaks lemmy_crawler as the field is not included in
the Lemmy API, but is required when attempting to parse API responses.
Should only use serde(skip) to avoid this problem
* use option
* add placeholders
* no unwrap
* Remove `actix_rt` & use standard tokio spawn
* Adjust rust log back down
* Format correctly
* Update cargo lock
* Add DB settings
* Change name and update to latest rev
* Clean up formatting changes
* Move `worker_count` and `worker_retry_count` to settings
* Update defaults
* Use `0.4.4` instead of git branch
* Adding hot_rank columns in place of function sorting.
- Creates hot_rank columns for post, comment, and community.
- Fixes#2932
* Updating all hot ranks on startup.
* Fixing post.url migration.
* Removing update_instance_software from startup.
* Adding post_rank query
* update diesel to 2.1.0 and diesel-async to 0.3.0 (issue #2882)
* get rid of deprecated RawValue type alias usage, explicitly import diesel types instead of using wildcard import
* Update diesel-derive-enum and diesel-derive-newtype libraries
* Revert manual changes to schema.rs
* Update to newer diesel-cli image in CI, run fix-clippy before schema comparison, update schema to one generated with diesel-cli 2.1.0 and clippyfixed
* Remove wildcard import lint, fix diesel schema pipeline
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Adding typescript generation for API. Fixes#2824
* Try to fix Ltree issue 1.
* Forgot a few types.
* Fixing api tests.
* Removing url_serde line.
* Manually deriving TS for some types.
* Adding diesel enums for SortType and ListingType
- Uses diesel-derive-enum.
- Adds diesel.toml , so we can again use the auto-generated schema.rs
- Fixes a lot of DB null issues and column ordering issues.
- Fixes#1136
- Also replaces RegistrationMode boilerplate.
* Fixing unit tests 1.
* Remove comment line.
* Before patch.
* Before again.
* Using patch file to fix diesel_ltree issue with diesel.toml
* Adding some yalc ignores
* Fixing RegistrationMode enums
* Adding woodpecker diesel schema check.
* Try adding openssl 1.
* Try using diesel-cli image 1
* Try using diesel-cli image 2
* Try using diesel-cli image 3
* Try using diesel-cli image 4
* Try using diesel-cli image 5
* Try using diesel-cli image 6
* Try using diesel-cli image 7
* Try using diesel-cli image 8
* Try using diesel-cli image 9
* Try using diesel-cli image 10
* Try using diesel-cli image 11
* Try using diesel-cli image 12
* Try using diesel-cli image 13
* update activitypub-federation crate to 0.4.0
* fixes
* apub compiles!
* everything compiling!
* almost done, federated follow failing
* some test fixes
* use release
* add code back in
* Combine prod and dev docker setups using build-arg
- Fixes#2603
* Dont use cache for release build.
* Adding 2FA / TOTP support.
- Fixes#2363
* Changed name to totp_2fa for clarity.
* Switch to sha256 for totp.
* Get rid of Safe Views, use serde_skip
- Also change the ViewToVec, to work with non-vector cases. Might be
necessary in preparation for #2763
- Fixes#2712
* Forgot one safe
---------
Co-authored-by: Nutomic <me@nutomic.com>
* Show deleted and removed posts for profile views. Fixes#2624
* Only showing non-deleted/removed posts for creator.
* Add a admin or mod check to views, to show deleted and removed posts.
- Also removed the pointless "blanking" functions
* Fix clippy
* Make hidden posts comment clearer.
* Fixing federation tests.
* Fixing fmt.
This is not necessary because the domain cant change, so we only
need to insert if no row exists for this domain.
Also fetch instance actor when parsing person, not only community
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Implement separate mod activities for feature, lock post
Also includes collection for featured posts. Later we also need
to do the same for Comment.distinguished
* some changes
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
- Other platforms can support additional language tags. Treat those
as None instead of throwing error
- deserialize_skip_error was implemented incorrectly and failed on
array values (6d9682f4e6)
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Speeding up comment-ltree migration, fixing index creation. Fixes#2664
* Adding some logging lines, fixing for missing posts.
* Adding more postgres config
* Fix error during site creation due to empty LocalSiteRateLimit update
* Move main function into lib.rs, to allow calling from other crates
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Implement federated user following (fixes#752)
* rewrite send_activity_in_community and add docs, remove default for column pending
* improve migration
* replace null values in db migration
* Specify dependencies and metadata for entire workspace (fixes#2474)
* building api_common shouldnt depend on diesel
* remove drone task
* try with quotes
* fix clippy
* more clippy
* Moving settings to Database.
- Moves many settings into the database. Fixes#2285
- Adds a local_site and instance table. Fixes#2365 . Fixes#2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.
* Fix weird clippy issue.
* Removing extra lines.
* Some fixes from suggestions.
* Fixing apub tests.
* Using instance creation helper function.
* Move forms to their own line.
* Trying to fix local_site_data, still broken.
* Testing out async
* Testing out async 2
* Fixing federation tests.
* Trying to fix check features 1.
* Starting on adding diesel async. 1/4th done.
* Added async to views and schema.
* Adding some more async
* Compiling now.
* Added diesel async. Fixes#2465
* Running clippy --fix
* Trying to fix cargo test on drone.
* Trying new muslrust.
* Trying a custom dns
* Trying a custom dns 2
* Trying a custom dns 3
* Trying a custom dns 4
* Trying a custom dns 5
* Trying a custom dns 6
* Trying a custom dns 7
* Addressing PR comments.
* Adding check_apub to all verify functions.
* Reverting back drone.
* Fixing merge
* Fix docker images.
* Adding missing discussion_languages.
* Trying to fix federation tests.
* Fix site setup user creation.
* Fix clippy
* Fix clippy 2
* Test api faster
* Try to fix 1
* Try to fix 2
* What are these lines about
* Trying to fix 3
* Moving federation test back to top.
* Remove logging cat.
* Moving settings to Database.
- Moves many settings into the database. Fixes#2285
- Adds a local_site and instance table. Fixes#2365 . Fixes#2368
- Separates SQL update an insert forms, to avoid runtime errors.
- Adds TypedBuilder to all the SQL forms, instead of default.
* Fix weird clippy issue.
* Removing extra lines.
* Some fixes from suggestions.
* Fixing apub tests.
* Using instance creation helper function.
* Move forms to their own line.
* Trying to fix local_site_data, still broken.
* Fixing federation tests.
* Trying to fix check features 1.
* Addressing PR comments.
* Adding check_apub to all verify functions.
* implement language tags for site/community in db and api
* add api checks for valid languages
* during db migration, update existing users, sites, communities to have all languages enabled
* init new users/communities with site languages (not all languages)
* federate site/community languages
* fix tests
* when updating site languages, limit community languages to this subset
also, when making a new post and subset of user lang, community lang
contains only one item, use that as post lang
* add tests for actor_language db functions
* include language list in siteview/communityview
* Fix some of the review comments
* Some more review changes
* Add todo about boxed query
* Add default_post_language to GetCommunityResponse
* Initial commit to bump diesel to 2.0.0-rc.0 and see what happens
* Add chrono feature from diesel
* db_schema crate is close to building?
* Upgrade diesel-derive-newtype
* Mostly modifying references to connections to be mutable ones; also used
new way to do migrations as suggested by the migration guide; a lot more
compiles now, though I can't figure out this tricky ToSql issue at the
moment
* Running clippy --fix
* Trying to fix drone clippy 1
* Fix clippy
* Upgrade clux-musl
* Trying to fix drone clippy 2
* Trying to fix drone clippy 3
* Trying to fix drone clippy 5
* Adding diesel table aliases, removing sql view hack. Fixes#2101
Co-authored-by: Steven Chu <stevenc1@gmail.com>
Co-authored-by: Nutomic <me@nutomic.com>
* Tag posts and comments with language (fixes#440)
* Untangle PostView tests
* Implement test for PostView language query
* Store languages directly in database
* finish moving languages into db, it compiles
* update post_view
* serde skip Language.id field
* add local_user_language table, other changes suggested in review
* add code for local_user_discussion_language_view
* Remove unnecessary clones in db view converteres
* Fixing up some table and join issues.
* Clearing the current languages.
* Fix formatting.
* update user languages in single transaction
* proper test for user language queries
* Some fixes for all / missing user languages. (#2404)
* Some fixes for all / missing user languages.
* Adding back in transaction.
* fix test
Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* First pass at adding comment trees.
- Extracted comment replies into its own table.
- Added ltree column to comment
- Added parent_id param to GetComments to fetch a tree branch
- No paging / limiting yet
* Adding child_count to comment_aggregates.
* Adding parent comment update counts
* Fix unit tests.
* Comment tree paging mostly done.
* Fix clippy
* Fix drone tests wrong postgres version.
* Fix unit tests.
* Add back in delete in unit test.
* Add postgres upgrade script.
* Fixing some PR comments.
* Move update ltree into Comment::create
* Updating based on comments.
* Fix send soft fail.
* Adding check for requests with no id or name, adding max limit.
* Consolidating a few functions.
* Fix page min
* Adding more websocket rate limits.
* Add check to GetCommunity
* Use a default message rate limit check.
* Adding a page and limit checker
* Fix clippy
* Fix clippy again
* Adding check for requests with no id or name, adding max limit.
* Consolidating a few functions.
* Fix page min
* Adding more websocket rate limits.
* Add check to GetCommunity
* Use a default message rate limit check.
* Adding a page and limit checker
* Fix clippy
* Fix clippy again
* Fix limit request.
* Move checks to inside limit_and_offset
* Fixing API tests.
* Change NotFound diesel errors to QueryBuilderError
* Specify minimum Rust version 1.57 (fixes#2333)
* use latest rust for clippy
* use rust 1.60 for clippy
* no add component
* use official rust docker
* comments
* First pass at adding admin purge. #904#1331
* Breaking out purge into 4 tables for the 4 purgeable types.
* Using CommunitySafe instead in view
* Fix db_schema features flags.
* Attempting to pass API key.
* Adding pictrs image purging
- Added pictrs_config block, for API_KEY
- Clear out image columns after purging
* Remove the remove_images field from a few of the purge API calls.
* Fix some suggestions by @nutomic.
* Add separate pictrs reqwest client.
* Update defaults.hjson
Co-authored-by: Nutomic <me@nutomic.com>
* Create example for apub lib
* some rewriting of apub lib
* Add LocalInstance struct for apub lib to avoid using Lemmy Settings
* Move ActorType trait to lemmy_apub, because its not needed in library
* Use reqwest_retry instead of custom impl, dont specify timeout on every send()
* Some improvements to example
* Moved inbox handling to library
* bug fixes
* Move context and serde helpers into library
* wip: example changes
* Add lemmy_utils feature to build only LemmyError
* Rename to activitypub_federation
* Remove lemmy_utils dep from activitypub_federation using generic error type
* Finish activitypub example
* Cleanup and fix tests
* Reorganize library files
* Remove ApubObject.to_tombstone()
* Extract activitypub library into separate git repository
* Derive default for api request structs, move type enums
* Simplify api by using enum types directly, instead of string
* Add default and clone for most api structs
* Initial working of hiding communities and adding a db entry for mod log
* Return mod log for hidden communities
* Clean up hidding communities PR
* use lower case like other migration files
* Formatting fix
* pass in admin id to list, make match logic the same in post_view as community_view. Dont force non null for reason
* Clean PR review stuff
* Change person_id to mod_person_id on hide community table
* Make bools optional, add a space for formating
Co-authored-by: Thor Odinson <odinson@asgard.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Implement instance actor
* wip: make site bans federate
* finish implementation and unit tests for federated bans
* start adding api tests
* fix api test
* remve site from GetCommunityResponse
* only federate site bans originating from user's home instance
* dont expose site.private_key in api
* Make webfinger case insensitive
* Make webfinger case insensitive
* Case insensitive domain name
* Case-insensitive webfinger
* formatting
Co-authored-by: Kradyz <k@radiz.nl>
* First pass at invite-only migration.
* Implement email verification (fixes#219)
* remove unwrap
* Adding views and functionality to registration application. #209
* Add private instance site column, and back end checks.
* Adding some message fields to LoginResponse
* Adding private instance to site setup.
* A few additions:
- Add a DeleteAccount response.
- RegistrationApplicationView now has the safe LocalUserSettings.
- Adding VerifyEmail to websocket API, added a proper response type.
* Adding and reorganizing some email helpers.
* A few fixes for private sites:
- Added a check_registration_application function.
- Only send a verification email if its been changed.
- VerifyEmail now returns LoginResponse.
- Deleting the old tokens after a successful email verify.
- If port is missing on email config, display a better error message.
* Version 0.15.0-rc.3
* Adding published to email_verification table.
* Adding fixes from comments.
* Version 0.15.0-rc.4
* Adding modlog private site check.
* Version 0.15.0-rc.6
Co-authored-by: Felix Ableitner <me@nutomic.com>
* Respond directly with LemmyError
Instrument Perform implementations for more precise traces
Use ApiError to format JSON errors when messages are present
Keep SpanTrace output in LemmyError Display impl
* Hide SpanTrace debug output from LemmyError
* Don't log when entering spans, only when leaving
* Update actix-web
* Update actix-rt
* Add newline after error info in LemmyError Display impl
* Propogate span information to blocking operations
* Instrument apub functions
* Use skip_all for more instrument attributes, don't skip 'self' in some api actions
* Make message a static string
* Send proper JSON over websocket
* Add 'message' to LemmyError display if present
* Use a quieter root span builder, don't pretty-print logs
* Keep passwords and emails out of logs
* Re-enable logging Login
* Instrument feeds
* Emit our own errors
* Move error log after status code recording
* Make Sensitive generic over the inner type
* Remove line that logged secrets
* Merge traits ToApub and FromApub into ApubObject
* Rewrite community outbox to use new fetcher
* Rewrite community moderators collection
* Rewrite tombstone
* Trying out rust-musl-builder for cargo publish
* Version 0.13.5-rc.1
* Try rust:nightly
* Version 0.13.5-rc.2
* Try rust slim
* Version 0.13.5-rc.3
* Try rust 1.51
* Version 0.13.5-rc.4
* Trying another nightly
* Version 0.13.5-rc.5
* Trying another fix
* Version 0.13.5-rc.6
* Allow fetching person from Pleroma, including test case (ref #1461)
* Added test case for parsing community from apub json
- fixed a bug with objectid (de)serialization
- fixed a bug with outbox fetching (ref #1582)
* Added apub test for post
* Ignore errors when reading community outbox (fixes#1582)
* Dont fetch community outbox/moderators during tests
* added test for lemmy comment
* Added federation test for pleroma comment
* Added html2md crate to parse comment html from pleroma (fixes#1461)
* some fixes for update_apub_test_files.sh
* Add tests for ToApub, private message, remove update script
* Delete objects from db at the end of each test
* First untested pass at reporting.
* Adding unit tests for post and comment report views
* Fix clippy
* Adding counts, creator_banned, and unresolved_only
* Adding my_vote to report views
* Fixing unit tests.
* Updating lemmy-js-client.
* Fix prod deploy script and clippy
* Try using buster.
* Using more generic and updated images.
* again.
* again.
* Try alternate env
* again.
* again.
* Version 0.11.4-rc.24
* again.
* Better old_tag
* Version 0.11.4-rc.27
* Updating cargo.lock
* Trying to run a cargo check to update the lock
* Version 0.11.4-rc.28
* A first pass at user / community blocking. #426
* Adding unit tests for person and community block.
* Moving migration
* Fixing creator_blocked for comment queries, added tests.
* Don't let a person block themselves
* Fix post creator_blocked
* Adding creator_blocked to PersonMentionView
* Moving blocked and follows to MyUserInfo
* Rename to local_user_view
* Add moderates to MyUserInfo
* Adding BlockCommunityResponse
* Fixing name, and check_person_block
* Fixing tests.
* Using type in Blockable trait.
* Changing recipient to target, adding unfollow to block action.
* Mark account as bot. Fixes#1357
* Fix clippy
* Federate bot status using actor `type` field
* fix clippy
Co-authored-by: Dessalines <tyhou13@gmx.com>
* origin/main:
revert Compose file version from 3.3 to 2.2
Adding more mem limits
bump memory limit of iframely
Remove extra category_id s . Fixes#1429
Fixing wrong user_ and community icon and banner urls.
Remove category from activitypub context
Adding a password length check to other API actions. (#1474)
Update test script
Use URL type in most outstanding struct fields (#1468)
Forbid usage of unwrap
Upgrade Rust version
Rewrite settings implementation. Fixes#1270 (#1433)
Rename `lemmy_structs` to `lemmy_api_structs`
# Conflicts:
# crates/db_schema/src/source/user.rs
* Use URL type in most outstanding struct fields
This fixes all known remaining cases where url fields are stored as
plain strings, with the exception of form fields where empty strings
are used as sentinels (see `diesel_option_overwrite_to_url`).
Tested for regressions in the federated docker setup attempting to
exercise all changed fields, including through apub federation.
Fixes#1385
* Add migration to fix blank-string post.url values to be null
This also then fixes#602
* Address review feedback
- Fixed some unwraps and err message formatting
- Bumped the `url` library to 2.2.1 to fix a bug with serde error
messages
- Add unit tests for the two diesel option override functions
- Fix migration teardown by adding a no-op
* Rename lemmy_db_queries::Url to lemmy_db_queries::DbUrl
* fix compile error
* box PostOrComment variants
Since DB tests execute diesel migrations automatically, concurrent
execution causes flaky failures from simultaneous migrations. This can
be worked around using `cargo test --workspace -- --test-threads=1`,
which is what the CI config does, but this is not intuitive for
newcomer developers and unnecessarily slows down the test suite for
the majority of tests which are safe to run concurrently. This fixes
this issue by integrating with the small test crate `serial_test` and
using it to explicitly mark DB tests to run sequentially while
allowing all other tests to run in parallel.
Additionally, this greatly improves the speed of `cargo test` by
disabling doc-tests in all crates, since these are aren't currently
used and cargo's doc-test pass, even when no doc-tests exist, has
significant overhead. On my machine, this change significantly
improves test suite times by about 85%, making it much more practical
to develop with tools like `cargo watch` auto-running tests.
* Adding some recurring lemmy tasks.
- Add active users by day, week, month, and half year to site and
community. Fixes#1195
- Periodically re-index the aggregates tables that use hot_rank.
Fixes#1384
- Clear out old activities (> 6 months). Fixes#1133
* Some cleanup, recalculating actives every hour.
