* Adding an image_details table to store image dimensions.
- Adds an image_details table, which stores the height,
width, and content_type for local and remote images.
- For LocalImages, this information already comes back with
the upload.
- For RemoteImages, it calls the pictrs details endpoint.
- Fixed some issues with proxying non-image urls.
- Fixes#3328
- Also fixes#4703
* Running sql format.
* Running fmt.
* Don't fetch metadata in background for local API requests.
* Dont export remote_image table to typescript.
* Cleaning up validate.
* Dont proxy url.
* Fixing tests, fixing issue with federated thumbnails.
* Fix tests.
* Updating corepack, fixing issue.
* Refactoring image inserts to use transactions.
* Use select exists again.
* Fixing imports.
* Fix test.
* Removing pointless backgrounded metadata generation version.
* Removing public pictrs details route.
* Fixing clippy.
* Fixing proxy image fetching. Fixes#4703
- This extracts only the proxy image fixes from #4704, leaving off
thumbnails.
* Fix test.
* Addressing PR comments.
* Address PR comments 2.
---------
Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
- Diesel ordinarily throws an error when no results are returned for a
single fetch, which is a bit confusing. This PR ensures that the
missing value cases are all caught, and wrapped with new LemmyErrors,
rather than diesel errors.
- Fixes#4601
* Fixing bug where comment replies wouldn't be sent to blocked instances.
- Instance blocks should only affect communities, not comments.
- Fixes#4590
* Revert "Fixing bug where comment replies wouldn't be sent to blocked instances."
This reverts commit 1349aa351a.
* Only block replies from the community's instance id.
- Also refactor send_local_notifs slightly, since it has to fetch the
community now.
- Fixes#4590
---------
Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
* Generate post thumbnail/metadata in background (ref #4529)
* fix api test
* Apply suggestions from code review
Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
* fix test
---------
Co-authored-by: SleeplessOne1917 <28871516+SleeplessOne1917@users.noreply.github.com>
* On registration, automatically set content languages from accept header
* no need to set site language or default language for new user anymore
* fix test
* fix langs
* avoid duplicate writing of new user languages
* Add a blocklist for URLs.
* Fix SQL format
* Make clippy happy.
* Use regex for URL matching.
* Escape regex chars in URLs.
* Use post for modification.
* Make URL block regex static and remove API routes.
* Add date fields to table and use transaction.
* Use Cache for blocklist.
* Rename check_links + move list to parameters of process_markdown.
* SQL format.
* Format, again.
* Remove println.
* Add API test.
* Set a shorter lifetime for regex in debug mode.
* Add missing macro.
* Update lemmy-js-client
* Update api_test/pnpm-lock.yaml
* Don't break other tests
* Use different URL for test
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: Nutomic <me@nutomic.com>
* Expose LemmyErrorType in lemmy_api_common
* Make conditional compilation gates for utils
* Make it so api_common doesn't pull in unnecessary deps
* Make error type non exhaustive
* Fix formatting
* Format toml
* Add some convenience derives to LemmyError
* Simplify features
* Fix CI compile error
---------
Co-authored-by: SleeplessOne1917 <insomnia-void@protonmail.com>
* Use the Accept-Language header to set new users' language
* Implement clippy suggestions
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Add support for local only community (fixes#1576)
* add filters and tests to db views
* dont federate local only community
* test get apub community http
* tests
* more checks
* wip
* api test
* fix tests
* change community.local_only column to visibility enum
(for private communities)
* sql fmt
* rename vars
* clippy
* fix tests
* update lib
* review
* fix js client version
* update client
* Add markdown rule to add rel=nofollow for all links
* Add markdown image rule to add local image proxy (fixes#1036)
* comments
* rewrite markdown image links working
* add comment
* perform markdown image processing in api/apub receivers
* clippy
* add db table to validate proxied links
* rewrite link fields for avatar, banner etc
* sql fmt
* proxy links received over federation
* add config option
* undo post.url rewriting, move http route definition
* add tests
* proxy images through pictrs
* testing
* cleanup request.rs file
* more cleanup (fixes#2611)
* include url content type when sending post over apub (fixes#2611)
* store post url content type in db
* should be media_type
* get rid of cache_remote_thumbnails setting, instead automatically
take thumbnail from federation data if available.
* fix tests
* add setting disable_external_link_previews
* federate post url as image depending on mime type
* change setting again
* machete
* invert
* support custom emoji
* clippy
* update defaults
* add image proxy test, fix test
* fix test
* clippy
* revert accidental changes
* address review
* clippy
* Markdown link rule-dess (#4356)
* Extracting opengraph_data to its own type.
* A few additions for markdown-link-rule.
---------
Co-authored-by: Nutomic <me@nutomic.com>
* fix setting
* use enum for image proxy setting
* fix test configs
* add config backwards compat
* clippy
* machete
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Prevent crates that shouldn't be published from being published
* Make dependent crates publishable
* Make dependent crates publishable
* fix toml formatting
* Removing publish=true
* Removing versioned deps.
---------
Co-authored-by: SleeplessOne1917 <insomnia-void@protonmail.com>
Co-authored-by: Dessalines <tyhou13@gmx.com>
* Make db queries for GET /api/v3/site in parallel (ref #4244)
* Cache site response
* machete
* Use try_join_with_pool macro
* machete
* taplo
* ttl 1s
* Merge /site_inbox into /inbox (fixes#4137)
Get rid of different inboxes, only use /inbox
Remove shared_inbox_url db columns
add code migration
move to db migration, fixes
machete
fix sql
drop inbox url unique constraints
Dont create auth cookie in backend (#4136)
dont change individual inboxes to shared inbox
Dont send comment reply to user who has community blocked. Fixes#3684 (#4096)
* Dont send comment reply to user who has community blocked. Fixes#3684
* Adding source instance block check.
* Adding api test.
* Addressing PR comments.
* move site inbox rewrite to db
* fix test
* clippy
* clippy 2
* fix test
* Handle invalid ban expires values (fixes#4045)
* Adding a few missing expire time checks. Fixing up time conversions. (#4051)
* Adding a few missing expire time checks. Fixing up time conversions.
* Increase settings export wait time.
* get rid of RemoveCommunity.expires
* fmt
* tests
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* wip
* stuff
* fmt
* fmt 2
* fmt 3
* fix default feature
* use Authorization header
* store ip and user agent for each login
* add list_logins endpoint
* serde(skip) for token
* fix api tests
* A few suggestions for login_token (#3991)
* A few suggestions.
* Fixing SQL format.
* review
* review
* rename cookie
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Remove explicit auth params (ref #3725)
Only take auth via header or cookie. This requires a new version
of lemmy-js-client for api tests to pass.
* rework api_crud
* remove remaining auth params, move logic to session middleware
* fmt, fix test
* update js client
* remove auth param from api tests
* Pass auth as header
* add !
* url vars, setHeader
* cleanup
* fmt
* update
* Updating for new lemmy-js-client.
---------
Co-authored-by: Dessalines <tyhou13@gmx.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* User can block instances (fixes#2397)
* update comments
* review comments
* use route
* update
* add api test
* update tests
* fix
* fix test
* ci
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
This removes the possibility of using a mix of sanitized and
non-sanitized values for `username` in code.
Signed-off-by: Apple Sheeple <AppleSheeple@github>
* persistent activity queue
* fixes
* fixes
* make federation workers function callable from outside
* log federation instances
* dead instance detection not needed here
* taplo fmt
* split federate bin/lib
* minor fix
* better logging
* log
* create struct to hold cancellable task for readability
* use boxfuture for readability
* reset submodule
* fix
* fix lint
* swap
* remove json column, use separate array columns instead
* some review comments
* make worker a struct for readability
* minor readability
* add local filter to community follower view
* remove separate lemmy_federate entry point
* fix remaining duration
* address review comments mostly
* fix lint
* upgrade actitypub-fed to simpler interface
* fix sql format
* increase delays a bit
* fixes after merge
* remove selectable
* fix instance selectable
* add comment
* start federation based on latest id at the time
* rename federate process args
* dead instances in one query
* filter follow+report activities by local
* remove synchronous federation
remove activity sender queue
* lint
* fix federation tests by waiting for results to change
* fix fed test
* fix comment report
* wait some more
* Apply suggestions from code review
Co-authored-by: SorteKanin <sortekanin@gmail.com>
* fix most remaining tests
* wait until private messages
* fix community tests
* fix community tests
* move arg parse
* use instance_id instead of domain in federation_queue_state table
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: SorteKanin <sortekanin@gmail.com>
* Revert "Automatically resolve report when post/comment is removed (#3850)"
This reverts commit f7f6766650.
* Automatically resolve reports using db trigger
* lint
* use mod log tables
* fix migration
* fix ci
* fix clippy
* Move admin flag from person to local_user (fixes#3060)
The person table is for federated data, but admin flag can only
apply to local users. Thats why it really belongs in the local_user
table. This will also prevent the federation code from accidentally
overwriting the admin flag
* fmt
* try to fix api tests
* lint
* fix person view
* ci
* ci
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Replace Option<bool> with bool for PostQuery and CommentQuery (#3819)
* Replace Option<bool> from all list queries (#3819)
---------
Co-authored-by: Freek van Zee <freek.van.zee@mediamonks.com>
* HTML sanitization in apub code
* Sanitize API inputs
* fmt
* Dont allow html a, img tags
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Remove SendActivity and Perform traits, rely on channel
These traits arent necessary anymore now that websocket is removed.
Removing them allows us to use normal actix http handler methods
which are much more flexible, and allow using different middlewares
as well as setting response attributes.
* compiling and create post federating
* clippy
* rename methods, join outgoing activities task
* fix api tests
* no unwrap
* conditional compile
* add back getrandom
* make crates optional
* fmt
* add new function build_post_response_deleted_allowed
* PostDelete uses new function build_post_response_deleted_allowed
* RemovePost uses new build_post_response_deleted_allowed function
* code comments about mod or admin flag having other use
* reformat "cargo +nightly fmt --all"
* Try using drone cache plugin
* Try another path
* Include volume
* Fix formatting
* Include fmt
* Exclude cargo dir from prettier
* Don't override cargo
* Just do check
* Add cache key
* Use different cache plugin
* Add clippy
* Try minio
* Add quotes
* Try adding secrets
* Try again
* Again
* Use correct secret formation
* Add back clippy
* Use secret for the root bucket name
* Try drone cache instead
* Add region
* Add path-style option
* Include cargo clippy
* Include everything again
* Fix formatting
* Don't run clippy twice
* Add `allow` statements for tests to pass
* Adjust endpoint to be a secret
* Fix prettier
* Merge & fix tests
* Try to restart the woodpecker test
* Change the ENV var name
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* change pool fields to parameters for list
* remove my_person_id and admin fields
* Change recipient id to list param
* Remove TypedBuilder from db_views and db_views_actor
* a lot
* merge
* Fix stuff broken by merge
* Get rid of repetitive `&mut *context.conn().await?`
* Add blank lines under each line with `conn =`
* Fix style mistakes (partial)
* Revert "Fix style mistakes (partial)"
This reverts commit 48a033b87f.
* Revert "Add blank lines under each line with `conn =`"
This reverts commit 773a6d3beb.
* Revert "Get rid of repetitive `&mut *context.conn().await?`"
This reverts commit d2c6263ea1.
* Use DbConn for CaptchaAnswer methods
* DbConn trait
* Remove more `&mut *`
* Fix stuff
* Re-run CI
* try to make ci start
* fix
* fix
* Fix api_common::utils
* Fix apub::activities::block
* Fix apub::api::resolve_object
* Fix some things
* Revert "Fix some things"
This reverts commit 2bf8574bc8.
* Revert "Fix apub::api::resolve_object"
This reverts commit 3e4059aabb.
* Revert "Fix apub::activities::block"
This reverts commit 3b02389abd.
* Revert "Fix api_common::utils"
This reverts commit 7dc73de613.
* Revert "Revert "Fix api_common::utils""
This reverts commit f740f115e5.
* Revert "Revert "Fix apub::activities::block""
This reverts commit 2ee206af7c.
* Revert "Revert "Fix apub::api::resolve_object""
This reverts commit 96ed8bf2e9.
* Fix fetch_local_site_data
* Fix get_comment_parent_creator
* Remove unused perma deleted text
* Fix routes::feeds
* Fix lib.rs
* Update lib.rs
* rerun ci
* Attempt to create custom GetConn and RunQueryDsl traits
* Start over
* Add GetConn trait
* aaaa
* Revert "aaaa"
This reverts commit acc9ca1aed.
* Revert "Revert "aaaa""
This reverts commit 443a2a00a5.
* still aaaaaaaaaaaaa
* Return to earlier thing
Revert "Add GetConn trait"
This reverts commit ab4e94aea5.
* Try to use DbPool enum
* Revert "Try to use DbPool enum"
This reverts commit e4d1712646.
* DbConn and DbPool enums (db_schema only fails to compile for tests)
* fmt
* Make functions take `&mut DbPool<'_>` and make db_schema tests compile
* Add try_join_with_pool macro and run fix-clippy on more crates
* Fix some errors
* I did it
* Remove function variants that take connection
* rerun ci
* rerun ci
* rerun ci
* Fix#3366: API does return plain HTML errors
* Fix Clippy errors
* Improve api response times by doing send_activity asynchronously (#3493)
* do send_activity after http response
* move to util function
* format
* fix prometheus
* make synchronous federation configurable
* cargo fmt
* empty
* empty
---------
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
* Updating `login.rs` with generic `incorrect_login` response. (#3549)
* Adding v0.18.1 and v0.18.0 release notes. (#3530)
* Update RELEASES.md (#3556)
added instruction to find the location of your docker directory (especially useful for those who used ansible since they never had to setup docker manually)
* Use async email sender (#3554)
* Upgrade all dependencies (#3526)
* Upgrade all dependencies
* as base64
* Adding phiresky to codeowners. (#3576)
* Error enum fixed (#3487)
* Create error type enum
* Replace magic string slices with LemmyErrorTypes
* Remove unused enum
* Add rename snake case to error enum
* Rename functions
* clippy
* Fix merge errors
* Serialize in PascalCase instead of snake_case
* Revert src/lib
* Add serialization tests
* Update translations
* Fix compilation error in test
* Fix another compilation error
* Add code for generating typescript types
* Various fixes to avoid breaking api
* impl From<LemmyErrorType> for LemmyError
* with_lemmy_type
* trigger ci
---------
Co-authored-by: SleeplessOne1917 <abias1122@gmail.com>
* Only update site_aggregates for local site (#3516)
* Fix#3501 - Fix aggregation counts for elements removed and deleted (#3543)
Two bugs were found and fixed:
- previously elements removal and deletion were counted as two separate disappearances
- removing comments did not affect post aggregations
* Use LemmyErrorType also make error_type compulsory
* Add missing import for jsonify_plain_text_errors
* Fix formatting
* Trying to make woodpecker run again
---------
Co-authored-by: phiresky <phireskyde+git@gmail.com>
Co-authored-by: Dessalines <dessalines@users.noreply.github.com>
Co-authored-by: rosenjcb <rosenjcb@gmail.com>
Co-authored-by: nixoye <12674582+nixoye@users.noreply.github.com>
Co-authored-by: dullbananas <dull.bananas0@gmail.com>
Co-authored-by: Nutomic <me@nutomic.com>
Co-authored-by: SleeplessOne1917 <abias1122@gmail.com>
Co-authored-by: Sander Saarend <sander@saarend.com>
Co-authored-by: Piotr Juszczyk <74842304+pijuszczyk@users.noreply.github.com>
With this change only http(s) schemes are allowed for post.url
field. This is checked for incoming api and federation requests.
Existing posts in database which are sent to clients are not
checked. Neither does it check urls in markdown.
* Fixes#2900 - Checks slur regex to see if it is too permissive along with small validation organization
* Clean up variable names, add handler for valid empty string usecase
* Update tests
* Create validation function and add tests
* Test clean up
* Use payload value vs local site value to prevent stunlocking
* Remove println added while testing
* Fall back to local site regex if not provided from request
* Attempt clean up of flaky comment_view tests
* Pull in latest submodule
* Move application, post check into functions, add more tests and improve test readability
---------
Co-authored-by: Nutomic <me@nutomic.com>
* Add separate Post check for is_valid_body_field
* Modify is_valid_body_check for posts only
* Fix check var reinit in validation.rs
* Extra empty line to rerun woodpecker with changes
* Change Option to bool, add false to non-post calls
* Woodpecker trick.. again
* Probable rust_fmt fail fixed
* cargo_clippy changes
* Missing space between = and if
* Remove ; after body length checks
* Remove `actix_rt` & use standard tokio spawn
* Adjust rust log back down
* Format correctly
* Update cargo lock
* Add DB settings
* Change name and update to latest rev
* Clean up formatting changes
* Move `worker_count` and `worker_retry_count` to settings
* Update defaults
* Use `0.4.4` instead of git branch
