LemmyNet
/
lemmy
3
2
Fork 3
Code Pull Requests 1 Releases Wiki Activity

Adding post and comment ap_id columns. #31

Closed
dessalines wants to merge 0 commits from federation_add_fed_columns into federation
pull from: federation_add_fed_columns
merge into: LemmyNet:federation
Conversation 1 Commits - Files Changed 196 +5966 -21407
196 changed files with 5966 additions and 21407 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.dockerignore vendored
View File

@ -2,7 +2,6 @@
ui/node_modules
server/target
docker/dev/volumes
docker/federation/volumes
docker/federation-test/volumes
.git
ansible

1
.gitignore vendored
View File

@ -6,7 +6,6 @@ ansible/passwords/
# docker build files
docker/lemmy_mine.hjson
docker/dev/env_deploy.sh
docker/federation/volumes
docker/federation-test/volumes
docker/dev/volumes

2
CODE_OF_CONDUCT.md vendored
View File

@ -30,6 +30,6 @@ In the Lemmy community we strive to go the extra step to look out for each other
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could’ve communicated better — remember that it’s your responsibility to make others comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
The enforcement policies listed above apply to all official Lemmy venues; including git repositories under [github.com/LemmyNet/lemmy](https://github.com/LemmyNet/lemmy) and [yerbamate.dev/LemmyNet/lemmy](https://yerbamate.dev/LemmyNet/lemmy), the [Matrix channel](https://matrix.to/#/!BZVTUuEiNmRcbFeLeI:matrix.org?via=matrix.org&via=privacytools.io&via=permaweb.io); and all instances under lemmy.ml. For other projects adopting the Rust Code of Conduct, please contact the maintainers of those projects for enforcement. If you wish to use this code of conduct for your own project, consider explicitly mentioning your moderation policy or making a copy with your own moderation policy so as to avoid confusion.
The enforcement policies listed above apply to all official Lemmy venues; including git repositories under [github.com/dessalines/lemmy](https://github.com/dessalines/lemmy) and [yerbamate.dev/dessalines/lemmy](https://yerbamate.dev/dessalines/lemmy), the [Matrix channel](https://matrix.to/#/!BZVTUuEiNmRcbFeLeI:matrix.org?via=matrix.org&via=privacytools.io&via=permaweb.io); and all instances under lemmy.ml. For other projects adopting the Rust Code of Conduct, please contact the maintainers of those projects for enforcement. If you wish to use this code of conduct for your own project, consider explicitly mentioning your moderation policy or making a copy with your own moderation policy so as to avoid confusion.
Adapted from the [Rust Code of Conduct](https://www.rust-lang.org/policies/code-of-conduct), which is based on the [Node.js Policy on Trolling](http://blog.izs.me/post/30036893703/policy-on-trolling) as well as the [Contributor Covenant v1.3.0](https://www.contributor-covenant.org/version/1/3/0/).

48
README.md vendored
View File

@ -1,12 +1,12 @@
<div align="center">
![GitHub tag (latest SemVer)](https://img.shields.io/github/tag/LemmyNet/lemmy.svg)
[![Build Status](https://travis-ci.org/LemmyNet/lemmy.svg?branch=master)](https://travis-ci.org/LemmyNet/lemmy)
[![GitHub issues](https://img.shields.io/github/issues-raw/LemmyNet/lemmy.svg)](https://github.com/LemmyNet/lemmy/issues)
![GitHub tag (latest SemVer)](https://img.shields.io/github/tag/dessalines/lemmy.svg)
[![Build Status](https://travis-ci.org/dessalines/lemmy.svg?branch=master)](https://travis-ci.org/dessalines/lemmy)
[![GitHub issues](https://img.shields.io/github/issues-raw/dessalines/lemmy.svg)](https://github.com/dessalines/lemmy/issues)
[![Docker Pulls](https://img.shields.io/docker/pulls/dessalines/lemmy.svg)](https://cloud.docker.com/repository/docker/dessalines/lemmy/)
[![Translation status](http://weblate.yerbamate.dev/widgets/lemmy/-/lemmy/svg-badge.svg)](http://weblate.yerbamate.dev/engage/lemmy/)
[![License](https://img.shields.io/github/license/LemmyNet/lemmy.svg)](LICENSE)
![GitHub stars](https://img.shields.io/github/stars/LemmyNet/lemmy?style=social)
[![License](https://img.shields.io/github/license/dessalines/lemmy.svg)](LICENSE)
![GitHub stars](https://img.shields.io/github/stars/dessalines/lemmy?style=social)
</div>
<p align="center">
@ -15,18 +15,18 @@
<h3 align="center"><a href="https://dev.lemmy.ml">Lemmy</a></h3>
<p align="center">
A link aggregator / Reddit clone for the fediverse.
A link aggregator / reddit clone for the fediverse.
<br />
<br />
<a href="https://dev.lemmy.ml">View Site</a>
·
<a href="https://dev.lemmy.ml/docs/index.html">Documentation</a>
·
<a href="https://github.com/LemmyNet/lemmy/issues">Report Bug</a>
<a href="https://github.com/dessalines/lemmy/issues">Report Bug</a>
·
<a href="https://github.com/LemmyNet/lemmy/issues">Request Feature</a>
<a href="https://github.com/dessalines/lemmy/issues">Request Feature</a>
·
<a href="https://github.com/LemmyNet/lemmy/blob/master/RELEASES.md">Releases</a>
<a href="https://github.com/dessalines/lemmy/blob/master/RELEASES.md">Releases</a>
</p>
</p>
@ -34,17 +34,17 @@
Front Page|Post
---|---
![main screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/main_screen.png)|![chat screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/chat_screen.png)
![main screen](https://i.imgur.com/kZSRcRu.png)|![chat screen](https://i.imgur.com/4XghNh6.png)
[Lemmy](https://github.com/LemmyNet/lemmy) is similar to sites like [Reddit](https://reddit.com), [Lobste.rs](https://lobste.rs), [Raddle](https://raddle.me), or [Hacker News](https://news.ycombinator.com/): you subscribe to forums you're interested in, post links and discussions, then vote, and comment on them. Behind the scenes, it is very different; anyone can easily run a server, and all these servers are federated (think email), and connected to the same universe, called the [Fediverse](https://en.wikipedia.org/wiki/Fediverse).
[Lemmy](https://github.com/dessalines/lemmy) is similar to sites like [Reddit](https://reddit.com), [Lobste.rs](https://lobste.rs), [Raddle](https://raddle.me), or [Hacker News](https://news.ycombinator.com/): you subscribe to forums you're interested in, post links and discussions, then vote, and comment on them. Behind the scenes, it is very different; anyone can easily run a server, and all these servers are federated (think email), and connected to the same universe, called the [Fediverse](https://en.wikipedia.org/wiki/Fediverse).
For a link aggregator, this means a user registered on one server can subscribe to forums on any other server, and can have discussions with users registered elsewhere.
The overall goal is to create an easily self-hostable, decentralized alternative to Reddit and other link aggregators, outside of their corporate control and meddling.
The overall goal is to create an easily self-hostable, decentralized alternative to reddit and other link aggregators, outside of their corporate control and meddling.
Each Lemmy server can set its own moderation policy; appointing site-wide admins, and community moderators to keep out the trolls, and foster a healthy, non-toxic environment where all can feel comfortable contributing.
Each lemmy server can set its own moderation policy; appointing site-wide admins, and community moderators to keep out the trolls, and foster a healthy, non-toxic environment where all can feel comfortable contributing.
*Note: Federation is still in active development and the WebSocket, as well as, HTTP API are currently unstable*
*Note: Federation is still in active development*
### Why's it called Lemmy?
@ -70,10 +70,10 @@ Each Lemmy server can set its own moderation policy; appointing site-wide admins
- Only a minimum of a username and password is required to sign up!
- User avatar support.
- Live-updating Comment threads.
- Full vote scores `(+/-)` like old Reddit.
- Full vote scores `(+/-)` like old reddit.
- Themes, including light, dark, and solarized.
- Emojis with autocomplete support. Start typing `:`
- User tagging using `@`, Community tagging using `!`.
- User tagging using `@`, Community tagging using `#`.
- Integrated image uploading in both posts and comments.
- A post can consist of a title and any combination of self text, a URL, or nothing else.
- Notifications, on comment replies and when you're tagged.
@ -108,9 +108,8 @@ Each Lemmy server can set its own moderation policy; appointing site-wide admins
Lemmy is free, open-source software, meaning no advertising, monetizing, or venture capital, ever. Your donations directly support full-time development of the project.
- [Support on Liberapay](https://liberapay.com/Lemmy).
- [Support on Liberapay.](https://liberapay.com/Lemmy)
- [Support on Patreon](https://www.patreon.com/dessalines).
- [Support on OpenCollective](https://opencollective.com/lemmy).
- [List of Sponsors](https://dev.lemmy.ml/sponsors).
### Crypto
@ -125,19 +124,16 @@ Lemmy is free, open-source software, meaning no advertising, monetizing, or vent
- [Docker Development](https://dev.lemmy.ml/docs/contributing_docker_development.html)
- [Local Development](https://dev.lemmy.ml/docs/contributing_local_development.html)
### Translations
### Translations
If you want to help with translating, take a look at [Weblate](https://weblate.yerbamate.dev/projects/lemmy/).
## Contact
- [Mastodon](https://mastodon.social/@LemmyDev)
- [Matrix](https://riot.im/app/#/room/#rust-reddit-fediverse:matrix.org)
## Code Mirrors
- [GitHub](https://github.com/LemmyNet/lemmy)
- [Gitea](https://yerbamate.dev/LemmyNet/lemmy)
- [Mastodon](https://mastodon.social/@LemmyDev) - [![Mastodon Follow](https://img.shields.io/mastodon/follow/810572?domain=https%3A%2F%2Fmastodon.social&style=social)](https://mastodon.social/@LemmyDev)
- [Matrix](https://riot.im/app/#/room/#rust-reddit-fediverse:matrix.org) - [![Matrix](https://img.shields.io/matrix/rust-reddit-fediverse:matrix.org.svg?label=matrix-chat)](https://riot.im/app/#/room/#rust-reddit-fediverse:matrix.org)
- [GitHub](https://github.com/dessalines/lemmy)
- [Gitea](https://yerbamate.dev/dessalines/lemmy)
- [GitLab](https://gitlab.com/dessalines/lemmy)
## Credits

67
RELEASES.md vendored
View File

@ -1,69 +1,6 @@
# Lemmy v0.7.0 Release (2020-06-23)
This release replaces [pictshare](https://github.com/HaschekSolutions/pictshare)
with [pict-rs](https://git.asonix.dog/asonix/pict-rs), which improves performance
and security.
Overall, since our last major release in January (v0.6.0), we have closed over
[100 issues!](https://github.com/LemmyNet/lemmy/milestone/16?closed=1)
- Site-wide list of recent comments
- Reconnecting websockets
- Many more themes, including a default light one.
- Expandable embeds for post links (and thumbnails), from
[iframely](https://github.com/itteco/iframely)
- Better icons
- Emoji autocomplete to post and message bodies, and an Emoji Picker
- Post body now searchable
- Community title and description is now searchable
- Simplified cross-posts
- Better documentation
- LOTS more languages
- Lots of bugs squashed
- And more ...
## Upgrading
Before starting the upgrade, make sure that you have a working backup of your
database and image files. See our
[documentation](https://dev.lemmy.ml/docs/administration_backup_and_restore.html)
for backup instructions.
**With Ansible:**
```
# deploy with ansible from your local lemmy git repo
git pull
cd ansible
ansible-playbook lemmy.yml
# connect via ssh to run the migration script
ssh your-server
cd /lemmy/
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/migrate-pictshare-to-pictrs.bash
chmod +x migrate-pictshare-to-pictrs.bash
sudo ./migrate-pictshare-to-pictrs.bash
```
**With manual Docker installation:**
```
# run these commands on your server
cd /lemmy
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/ansible/templates/nginx.conf
# Replace the {{ vars }}
sudo mv nginx.conf /etc/nginx/sites-enabled/lemmy.conf
sudo nginx -s reload
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/docker-compose.yml
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/migrate-pictshare-to-pictrs.bash
chmod +x migrate-pictshare-to-pictrs.bash
sudo bash migrate-pictshare-to-pictrs.bash
```
**Note:** After upgrading, all users need to reload the page, then logout and
login again, so that images are loaded correctly.
# Lemmy v0.6.0 Release (2020-01-16)
`v0.6.0` is here, and we've closed [41 issues!](https://github.com/LemmyNet/lemmy/milestone/15?closed=1)
`v0.6.0` is here, and we've closed [41 issues!](https://github.com/dessalines/lemmy/milestone/15?closed=1)
This is the biggest release by far:
@ -73,7 +10,7 @@ This is the biggest release by far:
- Can set a custom language.
- Lemmy-wide settings to disable downvotes, and close registration.
- A better documentation system, hosted in lemmy itself.
- [Huge DB performance gains](https://github.com/LemmyNet/lemmy/issues/411) (everthing down to < `30ms`) by using materialized views.
- [Huge DB performance gains](https://github.com/dessalines/lemmy/issues/411) (everthing down to < `30ms`) by using materialized views.
- Fixed major issue with similar post URL and title searching.
- Upgraded to Actix `2.0`
- Faster comment / post voting.

2
ansible/VERSION vendored
View File

@ -1 +1 @@
v0.7.6
v0.6.44

1
ansible/ansible.cfg vendored
View File

@ -1,6 +1,5 @@
[defaults]
inventory=inventory
interpreter_python=/usr/bin/python3
[ssh_connection]
pipelining = True

8
ansible/lemmy.yml vendored
View File

@ -24,11 +24,10 @@
creates: '/etc/letsencrypt/live/{{domain}}/privkey.pem'
- name: create lemmy folder
file: path={{item.path}} {{item.owner}} state=directory
file: path={{item.path}} state=directory
with_items:
- { path: '/lemmy/', owner: 'root' }
- { path: '/lemmy/volumes/', owner: 'root' }
- { path: '/lemmy/volumes/pictrs/', owner: '991' }
- { path: '/lemmy/' }
- { path: '/lemmy/volumes/' }
- block:
- name: add template files
@ -60,7 +59,6 @@
project_src: /lemmy/
state: present
pull: yes
remove_orphans: yes
- name: reload nginx with new config
shell: nginx -s reload

8
ansible/lemmy_dev.yml vendored
View File

@ -26,11 +26,10 @@
creates: '/etc/letsencrypt/live/{{domain}}/privkey.pem'
- name: create lemmy folder
file: path={{item.path}} owner={{item.owner}} state=directory
file: path={{item.path}} state=directory
with_items:
- { path: '/lemmy/', owner: 'root' }
- { path: '/lemmy/volumes/', owner: 'root' }
- { path: '/lemmy/volumes/pictrs/', owner: '991' }
- { path: '/lemmy/' }
- { path: '/lemmy/volumes/' }
- block:
- name: add template files
@ -89,7 +88,6 @@
project_src: /lemmy/
state: present
recreate: always
remove_orphans: yes
ignore_errors: yes
- name: reload nginx with new config

12
ansible/templates/config.hjson vendored
View File

@ -1,25 +1,13 @@
{
# for more info about the config, check out the documentation
# https://dev.lemmy.ml/docs/administration_configuration.html
# settings related to the postgresql database
database: {
# password to connect to postgres
password: "{{ postgres_password }}"
# host where postgres is running
host: "postgres"
}
# the domain name of your instance (eg "dev.lemmy.ml")
hostname: "{{ domain }}"
# json web token for authorization between server and client
jwt_secret: "{{ jwt_password }}"
# The location of the frontend
front_end_dir: "/app/dist"
# email sending configuration
email: {
# hostname of the smtp server
smtp_server: "postfix:25"
# address to send emails from, eg "noreply@your-instance.com"
smtp_from_address: "noreply@{{ domain }}"
use_tls: false
}

13
ansible/templates/docker-compose.yml vendored
View File

@ -12,7 +12,7 @@ services:
- ./lemmy.hjson:/config/config.hjson:ro
depends_on:
- postgres
- pictrs
- pictshare
- iframely
postgres:
@ -25,17 +25,16 @@ services:
- ./volumes/postgres:/var/lib/postgresql/data
restart: always
pictrs:
image: asonix/pictrs:amd64-v0.1.0-r9
user: 991:991
pictshare:
image: shtripok/pictshare:latest
ports:
- "127.0.0.1:8537:8080"
- "127.0.0.1:8537:80"
volumes:
- ./volumes/pictrs:/mnt
- ./volumes/pictshare:/usr/share/nginx/html/data
restart: always
iframely:
image: jolt/iframely:v1.4.3
image: dogbin/iframely:latest
ports:
- "127.0.0.1:8061:80"
volumes:

32
ansible/templates/nginx.conf vendored
View File

@ -1,5 +1,4 @@
proxy_cache_path /var/cache/lemmy_frontend levels=1:2 keys_zone=lemmy_frontend_cache:10m max_size=100m use_temp_path=off;
limit_req_zone $binary_remote_addr zone=lemmy_ratelimit:10m rate=1r/s;
server {
listen 80;
@ -37,7 +36,7 @@ server {
# It might be nice to compress JSON, but leaving that out to protect against potential
# compression+encryption information leak attacks like BREACH.
gzip on;
gzip_types text/css application/javascript image/svg+xml;
gzip_types text/css application/javascript;
gzip_vary on;
# Only connect to this site via HTTPS for the two years
@ -49,11 +48,8 @@ server {
add_header X-Frame-Options "DENY";
add_header X-XSS-Protection "1; mode=block";
# Upload limit for pictrs
client_max_body_size 20M;
# Rate limit
limit_req zone=lemmy_ratelimit burst=30 nodelay;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://0.0.0.0:8536;
@ -74,21 +70,15 @@ server {
proxy_cache_min_uses 5;
}
# Redirect pictshare images to pictrs
location ~ /pictshare/(.*)$ {
return 301 /pictrs/image/$1;
}
location /pictshare/ {
proxy_pass http://0.0.0.0:8537/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# pict-rs images
location /pictrs {
location /pictrs/image {
proxy_pass http://0.0.0.0:8537/image;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Block the import
return 403;
if ($request_uri ~ \.(?:ico|gif|jpe?g|png|webp|bmp|mp4)$) {
add_header Cache-Control "public, max-age=31536000, immutable";
}
}
location /iframely/ {

15
docker/dev/Dockerfile vendored
View File

@ -21,13 +21,17 @@ COPY server/Cargo.toml server/Cargo.lock ./
RUN sudo chown -R rust:rust .
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
RUN cargo build
RUN cargo build --release
RUN rm -f ./target/x86_64-unknown-linux-musl/release/deps/lemmy_server*
COPY server/src ./src/
COPY server/migrations ./migrations/
# Build for debug
RUN cargo build
# Build for release
RUN cargo build --frozen --release
# Get diesel-cli on there just in case
# RUN cargo install diesel_cli --no-default-features --features postgres
FROM ekidd/rust-musl-builder:1.42.0-openssl11 as docs
WORKDIR /app
@ -35,14 +39,15 @@ COPY docs ./docs
RUN sudo chown -R rust:rust .
RUN mdbook build docs/
FROM alpine:3.12
FROM alpine:3.10
# Install libpq for postgres
RUN apk add libpq
# Copy resources
COPY server/config/defaults.hjson /config/defaults.hjson
COPY --from=rust /app/server/target/x86_64-unknown-linux-musl/debug/lemmy_server /app/lemmy
COPY --from=rust /app/server/target/x86_64-unknown-linux-musl/release/lemmy_server /app/lemmy
COPY --from=docs /app/docs/book/ /app/dist/documentation/
COPY --from=node /app/ui/dist /app/dist

79
docker/dev/Dockerfile.aarch64 vendored Normal file
View File

@ -0,0 +1,79 @@
FROM node:10-jessie as node
WORKDIR /app/ui
# Cache deps
COPY ui/package.json ui/yarn.lock ./
RUN yarn install --pure-lockfile
# Build
COPY ui /app/ui
RUN yarn build
# contains qemu-*-static for cross-compilation
FROM multiarch/qemu-user-static as qemu
FROM arm64v8/rust:1.40-buster as rust
COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
#COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
# Install musl
#RUN apt-get update && apt-get install -y mc
#RUN apt-get install -y musl-tools mc
#libpq-dev mc
#RUN rustup target add ${TARGET}
# Cache deps
WORKDIR /app
RUN USER=root cargo new server
WORKDIR /app/server
COPY server/Cargo.toml server/Cargo.lock ./
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
RUN cargo build --release
# RUN cargo build
COPY server/src ./src/
COPY server/migrations ./migrations/
RUN rm -f ./target/release/deps/lemmy_server* ; rm -f ./target/debug/deps/lemmy_server*
# build for release
RUN cargo build --frozen --release
# RUN cargo build --frozen
# Get diesel-cli on there just in case
# RUN cargo install diesel_cli --no-default-features --features postgres
# RUN cp /app/server/target/debug/lemmy_server /app/server/ready
RUN cp /app/server/target/release/lemmy_server /app/server/ready
#FROM alpine:3.10
# debian because build with dynamic linking with debian:buster
FROM arm64v8/debian:buster-slim as lemmy
#COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
# Install libpq for postgres
#RUN apk add libpq
RUN apt-get update && apt-get install -y libpq5
RUN addgroup --gid 1000 lemmy
# for alpine
#RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
# for debian
RUN adduser --disabled-password --shell /bin/sh --uid 1000 --ingroup lemmy lemmy
# Copy resources
COPY server/config/defaults.hjson /config/defaults.hjson
COPY --from=rust /app/server/ready /app/lemmy
COPY --from=node /app/ui/dist /app/dist
RUN chown lemmy:lemmy /app/lemmy
USER lemmy
EXPOSE 8536
CMD ["/app/lemmy"]

79
docker/dev/Dockerfile.armv7hf vendored Normal file
View File

@ -0,0 +1,79 @@
FROM node:10-jessie as node
WORKDIR /app/ui
# Cache deps
COPY ui/package.json ui/yarn.lock ./
RUN yarn install --pure-lockfile
# Build
COPY ui /app/ui
RUN yarn build
# contains qemu-*-static for cross-compilation
FROM multiarch/qemu-user-static as qemu
FROM arm32v7/rust:1.37-buster as rust
#COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
# Install musl
#RUN apt-get update && apt-get install -y mc
#RUN apt-get install -y musl-tools mc
#libpq-dev mc
#RUN rustup target add ${TARGET}
# Cache deps
WORKDIR /app
RUN USER=root cargo new server
WORKDIR /app/server
COPY server/Cargo.toml server/Cargo.lock ./
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
#RUN cargo build --release
# RUN cargo build
RUN RUSTFLAGS='-Ccodegen-units=1' cargo build
COPY server/src ./src/
COPY server/migrations ./migrations/
RUN rm -f ./target/release/deps/lemmy_server* ; rm -f ./target/debug/deps/lemmy_server*
# build for release
#RUN cargo build --frozen --release
RUN cargo build --frozen
# Get diesel-cli on there just in case
# RUN cargo install diesel_cli --no-default-features --features postgres
RUN cp /app/server/target/debug/lemmy_server /app/server/ready
#RUN cp /app/server/target/release/lemmy_server /app/server/ready
#FROM alpine:3.10
# debian because build with dynamic linking with debian:buster
FROM arm32v7/debian:buster-slim as lemmy
COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
# Install libpq for postgres
#RUN apk add libpq
RUN apt-get update && apt-get install -y libpq5
RUN addgroup --gid 1000 lemmy
# for alpine
#RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
# for debian
RUN adduser --disabled-password --shell /bin/sh --uid 1000 --ingroup lemmy lemmy
# Copy resources
COPY server/config/defaults.hjson /config/defaults.hjson
COPY --from=rust /app/server/ready /app/lemmy
COPY --from=node /app/ui/dist /app/dist
RUN chown lemmy:lemmy /app/lemmy
USER lemmy
EXPOSE 8536
CMD ["/app/lemmy"]

88
docker/dev/Dockerfile.libc vendored Normal file
View File

@ -0,0 +1,88 @@
# can be build on x64, arm32, arm64 platforms
# to build on target platform run
# docker build -f Dockerfile.libc -t dessalines/lemmy:version ../..
#
# to use docker buildx run
# docker buildx build --platform linux/amd64,linux/arm64 -f Dockerfile.libc -t YOURNAME/lemmy --push ../..
FROM node:12-buster as node
# use this if use docker buildx
#FROM --platform=$BUILDPLATFORM node:12-buster as node
WORKDIR /app/ui
# Cache deps
COPY ui/package.json ui/yarn.lock ./
RUN yarn install --pure-lockfile --network-timeout 100000
# Build
COPY ui /app/ui
RUN yarn build
FROM rust:1.42 as rust
# Cache deps
WORKDIR /app
RUN USER=root cargo new server
WORKDIR /app/server
COPY server/Cargo.toml server/Cargo.lock ./
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
RUN cargo build --release
#RUN cargo build && \
# rm -f ./target/release/deps/lemmy_server* ; rm -f ./target/debug/deps/lemmy_server*
COPY server/src ./src/
COPY server/migrations ./migrations/
# build for release
# workaround for https://github.com/rust-lang/rust/issues/62896
#RUN RUSTFLAGS='-Ccodegen-units=1' cargo build --release
RUN cargo build --release --frozen
#RUN cargo build --frozen
# Get diesel-cli on there just in case
# RUN cargo install diesel_cli --no-default-features --features postgres
# make result place always the same for lemmy container
RUN cp /app/server/target/release/lemmy_server /app/server/ready
#RUN cp /app/server/target/debug/lemmy_server /app/server/ready
FROM rust:1.42 as docs
WORKDIR /app
# Build docs
COPY docs ./docs
RUN cargo install mdbook
RUN mdbook build docs/
#FROM alpine:3.10
# debian because build with dynamic linking with debian:buster
FROM debian:buster as lemmy
# Install libpq for postgres
#RUN apk add libpq
RUN apt-get update && apt-get install -y libpq5
RUN addgroup --gid 1000 lemmy
# for alpine
#RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
# for debian
RUN adduser --disabled-password --shell /bin/sh --uid 1000 --ingroup lemmy lemmy
# Copy resources
COPY server/config/defaults.hjson /config/defaults.hjson
COPY --from=node /app/ui/dist /app/dist
COPY --from=docs /app/docs/book/ /app/dist/documentation/
COPY --from=rust /app/server/ready /app/lemmy
RUN chown lemmy:lemmy /app/lemmy
USER lemmy
EXPOSE 8536
CMD ["/app/lemmy"]

76
docker/dev/deploy.sh vendored Executable file
View File

@ -0,0 +1,76 @@
#!/bin/sh
git checkout master
# Import translations
git fetch weblate
git merge weblate/master
# Creating the new tag
new_tag="$1"
third_semver=$(echo $new_tag | cut -d "." -f 3)
# Setting the version on the front end
cd ../../
echo "export const version: string = '$new_tag';" > "ui/src/version.ts"
git add "ui/src/version.ts"
# Setting the version on the backend
echo "pub const VERSION: &str = \"$new_tag\";" > "server/src/version.rs"
git add "server/src/version.rs"
# Setting the version for Ansible
echo $new_tag > "ansible/VERSION"
git add "ansible/VERSION"
cd docker/dev || exit
# Changing the docker-compose prod
sed -i "s/dessalines\/lemmy:.*/dessalines\/lemmy:$new_tag/" ../prod/docker-compose.yml
sed -i "s/dessalines\/lemmy:.*/dessalines\/lemmy:$new_tag/" ../../ansible/templates/docker-compose.yml
git add ../prod/docker-compose.yml
git add ../../ansible/templates/docker-compose.yml
# The commit
git commit -m"Version $new_tag"
git tag $new_tag
# Rebuilding docker
docker-compose build
docker tag dev_lemmy:latest dessalines/lemmy:x64-$new_tag
docker push dessalines/lemmy:x64-$new_tag
# Build for Raspberry Pi / other archs
# Arm currently not working
# docker build -t lemmy:armv7hf -f Dockerfile.armv7hf ../../
# docker tag lemmy:armv7hf dessalines/lemmy:armv7hf-$new_tag
# docker push dessalines/lemmy:armv7hf-$new_tag
# aarch64
# Only do this on major releases (IE the third semver is 0)
if [ $third_semver -eq 0 ]; then
# Registering qemu binaries
docker run --rm --privileged multiarch/qemu-user-static:register --reset
docker build -t lemmy:aarch64 -f Dockerfile.aarch64 ../../
docker tag lemmy:aarch64 dessalines/lemmy:arm64-$new_tag
docker push dessalines/lemmy:arm64-$new_tag
fi
# Creating the manifest for the multi-arch build
if [ $third_semver -eq 0 ]; then
docker manifest create dessalines/lemmy:$new_tag \
dessalines/lemmy:x64-$new_tag \
dessalines/lemmy:arm64-$new_tag
else
docker manifest create dessalines/lemmy:$new_tag \
dessalines/lemmy:x64-$new_tag
fi
docker manifest push dessalines/lemmy:$new_tag
# Push
git push origin $new_tag
git push
# Pushing to any ansible deploys
cd ../../ansible || exit
ansible-playbook lemmy.yml --become

12
docker/dev/dev_deploy.sh vendored Executable file
View File

@ -0,0 +1,12 @@
#!/bin/sh
# Building from the dev branch for dev servers
git checkout dev
# Rebuilding dev docker
docker-compose build
docker tag dev_lemmy:latest dessalines/lemmy:dev
docker push dessalines/lemmy:dev
# SSH and pull it
ssh $LEMMY_USER@$LEMMY_HOST "cd ~/git/lemmy/docker/dev && docker pull dessalines/lemmy:dev && docker-compose up -d"

36
docker/dev/docker-compose.yml vendored
View File

@ -1,6 +1,15 @@
version: '3.3'
services:
postgres:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres:/var/lib/postgresql/data
restart: always
lemmy:
build:
@ -12,35 +21,22 @@ services:
environment:
- RUST_LOG=debug
volumes:
- ../lemmy.hjson:/config/config.hjson
- ../lemmy.hjson:/config/config.hjson:ro
depends_on:
- pictrs
- postgres
- pictshare
- iframely
postgres:
image: postgres:12-alpine
pictshare:
image: shtripok/pictshare:latest
ports:
- "127.0.0.1:5432:5432"
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
- "127.0.0.1:8537:80"
volumes:
- ./volumes/postgres:/var/lib/postgresql/data
restart: always
pictrs:
image: asonix/pictrs:v0.1.13-r0
ports:
- "127.0.0.1:8537:8080"
user: 991:991
volumes:
- ./volumes/pictrs:/mnt
- ./volumes/pictshare:/usr/share/nginx/html/data
restart: always
iframely:
image: jolt/iframely:v1.4.3
image: dogbin/iframely:latest
ports:
- "127.0.0.1:8061:80"
volumes:

4
docker/dev/docker_update.sh vendored
View File

@ -1,6 +1,2 @@
#!/bin/sh
set -e
export COMPOSE_DOCKER_CLI_BUILD=1
export DOCKER_BUILDKIT=1
docker-compose up -d --no-deps --build

18
docker/dev/test_deploy.sh vendored
View File

@ -1,18 +0,0 @@
#!/bin/bash
set -e
BRANCH=$1
git checkout $BRANCH
export COMPOSE_DOCKER_CLI_BUILD=1
export DOCKER_BUILDKIT=1
# Rebuilding dev docker
sudo docker build ../../ -f . -t "dessalines/lemmy:$BRANCH"
sudo docker push "dessalines/lemmy:$BRANCH"
# Run the playbook
pushd ../../../lemmy-ansible
ansible-playbook -i test playbooks/site.yml
popd

4
docker/federation/Dockerfile → docker/federation-test/Dockerfile vendored
View File

@ -1,9 +1,9 @@
FROM ekidd/rust-musl-builder:1.42.0-openssl11
FROM ekidd/rust-musl-builder:1.38.0-openssl11
USER root
RUN mkdir /app/dist/documentation/ -p \
&& addgroup --gid 1001 lemmy \
&& adduser --gecos "" --disabled-password --shell /bin/sh -u 1001 --ingroup lemmy lemmy
&& adduser --disabled-password --shell /bin/sh -u 1001 --ingroup lemmy lemmy
# Copy resources
COPY server/config/defaults.hjson /app/config/defaults.hjson

83
docker/federation-test/docker-compose.yml vendored Normal file
View File

@ -0,0 +1,83 @@
version: '3.3'
services:
nginx:
image: nginx:1.17-alpine
ports:
- "8540:8540"
- "8550:8550"
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
depends_on:
- lemmy_alpha
- pictshare_alpha
- lemmy_beta
- pictshare_beta
- iframely
restart: "always"
lemmy_alpha:
image: lemmy-federation-test:latest
environment:
- LEMMY_HOSTNAME=lemmy_alpha:8540
- LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_alpha:5432/lemmy
- LEMMY_JWT_SECRET=changeme
- LEMMY_FRONT_END_DIR=/app/dist
- LEMMY_FEDERATION__ENABLED=true
- LEMMY_FEDERATION__FOLLOWED_INSTANCES=lemmy_beta:8550
- LEMMY_FEDERATION__TLS_ENABLED=false
- LEMMY_PORT=8540
- RUST_BACKTRACE=1
restart: always
depends_on:
- postgres_alpha
postgres_alpha:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres_alpha:/var/lib/postgresql/data
restart: always
pictshare_alpha:
image: shtripok/pictshare:latest
volumes:
- ./volumes/pictshare_alpha:/usr/share/nginx/html/data
restart: always
lemmy_beta:
image: lemmy-federation-test:latest
environment:
- LEMMY_HOSTNAME=lemmy_beta:8550
- LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_beta:5432/lemmy
- LEMMY_JWT_SECRET=changeme
- LEMMY_FRONT_END_DIR=/app/dist
- LEMMY_FEDERATION__ENABLED=true
- LEMMY_FEDERATION__FOLLOWED_INSTANCES=lemmy_alpha:8540
- LEMMY_FEDERATION__TLS_ENABLED=false
- LEMMY_PORT=8550
- RUST_BACKTRACE=1
restart: always
depends_on:
- postgres_beta
postgres_beta:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres_beta:/var/lib/postgresql/data
restart: always
pictshare_beta:
image: shtripok/pictshare:latest
volumes:
- ./volumes/pictshare_beta:/usr/share/nginx/html/data
restart: always
iframely:
image: dogbin/iframely:latest
volumes:
- ../iframely.config.local.js:/iframely/config.local.js:ro
restart: always

73
docker/federation-test/nginx.conf vendored Normal file
View File

@ -0,0 +1,73 @@
events {
worker_connections 1024;
}
http {
server {
listen 8540;
server_name 127.0.0.1;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://lemmy_alpha:8540;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
location /pictshare/ {
proxy_pass http://pictshare_alpha:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /iframely/ {
proxy_pass http://iframely:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 8550;
server_name 127.0.0.1;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://lemmy_beta:8550;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
location /pictshare/ {
proxy_pass http://pictshare_beta:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /iframely/ {
proxy_pass http://iframely:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}

14
docker/federation-test/run-federation-test.bash vendored Executable file
View File

@ -0,0 +1,14 @@
#!/bin/bash
set -e
pushd ../../ui/ || exit
yarn build
popd || exit
pushd ../../server/ || exit
cargo build
popd || exit
sudo docker build ../../ -f Dockerfile -t lemmy-federation-test:latest
sudo docker-compose up

32
docker/federation-test/run-tests.sh vendored
View File

@ -1,32 +0,0 @@
#!/bin/bash
set -e
pushd ../../server/
cargo build
popd
pushd ../../ui
yarn
popd
mkdir -p volumes/pictrs_{alpha,beta,gamma}
sudo chown -R 991:991 volumes/pictrs_{alpha,beta,gamma}
sudo docker build ../../ --file ../federation/Dockerfile --tag lemmy-federation:latest
sudo mkdir -p volumes/pictrs_alpha
sudo chown -R 991:991 volumes/pictrs_alpha
sudo docker-compose --file ../federation/docker-compose.yml --project-directory . up -d
pushd ../../ui
echo "Waiting for Lemmy to start..."
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8540/api/v1/site')" != "200" ]]; do sleep 1; done
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8550/api/v1/site')" != "200" ]]; do sleep 1; done
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8560/api/v1/site')" != "200" ]]; do sleep 1; done
yarn api-test || true
popd
sudo docker-compose --file ../federation/docker-compose.yml --project-directory . down
sudo rm -r volumes/

19
docker/federation-test/servers.sh vendored
View File

@ -1,19 +0,0 @@
#!/bin/bash
set -e
sudo rm -rf volumes
pushd ../../server/
cargo build
popd
pushd ../../ui
yarn
popd
mkdir -p volumes/pictrs_{alpha,beta,gamma}
sudo chown -R 991:991 volumes/pictrs_{alpha,beta,gamma}
sudo docker build ../../ --file ../federation/Dockerfile --tag lemmy-federation:latest
sudo docker-compose --file ../federation/docker-compose.yml --project-directory . up

10
docker/federation-test/tests.sh vendored
View File

@ -1,10 +0,0 @@
#!/bin/bash
set -xe
pushd ../../ui
echo "Waiting for Lemmy to start..."
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8540/api/v1/site')" != "200" ]]; do sleep 1; done
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8550/api/v1/site')" != "200" ]]; do sleep 1; done
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8560/api/v1/site')" != "200" ]]; do sleep 1; done
yarn api-test || true
popd

112
docker/federation/docker-compose.yml vendored
View File

@ -1,112 +0,0 @@
version: '3.3'
services:
nginx:
image: nginx:1.17-alpine
ports:
- "8540:8540"
- "8550:8550"
- "8560:8560"
volumes:
# Hack to make this work from both docker/federation/ and docker/federation-test/
- ../federation/nginx.conf:/etc/nginx/nginx.conf
restart: on-failure
depends_on:
- lemmy-alpha
- pictrs
- lemmy-beta
- lemmy-gamma
- iframely
pictrs:
restart: always
image: asonix/pictrs:v0.1.13-r0
user: 991:991
volumes:
- ./volumes/pictrs_alpha:/mnt
lemmy-alpha:
image: lemmy-federation:latest
environment:
- LEMMY_HOSTNAME=lemmy-alpha:8540
- LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_alpha:5432/lemmy
- LEMMY_JWT_SECRET=changeme
- LEMMY_FRONT_END_DIR=/app/dist
- LEMMY_FEDERATION__ENABLED=true
- LEMMY_FEDERATION__TLS_ENABLED=false
- LEMMY_FEDERATION__ALLOWED_INSTANCES=lemmy-beta,lemmy-gamma
- LEMMY_PORT=8540
- LEMMY_SETUP__ADMIN_USERNAME=lemmy_alpha
- LEMMY_SETUP__ADMIN_PASSWORD=lemmy
- LEMMY_SETUP__SITE_NAME=lemmy-alpha
- RUST_BACKTRACE=1
- RUST_LOG=debug
depends_on:
- postgres_alpha
postgres_alpha:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres_alpha:/var/lib/postgresql/data
lemmy-beta:
image: lemmy-federation:latest
environment:
- LEMMY_HOSTNAME=lemmy-beta:8550
- LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_beta:5432/lemmy
- LEMMY_JWT_SECRET=changeme
- LEMMY_FRONT_END_DIR=/app/dist
- LEMMY_FEDERATION__ENABLED=true
- LEMMY_FEDERATION__TLS_ENABLED=false
- LEMMY_FEDERATION__ALLOWED_INSTANCES=lemmy-alpha,lemmy-gamma
- LEMMY_PORT=8550
- LEMMY_SETUP__ADMIN_USERNAME=lemmy_beta
- LEMMY_SETUP__ADMIN_PASSWORD=lemmy
- LEMMY_SETUP__SITE_NAME=lemmy-beta
- RUST_BACKTRACE=1
- RUST_LOG=debug
depends_on:
- postgres_beta
postgres_beta:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres_beta:/var/lib/postgresql/data
lemmy-gamma:
image: lemmy-federation:latest
environment:
- LEMMY_HOSTNAME=lemmy-gamma:8560
- LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_gamma:5432/lemmy
- LEMMY_JWT_SECRET=changeme
- LEMMY_FRONT_END_DIR=/app/dist
- LEMMY_FEDERATION__ENABLED=true
- LEMMY_FEDERATION__TLS_ENABLED=false
- LEMMY_FEDERATION__ALLOWED_INSTANCES=lemmy-alpha,lemmy-beta
- LEMMY_PORT=8560
- LEMMY_SETUP__ADMIN_USERNAME=lemmy_gamma
- LEMMY_SETUP__ADMIN_PASSWORD=lemmy
- LEMMY_SETUP__SITE_NAME=lemmy-gamma
- RUST_BACKTRACE=1
- RUST_LOG=debug
depends_on:
- postgres_gamma
postgres_gamma:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres_gamma:/var/lib/postgresql/data
iframely:
image: jolt/iframely:v1.4.3
volumes:
- ../iframely.config.local.js:/iframely/config.local.js:ro

125
docker/federation/nginx.conf vendored
View File

@ -1,125 +0,0 @@
events {
worker_connections 1024;
}
http {
server {
listen 8540;
server_name 127.0.0.1;
access_log off;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://lemmy-alpha:8540;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# pict-rs images
location /pictrs {
location /pictrs/image {
proxy_pass http://pictrs:8080/image;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Block the import
return 403;
}
location /iframely/ {
proxy_pass http://iframely:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 8550;
server_name 127.0.0.1;
access_log off;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://lemmy-beta:8550;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# pict-rs images
location /pictrs {
location /pictrs/image {
proxy_pass http://pictrs:8080/image;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Block the import
return 403;
}
location /iframely/ {
proxy_pass http://iframely:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 8560;
server_name 127.0.0.1;
access_log off;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://lemmy-gamma:8560;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# pict-rs images
location /pictrs {
location /pictrs/image {
proxy_pass http://pictrs:8080/image;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Block the import
return 403;
}
location /iframely/ {
proxy_pass http://iframely:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}

28
docker/federation/run-federation-test.bash vendored
View File

@ -1,28 +0,0 @@
#!/bin/bash
set -e
# already start rust build in the background
pushd ../../server/ || exit
cargo build &
popd || exit
if [ "$1" = "-yarn" ]; then
pushd ../../ui/ || exit
yarn
yarn build
popd || exit
fi
# wait for rust build to finish
pushd ../../server/ || exit
cargo build
popd || exit
sudo docker build ../../ --file Dockerfile -t lemmy-federation:latest
for Item in alpha beta gamma ; do
sudo mkdir -p volumes/pictrs_$Item
sudo chown -R 991:991 volumes/pictrs_$Item
done
sudo docker-compose up

59
docker/lemmy.hjson vendored
View File

@ -1,7 +1,18 @@
{
# for more info about the config, check out the documentation
# https://dev.lemmy.ml/docs/administration_configuration.html
database: {
# username to connect to postgres
user: "lemmy"
# password to connect to postgres
password: "password"
# host where postgres is running
host: "postgres"
# port where postgres can be accessed
port: 5432
# name of the postgres database for lemmy
database: "lemmy"
# maximum number of active sql connections
pool_size: 5
}
# the domain name of your instance (eg "dev.lemmy.ml")
hostname: "my_domain"
# address where lemmy should listen for incoming requests
@ -10,19 +21,35 @@
port: 8536
# json web token for authorization between server and client
jwt_secret: "changeme"
# settings related to the postgresql database
database: {
# name of the postgres database for lemmy
database: "lemmy"
# username to connect to postgres
user: "lemmy"
# password to connect to postgres
password: "password"
# host where postgres is running
host: "postgres"
}
# The location of the frontend
# The dir for the front end
front_end_dir: "/app/dist"
# whether to enable activitypub federation. this feature is in alpha, do not enable in production, as might
# cause problems like remote instances fetching and permanently storing bad data.
federation_enabled: false
# rate limits for various user actions, by user ip
rate_limit: {
# maximum number of messages created in interval
message: 180
# interval length for message limit
message_per_second: 60
# maximum number of posts created in interval
post: 6
# interval length for post limit
post_per_second: 600
# maximum number of registrations in interval
register: 3
# interval length for registration limit
register_per_second: 3600
}
# # optional: parameters for automatic configuration of new instance (only used at first start)
# setup: {
# # username for the admin user
# admin_username: "lemmy"
# # password for the admin user
# admin_password: "lemmy"
# # name of the site (can be changed later)
# site_name: "Lemmy Test"
# }
# # optional: email sending configuration
# email: {
# # hostname of the smtp server
@ -31,7 +58,7 @@
# smtp_login: ""
# # password to login to the smtp server
# smtp_password: ""
# # address to send emails from, eg "noreply@your-instance.com"
# # address to send emails from, eg "info@your-instance.com"
# smtp_from_address: ""
# }
}

64
docker/prod/Dockerfile vendored
View File

@ -1,64 +0,0 @@
ARG RUST_BUILDER_IMAGE=shtripok/rust-musl-builder:arm
FROM $RUST_BUILDER_IMAGE as rust
#ARG RUSTRELEASEDIR="debug"
ARG RUSTRELEASEDIR="release"
# Cache deps
WORKDIR /app
RUN sudo chown -R rust:rust .
RUN USER=root cargo new server
WORKDIR /app/server
COPY --chown=rust:rust server/Cargo.toml server/Cargo.lock ./
#RUN sudo chown -R rust:rust .
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
RUN cargo build --release
RUN rm -f ./target/$CARGO_BUILD_TARGET/$RUSTRELEASEDIR/deps/lemmy_server*
COPY --chown=rust:rust server/src ./src/
COPY --chown=rust:rust server/migrations ./migrations/
# build for release
# workaround for https://github.com/rust-lang/rust/issues/62896
RUN cargo build --frozen --release
# reduce binary size
RUN strip ./target/$CARGO_BUILD_TARGET/$RUSTRELEASEDIR/lemmy_server
RUN cp ./target/$CARGO_BUILD_TARGET/$RUSTRELEASEDIR/lemmy_server /app/server/
FROM $RUST_BUILDER_IMAGE as docs
WORKDIR /app
COPY --chown=rust:rust docs ./docs
RUN mdbook build docs/
FROM node:12-buster as node
WORKDIR /app/ui
# Cache deps
COPY ui/package.json ui/yarn.lock ./
RUN yarn install --pure-lockfile --network-timeout 600000
# Build
COPY ui /app/ui
RUN yarn build
FROM alpine:3.12 as lemmy
# Install libpq for postgres
RUN apk add libpq
RUN addgroup -g 1000 lemmy
RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
# Copy resources
COPY --chown=lemmy:lemmy server/config/defaults.hjson /config/defaults.hjson
COPY --chown=lemmy:lemmy --from=rust /app/server/lemmy_server /app/lemmy
COPY --chown=lemmy:lemmy --from=docs /app/docs/book/ /app/dist/documentation/
COPY --chown=lemmy:lemmy --from=node /app/ui/dist /app/dist
RUN chown lemmy:lemmy /app/lemmy
USER lemmy
EXPOSE 8536
CMD ["/app/lemmy"]

60
docker/prod/deploy.sh vendored
View File

@ -1,60 +0,0 @@
#!/bin/sh
set -e
git checkout master
# Import translations
git fetch weblate
git merge weblate/master
# Creating the new tag
new_tag="$1"
third_semver=$(echo $new_tag | cut -d "." -f 3)
# Setting the version on the front end
cd ../../
echo "export const version: string = '$new_tag';" > "ui/src/version.ts"
git add "ui/src/version.ts"
# Setting the version on the backend
echo "pub const VERSION: &str = \"$new_tag\";" > "server/src/version.rs"
git add "server/src/version.rs"
# Setting the version for Ansible
echo $new_tag > "ansible/VERSION"
git add "ansible/VERSION"
cd docker/prod || exit
# Changing the docker-compose prod
sed -i "s/dessalines\/lemmy:.*/dessalines\/lemmy:$new_tag/" ../prod/docker-compose.yml
sed -i "s/dessalines\/lemmy:.*/dessalines\/lemmy:$new_tag/" ../../ansible/templates/docker-compose.yml
git add ../prod/docker-compose.yml
git add ../../ansible/templates/docker-compose.yml
# The commit
git commit -m"Version $new_tag"
git tag $new_tag
export COMPOSE_DOCKER_CLI_BUILD=1
export DOCKER_BUILDKIT=1
# Rebuilding docker
if [ $third_semver -eq 0 ]; then
# TODO get linux/arm/v7 build working
# Build for Raspberry Pi / other archs too
docker buildx build --platform linux/amd64,linux/arm64 ../../ \
--file Dockerfile \
--tag dessalines/lemmy:$new_tag \
--push
else
docker buildx build --platform linux/amd64 ../../ \
--file Dockerfile \
--tag dessalines/lemmy:$new_tag \
--push
fi
# Push
git push origin $new_tag
git push
# Pushing to any ansible deploys
cd ../../../lemmy-ansible || exit
ansible-playbook -i prod playbooks/site.yml --vault-password-file vault_pass

22
docker/prod/docker-compose.yml vendored
View File

@ -1,4 +1,4 @@
version: '2.2'
version: '3.3'
services:
postgres:
@ -12,33 +12,31 @@ services:
restart: always
lemmy:
image: dessalines/lemmy:v0.7.6
image: dessalines/lemmy:v0.6.44
ports:
- "127.0.0.1:8536:8536"
restart: always
environment:
- RUST_LOG=error
volumes:
- ./lemmy.hjson:/config/config.hjson
- ./lemmy.hjson:/config/config.hjson:ro
depends_on:
- postgres
- pictrs
- pictshare
- iframely
pictrs:
image: asonix/pictrs:v0.1.13-r0
ports:
- "127.0.0.1:8537:8080"
user: 991:991
pictshare:
image: shtripok/pictshare:latest
ports:
- "127.0.0.1:8537:80"
volumes:
- ./volumes/pictrs:/mnt
- ./volumes/pictshare:/usr/share/nginx/html/data
restart: always
iframely:
image: jolt/iframely:v1.4.3
image: dogbin/iframely:latest
ports:
- "127.0.0.1:8061:80"
volumes:
- ./iframely.config.local.js:/iframely/config.local.js:ro
restart: always
mem_limit: 100m

60
docker/prod/migrate-pictshare-to-pictrs.bash vendored
View File

@ -1,60 +0,0 @@
#!/bin/bash
set -e
if [[ $(id -u) != 0 ]]; then
echo "This migration needs to be run as root"
exit
fi
if [[ ! -f docker-compose.yml ]]; then
echo "No docker-compose.yml found in current directory. Is this the right folder?"
exit
fi
# Fixing pictrs permissions
mkdir -p volumes/pictrs
sudo chown -R 991:991 volumes/pictrs
echo "Restarting docker-compose, making sure that pictrs is started and pictshare is removed"
docker-compose up -d --remove-orphans
if [[ -z $(docker-compose ps | grep pictrs) ]]; then
echo "Pict-rs is not running, make sure you update Lemmy first"
exit
fi
# echo "Stopping Lemmy so that users dont upload new images during the migration"
# docker-compose stop lemmy
pushd volumes/pictshare/
echo "Importing pictshare images to pict-rs..."
IMAGE_NAMES=*
for image in $IMAGE_NAMES; do
IMAGE_PATH="$(pwd)/$image/$image"
if [[ ! -f $IMAGE_PATH ]]; then
continue
fi
echo -e "\nImporting $IMAGE_PATH"
ret=0
curl --silent --fail -F "images[]=@$IMAGE_PATH" http://127.0.0.1:8537/import || ret=$?
if [[ $ret != 0 ]]; then
echo "Error for $IMAGE_PATH : $ret"
fi
done
echo "Fixing permissions on pictshare folder"
find . -type d -exec chmod 755 {} \;
find . -type f -exec chmod 644 {} \;
popd
echo "Rewrite image links in Lemmy database"
docker-compose exec -u postgres postgres psql -U lemmy -c "UPDATE user_ SET avatar = REPLACE(avatar, 'pictshare', 'pictrs/image') WHERE avatar is not null;"
docker-compose exec -u postgres postgres psql -U lemmy -c "UPDATE post SET url = REPLACE(url, 'pictshare', 'pictrs/image') WHERE url is not null;"
echo "Moving pictshare data folder to pictshare_backup"
mv volumes/pictshare volumes/pictshare_backup
echo "Migration done, starting Lemmy again"
echo "If everything went well, you can delete ./volumes/pictshare_backup/"
docker-compose start lemmy

BIN
docs/img/chat_screen.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 78 KiB

BIN
docs/img/main_screen.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 92 KiB

BIN
docs/img/rank_algorithm.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 54 KiB

2
docs/src/SUMMARY.md vendored
View File

@ -10,11 +10,9 @@
- [Install with Ansible](administration_install_ansible.md)
- [Install with Kubernetes](administration_install_kubernetes.md)
- [Configuration](administration_configuration.md)
- [Backup and Restore](administration_backup_and_restore.md)
- [Contributing](contributing.md)
- [Docker Development](contributing_docker_development.md)
- [Local Development](contributing_local_development.md)
- [Tests](contributing_tests.md)
- [Federation Development](contributing_federation_development.md)
- [Websocket/HTTP API](contributing_websocket_http_api.md)
- [ActivityPub API Outline](contributing_apub_api_outline.md)

4
docs/src/about.md vendored
View File

@ -2,9 +2,9 @@
Front Page|Post
---|---
![main screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/main_screen.png)|![chat screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/chat_screen.png)
![main screen](https://i.imgur.com/kZSRcRu.png)|![chat screen](https://i.imgur.com/4XghNh6.png)
[Lemmy](https://github.com/LemmyNet/lemmy) is similar to sites like [Reddit](https://reddit.com), [Lobste.rs](https://lobste.rs), [Raddle](https://raddle.me), or [Hacker News](https://news.ycombinator.com/): you subscribe to forums you're interested in, post links and discussions, then vote, and comment on them. Behind the scenes, it is very different; anyone can easily run a server, and all these servers are federated (think email), and connected to the same universe, called the [Fediverse](https://en.wikipedia.org/wiki/Fediverse).
[Lemmy](https://github.com/dessalines/lemmy) is similar to sites like [Reddit](https://reddit.com), [Lobste.rs](https://lobste.rs), [Raddle](https://raddle.me), or [Hacker News](https://news.ycombinator.com/): you subscribe to forums you're interested in, post links and discussions, then vote, and comment on them. Behind the scenes, it is very different; anyone can easily run a server, and all these servers are federated (think email), and connected to the same universe, called the [Fediverse](https://en.wikipedia.org/wiki/Fediverse).
For a link aggregator, this means a user registered on one server can subscribe to forums on any other server, and can have discussions with users registered elsewhere.

1
docs/src/about_goals.md vendored
View File

@ -51,4 +51,3 @@
- [Activitypub implementers guide](https://socialhub.activitypub.rocks/t/draft-guide-for-new-activitypub-implementers/479)
- [Data storage questions](https://socialhub.activitypub.rocks/t/data-storage-questions/579/3)
- [Activitypub as it has been understood](https://flak.tedunangst.com/post/ActivityPub-as-it-has-been-understood)
- [Asonix http signatures in rust](https://git.asonix.dog/Aardwolf/http-signature-normalization)

2
docs/src/about_guide.md vendored
View File

@ -3,7 +3,7 @@
Start typing...
- `@a_user_name` to get a list of usernames.
- `!a_community` to get a list of communities.
- `#a_community` to get a list of communities.
- `:emoji` to get a list of emojis.
## Sorting

2
docs/src/about_ranking.md vendored
View File

@ -26,4 +26,4 @@ Gravity = Decay gravity, 1.8 is default
A plot of rank over 24 hours, of scores of 1, 5, 10, 100, 1000, with a scale factor of 10k.
![](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/rank_algorithm.png)
![](https://i.imgur.com/w8oBLlL.png)

44
docs/src/administration_backup_and_restore.md vendored
View File

@ -1,44 +0,0 @@
# Backup and Restore Guide
## Docker and Ansible
When using docker or ansible, there should be a `volumes` folder, which contains both the database, and all the pictures. Copy this folder to the new instance to restore your data.
### Incremental Database backup
To incrementally backup the DB to an `.sql` file, you can run:
```bash
docker exec -t FOLDERNAME_postgres_1 pg_dumpall -c -U lemmy > lemmy_dump_`date +%Y-%m-%d"_"%H_%M_%S`.sql
```
### A Sample backup script
```bash
#!/bin/sh
# DB Backup
ssh MY_USER@MY_IP "docker exec -t FOLDERNAME_postgres_1 pg_dumpall -c -U lemmy" > ~/BACKUP_LOCATION/INSTANCE_NAME_dump_`date +%Y-%m-%d"_"%H_%M_%S`.sql
# Volumes folder Backup
rsync -avP -zz --rsync-path="sudo rsync" MY_USER@MY_IP:/LEMMY_LOCATION/volumes ~/BACKUP_LOCATION/FOLDERNAME
```
### Restoring the DB
If you need to restore from a `pg_dumpall` file, you need to first clear out your existing database
```bash
# Drop the existing DB
docker exec -i FOLDERNAME_postgres_1 psql -U lemmy -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public;"
# Restore from the .sql backup
cat db_dump.sql | docker exec -i FOLDERNAME_postgres_1 psql -U lemmy # restores the db
# This also might be necessary when doing a db import with a different password.
docker exec -i FOLDERNAME_postgres_1 psql -U lemmy -c "alter user lemmy with password 'bleh'"
```
## More resources
- https://stackoverflow.com/questions/24718706/backup-restore-a-dockerized-postgresql-database

17
docs/src/administration_configuration.md vendored
View File

@ -1,20 +1,13 @@
# Configuration
The configuration is based on the file
[defaults.hjson](https://yerbamate.dev/LemmyNet/lemmy/src/branch/master/server/config/defaults.hjson).
This file also contains documentation for all the available options. To override the defaults, you
can copy the options you want to change into your local `config.hjson` file.
The configuration is based on the file [defaults.hjson](server/config/defaults.hjson). This file also contains documentation for all the available options. To override the defaults, you can copy the options you want to change into your local `config.hjson` file.
Additionally, you can override any config files with environment variables. These have the same
name as the config options, and are prefixed with `LEMMY_`. For example, you can override the
`database.password` with `LEMMY__DATABASE__POOL_SIZE=10`.
Additionally, you can override any config files with environment variables. These have the same name as the config options, and are prefixed with `LEMMY_`. For example, you can override the `database.password` with
`LEMMY__DATABASE__POOL_SIZE=10`.
An additional option `LEMMY_DATABASE_URL` is available, which can be used with a PostgreSQL
connection string like `postgres://lemmy:password@lemmy_db:5432/lemmy`, passing all connection
details at once.
An additional option `LEMMY_DATABASE_URL` is available, which can be used with a PostgreSQL connection string like `postgres://lemmy:password@lemmy_db:5432/lemmy`, passing all connection details at once.
If the Docker container is not used, manually create the database specified above by running the
following commands:
If the Docker container is not used, manually create the database specified above by running the following commands:
```bash
cd server

7
docs/src/administration_install_ansible.md vendored
View File

@ -7,13 +7,10 @@ First, you need to [install Ansible on your local computer](https://docs.ansible
Then run the following commands on your local computer:
```bash
git clone https://github.com/LemmyNet/lemmy.git
git clone https://github.com/dessalines/lemmy.git
cd lemmy/ansible/
cp inventory.example inventory
nano inventory # enter your server, domain, contact email
# If the command below fails, you may need to comment out this line
# In the ansible.cfg file:
# interpreter_python=/usr/bin/python3
ansible-playbook lemmy.yml --become
```
@ -22,4 +19,4 @@ To update to a new version, just run the following in your local Lemmy repo:
git pull origin master
cd ansible
ansible-playbook lemmy.yml --become
```
```

22
docs/src/administration_install_docker.md vendored
View File

@ -6,25 +6,19 @@ Make sure you have both docker and docker-compose(>=`1.24.0`) installed. On Ubun
# create a folder for the lemmy files. the location doesnt matter, you can put this anywhere you want
mkdir /lemmy
cd /lemmy
# download default config files
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/docker-compose.yml
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/lemmy.hjson
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/iframely.config.local.js
# Set correct permissions for pictrs folder
mkdir -p volumes/pictrs
sudo chown -R 991:991 volumes/pictrs
wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/prod/docker-compose.yml
wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/lemmy.hjson
wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/iframely.config.local.js
docker-compose up -d
```
After this, have a look at the [config file](administration_configuration.md) named `lemmy.hjson`, and adjust it, in particular the hostname, and possibly the db password. Then run:
After this, have a look at the [config file](administration_configuration.md) named `lemmy.hjson`, and adjust it, in particular the hostname.
`docker-compose up -d`
To make Lemmy available outside the server, you need to setup a reverse proxy, like Nginx. [A sample nginx config](https://raw.githubusercontent.com/LemmyNet/lemmy/master/ansible/templates/nginx.conf), could be setup with:
To make Lemmy available outside the server, you need to setup a reverse proxy, like Nginx. [A sample nginx config](/ansible/templates/nginx.conf), could be setup with:
```bash
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/ansible/templates/nginx.conf
wget https://raw.githubusercontent.com/dessalines/lemmy/master/ansible/templates/nginx.conf
# Replace the {{ vars }}
sudo mv nginx.conf /etc/nginx/sites-enabled/lemmy.conf
```
@ -36,6 +30,6 @@ You will also need to setup TLS, for example with [Let's Encrypt](https://letsen
To update to the newest version, you can manually change the version in `docker-compose.yml`. Alternatively, fetch the latest version from our git repo:
```bash
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/docker-compose.yml
wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/prod/docker-compose.yml
docker-compose up -d
```

9
docs/src/contributing.md vendored
View File

@ -4,14 +4,13 @@ Information about contributing to Lemmy, whether it is translating, testing, des
## Issue tracking / Repositories
- [GitHub (for issues and pull requests)](https://github.com/LemmyNet/lemmy)
- [Gitea (only for pull requests)](https://yerbamate.dev/LemmyNet/lemmy)
- [GitLab (only code-mirror)](https://gitlab.com/dessalines/lemmy)
- [GitHub (for issues)](https://github.com/dessalines/lemmy)
- [Gitea](https://yerbamate.dev/dessalines/lemmy)
- [GitLab](https://gitlab.com/dessalines/lemmy)
## Translating
Check out [Lemmy's Weblate](https://weblate.yerbamate.dev/projects/lemmy/) for translations.
Go [here](https://github.com/dessalines/lemmy#translations) for translation instructions.
## Architecture

17
docs/src/contributing_docker_development.md vendored
View File

@ -3,22 +3,11 @@
## Running
```bash
sudo apt install git docker-compose
git clone https://github.com/LemmyNet/lemmy
git clone https://github.com/dessalines/lemmy
cd lemmy/docker/dev
sudo docker-compose up --no-deps --build
./docker_update.sh # This builds and runs it, updating for your changes
```
and go to http://localhost:8536.
To speed up the Docker compile, add the following to `/etc/docker/daemon.json` and restart Docker.
```
{
"features": {
"buildkit": true
}
}
```
If the build is still too slow, you will have to use a
[local build](contributing_local_development.md) instead.
Note that compile times when changing `Cargo.toml` are relatively long with Docker, because builds can't be incrementally cached. If this is a problem for you, you should use [Local Development](contributing_local_development.md).

35
docs/src/contributing_federation_development.md vendored
View File

@ -5,17 +5,17 @@
If you don't have a local clone of the Lemmy repo yet, just run the following command:
```bash
git clone https://github.com/LemmyNet/lemmy -b federation
git clone https://yerbamate.dev/nutomic/lemmy.git -b federation
```
If you already have the Lemmy repo cloned, you need to add a new remote:
```bash
git remote add federation https://github.com/LemmyNet/lemmy
git remote add federation https://yerbamate.dev/nutomic/lemmy.git
git checkout federation
git pull federation federation
```
## Running locally
## Running
You need to have the following packages installed, the Docker service needs to be running.
@ -31,30 +31,7 @@ cd dev/federation-test
```
After the build is finished and the docker-compose setup is running, open [127.0.0.1:8540](http://127.0.0.1:8540) and
[127.0.0.1:8550](http://127.0.0.1:8550) in your browser to use the test instances. You can login as admin with
username `lemmy_alpha` and `lemmy_beta` respectively, with password `lemmy`.
[127.0.0.1:8541](http://127.0.0.1:8541) in your browser to use the test instances. You can login as admin with
username `lemmy` and password `lemmy`, or create new accounts.
## Running on a server
Note that federation is currently in alpha. Only use it for testing, not on any production server, and be aware
that you might have to wipe the instance data at one point or another.
Follow the normal installation instructions, either with [Ansible](administration_install_ansible.md) or
[manually](administration_install_docker.md). Then replace the line `image: dessalines/lemmy:v0.x.x` in
`/lemmy/docker-compose.yml` with `image: dessalines/lemmy:federation`. Also add the following in
`/lemmy/lemmy.hjson`:
```
federation: {
enabled: true
allowed_instances: example.com
}
```
Afterwards, and whenver you want to update to the latest version, run these commands on the server:
```
cd /lemmy/
sudo docker-compose pull
sudo docker-compose up -d
```
Please get in touch if you want to contribute to this, so we can coordinate things and avoid duplicate work.

74
docs/src/contributing_local_development.md vendored
View File

@ -1,67 +1,31 @@
### Ubuntu
#### Requirements
- [Rust](https://www.rust-lang.org/)
- [Yarn](https://yarnpkg.com/en/)
- [Postgres](https://www.postgresql.org/)
#### Build requirements:
```
sudo apt install git cargo libssl-dev pkg-config libpq-dev yarn curl gnupg2 git
# install yarn
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt update && sudo apt install yarn
```
#### Set up Postgres DB
#### Get the source code
```
git clone https://github.com/LemmyNet/lemmy.git
# or alternatively from gitea
# git clone https://yerbamate.dev/LemmyNet/lemmy.git
```
All the following commands need to be run either in `lemmy/server` or `lemmy/ui`, as indicated
by the `cd` command.
#### Build the backend (Rust)
```
```bash
cd server
cargo build
# for development, use `cargo check` instead)
./db-init.sh
```
#### Build the frontend (Typescript)
```
cd ui
yarn
yarn build
```
Or run the commands manually:
#### Setup postgresql
```
sudo apt install postgresql
sudo systemctl start postgresql
# initialize postgres database
sudo -u postgres psql -c "create user lemmy with password 'password' superuser;" -U postgres
sudo -u postgres psql -c 'create database lemmy with owner lemmy;' -U postgres
```bash
psql -c "create user lemmy with password 'password' superuser;" -U postgres
psql -c 'create database lemmy with owner lemmy;' -U postgres
export LEMMY_DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
# or execute server/db-init.sh
```
#### Run a local development instance
```
# run each of these in a seperate terminal
cd server && cargo run
ui & yarn start
```
#### Running
Then open [localhost:4444](http://localhost:4444) in your browser. It will auto-refresh if you edit
any frontend files. For backend coding, you will have to rerun `cargo run`. You can use
`cargo check` as a faster way to find compilation errors.
To speed up incremental builds, you can add the following to `~/.cargo/config`:
```bash
git clone https://github.com/dessalines/lemmy
cd lemmy
./install.sh
# For live coding, where both the front and back end, automagically reload on any save, do:
# cd ui && yarn start
# cd server && cargo watch -x run
```
[target.x86_64-unknown-linux-gnu]
rustflags = ["-Clink-arg=-fuse-ld=lld"]
```
Note that this setup doesn't include image uploads or link previews (provided by pict-rs and
iframely respectively). If you want to test those, you should use the
[Docker development](contributing_docker_development.md).

18
docs/src/contributing_tests.md vendored
View File

@ -1,18 +0,0 @@
### Tests
#### Rust
After installing [local development dependencies](contributing_local_development.md), run the
following commands in the `server` subfolder:
```bash
psql -U lemmy -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public;"
export DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
diesel migration run
RUST_TEST_THREADS=1 cargo test
```
### Federation
Install the [Docker development dependencies](contributing_docker_development.md), and execute
`docker/federation-test/run-tests.sh`

101
docs/src/contributing_websocket_http_api.md vendored
View File

@ -1,6 +1,6 @@
# Lemmy API
*Note: this may lag behind the actual API endpoints [here](../server/src/api). The API should be considered unstable and may change any time.*
*Note: this may lag behind the actual API endpoints [here](../server/src/api).*
<!-- toc -->
@ -92,93 +92,85 @@
- [Request](#request-17)
- [Response](#response-17)
- [HTTP](#http-18)
+ [Get Site Config](#get-site-config)
* [Community](#community)
+ [Get Community](#get-community)
- [Request](#request-18)
- [Response](#response-18)
- [HTTP](#http-19)
+ [Save Site Config](#save-site-config)
+ [Create Community](#create-community)
- [Request](#request-19)
- [Response](#response-19)
- [HTTP](#http-20)
* [Community](#community)
+ [Get Community](#get-community)
+ [List Communities](#list-communities)
- [Request](#request-20)
- [Response](#response-20)
- [HTTP](#http-21)
+ [Create Community](#create-community)
+ [Ban from Community](#ban-from-community)
- [Request](#request-21)
- [Response](#response-21)
- [HTTP](#http-22)
+ [List Communities](#list-communities)
+ [Add Mod to Community](#add-mod-to-community)
- [Request](#request-22)
- [Response](#response-22)
- [HTTP](#http-23)
+ [Ban from Community](#ban-from-community)
+ [Edit Community](#edit-community)
- [Request](#request-23)
- [Response](#response-23)
- [HTTP](#http-24)
+ [Add Mod to Community](#add-mod-to-community)
+ [Follow Community](#follow-community)
- [Request](#request-24)
- [Response](#response-24)
- [HTTP](#http-25)
+ [Edit Community](#edit-community)
+ [Get Followed Communities](#get-followed-communities)
- [Request](#request-25)
- [Response](#response-25)
- [HTTP](#http-26)
+ [Follow Community](#follow-community)
+ [Transfer Community](#transfer-community)
- [Request](#request-26)
- [Response](#response-26)
- [HTTP](#http-27)
+ [Get Followed Communities](#get-followed-communities)
* [Post](#post)
+ [Create Post](#create-post)
- [Request](#request-27)
- [Response](#response-27)
- [HTTP](#http-28)
+ [Transfer Community](#transfer-community)
+ [Get Post](#get-post)
- [Request](#request-28)
- [Response](#response-28)
- [HTTP](#http-29)
* [Post](#post)
+ [Create Post](#create-post)
+ [Get Posts](#get-posts)
- [Request](#request-29)
- [Response](#response-29)
- [HTTP](#http-30)
+ [Get Post](#get-post)
+ [Create Post Like](#create-post-like)
- [Request](#request-30)
- [Response](#response-30)
- [HTTP](#http-31)
+ [Get Posts](#get-posts)
+ [Edit Post](#edit-post)
- [Request](#request-31)
- [Response](#response-31)
- [HTTP](#http-32)
+ [Create Post Like](#create-post-like)
+ [Save Post](#save-post)
- [Request](#request-32)
- [Response](#response-32)
- [HTTP](#http-33)
+ [Edit Post](#edit-post)
* [Comment](#comment)
+ [Create Comment](#create-comment)
- [Request](#request-33)
- [Response](#response-33)
- [HTTP](#http-34)
+ [Save Post](#save-post)
+ [Edit Comment](#edit-comment)
- [Request](#request-34)
- [Response](#response-34)
- [HTTP](#http-35)
* [Comment](#comment)
+ [Create Comment](#create-comment)
+ [Save Comment](#save-comment)
- [Request](#request-35)
- [Response](#response-35)
- [HTTP](#http-36)
+ [Edit Comment](#edit-comment)
+ [Create Comment Like](#create-comment-like)
- [Request](#request-36)
- [Response](#response-36)
- [HTTP](#http-37)
+ [Save Comment](#save-comment)
- [Request](#request-37)
- [Response](#response-37)
- [HTTP](#http-38)
+ [Create Comment Like](#create-comment-like)
- [Request](#request-38)
- [Response](#response-38)
- [HTTP](#http-39)
* [RSS / Atom feeds](#rss--atom-feeds)
+ [All](#all)
+ [Community](#community-1)
@ -787,53 +779,6 @@ Search types are `All, Comments, Posts, Communities, Users, Url`
`POST /site/transfer`
#### Get Site Config
##### Request
```rust
{
op: "GetSiteConfig",
data: {
auth: String
}
}
```
##### Response
```rust
{
op: "GetSiteConfig",
data: {
config_hjson: String,
}
}
```
##### HTTP
`GET /site/config`
#### Save Site Config
##### Request
```rust
{
op: "SaveSiteConfig",
data: {
config_hjson: String,
auth: String
}
}
```
##### Response
```rust
{
op: "SaveSiteConfig",
data: {
config_hjson: String,
}
}
```
##### HTTP
`PUT /site/config`
### Community
#### Get Community
##### Request

9
docs/src/lemmy_council.md vendored
View File

@ -1,7 +1,6 @@
# Lemmy Council
- A group of lemmy developers and users that use a well-defined democratic process to steer the project in a positive direction, keep it aligned to community goals, and resolve conflicts.
- Council members are also added as administrators to any official Lemmy instances.
- A group of lemmy developers and users that use a well-defined democratic process to steer the project in a positive direction, keep it aligned to community goals, and resolve conflicts.
## Voting / Decision-Making
@ -50,7 +49,5 @@
## Member List / Contact Info
General Contact [@LemmyDev Mastodon](https://mastodon.social/@LemmyDev)
- [Dessalines](https://dev.lemmy.ml/u/dessalines)
- [Nutomic](https://dev.lemmy.ml/u/nutomic)
- [AgreeableLandscape](https://dev.lemmy.ml/u/AgreeableLandscape)
- [fruechtchen](https://dev.lemmy.ml/u/fruechtchen)
- Dessalines [Matrix](https://matrix.to/#/@happydooby:matrix.org)
- Nutomic [Matrix](https://matrix.to/#/@nutomic:matrix.org), [Mastodon](https://radical.town/@felix)

73
install.sh vendored
View File

@ -1,4 +1,4 @@
#!/bin/sh
#!/bin/bash
set -e
# Set the database variable to the default first.
@ -10,55 +10,25 @@ export LEMMY_DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
export JWT_SECRET=changeme
export HOSTNAME=rrr
yes_no_prompt_invalid() {
echo "Invalid input. Please enter either \"y\" or \"n\"." 1>&2
}
ask_to_init_db() {
init_db_valid=0
init_db_final=0
while [ "$init_db_valid" == 0 ]
do
read -p "Initialize database (y/n)? " init_db
case "$init_db" in
[yY]* ) init_db_valid=1; init_db_final=1;;
[nN]* ) init_db_valid=1; init_db_final=0;;
* ) yes_no_prompt_invalid;;
esac
echo
done
if [ "$init_db_final" = 1 ]
then
source ./server/db-init.sh
read -n 1 -s -r -p "Press ANY KEY to continue execution of this script, press CTRL+C to quit..."
echo
fi
}
ask_to_auto_reload() {
auto_reload_valid=0
auto_reload_final=0
while [ "$auto_reload_valid" == 0 ]
do
echo "Automagically reload the project when source files are changed?"
echo "ONLY ENABLE THIS FOR DEVELOPMENT!"
read -p "(y/n) " auto_reload
case "$auto_reload" in
[yY]* ) auto_reload_valid=1; auto_reload_final=1;;
[nN]* ) auto_reload_valid=1; auto_reload_final=0;;
* ) yes_no_prompt_invalid;;
esac
echo
done
if [ "$auto_reload_final" = 1 ]
then
cd ui && yarn start
cd server && cargo watch -x run
fi
}
# Optionally initialize the database
ask_to_init_db
init_db_valid=0
init_db_final=0
while [ "$init_db_valid" == 0 ]
do
read -p "Initialize database (y/n)? " init_db
case "${init_db,,}" in
y|yes ) init_db_valid=1; init_db_final=1;;
n|no ) init_db_valid=1; init_db_final=0;;
* ) echo "Invalid input" 1>&2;;
esac
echo
done
if [ "$init_db_final" = 1 ]
then
source ./server/db-init.sh
read -n 1 -s -r -p "Press ANY KEY to continue execution of this script, press CTRL+C to quit..."
echo
fi
# Build the web client
cd ui
@ -69,5 +39,6 @@ yarn build
cd ../server
RUST_LOG=debug cargo run
# For live coding, where both the front and back end, automagically reload on any save
ask_to_auto_reload
# For live coding, where both the front and back end, automagically reload on any save, do:
# cd ui && yarn start
# cd server && cargo watch -x run

5
server/.rustfmt.toml vendored
View File

@ -1,5 +1,2 @@
tab_spaces = 2
edition="2018"
imports_layout="HorizontalVertical"
merge_imports=true
reorder_imports=true
edition="2018"

3079
server/Cargo.lock generated vendored
View File

File diff suppressed because it is too large Load Diff

48
server/Cargo.toml vendored
View File

@ -1,30 +1,24 @@
[package]
name = "lemmy_server"
version = "0.0.1"
authors = ["Dessalines <tyhou13@gmx.com>"]
authors = ["Dessalines <happydooby@gmail.com>"]
edition = "2018"
[profile.release]
lto = true
[dependencies]
diesel = { version = "1.4.4", features = ["postgres","chrono","r2d2","64-column-tables","serde_json"] }
diesel = { version = "1.4.2", features = ["postgres","chrono", "r2d2", "64-column-tables"] }
diesel_migrations = "1.4.0"
dotenv = "0.15.0"
activitystreams = "0.6.2"
activitystreams-new = { git = "https://git.asonix.dog/asonix/activitystreams-sketch" }
activitystreams-ext = { git = "https://git.asonix.dog/asonix/activitystreams-ext" }
bcrypt = "0.8.0"
activitystreams = "0.5.0-alpha.16"
bcrypt = "0.6.2"
chrono = { version = "0.4.7", features = ["serde"] }
serde_json = { version = "1.0.52", features = ["preserve_order"]}
failure = "0.1.8"
failure = "0.1.5"
serde_json = { version = "1.0.48", features = ["preserve_order"]}
serde = { version = "1.0.105", features = ["derive"] }
actix = "0.10.0-alpha.2"
actix-web = { version = "3.0.0-alpha.3", features = ["rustls"] }
actix-files = "0.3.0-alpha.1"
actix-web-actors = "3.0.0-alpha.1"
actix-rt = "1.1.1"
awc = "2.0.0-alpha.2"
actix = "0.9.0"
actix-web = "2.0.0"
actix-files = "0.2.1"
actix-web-actors = "2.0.0"
actix-rt = "1.0.0"
log = "0.4.0"
env_logger = "0.7.1"
rand = "0.7.3"
@ -33,21 +27,15 @@ strum_macros = "0.18.0"
jsonwebtoken = "7.0.1"
regex = "1.3.5"
lazy_static = "1.3.0"
lettre = "0.9.3"
lettre_email = "0.9.4"
lettre = "0.9.2"
lettre_email = "0.9.2"
sha2 = "0.8.1"
rss = "1.9.0"
htmlescape = "0.3.1"
url = { version = "2.1.1", features = ["serde"] }
config = {version = "0.10.1", default-features = false, features = ["hjson"] }
config = "0.10.1"
hjson = "0.8.2"
url = "2.1.1"
percent-encoding = "2.1.0"
isahc = "0.9"
comrak = "0.7"
openssl = "0.10"
http = "0.2.1"
http-signature-normalization-actix = { version = "0.4.0-alpha.0", default-features = false, features = ["sha-2"] }
base64 = "0.12.1"
tokio = "0.2.21"
futures = "0.3.5"
itertools = "0.9.0"
uuid = { version = "0.8", features = ["serde", "v4"] }
sha2 = "0.9"
async-trait = "0.1.36"

12
server/config/defaults.hjson vendored
View File

@ -33,7 +33,7 @@
port: 8536
# json web token for authorization between server and client
jwt_secret: "changeme"
# The location of the frontend
# The dir for the front end
front_end_dir: "../ui/dist"
# rate limits for various user actions, by user ip
rate_limit: {
@ -54,22 +54,20 @@
federation: {
# whether to enable activitypub federation. this feature is in alpha, do not enable in production.
enabled: false
# comma seperated list of instances to follow
followed_instances: ""
# whether tls is required for activitypub. only disable this for debugging, never for producion.
tls_enabled: true
# comma seperated list of instances with which federation is allowed
allowed_instances: ""
}
# # email sending configuration
# email: {
# # hostname and port of the smtp server
# # hostname of the smtp server
# smtp_server: ""
# # login name for smtp server
# smtp_login: ""
# # password to login to the smtp server
# smtp_password: ""
# # address to send emails from, eg "noreply@your-instance.com"
# # address to send emails from, eg "info@your-instance.com"
# smtp_from_address: ""
# # whether or not smtp connections should use tls
# use_tls: true
# }
}

113
server/db-init.sh vendored
View File

@ -1,106 +1,43 @@
#!/bin/sh
#!/bin/bash
# Default configurations
username=lemmy
dbname=lemmy
port=5432
yes_no_prompt_invalid() {
echo "Invalid input. Please enter either \"y\" or \"n\"." 1>&2
}
password=""
password_confirm=""
password_valid=0
print_config() {
echo " database name: $dbname"
echo " username: $username"
echo " port: $port"
}
ask_for_db_config() {
echo "The default database configuration is:"
print_config
while [ "$password_valid" == 0 ]
do
read -p "Enter database password: " -s password
echo
default_config_final=0
default_config_valid=0
while [ "$default_config_valid" == 0 ]
do
read -p "Use this configuration (y/n)? " default_config
case "$default_config" in
[yY]* ) default_config_valid=1; default_config_final=1;;
[nN]* ) default_config_valid=1; default_config_final=0;;
* ) yes_no_prompt_invalid;;
esac
echo
done
read -p "Verify database password: " -s password_confirm
echo
echo
if [ "$default_config_final" == 0 ]
# Start the loop from the top if either check fails
if [ -z "$password" ]
then
config_ok_final=0
while [ "$config_ok_final" == 0 ]
do
read -p "Database name: " dbname
read -p "Username: " username
read -p "Port: " port
#echo
#echo "The database configuration is:"
#print_config
#echo
config_ok_valid=0
while [ "$config_ok_valid" == 0 ]
do
read -p "Use this configuration (y/n)? " config_ok
case "$config_ok" in
[yY]* ) config_ok_valid=1; config_ok_final=1;;
[nN]* ) config_ok_valid=1; config_ok_final=0;;
* ) yes_no_prompt_invalid;;
esac
echo
done
done
echo "Error: Password cannot be empty." 1>&2
echo
continue
fi
}
ask_for_password() {
password=""
password_confirm=""
password_valid=0
while [ "$password_valid" == 0 ]
do
read -p "Enter database password: " -s password
if [ "$password" != "$password_confirm" ]
then
echo "Error: Passwords don't match." 1>&2
echo
continue
fi
read -p "Verify database password: " -s password_confirm
echo
echo
# Set the password_valid variable to break out of the loop
password_valid=1
done
# Start the loop from the top if either check fails
if [ -z "$password" ]
then
echo "Error: Password cannot be empty." 1>&2
echo
continue
fi
if [ "$password" != "$password_confirm" ]
then
echo "Error: Passwords don't match." 1>&2
echo
continue
fi
# Set the password_valid variable to break out of the loop
password_valid=1
done
}
ask_for_db_config
ask_for_password
psql -c "CREATE USER $username WITH PASSWORD '$password' SUPERUSER;" -U postgres
psql -c "CREATE DATABASE $dbname WITH OWNER $username;" -U postgres
psql -c 'CREATE DATABASE $dbname WITH OWNER $username;' -U postgres
export LEMMY_DATABASE_URL=postgres://$username:$password@localhost:$port/$dbname
echo "The database URL is $LEMMY_DATABASE_URL"
echo $LEMMY_DATABASE_URL

4
server/migrations/2020-03-26-192410_add_activitypub_tables/up.sql vendored
View File

@ -15,7 +15,7 @@ create unique index idx_activity_unique_apid on activity ((data ->> 'id'::text))
-- Add federation columns to the two actor tables
alter table user_
-- TODO uniqueness constraints should be added on these 3 columns later
add column actor_id character varying(255) not null default 'http://fake.com', -- This needs to be checked and updated in code, building from the site url if local
add column actor_id character varying(255) not null default 'changeme', -- This needs to be checked and updated in code, building from the site url if local
add column bio text, -- not on community, already has description
add column local boolean not null default true,
add column private_key text, -- These need to be generated from code
@ -25,7 +25,7 @@ add column last_refreshed_at timestamp not null default now() -- Used to re-fetc
-- Community
alter table community
add column actor_id character varying(255) not null default 'http://fake.com', -- This needs to be checked and updated in code, building from the site url if local
add column actor_id character varying(255) not null default 'changeme', -- This needs to be checked and updated in code, building from the site url if local
add column local boolean not null default true,
add column private_key text, -- These need to be generated from code
add column public_key text,

4
server/migrations/2020-04-03-194936_add_activitypub_for_posts_and_comments/up.sql vendored
View File

@ -2,13 +2,13 @@
alter table post
-- TODO uniqueness constraints should be added on these 3 columns later
add column ap_id character varying(255) not null default 'http://fake.com', -- This needs to be checked and updated in code, building from the site url if local
add column ap_id character varying(255) not null default 'changeme', -- This needs to be checked and updated in code, building from the site url if local
add column local boolean not null default true
;
alter table comment
-- TODO uniqueness constraints should be added on these 3 columns later
add column ap_id character varying(255) not null default 'http://fake.com', -- This needs to be checked and updated in code, building from the site url if local
add column ap_id character varying(255) not null default 'changeme', -- This needs to be checked and updated in code, building from the site url if local
add column local boolean not null default true
;

2
server/migrations/2020-04-07-135912_add_user_community_apub_constraints/down.sql vendored
View File

@ -2,7 +2,7 @@
drop view user_view cascade;
alter table user_
add column fedi_name varchar(40) not null default 'http://fake.com';
add column fedi_name varchar(40) not null default 'changeme';
alter table user_
add constraint user__name_fedi_name_key unique (name, fedi_name);

440
server/migrations/2020-04-14-163701_update_views_for_activitypub/down.sql vendored
View File

@ -1,440 +0,0 @@
-- user_view
drop view user_view cascade;
create view user_view as
select
u.id,
u.name,
u.avatar,
u.email,
u.matrix_user_id,
u.admin,
u.banned,
u.show_avatars,
u.send_notifications_to_email,
u.published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;
create materialized view user_mview as select * from user_view;
create unique index idx_user_mview_id on user_mview (id);
-- community_view
drop view community_aggregates_view cascade;
create view community_aggregates_view as
select c.*,
(select name from user_ u where c.creator_id = u.id) as creator_name,
(select avatar from user_ u where c.creator_id = u.id) as creator_avatar,
(select name from category ct where c.category_id = ct.id) as category_name,
(select count(*) from community_follower cf where cf.community_id = c.id) as number_of_subscribers,
(select count(*) from post p where p.community_id = c.id) as number_of_posts,
(select count(*) from comment co, post p where c.id = p.community_id and p.id = co.post_id) as number_of_comments,
hot_rank((select count(*) from community_follower cf where cf.community_id = c.id), c.published) as hot_rank
from community c;
create materialized view community_aggregates_mview as select * from community_aggregates_view;
create unique index idx_community_aggregates_mview_id on community_aggregates_mview (id);
create view community_view as
with all_community as
(
select
ca.*
from community_aggregates_view ca
)
select
ac.*,
u.id as user_id,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
from user_ u
cross join all_community ac
union all
select
ac.*,
null as user_id,
null as subscribed
from all_community ac
;
create view community_mview as
with all_community as
(
select
ca.*
from community_aggregates_mview ca
)
select
ac.*,
u.id as user_id,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
from user_ u
cross join all_community ac
union all
select
ac.*,
null as user_id,
null as subscribed
from all_community ac
;
-- community views
drop view community_moderator_view;
drop view community_follower_view;
drop view community_user_ban_view;
create view community_moderator_view as
select *,
(select name from user_ u where cm.user_id = u.id) as user_name,
(select avatar from user_ u where cm.user_id = u.id),
(select name from community c where cm.community_id = c.id) as community_name
from community_moderator cm;
create view community_follower_view as
select *,
(select name from user_ u where cf.user_id = u.id) as user_name,
(select avatar from user_ u where cf.user_id = u.id),
(select name from community c where cf.community_id = c.id) as community_name
from community_follower cf;
create view community_user_ban_view as
select *,
(select name from user_ u where cm.user_id = u.id) as user_name,
(select avatar from user_ u where cm.user_id = u.id),
(select name from community c where cm.community_id = c.id) as community_name
from community_user_ban cm;
-- post_view
drop view post_view;
drop view post_mview;
drop materialized view post_aggregates_mview;
drop view post_aggregates_view;
-- regen post view
create view post_aggregates_view as
select
p.*,
(select u.banned from user_ u where p.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb where p.creator_id = cb.user_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where p.creator_id = user_.id) as creator_name,
(select avatar from user_ where p.creator_id = user_.id) as creator_avatar,
(select name from community where p.community_id = community.id) as community_name,
(select removed from community c where p.community_id = c.id) as community_removed,
(select deleted from community c where p.community_id = c.id) as community_deleted,
(select nsfw from community c where p.community_id = c.id) as community_nsfw,
(select count(*) from comment where comment.post_id = p.id) as number_of_comments,
coalesce(sum(pl.score), 0) as score,
count (case when pl.score = 1 then 1 else null end) as upvotes,
count (case when pl.score = -1 then 1 else null end) as downvotes,
hot_rank(coalesce(sum(pl.score) , 0),
(
case when (p.published < ('now'::timestamp - '1 month'::interval)) then p.published -- Prevents necro-bumps
else greatest(c.recent_comment_time, p.published)
end
)
) as hot_rank,
(
case when (p.published < ('now'::timestamp - '1 month'::interval)) then p.published -- Prevents necro-bumps
else greatest(c.recent_comment_time, p.published)
end
) as newest_activity_time
from post p
left join post_like pl on p.id = pl.post_id
left join (
select post_id,
max(published) as recent_comment_time
from comment
group by 1
) c on p.id = c.post_id
group by p.id, c.recent_comment_time;
create materialized view post_aggregates_mview as select * from post_aggregates_view;
create unique index idx_post_aggregates_mview_id on post_aggregates_mview (id);
create view post_view as
with all_post as (
select
pa.*
from post_aggregates_view pa
)
select
ap.*,
u.id as user_id,
coalesce(pl.score, 0) as my_vote,
(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
from user_ u
cross join all_post ap
left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
union all
select
ap.*,
null as user_id,
null as my_vote,
null as subscribed,
null as read,
null as saved
from all_post ap
;
create view post_mview as
with all_post as (
select
pa.*
from post_aggregates_mview pa
)
select
ap.*,
u.id as user_id,
coalesce(pl.score, 0) as my_vote,
(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
from user_ u
cross join all_post ap
left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
union all
select
ap.*,
null as user_id,
null as my_vote,
null as subscribed,
null as read,
null as saved
from all_post ap
;
-- reply_view, comment_view, user_mention
drop view reply_view;
drop view user_mention_view;
drop view user_mention_mview;
drop view comment_view;
drop view comment_mview;
drop materialized view comment_aggregates_mview;
drop view comment_aggregates_view;
-- reply and comment view
create view comment_aggregates_view as
select
c.*,
(select community_id from post p where p.id = c.post_id),
(select co.name from post p, community co where p.id = c.post_id and p.community_id = co.id) as community_name,
(select u.banned from user_ u where c.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb, post p where c.creator_id = cb.user_id and p.id = c.post_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where c.creator_id = user_.id) as creator_name,
(select avatar from user_ where c.creator_id = user_.id) as creator_avatar,
coalesce(sum(cl.score), 0) as score,
count (case when cl.score = 1 then 1 else null end) as upvotes,
count (case when cl.score = -1 then 1 else null end) as downvotes,
hot_rank(coalesce(sum(cl.score) , 0), c.published) as hot_rank
from comment c
left join comment_like cl on c.id = cl.comment_id
group by c.id;
create materialized view comment_aggregates_mview as select * from comment_aggregates_view;
create unique index idx_comment_aggregates_mview_id on comment_aggregates_mview (id);
create view comment_view as
with all_comment as
(
select
ca.*
from comment_aggregates_view ca
)
select
ac.*,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.community_id = cf.community_id) as subscribed,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
union all
select
ac.*,
null as user_id,
null as my_vote,
null as subscribed,
null as saved
from all_comment ac
;
create view comment_mview as
with all_comment as
(
select
ca.*
from comment_aggregates_mview ca
)
select
ac.*,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.community_id = cf.community_id) as subscribed,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
union all
select
ac.*,
null as user_id,
null as my_vote,
null as subscribed,
null as saved
from all_comment ac
;
-- Do the reply_view referencing the comment_mview
create view reply_view as
with closereply as (
select
c2.id,
c2.creator_id as sender_id,
c.creator_id as recipient_id
from comment c
inner join comment c2 on c.id = c2.parent_id
where c2.creator_id != c.creator_id
-- Do union where post is null
union
select
c.id,
c.creator_id as sender_id,
p.creator_id as recipient_id
from comment c, post p
where c.post_id = p.id and c.parent_id is null and c.creator_id != p.creator_id
)
select cv.*,
closereply.recipient_id
from comment_mview cv, closereply
where closereply.id = cv.id
;
-- user mention
create view user_mention_view as
select
c.id,
um.id as user_mention_id,
c.creator_id,
c.post_id,
c.parent_id,
c.content,
c.removed,
um.read,
c.published,
c.updated,
c.deleted,
c.community_id,
c.community_name,
c.banned,
c.banned_from_community,
c.creator_name,
c.creator_avatar,
c.score,
c.upvotes,
c.downvotes,
c.hot_rank,
c.user_id,
c.my_vote,
c.saved,
um.recipient_id
from user_mention um, comment_view c
where um.comment_id = c.id;
create view user_mention_mview as
with all_comment as
(
select
ca.*
from comment_aggregates_mview ca
)
select
ac.id,
um.id as user_mention_id,
ac.creator_id,
ac.post_id,
ac.parent_id,
ac.content,
ac.removed,
um.read,
ac.published,
ac.updated,
ac.deleted,
ac.community_id,
ac.community_name,
ac.banned,
ac.banned_from_community,
ac.creator_name,
ac.creator_avatar,
ac.score,
ac.upvotes,
ac.downvotes,
ac.hot_rank,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved,
um.recipient_id
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
left join user_mention um on um.comment_id = ac.id
union all
select
ac.id,
um.id as user_mention_id,
ac.creator_id,
ac.post_id,
ac.parent_id,
ac.content,
ac.removed,
um.read,
ac.published,
ac.updated,
ac.deleted,
ac.community_id,
ac.community_name,
ac.banned,
ac.banned_from_community,
ac.creator_name,
ac.creator_avatar,
ac.score,
ac.upvotes,
ac.downvotes,
ac.hot_rank,
null as user_id,
null as my_vote,
null as saved,
um.recipient_id
from all_comment ac
left join user_mention um on um.comment_id = ac.id
;

497
server/migrations/2020-04-14-163701_update_views_for_activitypub/up.sql vendored
View File

@ -1,497 +0,0 @@
-- user_view
drop view user_view cascade;
create view user_view as
select
u.id,
u.actor_id,
u.name,
u.avatar,
u.email,
u.matrix_user_id,
u.bio,
u.local,
u.admin,
u.banned,
u.show_avatars,
u.send_notifications_to_email,
u.published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;
create materialized view user_mview as select * from user_view;
create unique index idx_user_mview_id on user_mview (id);
-- community_view
drop view community_aggregates_view cascade;
create view community_aggregates_view as
-- Now that there's public and private keys, you have to be explicit here
select c.id,
c.name,
c.title,
c.description,
c.category_id,
c.creator_id,
c.removed,
c.published,
c.updated,
c.deleted,
c.nsfw,
c.actor_id,
c.local,
c.last_refreshed_at,
(select actor_id from user_ u where c.creator_id = u.id) as creator_actor_id,
(select local from user_ u where c.creator_id = u.id) as creator_local,
(select name from user_ u where c.creator_id = u.id) as creator_name,
(select avatar from user_ u where c.creator_id = u.id) as creator_avatar,
(select name from category ct where c.category_id = ct.id) as category_name,
(select count(*) from community_follower cf where cf.community_id = c.id) as number_of_subscribers,
(select count(*) from post p where p.community_id = c.id) as number_of_posts,
(select count(*) from comment co, post p where c.id = p.community_id and p.id = co.post_id) as number_of_comments,
hot_rank((select count(*) from community_follower cf where cf.community_id = c.id), c.published) as hot_rank
from community c;
create materialized view community_aggregates_mview as select * from community_aggregates_view;
create unique index idx_community_aggregates_mview_id on community_aggregates_mview (id);
create view community_view as
with all_community as
(
select
ca.*
from community_aggregates_view ca
)
select
ac.*,
u.id as user_id,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
from user_ u
cross join all_community ac
union all
select
ac.*,
null as user_id,
null as subscribed
from all_community ac
;
create view community_mview as
with all_community as
(
select
ca.*
from community_aggregates_mview ca
)
select
ac.*,
u.id as user_id,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
from user_ u
cross join all_community ac
union all
select
ac.*,
null as user_id,
null as subscribed
from all_community ac
;
-- community views
drop view community_moderator_view;
drop view community_follower_view;
drop view community_user_ban_view;
create view community_moderator_view as
select *,
(select actor_id from user_ u where cm.user_id = u.id) as user_actor_id,
(select local from user_ u where cm.user_id = u.id) as user_local,
(select name from user_ u where cm.user_id = u.id) as user_name,
(select avatar from user_ u where cm.user_id = u.id),
(select actor_id from community c where cm.community_id = c.id) as community_actor_id,
(select local from community c where cm.community_id = c.id) as community_local,
(select name from community c where cm.community_id = c.id) as community_name
from community_moderator cm;
create view community_follower_view as
select *,
(select actor_id from user_ u where cf.user_id = u.id) as user_actor_id,
(select local from user_ u where cf.user_id = u.id) as user_local,
(select name from user_ u where cf.user_id = u.id) as user_name,
(select avatar from user_ u where cf.user_id = u.id),
(select actor_id from community c where cf.community_id = c.id) as community_actor_id,
(select local from community c where cf.community_id = c.id) as community_local,
(select name from community c where cf.community_id = c.id) as community_name
from community_follower cf;
create view community_user_ban_view as
select *,
(select actor_id from user_ u where cm.user_id = u.id) as user_actor_id,
(select local from user_ u where cm.user_id = u.id) as user_local,
(select name from user_ u where cm.user_id = u.id) as user_name,
(select avatar from user_ u where cm.user_id = u.id),
(select actor_id from community c where cm.community_id = c.id) as community_actor_id,
(select local from community c where cm.community_id = c.id) as community_local,
(select name from community c where cm.community_id = c.id) as community_name
from community_user_ban cm;
-- post_view
drop view post_view;
drop view post_mview;
drop materialized view post_aggregates_mview;
drop view post_aggregates_view;
-- regen post view
create view post_aggregates_view as
select
p.*,
(select u.banned from user_ u where p.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb where p.creator_id = cb.user_id and p.community_id = cb.community_id) as banned_from_community,
(select actor_id from user_ where p.creator_id = user_.id) as creator_actor_id,
(select local from user_ where p.creator_id = user_.id) as creator_local,
(select name from user_ where p.creator_id = user_.id) as creator_name,
(select avatar from user_ where p.creator_id = user_.id) as creator_avatar,
(select actor_id from community where p.community_id = community.id) as community_actor_id,
(select local from community where p.community_id = community.id) as community_local,
(select name from community where p.community_id = community.id) as community_name,
(select removed from community c where p.community_id = c.id) as community_removed,
(select deleted from community c where p.community_id = c.id) as community_deleted,
(select nsfw from community c where p.community_id = c.id) as community_nsfw,
(select count(*) from comment where comment.post_id = p.id) as number_of_comments,
coalesce(sum(pl.score), 0) as score,
count (case when pl.score = 1 then 1 else null end) as upvotes,
count (case when pl.score = -1 then 1 else null end) as downvotes,
hot_rank(coalesce(sum(pl.score) , 0),
(
case when (p.published < ('now'::timestamp - '1 month'::interval)) then p.published -- Prevents necro-bumps
else greatest(c.recent_comment_time, p.published)
end
)
) as hot_rank,
(
case when (p.published < ('now'::timestamp - '1 month'::interval)) then p.published -- Prevents necro-bumps
else greatest(c.recent_comment_time, p.published)
end
) as newest_activity_time
from post p
left join post_like pl on p.id = pl.post_id
left join (
select post_id,
max(published) as recent_comment_time
from comment
group by 1
) c on p.id = c.post_id
group by p.id, c.recent_comment_time;
create materialized view post_aggregates_mview as select * from post_aggregates_view;
create unique index idx_post_aggregates_mview_id on post_aggregates_mview (id);
create view post_view as
with all_post as (
select
pa.*
from post_aggregates_view pa
)
select
ap.*,
u.id as user_id,
coalesce(pl.score, 0) as my_vote,
(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
from user_ u
cross join all_post ap
left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
union all
select
ap.*,
null as user_id,
null as my_vote,
null as subscribed,
null as read,
null as saved
from all_post ap
;
create view post_mview as
with all_post as (
select
pa.*
from post_aggregates_mview pa
)
select
ap.*,
u.id as user_id,
coalesce(pl.score, 0) as my_vote,
(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
from user_ u
cross join all_post ap
left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
union all
select
ap.*,
null as user_id,
null as my_vote,
null as subscribed,
null as read,
null as saved
from all_post ap
;
-- reply_view, comment_view, user_mention
drop view reply_view;
drop view user_mention_view;
drop view user_mention_mview;
drop view comment_view;
drop view comment_mview;
drop materialized view comment_aggregates_mview;
drop view comment_aggregates_view;
-- reply and comment view
create view comment_aggregates_view as
select
c.*,
(select community_id from post p where p.id = c.post_id),
(select co.actor_id from post p, community co where p.id = c.post_id and p.community_id = co.id) as community_actor_id,
(select co.local from post p, community co where p.id = c.post_id and p.community_id = co.id) as community_local,
(select co.name from post p, community co where p.id = c.post_id and p.community_id = co.id) as community_name,
(select u.banned from user_ u where c.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb, post p where c.creator_id = cb.user_id and p.id = c.post_id and p.community_id = cb.community_id) as banned_from_community,
(select actor_id from user_ where c.creator_id = user_.id) as creator_actor_id,
(select local from user_ where c.creator_id = user_.id) as creator_local,
(select name from user_ where c.creator_id = user_.id) as creator_name,
(select avatar from user_ where c.creator_id = user_.id) as creator_avatar,
coalesce(sum(cl.score), 0) as score,
count (case when cl.score = 1 then 1 else null end) as upvotes,
count (case when cl.score = -1 then 1 else null end) as downvotes,
hot_rank(coalesce(sum(cl.score) , 0), c.published) as hot_rank
from comment c
left join comment_like cl on c.id = cl.comment_id
group by c.id;
create materialized view comment_aggregates_mview as select * from comment_aggregates_view;
create unique index idx_comment_aggregates_mview_id on comment_aggregates_mview (id);
create view comment_view as
with all_comment as
(
select
ca.*
from comment_aggregates_view ca
)
select
ac.*,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.community_id = cf.community_id) as subscribed,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
union all
select
ac.*,
null as user_id,
null as my_vote,
null as subscribed,
null as saved
from all_comment ac
;
create view comment_mview as
with all_comment as
(
select
ca.*
from comment_aggregates_mview ca
)
select
ac.*,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.community_id = cf.community_id) as subscribed,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
union all
select
ac.*,
null as user_id,
null as my_vote,
null as subscribed,
null as saved
from all_comment ac
;
-- Do the reply_view referencing the comment_mview
create view reply_view as
with closereply as (
select
c2.id,
c2.creator_id as sender_id,
c.creator_id as recipient_id
from comment c
inner join comment c2 on c.id = c2.parent_id
where c2.creator_id != c.creator_id
-- Do union where post is null
union
select
c.id,
c.creator_id as sender_id,
p.creator_id as recipient_id
from comment c, post p
where c.post_id = p.id and c.parent_id is null and c.creator_id != p.creator_id
)
select cv.*,
closereply.recipient_id
from comment_mview cv, closereply
where closereply.id = cv.id
;
-- user mention
create view user_mention_view as
select
c.id,
um.id as user_mention_id,
c.creator_id,
c.creator_actor_id,
c.creator_local,
c.post_id,
c.parent_id,
c.content,
c.removed,
um.read,
c.published,
c.updated,
c.deleted,
c.community_id,
c.community_actor_id,
c.community_local,
c.community_name,
c.banned,
c.banned_from_community,
c.creator_name,
c.creator_avatar,
c.score,
c.upvotes,
c.downvotes,
c.hot_rank,
c.user_id,
c.my_vote,
c.saved,
um.recipient_id,
(select actor_id from user_ u where u.id = um.recipient_id) as recipient_actor_id,
(select local from user_ u where u.id = um.recipient_id) as recipient_local
from user_mention um, comment_view c
where um.comment_id = c.id;
create view user_mention_mview as
with all_comment as
(
select
ca.*
from comment_aggregates_mview ca
)
select
ac.id,
um.id as user_mention_id,
ac.creator_id,
ac.creator_actor_id,
ac.creator_local,
ac.post_id,
ac.parent_id,
ac.content,
ac.removed,
um.read,
ac.published,
ac.updated,
ac.deleted,
ac.community_id,
ac.community_actor_id,
ac.community_local,
ac.community_name,
ac.banned,
ac.banned_from_community,
ac.creator_name,
ac.creator_avatar,
ac.score,
ac.upvotes,
ac.downvotes,
ac.hot_rank,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved,
um.recipient_id,
(select actor_id from user_ u where u.id = um.recipient_id) as recipient_actor_id,
(select local from user_ u where u.id = um.recipient_id) as recipient_local
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
left join user_mention um on um.comment_id = ac.id
union all
select
ac.id,
um.id as user_mention_id,
ac.creator_id,
ac.creator_actor_id,
ac.creator_local,
ac.post_id,
ac.parent_id,
ac.content,
ac.removed,
um.read,
ac.published,
ac.updated,
ac.deleted,
ac.community_id,
ac.community_actor_id,
ac.community_local,
ac.community_name,
ac.banned,
ac.banned_from_community,
ac.creator_name,
ac.creator_avatar,
ac.score,
ac.upvotes,
ac.downvotes,
ac.hot_rank,
null as user_id,
null as my_vote,
null as saved,
um.recipient_id,
(select actor_id from user_ u where u.id = um.recipient_id) as recipient_actor_id,
(select local from user_ u where u.id = um.recipient_id) as recipient_local
from all_comment ac
left join user_mention um on um.comment_id = ac.id
;

4
server/migrations/2020-04-21-123957_remove_unique_user_constraints/down.sql vendored
View File

@ -1,4 +0,0 @@
-- The username index
drop index idx_user_name_lower_actor_id;
create unique index idx_user_name_lower on user_ (lower(name));

2
server/migrations/2020-04-21-123957_remove_unique_user_constraints/up.sql vendored
View File

@ -1,2 +0,0 @@
drop index idx_user_name_lower;
create unique index idx_user_name_lower_actor_id on user_ (lower(name), lower(actor_id));

21
server/migrations/2020-05-05-210233_add_activitypub_for_private_messages/down.sql vendored
View File

@ -1,21 +0,0 @@
drop materialized view private_message_mview;
drop view private_message_view;
alter table private_message
drop column ap_id,
drop column local;
create view private_message_view as
select
pm.*,
u.name as creator_name,
u.avatar as creator_avatar,
u2.name as recipient_name,
u2.avatar as recipient_avatar
from private_message pm
inner join user_ u on u.id = pm.creator_id
inner join user_ u2 on u2.id = pm.recipient_id;
create materialized view private_message_mview as select * from private_message_view;
create unique index idx_private_message_mview_id on private_message_mview (id);

25
server/migrations/2020-05-05-210233_add_activitypub_for_private_messages/up.sql vendored
View File

@ -1,25 +0,0 @@
alter table private_message
add column ap_id character varying(255) not null default 'http://fake.com', -- This needs to be checked and updated in code, building from the site url if local
add column local boolean not null default true
;
drop materialized view private_message_mview;
drop view private_message_view;
create view private_message_view as
select
pm.*,
u.name as creator_name,
u.avatar as creator_avatar,
u.actor_id as creator_actor_id,
u.local as creator_local,
u2.name as recipient_name,
u2.avatar as recipient_avatar,
u2.actor_id as recipient_actor_id,
u2.local as recipient_local
from private_message pm
inner join user_ u on u.id = pm.creator_id
inner join user_ u2 on u2.id = pm.recipient_id;
create materialized view private_message_mview as select * from private_message_view;
create unique index idx_private_message_mview_id on private_message_mview (id);

6
server/query_testing/apache_bench_report.sh vendored
View File

@ -11,12 +11,6 @@ declare -a arr=(
"https://torrents-csv.ml/service/search?q=wheel&page=1&type_=torrent"
)
## check if ab installed
if ! [ -x "$(command -v ab)" ]; then
echo 'Error: ab (Apache Bench) is not installed. https://httpd.apache.org/docs/2.4/programs/ab.html' >&2
exit 1
fi
## now loop through the above array
for i in "${arr[@]}"
do

6
server/query_testing/api_benchmark.sh vendored
View File

@ -15,12 +15,6 @@ declare -a arr=(
"/api/v1/post/list?sort=Hot&type_=All"
)
## check if ab installed
if ! [ -x "$(command -v ab)" ]; then
echo 'Error: ab (Apache Bench) is not installed. https://httpd.apache.org/docs/2.4/programs/ab.html' >&2
exit 1
fi
## now loop through the above array
for path in "${arr[@]}"
do

677
server/src/api/comment.rs
View File

@ -1,39 +1,8 @@
use crate::{
api::{APIError, Oper, Perform},
apub::{ApubLikeableType, ApubObjectType},
blocking,
db::{
comment::*,
comment_view::*,
community_view::*,
moderator::*,
post::*,
site_view::*,
user::*,
user_mention::*,
user_view::*,
Crud,
Likeable,
ListingType,
Saveable,
SortType,
},
naive_now,
remove_slurs,
scrape_text_for_mentions,
send_email,
settings::Settings,
websocket::{
server::{JoinCommunityRoom, SendComment},
UserOperation,
WebsocketInfo,
},
DbPool,
LemmyError,
MentionData,
};
use super::*;
use crate::send_email;
use crate::settings::Settings;
use diesel::PgConnection;
use log::error;
use serde::{Deserialize, Serialize};
use std::str::FromStr;
#[derive(Serialize, Deserialize)]
@ -95,15 +64,8 @@ pub struct GetCommentsResponse {
comments: Vec<CommentView>,
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<CreateComment> {
type Response = CommentResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<CommentResponse, LemmyError> {
impl Perform<CommentResponse> for Oper<CreateComment> {
fn perform(&self, conn: &PgConnection) -> Result<CommentResponse, Error> {
let data: &CreateComment = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -113,6 +75,19 @@ impl Perform for Oper<CreateComment> {
let user_id = claims.id;
let hostname = &format!("https://{}", Settings::get().hostname);
// Check for a community ban
let post = Post::read(&conn, data.post_id)?;
if CommunityUserBanView::get(&conn, user_id, post.community_id).is_ok() {
return Err(APIError::err("community_ban").into());
}
// Check for a site ban
if UserView::read(&conn, user_id)?.banned {
return Err(APIError::err("site_ban").into());
}
let content_slurs_removed = remove_slurs(&data.content.to_owned());
let comment_form = CommentForm {
@ -123,54 +98,122 @@ impl Perform for Oper<CreateComment> {
removed: None,
deleted: None,
read: None,
published: None,
updated: None,
ap_id: "http://fake.com".into(),
ap_id: "changeme".into(),
local: true,
};
// Check for a community ban
let post_id = data.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let is_banned =
move |conn: &'_ _| CommunityUserBanView::get(conn, user_id, community_id).is_ok();
if blocking(pool, is_banned).await? {
return Err(APIError::err("community_ban").into());
}
// Check for a site ban
let user = blocking(pool, move |conn| User_::read(&conn, user_id)).await??;
if user.banned {
return Err(APIError::err("site_ban").into());
}
let comment_form2 = comment_form.clone();
let inserted_comment =
match blocking(pool, move |conn| Comment::create(&conn, &comment_form2)).await? {
Ok(comment) => comment,
Err(_e) => return Err(APIError::err("couldnt_create_comment").into()),
};
let inserted_comment_id = inserted_comment.id;
let updated_comment: Comment = match blocking(pool, move |conn| {
Comment::update_ap_id(&conn, inserted_comment_id)
})
.await?
{
let inserted_comment = match Comment::create(&conn, &comment_form) {
Ok(comment) => comment,
Err(_e) => return Err(APIError::err("couldnt_create_comment").into()),
};
updated_comment
.send_create(&user, &self.client, pool)
.await?;
match Comment::update_ap_id(&conn, inserted_comment.id) {
Ok(comment) => comment,
Err(_e) => return Err(APIError::err("couldnt_create_comment").into()),
};
let mut recipient_ids = Vec::new();
// Scan the comment for user mentions, add those rows
let mentions = scrape_text_for_mentions(&comment_form.content);
let recipient_ids =
send_local_notifs(mentions, updated_comment.clone(), user.clone(), post, pool).await?;
let extracted_usernames = extract_usernames(&comment_form.content);
for username_mention in &extracted_usernames {
if let Ok(mention_user) = User_::read_from_name(&conn, (*username_mention).to_string()) {
// You can't mention yourself
// At some point, make it so you can't tag the parent creator either
// This can cause two notifications, one for reply and the other for mention
if mention_user.id != user_id {
recipient_ids.push(mention_user.id);
let user_mention_form = UserMentionForm {
recipient_id: mention_user.id,
comment_id: inserted_comment.id,
read: None,
};
// Allow this to fail softly, since comment edits might re-update or replace it
// Let the uniqueness handle this fail
match UserMention::create(&conn, &user_mention_form) {
Ok(_mention) => (),
Err(_e) => error!("{}", &_e),
};
// Send an email to those users that have notifications on
if mention_user.send_notifications_to_email {
if let Some(mention_email) = mention_user.email {
let subject = &format!(
"{} - Mentioned by {}",
Settings::get().hostname,
claims.username
);
let html = &format!(
"<h1>User Mention</h1><br><div>{} - {}</div><br><a href={}/inbox>inbox</a>",
claims.username, comment_form.content, hostname
);
match send_email(subject, &mention_email, &mention_user.name, html) {
Ok(_o) => _o,
Err(e) => error!("{}", e),
};
}
}
}
}
}
// Send notifs to the parent commenter / poster
match data.parent_id {
Some(parent_id) => {
let parent_comment = Comment::read(&conn, parent_id)?;
if parent_comment.creator_id != user_id {
let parent_user = User_::read(&conn, parent_comment.creator_id)?;
recipient_ids.push(parent_user.id);
if parent_user.send_notifications_to_email {
if let Some(comment_reply_email) = parent_user.email {
let subject = &format!(
"{} - Reply from {}",
Settings::get().hostname,
claims.username
);
let html = &format!(
"<h1>Comment Reply</h1><br><div>{} - {}</div><br><a href={}/inbox>inbox</a>",
claims.username, comment_form.content, hostname
);
match send_email(subject, &comment_reply_email, &parent_user.name, html) {
Ok(_o) => _o,
Err(e) => error!("{}", e),
};
}
}
}
}
// Its a post
None => {
if post.creator_id != user_id {
let parent_user = User_::read(&conn, post.creator_id)?;
recipient_ids.push(parent_user.id);
if parent_user.send_notifications_to_email {
if let Some(post_reply_email) = parent_user.email {
let subject = &format!(
"{} - Reply from {}",
Settings::get().hostname,
claims.username
);
let html = &format!(
"<h1>Post Reply</h1><br><div>{} - {}</div><br><a href={}/inbox>inbox</a>",
claims.username, comment_form.content, hostname
);
match send_email(subject, &post_reply_email, &parent_user.name, html) {
Ok(_o) => _o,
Err(e) => error!("{}", e),
};
}
}
}
}
};
// You like your own comment by default
let like_form = CommentLikeForm {
@ -180,48 +223,22 @@ impl Perform for Oper<CreateComment> {
score: 1,
};
let like = move |conn: &'_ _| CommentLike::like(&conn, &like_form);
if blocking(pool, like).await?.is_err() {
return Err(APIError::err("couldnt_like_comment").into());
}
updated_comment.send_like(&user, &self.client, pool).await?;
let comment_view = blocking(pool, move |conn| {
CommentView::read(&conn, inserted_comment.id, Some(user_id))
})
.await??;
let mut res = CommentResponse {
comment: comment_view,
recipient_ids,
let _inserted_like = match CommentLike::like(&conn, &like_form) {
Ok(like) => like,
Err(_e) => return Err(APIError::err("couldnt_like_comment").into()),
};
if let Some(ws) = websocket_info {
ws.chatserver.do_send(SendComment {
op: UserOperation::CreateComment,
comment: res.clone(),
my_id: ws.id,
});
let comment_view = CommentView::read(&conn, inserted_comment.id, Some(user_id))?;
// strip out the recipient_ids, so that
// users don't get double notifs
res.recipient_ids = Vec::new();
}
Ok(res)
Ok(CommentResponse {
comment: comment_view,
recipient_ids,
})
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<EditComment> {
type Response = CommentResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<CommentResponse, LemmyError> {
impl Perform<CommentResponse> for Oper<EditComment> {
fn perform(&self, conn: &PgConnection) -> Result<CommentResponse, Error> {
let data: &EditComment = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -231,57 +248,38 @@ impl Perform for Oper<EditComment> {
let user_id = claims.id;
let user = blocking(pool, move |conn| User_::read(&conn, user_id)).await??;
let edit_id = data.edit_id;
let orig_comment =
blocking(pool, move |conn| CommentView::read(&conn, edit_id, None)).await??;
let orig_comment = CommentView::read(&conn, data.edit_id, None)?;
// You are allowed to mark the comment as read even if you're banned.
if data.read.is_none() {
// Verify its the creator or a mod, or an admin
let mut editors: Vec<i32> = vec![data.creator_id];
let community_id = orig_comment.community_id;
editors.append(
&mut blocking(pool, move |conn| {
Ok(
CommunityModeratorView::for_community(&conn, community_id)?
.into_iter()
.map(|m| m.user_id)
.collect(),
) as Result<_, LemmyError>
})
.await??,
);
editors.append(
&mut blocking(pool, move |conn| {
Ok(UserView::admins(conn)?.into_iter().map(|a| a.id).collect()) as Result<_, LemmyError>
})
.await??,
&mut CommunityModeratorView::for_community(&conn, orig_comment.community_id)?
.into_iter()
.map(|m| m.user_id)
.collect(),
);
editors.append(&mut UserView::admins(&conn)?.into_iter().map(|a| a.id).collect());
if !editors.contains(&user_id) {
return Err(APIError::err("no_comment_edit_allowed").into());
}
// Check for a community ban
let community_id = orig_comment.community_id;
let is_banned =
move |conn: &'_ _| CommunityUserBanView::get(conn, user_id, community_id).is_ok();
if blocking(pool, is_banned).await? {
if CommunityUserBanView::get(&conn, user_id, orig_comment.community_id).is_ok() {
return Err(APIError::err("community_ban").into());
}
// Check for a site ban
if user.banned {
if UserView::read(&conn, user_id)?.banned {
return Err(APIError::err("site_ban").into());
}
}
let content_slurs_removed = remove_slurs(&data.content.to_owned());
let edit_id = data.edit_id;
let read_comment = blocking(pool, move |conn| Comment::read(conn, edit_id)).await??;
let read_comment = Comment::read(&conn, data.edit_id)?;
let comment_form = CommentForm {
content: content_slurs_removed,
@ -291,7 +289,6 @@ impl Perform for Oper<EditComment> {
removed: data.removed.to_owned(),
deleted: data.deleted.to_owned(),
read: data.read.to_owned(),
published: None,
updated: if data.read.is_some() {
orig_comment.updated
} else {
@ -301,48 +298,58 @@ impl Perform for Oper<EditComment> {
local: read_comment.local,
};
let edit_id = data.edit_id;
let comment_form2 = comment_form.clone();
let updated_comment = match blocking(pool, move |conn| {
Comment::update(conn, edit_id, &comment_form2)
})
.await?
{
let _updated_comment = match Comment::update(&conn, data.edit_id, &comment_form) {
Ok(comment) => comment,
Err(_e) => return Err(APIError::err("couldnt_update_comment").into()),
};
if let Some(deleted) = data.deleted.to_owned() {
if deleted {
updated_comment
.send_delete(&user, &self.client, pool)
.await?;
} else {
updated_comment
.send_undo_delete(&user, &self.client, pool)
.await?;
let mut recipient_ids = Vec::new();
// Scan the comment for user mentions, add those rows
let extracted_usernames = extract_usernames(&comment_form.content);
for username_mention in &extracted_usernames {
let mention_user = User_::read_from_name(&conn, (*username_mention).to_string());
if mention_user.is_ok() {
let mention_user_id = mention_user?.id;
// You can't mention yourself
// At some point, make it so you can't tag the parent creator either
// This can cause two notifications, one for reply and the other for mention
if mention_user_id != user_id {
recipient_ids.push(mention_user_id);
let user_mention_form = UserMentionForm {
recipient_id: mention_user_id,
comment_id: data.edit_id,
read: None,
};
// Allow this to fail softly, since comment edits might re-update or replace it
// Let the uniqueness handle this fail
match UserMention::create(&conn, &user_mention_form) {
Ok(_mention) => (),
Err(_e) => error!("{}", &_e),
}
}
}
} else if let Some(removed) = data.removed.to_owned() {
if removed {
updated_comment
.send_remove(&user, &self.client, pool)
.await?;
} else {
updated_comment
.send_undo_remove(&user, &self.client, pool)
.await?;
}
} else {
updated_comment
.send_update(&user, &self.client, pool)
.await?;
}
let post_id = data.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let mentions = scrape_text_for_mentions(&comment_form.content);
let recipient_ids = send_local_notifs(mentions, updated_comment, user, post, pool).await?;
// Add to recipient ids
match data.parent_id {
Some(parent_id) => {
let parent_comment = Comment::read(&conn, parent_id)?;
if parent_comment.creator_id != user_id {
let parent_user = User_::read(&conn, parent_comment.creator_id)?;
recipient_ids.push(parent_user.id);
}
}
None => {
let post = Post::read(&conn, data.post_id)?;
recipient_ids.push(post.creator_id);
}
}
// Mod tables
if let Some(removed) = data.removed.to_owned() {
@ -352,45 +359,20 @@ impl Perform for Oper<EditComment> {
removed: Some(removed),
reason: data.reason.to_owned(),
};
blocking(pool, move |conn| ModRemoveComment::create(conn, &form)).await??;
ModRemoveComment::create(&conn, &form)?;
}
let edit_id = data.edit_id;
let comment_view = blocking(pool, move |conn| {
CommentView::read(conn, edit_id, Some(user_id))
})
.await??;
let comment_view = CommentView::read(&conn, data.edit_id, Some(user_id))?;
let mut res = CommentResponse {
Ok(CommentResponse {
comment: comment_view,
recipient_ids,
};
if let Some(ws) = websocket_info {
ws.chatserver.do_send(SendComment {
op: UserOperation::EditComment,
comment: res.clone(),
my_id: ws.id,
});
// strip out the recipient_ids, so that
// users don't get double notifs
res.recipient_ids = Vec::new();
}
Ok(res)
})
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<SaveComment> {
type Response = CommentResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<CommentResponse, LemmyError> {
impl Perform<CommentResponse> for Oper<SaveComment> {
fn perform(&self, conn: &PgConnection) -> Result<CommentResponse, Error> {
let data: &SaveComment = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -406,22 +388,18 @@ impl Perform for Oper<SaveComment> {
};
if data.save {
let save_comment = move |conn: &'_ _| CommentSaved::save(conn, &comment_saved_form);
if blocking(pool, save_comment).await?.is_err() {
return Err(APIError::err("couldnt_save_comment").into());
}
match CommentSaved::save(&conn, &comment_saved_form) {
Ok(comment) => comment,
Err(_e) => return Err(APIError::err("couldnt_save_comment").into()),
};
} else {
let unsave_comment = move |conn: &'_ _| CommentSaved::unsave(conn, &comment_saved_form);
if blocking(pool, unsave_comment).await?.is_err() {
return Err(APIError::err("couldnt_save_comment").into());
}
match CommentSaved::unsave(&conn, &comment_saved_form) {
Ok(comment) => comment,
Err(_e) => return Err(APIError::err("couldnt_save_comment").into()),
};
}
let comment_id = data.comment_id;
let comment_view = blocking(pool, move |conn| {
CommentView::read(conn, comment_id, Some(user_id))
})
.await??;
let comment_view = CommentView::read(&conn, data.comment_id, Some(user_id))?;
Ok(CommentResponse {
comment: comment_view,
@ -430,15 +408,8 @@ impl Perform for Oper<SaveComment> {
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<CreateCommentLike> {
type Response = CommentResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<CommentResponse, LemmyError> {
impl Perform<CommentResponse> for Oper<CreateCommentLike> {
fn perform(&self, conn: &PgConnection) -> Result<CommentResponse, Error> {
let data: &CreateCommentLike = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -452,40 +423,31 @@ impl Perform for Oper<CreateCommentLike> {
// Don't do a downvote if site has downvotes disabled
if data.score == -1 {
let site = blocking(pool, move |conn| SiteView::read(conn)).await??;
let site = SiteView::read(&conn)?;
if !site.enable_downvotes {
return Err(APIError::err("downvotes_disabled").into());
}
}
// Check for a community ban
let post_id = data.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let is_banned =
move |conn: &'_ _| CommunityUserBanView::get(conn, user_id, community_id).is_ok();
if blocking(pool, is_banned).await? {
let post = Post::read(&conn, data.post_id)?;
if CommunityUserBanView::get(&conn, user_id, post.community_id).is_ok() {
return Err(APIError::err("community_ban").into());
}
// Check for a site ban
let user = blocking(pool, move |conn| User_::read(conn, user_id)).await??;
if user.banned {
if UserView::read(&conn, user_id)?.banned {
return Err(APIError::err("site_ban").into());
}
let comment_id = data.comment_id;
let comment = blocking(pool, move |conn| Comment::read(conn, comment_id)).await??;
let comment = Comment::read(&conn, data.comment_id)?;
// Add to recipient ids
match comment.parent_id {
Some(parent_id) => {
let parent_comment = blocking(pool, move |conn| Comment::read(conn, parent_id)).await??;
let parent_comment = Comment::read(&conn, parent_id)?;
if parent_comment.creator_id != user_id {
let parent_user = blocking(pool, move |conn| {
User_::read(conn, parent_comment.creator_id)
})
.await??;
let parent_user = User_::read(&conn, parent_comment.creator_id)?;
recipient_ids.push(parent_user.id);
}
}
@ -502,64 +464,29 @@ impl Perform for Oper<CreateCommentLike> {
};
// Remove any likes first
let like_form2 = like_form.clone();
blocking(pool, move |conn| CommentLike::remove(conn, &like_form2)).await??;
CommentLike::remove(&conn, &like_form)?;
// Only add the like if the score isnt 0
let do_add = like_form.score != 0 && (like_form.score == 1 || like_form.score == -1);
if do_add {
let like_form2 = like_form.clone();
let like = move |conn: &'_ _| CommentLike::like(conn, &like_form2);
if blocking(pool, like).await?.is_err() {
return Err(APIError::err("couldnt_like_comment").into());
}
if like_form.score == 1 {
comment.send_like(&user, &self.client, pool).await?;
} else if like_form.score == -1 {
comment.send_dislike(&user, &self.client, pool).await?;
}
} else {
comment.send_undo_like(&user, &self.client, pool).await?;
let _inserted_like = match CommentLike::like(&conn, &like_form) {
Ok(like) => like,
Err(_e) => return Err(APIError::err("couldnt_like_comment").into()),
};
}
// Have to refetch the comment to get the current state
let comment_id = data.comment_id;
let liked_comment = blocking(pool, move |conn| {
CommentView::read(conn, comment_id, Some(user_id))
})
.await??;
let liked_comment = CommentView::read(&conn, data.comment_id, Some(user_id))?;
let mut res = CommentResponse {
Ok(CommentResponse {
comment: liked_comment,
recipient_ids,
};
if let Some(ws) = websocket_info {
ws.chatserver.do_send(SendComment {
op: UserOperation::CreateCommentLike,
comment: res.clone(),
my_id: ws.id,
});
// strip out the recipient_ids, so that
// users don't get double notifs
res.recipient_ids = Vec::new();
}
Ok(res)
})
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<GetComments> {
type Response = GetCommentsResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<GetCommentsResponse, LemmyError> {
impl Perform<GetCommentsResponse> for Oper<GetComments> {
fn perform(&self, conn: &PgConnection) -> Result<GetCommentsResponse, Error> {
let data: &GetComments = &self.data;
let user_claims: Option<Claims> = match &data.auth {
@ -578,157 +505,19 @@ impl Perform for Oper<GetComments> {
let type_ = ListingType::from_str(&data.type_)?;
let sort = SortType::from_str(&data.sort)?;
let community_id = data.community_id;
let page = data.page;
let limit = data.limit;
let comments = blocking(pool, move |conn| {
CommentQueryBuilder::create(conn)
.listing_type(type_)
.sort(&sort)
.for_community_id(community_id)
.my_user_id(user_id)
.page(page)
.limit(limit)
.list()
})
.await?;
let comments = match comments {
let comments = match CommentQueryBuilder::create(&conn)
.listing_type(type_)
.sort(&sort)
.for_community_id(data.community_id)
.my_user_id(user_id)
.page(data.page)
.limit(data.limit)
.list()
{
Ok(comments) => comments,
Err(_) => return Err(APIError::err("couldnt_get_comments").into()),
Err(_e) => return Err(APIError::err("couldnt_get_comments").into()),
};
if let Some(ws) = websocket_info {
// You don't need to join the specific community room, bc this is already handled by
// GetCommunity
if data.community_id.is_none() {
if let Some(id) = ws.id {
// 0 is the "all" community
ws.chatserver.do_send(JoinCommunityRoom {
community_id: 0,
id,
});
}
}
}
Ok(GetCommentsResponse { comments })
}
}
pub async fn send_local_notifs(
mentions: Vec<MentionData>,
comment: Comment,
user: User_,
post: Post,
pool: &DbPool,
) -> Result<Vec<i32>, LemmyError> {
let ids = blocking(pool, move |conn| {
do_send_local_notifs(conn, &mentions, &comment, &user, &post)
})
.await?;
Ok(ids)
}
fn do_send_local_notifs(
conn: &diesel::PgConnection,
mentions: &[MentionData],
comment: &Comment,
user: &User_,
post: &Post,
) -> Vec<i32> {
let mut recipient_ids = Vec::new();
let hostname = &format!("https://{}", Settings::get().hostname);
// Send the local mentions
for mention in mentions
.iter()
.filter(|m| m.is_local() && m.name.ne(&user.name))
.collect::<Vec<&MentionData>>()
{
if let Ok(mention_user) = User_::read_from_name(&conn, &mention.name) {
// TODO
// At some point, make it so you can't tag the parent creator either
// This can cause two notifications, one for reply and the other for mention
recipient_ids.push(mention_user.id);
let user_mention_form = UserMentionForm {
recipient_id: mention_user.id,
comment_id: comment.id,
read: None,
};
// Allow this to fail softly, since comment edits might re-update or replace it
// Let the uniqueness handle this fail
match UserMention::create(&conn, &user_mention_form) {
Ok(_mention) => (),
Err(_e) => error!("{}", &_e),
};
// Send an email to those users that have notifications on
if mention_user.send_notifications_to_email {
if let Some(mention_email) = mention_user.email {
let subject = &format!("{} - Mentioned by {}", Settings::get().hostname, user.name,);
let html = &format!(
"<h1>User Mention</h1><br><div>{} - {}</div><br><a href={}/inbox>inbox</a>",
user.name, comment.content, hostname
);
match send_email(subject, &mention_email, &mention_user.name, html) {
Ok(_o) => _o,
Err(e) => error!("{}", e),
};
}
}
}
}
// Send notifs to the parent commenter / poster
match comment.parent_id {
Some(parent_id) => {
if let Ok(parent_comment) = Comment::read(&conn, parent_id) {
if parent_comment.creator_id != user.id {
if let Ok(parent_user) = User_::read(&conn, parent_comment.creator_id) {
recipient_ids.push(parent_user.id);
if parent_user.send_notifications_to_email {
if let Some(comment_reply_email) = parent_user.email {
let subject = &format!("{} - Reply from {}", Settings::get().hostname, user.name,);
let html = &format!(
"<h1>Comment Reply</h1><br><div>{} - {}</div><br><a href={}/inbox>inbox</a>",
user.name, comment.content, hostname
);
match send_email(subject, &comment_reply_email, &parent_user.name, html) {
Ok(_o) => _o,
Err(e) => error!("{}", e),
};
}
}
}
}
}
}
// Its a post
None => {
if post.creator_id != user.id {
if let Ok(parent_user) = User_::read(&conn, post.creator_id) {
recipient_ids.push(parent_user.id);
if parent_user.send_notifications_to_email {
if let Some(post_reply_email) = parent_user.email {
let subject = &format!("{} - Reply from {}", Settings::get().hostname, user.name,);
let html = &format!(
"<h1>Post Reply</h1><br><div>{} - {}</div><br><a href={}/inbox>inbox</a>",
user.name, comment.content, hostname
);
match send_email(subject, &post_reply_email, &parent_user.name, html) {
Ok(_o) => _o,
Err(e) => error!("{}", e),
};
}
}
}
}
}
};
recipient_ids
}

554
server/src/api/community.rs
View File

@ -1,28 +1,8 @@
use super::*;
use crate::{
api::{APIError, Oper, Perform},
apub::{
extensions::signatures::generate_actor_keypair,
make_apub_endpoint,
ActorType,
EndpointType,
},
blocking,
db::{Bannable, Crud, Followable, Joinable, SortType},
is_valid_community_name,
naive_from_unix,
naive_now,
slur_check,
slurs_vec_to_str,
websocket::{
server::{JoinCommunityRoom, SendCommunityRoomMessage},
UserOperation,
WebsocketInfo,
},
DbPool,
LemmyError,
};
use serde::{Deserialize, Serialize};
use crate::apub::puller::{fetch_all_communities, fetch_remote_community};
use crate::apub::{gen_keypair_str, make_apub_endpoint, EndpointType};
use crate::settings::Settings;
use diesel::PgConnection;
use std::str::FromStr;
#[derive(Serialize, Deserialize)]
@ -61,6 +41,7 @@ pub struct ListCommunities {
pub page: Option<i64>,
pub limit: Option<i64>,
pub auth: Option<String>,
pub local_only: Option<bool>,
}
#[derive(Serialize, Deserialize, Debug)]
@ -78,7 +59,7 @@ pub struct BanFromCommunity {
auth: String,
}
#[derive(Serialize, Deserialize, Clone)]
#[derive(Serialize, Deserialize)]
pub struct BanFromCommunityResponse {
user: UserView,
banned: bool,
@ -92,7 +73,7 @@ pub struct AddModToCommunity {
auth: String,
}
#[derive(Serialize, Deserialize, Clone)]
#[derive(Serialize, Deserialize)]
pub struct AddModToCommunityResponse {
moderators: Vec<CommunityModeratorView>,
}
@ -136,17 +117,17 @@ pub struct TransferCommunity {
auth: String,
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<GetCommunity> {
type Response = GetCommunityResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<GetCommunityResponse, LemmyError> {
impl Perform<GetCommunityResponse> for Oper<GetCommunity> {
fn perform(&self, conn: &PgConnection) -> Result<GetCommunityResponse, Error> {
let data: &GetCommunity = &self.data;
if data.name.is_some()
&& Settings::get().federation.enabled
&& data.name.as_ref().unwrap().contains('@')
{
return fetch_remote_community(data.name.as_ref().unwrap());
}
let user_id: Option<i32> = match &data.auth {
Some(auth) => match Claims::decode(&auth) {
Ok(claims) => {
@ -158,81 +139,47 @@ impl Perform for Oper<GetCommunity> {
None => None,
};
let name = data.name.to_owned().unwrap_or_else(|| "main".to_string());
let community = match data.id {
Some(id) => blocking(pool, move |conn| Community::read(conn, id)).await??,
None => match blocking(pool, move |conn| Community::read_from_name(conn, &name)).await? {
Ok(community) => community,
Err(_e) => return Err(APIError::err("couldnt_find_community").into()),
},
let community_id = match data.id {
Some(id) => id,
None => {
match Community::read_from_name(
&conn,
data.name.to_owned().unwrap_or_else(|| "main".to_string()),
) {
Ok(community) => community.id,
Err(_e) => return Err(APIError::err("couldnt_find_community").into()),
}
}
};
let community_id = community.id;
let community_view = match blocking(pool, move |conn| {
CommunityView::read(conn, community_id, user_id)
})
.await?
{
let community_view = match CommunityView::read(&conn, community_id, user_id) {
Ok(community) => community,
Err(_e) => return Err(APIError::err("couldnt_find_community").into()),
};
let community_id = community.id;
let moderators: Vec<CommunityModeratorView> = match blocking(pool, move |conn| {
CommunityModeratorView::for_community(conn, community_id)
})
.await?
{
let moderators = match CommunityModeratorView::for_community(&conn, community_id) {
Ok(moderators) => moderators,
Err(_e) => return Err(APIError::err("couldnt_find_community").into()),
};
let site = blocking(pool, move |conn| Site::read(conn, 1)).await??;
let site_creator_id = site.creator_id;
let mut admins = blocking(pool, move |conn| UserView::admins(conn)).await??;
let site_creator_id = Site::read(&conn, 1)?.creator_id;
let mut admins = UserView::admins(&conn)?;
let creator_index = admins.iter().position(|r| r.id == site_creator_id).unwrap();
let creator_user = admins.remove(creator_index);
admins.insert(0, creator_user);
let online = if let Some(ws) = websocket_info {
if let Some(id) = ws.id {
ws.chatserver.do_send(JoinCommunityRoom {
community_id: community.id,
id,
});
}
// TODO
1
// let fut = async {
// ws.chatserver.send(GetCommunityUsersOnline {community_id}).await.unwrap()
// };
// Runtime::new().unwrap().block_on(fut)
} else {
0
};
let res = GetCommunityResponse {
// Return the jwt
Ok(GetCommunityResponse {
community: community_view,
moderators,
admins,
online,
};
// Return the jwt
Ok(res)
online: 0,
})
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<CreateCommunity> {
type Response = CommunityResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<CommunityResponse, LemmyError> {
impl Perform<CommunityResponse> for Oper<CreateCommunity> {
fn perform(&self, conn: &PgConnection) -> Result<CommunityResponse, Error> {
let data: &CreateCommunity = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -254,20 +201,15 @@ impl Perform for Oper<CreateCommunity> {
}
}
if !is_valid_community_name(&data.name) {
return Err(APIError::err("invalid_community_name").into());
}
let user_id = claims.id;
// Check for a site ban
let user_view = blocking(pool, move |conn| UserView::read(conn, user_id)).await??;
if user_view.banned {
if UserView::read(&conn, user_id)?.banned {
return Err(APIError::err("site_ban").into());
}
// When you create a community, make sure the user becomes a moderator and a follower
let keypair = generate_actor_keypair()?;
let (community_public_key, community_private_key) = gen_keypair_str();
let community_form = CommunityForm {
name: data.name.to_owned(),
@ -281,42 +223,39 @@ impl Perform for Oper<CreateCommunity> {
updated: None,
actor_id: make_apub_endpoint(EndpointType::Community, &data.name).to_string(),
local: true,
private_key: Some(keypair.private_key),
public_key: Some(keypair.public_key),
private_key: Some(community_private_key),
public_key: Some(community_public_key),
last_refreshed_at: None,
published: None,
};
let inserted_community =
match blocking(pool, move |conn| Community::create(conn, &community_form)).await? {
Ok(community) => community,
Err(_e) => return Err(APIError::err("community_already_exists").into()),
};
let inserted_community = match Community::create(&conn, &community_form) {
Ok(community) => community,
Err(_e) => return Err(APIError::err("community_already_exists").into()),
};
let community_moderator_form = CommunityModeratorForm {
community_id: inserted_community.id,
user_id,
};
let join = move |conn: &'_ _| CommunityModerator::join(conn, &community_moderator_form);
if blocking(pool, join).await?.is_err() {
return Err(APIError::err("community_moderator_already_exists").into());
}
let _inserted_community_moderator =
match CommunityModerator::join(&conn, &community_moderator_form) {
Ok(user) => user,
Err(_e) => return Err(APIError::err("community_moderator_already_exists").into()),
};
let community_follower_form = CommunityFollowerForm {
community_id: inserted_community.id,
user_id,
};
let follow = move |conn: &'_ _| CommunityFollower::follow(conn, &community_follower_form);
if blocking(pool, follow).await?.is_err() {
return Err(APIError::err("community_follower_already_exists").into());
}
let _inserted_community_follower =
match CommunityFollower::follow(&conn, &community_follower_form) {
Ok(user) => user,
Err(_e) => return Err(APIError::err("community_follower_already_exists").into()),
};
let community_view = blocking(pool, move |conn| {
CommunityView::read(conn, inserted_community.id, Some(user_id))
})
.await??;
let community_view = CommunityView::read(&conn, inserted_community.id, Some(user_id))?;
Ok(CommunityResponse {
community: community_view,
@ -324,15 +263,8 @@ impl Perform for Oper<CreateCommunity> {
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<EditCommunity> {
type Response = CommunityResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<CommunityResponse, LemmyError> {
impl Perform<CommunityResponse> for Oper<EditCommunity> {
fn perform(&self, conn: &PgConnection) -> Result<CommunityResponse, Error> {
let data: &EditCommunity = &self.data;
if let Err(slurs) = slur_check(&data.name) {
@ -354,40 +286,27 @@ impl Perform for Oper<EditCommunity> {
Err(_e) => return Err(APIError::err("not_logged_in").into()),
};
if !is_valid_community_name(&data.name) {
return Err(APIError::err("invalid_community_name").into());
}
let user_id = claims.id;
// Check for a site ban
let user = blocking(pool, move |conn| User_::read(conn, user_id)).await??;
if user.banned {
if UserView::read(&conn, user_id)?.banned {
return Err(APIError::err("site_ban").into());
}
// Verify its a mod
let edit_id = data.edit_id;
let mut editors: Vec<i32> = Vec::new();
editors.append(
&mut blocking(pool, move |conn| {
CommunityModeratorView::for_community(conn, edit_id)
.map(|v| v.into_iter().map(|m| m.user_id).collect())
})
.await??,
);
editors.append(
&mut blocking(pool, move |conn| {
UserView::admins(conn).map(|v| v.into_iter().map(|a| a.id).collect())
})
.await??,
&mut CommunityModeratorView::for_community(&conn, data.edit_id)?
.into_iter()
.map(|m| m.user_id)
.collect(),
);
editors.append(&mut UserView::admins(&conn)?.into_iter().map(|a| a.id).collect());
if !editors.contains(&user_id) {
return Err(APIError::err("no_community_edit_allowed").into());
}
let edit_id = data.edit_id;
let read_community = blocking(pool, move |conn| Community::read(conn, edit_id)).await??;
let read_community = Community::read(&conn, data.edit_id)?;
let community_form = CommunityForm {
name: data.name.to_owned(),
@ -404,15 +323,9 @@ impl Perform for Oper<EditCommunity> {
private_key: read_community.private_key,
public_key: read_community.public_key,
last_refreshed_at: None,
published: None,
};
let edit_id = data.edit_id;
let updated_community = match blocking(pool, move |conn| {
Community::update(conn, edit_id, &community_form)
})
.await?
{
let _updated_community = match Community::update(&conn, data.edit_id, &community_form) {
Ok(community) => community,
Err(_e) => return Err(APIError::err("couldnt_update_community").into()),
};
@ -430,70 +343,28 @@ impl Perform for Oper<EditCommunity> {
reason: data.reason.to_owned(),
expires,
};
blocking(pool, move |conn| ModRemoveCommunity::create(conn, &form)).await??;
ModRemoveCommunity::create(&conn, &form)?;
}
if let Some(deleted) = data.deleted.to_owned() {
if deleted {
updated_community
.send_delete(&user, &self.client, pool)
.await?;
} else {
updated_community
.send_undo_delete(&user, &self.client, pool)
.await?;
}
} else if let Some(removed) = data.removed.to_owned() {
if removed {
updated_community
.send_remove(&user, &self.client, pool)
.await?;
} else {
updated_community
.send_undo_remove(&user, &self.client, pool)
.await?;
}
}
let community_view = CommunityView::read(&conn, data.edit_id, Some(user_id))?;
let edit_id = data.edit_id;
let community_view = blocking(pool, move |conn| {
CommunityView::read(conn, edit_id, Some(user_id))
})
.await??;
let res = CommunityResponse {
Ok(CommunityResponse {
community: community_view,
};
if let Some(ws) = websocket_info {
// Strip out the user id and subscribed when sending to others
let mut res_sent = res.clone();
res_sent.community.user_id = None;
res_sent.community.subscribed = None;
ws.chatserver.do_send(SendCommunityRoomMessage {
op: UserOperation::EditCommunity,
response: res_sent,
community_id: data.edit_id,
my_id: ws.id,
});
}
Ok(res)
})
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<ListCommunities> {
type Response = ListCommunitiesResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<ListCommunitiesResponse, LemmyError> {
impl Perform<ListCommunitiesResponse> for Oper<ListCommunities> {
fn perform(&self, conn: &PgConnection) -> Result<ListCommunitiesResponse, Error> {
let data: &ListCommunities = &self.data;
let local_only = data.local_only.unwrap_or(false);
if Settings::get().federation.enabled && !local_only {
return Ok(ListCommunitiesResponse {
communities: fetch_all_communities()?,
});
}
let user_claims: Option<Claims> = match &data.auth {
Some(auth) => match Claims::decode(&auth) {
Ok(claims) => Some(claims.claims),
@ -514,33 +385,21 @@ impl Perform for Oper<ListCommunities> {
let sort = SortType::from_str(&data.sort)?;
let page = data.page;
let limit = data.limit;
let communities = blocking(pool, move |conn| {
CommunityQueryBuilder::create(conn)
.sort(&sort)
.for_user(user_id)
.show_nsfw(show_nsfw)
.page(page)
.limit(limit)
.list()
})
.await??;
let communities = CommunityQueryBuilder::create(&conn)
.sort(&sort)
.for_user(user_id)
.show_nsfw(show_nsfw)
.page(data.page)
.limit(data.limit)
.list()?;
// Return the jwt
Ok(ListCommunitiesResponse { communities })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<FollowCommunity> {
type Response = CommunityResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<CommunityResponse, LemmyError> {
impl Perform<CommunityResponse> for Oper<FollowCommunity> {
fn perform(&self, conn: &PgConnection) -> Result<CommunityResponse, Error> {
let data: &FollowCommunity = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -550,53 +409,24 @@ impl Perform for Oper<FollowCommunity> {
let user_id = claims.id;
let community_id = data.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let community_follower_form = CommunityFollowerForm {
community_id: data.community_id,
user_id,
};
if community.local {
if data.follow {
let follow = move |conn: &'_ _| CommunityFollower::follow(conn, &community_follower_form);
if blocking(pool, follow).await?.is_err() {
return Err(APIError::err("community_follower_already_exists").into());
}
} else {
let unfollow =
move |conn: &'_ _| CommunityFollower::unfollow(conn, &community_follower_form);
if blocking(pool, unfollow).await?.is_err() {
return Err(APIError::err("community_follower_already_exists").into());
}
}
if data.follow {
match CommunityFollower::follow(&conn, &community_follower_form) {
Ok(user) => user,
Err(_e) => return Err(APIError::err("community_follower_already_exists").into()),
};
} else {
let user = blocking(pool, move |conn| User_::read(conn, user_id)).await??;
if data.follow {
// Dont actually add to the community followers here, because you need
// to wait for the accept
user
.send_follow(&community.actor_id, &self.client, pool)
.await?;
} else {
user
.send_unfollow(&community.actor_id, &self.client, pool)
.await?;
let unfollow =
move |conn: &'_ _| CommunityFollower::unfollow(conn, &community_follower_form);
if blocking(pool, unfollow).await?.is_err() {
return Err(APIError::err("community_follower_already_exists").into());
}
}
// TODO: this needs to return a "pending" state, until Accept is received from the remote server
match CommunityFollower::ignore(&conn, &community_follower_form) {
Ok(user) => user,
Err(_e) => return Err(APIError::err("community_follower_already_exists").into()),
};
}
let community_id = data.community_id;
let community_view = blocking(pool, move |conn| {
CommunityView::read(conn, community_id, Some(user_id))
})
.await??;
let community_view = CommunityView::read(&conn, data.community_id, Some(user_id))?;
Ok(CommunityResponse {
community: community_view,
@ -604,15 +434,8 @@ impl Perform for Oper<FollowCommunity> {
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<GetFollowedCommunities> {
type Response = GetFollowedCommunitiesResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<GetFollowedCommunitiesResponse, LemmyError> {
impl Perform<GetFollowedCommunitiesResponse> for Oper<GetFollowedCommunities> {
fn perform(&self, conn: &PgConnection) -> Result<GetFollowedCommunitiesResponse, Error> {
let data: &GetFollowedCommunities = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -622,29 +445,19 @@ impl Perform for Oper<GetFollowedCommunities> {
let user_id = claims.id;
let communities = match blocking(pool, move |conn| {
CommunityFollowerView::for_user(conn, user_id)
})
.await?
{
Ok(communities) => communities,
_ => return Err(APIError::err("system_err_login").into()),
};
let communities: Vec<CommunityFollowerView> =
match CommunityFollowerView::for_user(&conn, user_id) {
Ok(communities) => communities,
Err(_e) => return Err(APIError::err("system_err_login").into()),
};
// Return the jwt
Ok(GetFollowedCommunitiesResponse { communities })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<BanFromCommunity> {
type Response = BanFromCommunityResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<BanFromCommunityResponse, LemmyError> {
impl Perform<BanFromCommunityResponse> for Oper<BanFromCommunity> {
fn perform(&self, conn: &PgConnection) -> Result<BanFromCommunityResponse, Error> {
let data: &BanFromCommunity = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -660,15 +473,15 @@ impl Perform for Oper<BanFromCommunity> {
};
if data.ban {
let ban = move |conn: &'_ _| CommunityUserBan::ban(conn, &community_user_ban_form);
if blocking(pool, ban).await?.is_err() {
return Err(APIError::err("community_user_already_banned").into());
}
match CommunityUserBan::ban(&conn, &community_user_ban_form) {
Ok(user) => user,
Err(_e) => return Err(APIError::err("community_user_already_banned").into()),
};
} else {
let unban = move |conn: &'_ _| CommunityUserBan::unban(conn, &community_user_ban_form);
if blocking(pool, unban).await?.is_err() {
return Err(APIError::err("community_user_already_banned").into());
}
match CommunityUserBan::unban(&conn, &community_user_ban_form) {
Ok(user) => user,
Err(_e) => return Err(APIError::err("community_user_already_banned").into()),
};
}
// Mod tables
@ -685,38 +498,19 @@ impl Perform for Oper<BanFromCommunity> {
banned: Some(data.ban),
expires,
};
blocking(pool, move |conn| ModBanFromCommunity::create(conn, &form)).await??;
ModBanFromCommunity::create(&conn, &form)?;
let user_id = data.user_id;
let user_view = blocking(pool, move |conn| UserView::read(conn, user_id)).await??;
let user_view = UserView::read(&conn, data.user_id)?;
let res = BanFromCommunityResponse {
Ok(BanFromCommunityResponse {
user: user_view,
banned: data.ban,
};
if let Some(ws) = websocket_info {
ws.chatserver.do_send(SendCommunityRoomMessage {
op: UserOperation::BanFromCommunity,
response: res.clone(),
community_id: data.community_id,
my_id: ws.id,
});
}
Ok(res)
})
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<AddModToCommunity> {
type Response = AddModToCommunityResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<AddModToCommunityResponse, LemmyError> {
impl Perform<AddModToCommunityResponse> for Oper<AddModToCommunity> {
fn perform(&self, conn: &PgConnection) -> Result<AddModToCommunityResponse, Error> {
let data: &AddModToCommunity = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -732,15 +526,15 @@ impl Perform for Oper<AddModToCommunity> {
};
if data.added {
let join = move |conn: &'_ _| CommunityModerator::join(conn, &community_moderator_form);
if blocking(pool, join).await?.is_err() {
return Err(APIError::err("community_moderator_already_exists").into());
}
match CommunityModerator::join(&conn, &community_moderator_form) {
Ok(user) => user,
Err(_e) => return Err(APIError::err("community_moderator_already_exists").into()),
};
} else {
let leave = move |conn: &'_ _| CommunityModerator::leave(conn, &community_moderator_form);
if blocking(pool, leave).await?.is_err() {
return Err(APIError::err("community_moderator_already_exists").into());
}
match CommunityModerator::leave(&conn, &community_moderator_form) {
Ok(user) => user,
Err(_e) => return Err(APIError::err("community_moderator_already_exists").into()),
};
}
// Mod tables
@ -750,38 +544,16 @@ impl Perform for Oper<AddModToCommunity> {
community_id: data.community_id,
removed: Some(!data.added),
};
blocking(pool, move |conn| ModAddCommunity::create(conn, &form)).await??;
ModAddCommunity::create(&conn, &form)?;
let community_id = data.community_id;
let moderators = blocking(pool, move |conn| {
CommunityModeratorView::for_community(conn, community_id)
})
.await??;
let moderators = CommunityModeratorView::for_community(&conn, data.community_id)?;
let res = AddModToCommunityResponse { moderators };
if let Some(ws) = websocket_info {
ws.chatserver.do_send(SendCommunityRoomMessage {
op: UserOperation::AddModToCommunity,
response: res.clone(),
community_id: data.community_id,
my_id: ws.id,
});
}
Ok(res)
Ok(AddModToCommunityResponse { moderators })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<TransferCommunity> {
type Response = GetCommunityResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<GetCommunityResponse, LemmyError> {
impl Perform<GetCommunityResponse> for Oper<TransferCommunity> {
fn perform(&self, conn: &PgConnection) -> Result<GetCommunityResponse, Error> {
let data: &TransferCommunity = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -791,14 +563,10 @@ impl Perform for Oper<TransferCommunity> {
let user_id = claims.id;
let community_id = data.community_id;
let read_community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let site_creator_id =
blocking(pool, move |conn| Site::read(conn, 1).map(|s| s.creator_id)).await??;
let mut admins = blocking(pool, move |conn| UserView::admins(conn)).await??;
let read_community = Community::read(&conn, data.community_id)?;
let site_creator_id = Site::read(&conn, 1)?.creator_id;
let mut admins = UserView::admins(&conn)?;
let creator_index = admins.iter().position(|r| r.id == site_creator_id).unwrap();
let creator_user = admins.remove(creator_index);
admins.insert(0, creator_user);
@ -813,7 +581,7 @@ impl Perform for Oper<TransferCommunity> {
title: read_community.title,
description: read_community.description,
category_id: read_community.category_id,
creator_id: data.user_id, // This makes the new user the community creator
creator_id: data.user_id,
removed: None,
deleted: None,
nsfw: read_community.nsfw,
@ -823,21 +591,15 @@ impl Perform for Oper<TransferCommunity> {
private_key: read_community.private_key,
public_key: read_community.public_key,
last_refreshed_at: None,
published: None,
};
let community_id = data.community_id;
let update = move |conn: &'_ _| Community::update(conn, community_id, &community_form);
if blocking(pool, update).await?.is_err() {
return Err(APIError::err("couldnt_update_community").into());
let _updated_community = match Community::update(&conn, data.community_id, &community_form) {
Ok(community) => community,
Err(_e) => return Err(APIError::err("couldnt_update_community").into()),
};
// You also have to re-do the community_moderator table, reordering it.
let community_id = data.community_id;
let mut community_mods = blocking(pool, move |conn| {
CommunityModeratorView::for_community(conn, community_id)
})
.await??;
let mut community_mods = CommunityModeratorView::for_community(&conn, data.community_id)?;
let creator_index = community_mods
.iter()
.position(|r| r.user_id == data.user_id)
@ -845,23 +607,19 @@ impl Perform for Oper<TransferCommunity> {
let creator_user = community_mods.remove(creator_index);
community_mods.insert(0, creator_user);
let community_id = data.community_id;
blocking(pool, move |conn| {
CommunityModerator::delete_for_community(conn, community_id)
})
.await??;
CommunityModerator::delete_for_community(&conn, data.community_id)?;
// TODO: this should probably be a bulk operation
for cmod in &community_mods {
let community_moderator_form = CommunityModeratorForm {
community_id: cmod.community_id,
user_id: cmod.user_id,
};
let join = move |conn: &'_ _| CommunityModerator::join(conn, &community_moderator_form);
if blocking(pool, join).await?.is_err() {
return Err(APIError::err("community_moderator_already_exists").into());
}
let _inserted_community_moderator =
match CommunityModerator::join(&conn, &community_moderator_form) {
Ok(user) => user,
Err(_e) => return Err(APIError::err("community_moderator_already_exists").into()),
};
}
// Mod tables
@ -871,24 +629,14 @@ impl Perform for Oper<TransferCommunity> {
community_id: data.community_id,
removed: Some(false),
};
blocking(pool, move |conn| ModAddCommunity::create(conn, &form)).await??;
ModAddCommunity::create(&conn, &form)?;
let community_id = data.community_id;
let community_view = match blocking(pool, move |conn| {
CommunityView::read(conn, community_id, Some(user_id))
})
.await?
{
let community_view = match CommunityView::read(&conn, data.community_id, Some(user_id)) {
Ok(community) => community,
Err(_e) => return Err(APIError::err("couldnt_find_community").into()),
};
let community_id = data.community_id;
let moderators = match blocking(pool, move |conn| {
CommunityModeratorView::for_community(conn, community_id)
})
.await?
{
let moderators = match CommunityModeratorView::for_community(&conn, data.community_id) {
Ok(moderators) => moderators,
Err(_e) => return Err(APIError::err("couldnt_find_community").into()),
};

49
server/src/api/mod.rs
View File

@ -1,10 +1,29 @@
use crate::db::category::*;
use crate::db::comment::*;
use crate::db::comment_view::*;
use crate::db::community::*;
use crate::db::community_view::*;
use crate::db::moderator::*;
use crate::db::moderator_views::*;
use crate::db::password_reset_request::*;
use crate::db::post::*;
use crate::db::post_view::*;
use crate::db::private_message::*;
use crate::db::private_message_view::*;
use crate::db::site::*;
use crate::db::site_view::*;
use crate::db::user::*;
use crate::db::user_mention::*;
use crate::db::user_mention_view::*;
use crate::db::user_view::*;
use crate::db::*;
use crate::{
db::{community::*, community_view::*, moderator::*, site::*, user::*, user_view::*},
websocket::WebsocketInfo,
DbPool,
LemmyError,
extract_usernames, fetch_iframely_and_pictshare_data, naive_from_unix, naive_now, remove_slurs,
slur_check, slurs_vec_to_str,
};
use actix_web::client::Client;
use diesel::PgConnection;
use failure::Error;
use serde::{Deserialize, Serialize};
pub mod comment;
pub mod community;
@ -28,22 +47,16 @@ impl APIError {
pub struct Oper<T> {
data: T,
client: Client,
}
impl<Data> Oper<Data> {
pub fn new(data: Data, client: Client) -> Oper<Data> {
Oper { data, client }
impl<T> Oper<T> {
pub fn new(data: T) -> Oper<T> {
Oper { data }
}
}
#[async_trait::async_trait(?Send)]
pub trait Perform {
type Response: serde::ser::Serialize + Send;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<Self::Response, LemmyError>;
pub trait Perform<T> {
fn perform(&self, conn: &PgConnection) -> Result<T, Error>
where
T: Sized;
}

426
server/src/api/post.rs
View File

@ -1,39 +1,9 @@
use crate::{
api::{APIError, Oper, Perform},
apub::{ApubLikeableType, ApubObjectType},
blocking,
db::{
comment_view::*,
community_view::*,
moderator::*,
post::*,
post_view::*,
site::*,
site_view::*,
user::*,
user_view::*,
Crud,
Likeable,
ListingType,
Saveable,
SortType,
},
fetch_iframely_and_pictrs_data,
naive_now,
slur_check,
slurs_vec_to_str,
websocket::{
server::{JoinCommunityRoom, JoinPostRoom, SendPost},
UserOperation,
WebsocketInfo,
},
DbPool,
LemmyError,
};
use serde::{Deserialize, Serialize};
use super::*;
use crate::settings::Settings;
use diesel::PgConnection;
use std::str::FromStr;
#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize)]
pub struct CreatePost {
name: String,
url: Option<String>,
@ -110,15 +80,8 @@ pub struct SavePost {
auth: String,
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<CreatePost> {
type Response = PostResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<PostResponse, LemmyError> {
impl Perform<PostResponse> for Oper<CreatePost> {
fn perform(&self, conn: &PgConnection) -> Result<PostResponse, Error> {
let data: &CreatePost = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -139,22 +102,18 @@ impl Perform for Oper<CreatePost> {
let user_id = claims.id;
// Check for a community ban
let community_id = data.community_id;
let is_banned =
move |conn: &'_ _| CommunityUserBanView::get(conn, user_id, community_id).is_ok();
if blocking(pool, is_banned).await? {
if CommunityUserBanView::get(&conn, user_id, data.community_id).is_ok() {
return Err(APIError::err("community_ban").into());
}
// Check for a site ban
let user = blocking(pool, move |conn| User_::read(conn, user_id)).await??;
if user.banned {
if UserView::read(&conn, user_id)?.banned {
return Err(APIError::err("site_ban").into());
}
// Fetch Iframely and pictrs cached image
let (iframely_title, iframely_description, iframely_html, pictrs_thumbnail) =
fetch_iframely_and_pictrs_data(&self.client, data.url.to_owned()).await;
// Fetch Iframely and Pictshare cached image
let (iframely_title, iframely_description, iframely_html, pictshare_thumbnail) =
fetch_iframely_and_pictshare_data(data.url.to_owned());
let post_form = PostForm {
name: data.name.to_owned(),
@ -171,13 +130,12 @@ impl Perform for Oper<CreatePost> {
embed_title: iframely_title,
embed_description: iframely_description,
embed_html: iframely_html,
thumbnail_url: pictrs_thumbnail,
ap_id: "http://fake.com".into(),
thumbnail_url: pictshare_thumbnail,
ap_id: "changeme".into(),
local: true,
published: None,
};
let inserted_post = match blocking(pool, move |conn| Post::create(conn, &post_form)).await? {
let inserted_post = match Post::create(&conn, &post_form) {
Ok(post) => post,
Err(e) => {
let err_type = if e.to_string() == "value too long for type character varying(200)" {
@ -190,14 +148,10 @@ impl Perform for Oper<CreatePost> {
}
};
let inserted_post_id = inserted_post.id;
let updated_post =
match blocking(pool, move |conn| Post::update_ap_id(conn, inserted_post_id)).await? {
Ok(post) => post,
Err(_e) => return Err(APIError::err("couldnt_create_post").into()),
};
updated_post.send_create(&user, &self.client, pool).await?;
match Post::update_ap_id(&conn, inserted_post.id) {
Ok(post) => post,
Err(_e) => return Err(APIError::err("couldnt_create_post").into()),
};
// They like their own post by default
let like_form = PostLikeForm {
@ -206,47 +160,24 @@ impl Perform for Oper<CreatePost> {
score: 1,
};
let like = move |conn: &'_ _| PostLike::like(conn, &like_form);
if blocking(pool, like).await?.is_err() {
return Err(APIError::err("couldnt_like_post").into());
}
updated_post.send_like(&user, &self.client, pool).await?;
// Only add the like if the score isnt 0
let _inserted_like = match PostLike::like(&conn, &like_form) {
Ok(like) => like,
Err(_e) => return Err(APIError::err("couldnt_like_post").into()),
};
// Refetch the view
let inserted_post_id = inserted_post.id;
let post_view = match blocking(pool, move |conn| {
PostView::read(conn, inserted_post_id, Some(user_id))
})
.await?
{
let post_view = match PostView::read(&conn, inserted_post.id, Some(user_id)) {
Ok(post) => post,
Err(_e) => return Err(APIError::err("couldnt_find_post").into()),
};
let res = PostResponse { post: post_view };
if let Some(ws) = websocket_info {
ws.chatserver.do_send(SendPost {
op: UserOperation::CreatePost,
post: res.clone(),
my_id: ws.id,
});
}
Ok(res)
Ok(PostResponse { post: post_view })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<GetPost> {
type Response = GetPostResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<GetPostResponse, LemmyError> {
impl Perform<GetPostResponse> for Oper<GetPost> {
fn perform(&self, conn: &PgConnection) -> Result<GetPostResponse, Error> {
let data: &GetPost = &self.data;
let user_id: Option<i32> = match &data.auth {
@ -260,60 +191,27 @@ impl Perform for Oper<GetPost> {
None => None,
};
let id = data.id;
let post_view = match blocking(pool, move |conn| PostView::read(conn, id, user_id)).await? {
let post_view = match PostView::read(&conn, data.id, user_id) {
Ok(post) => post,
Err(_e) => return Err(APIError::err("couldnt_find_post").into()),
};
let id = data.id;
let comments = blocking(pool, move |conn| {
CommentQueryBuilder::create(conn)
.for_post_id(id)
.my_user_id(user_id)
.limit(9999)
.list()
})
.await??;
let comments = CommentQueryBuilder::create(&conn)
.for_post_id(data.id)
.my_user_id(user_id)
.limit(9999)
.list()?;
let community_id = post_view.community_id;
let community = blocking(pool, move |conn| {
CommunityView::read(conn, community_id, user_id)
})
.await??;
let community = CommunityView::read(&conn, post_view.community_id, user_id)?;
let community_id = post_view.community_id;
let moderators = blocking(pool, move |conn| {
CommunityModeratorView::for_community(conn, community_id)
})
.await??;
let moderators = CommunityModeratorView::for_community(&conn, post_view.community_id)?;
let site_creator_id =
blocking(pool, move |conn| Site::read(conn, 1).map(|s| s.creator_id)).await??;
let mut admins = blocking(pool, move |conn| UserView::admins(conn)).await??;
let site_creator_id = Site::read(&conn, 1)?.creator_id;
let mut admins = UserView::admins(&conn)?;
let creator_index = admins.iter().position(|r| r.id == site_creator_id).unwrap();
let creator_user = admins.remove(creator_index);
admins.insert(0, creator_user);
let online = if let Some(ws) = websocket_info {
if let Some(id) = ws.id {
ws.chatserver.do_send(JoinPostRoom {
post_id: data.id,
id,
});
}
// TODO
1
// let fut = async {
// ws.chatserver.send(GetPostUsersOnline {post_id: data.id}).await.unwrap()
// };
// Runtime::new().unwrap().block_on(fut)
} else {
0
};
// Return the jwt
Ok(GetPostResponse {
post: post_view,
@ -321,22 +219,20 @@ impl Perform for Oper<GetPost> {
community,
moderators,
admins,
online,
online: 0,
})
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<GetPosts> {
type Response = GetPostsResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<GetPostsResponse, LemmyError> {
impl Perform<GetPostsResponse> for Oper<GetPosts> {
fn perform(&self, conn: &PgConnection) -> Result<GetPostsResponse, Error> {
let data: &GetPosts = &self.data;
if Settings::get().federation.enabled {
// TODO: intercept here (but the type is wrong)
//get_remote_community_posts(get_posts.community_id.unwrap())
}
let user_claims: Option<Claims> = match &data.auth {
Some(auth) => match Claims::decode(&auth) {
Ok(claims) => Some(claims.claims),
@ -358,53 +254,26 @@ impl Perform for Oper<GetPosts> {
let type_ = ListingType::from_str(&data.type_)?;
let sort = SortType::from_str(&data.sort)?;
let page = data.page;
let limit = data.limit;
let community_id = data.community_id;
let posts = match blocking(pool, move |conn| {
PostQueryBuilder::create(conn)
.listing_type(type_)
.sort(&sort)
.show_nsfw(show_nsfw)
.for_community_id(community_id)
.my_user_id(user_id)
.page(page)
.limit(limit)
.list()
})
.await?
let posts = match PostQueryBuilder::create(&conn)
.listing_type(type_)
.sort(&sort)
.show_nsfw(show_nsfw)
.for_community_id(data.community_id)
.my_user_id(user_id)
.page(data.page)
.limit(data.limit)
.list()
{
Ok(posts) => posts,
Err(_e) => return Err(APIError::err("couldnt_get_posts").into()),
};
if let Some(ws) = websocket_info {
// You don't need to join the specific community room, bc this is already handled by
// GetCommunity
if data.community_id.is_none() {
if let Some(id) = ws.id {
// 0 is the "all" community
ws.chatserver.do_send(JoinCommunityRoom {
community_id: 0,
id,
});
}
}
}
Ok(GetPostsResponse { posts })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<CreatePostLike> {
type Response = PostResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<PostResponse, LemmyError> {
impl Perform<PostResponse> for Oper<CreatePostLike> {
fn perform(&self, conn: &PgConnection) -> Result<PostResponse, Error> {
let data: &CreatePostLike = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -416,26 +285,20 @@ impl Perform for Oper<CreatePostLike> {
// Don't do a downvote if site has downvotes disabled
if data.score == -1 {
let site = blocking(pool, move |conn| SiteView::read(conn)).await??;
let site = SiteView::read(&conn)?;
if !site.enable_downvotes {
return Err(APIError::err("downvotes_disabled").into());
}
}
// Check for a community ban
let post_id = data.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let is_banned =
move |conn: &'_ _| CommunityUserBanView::get(conn, user_id, community_id).is_ok();
if blocking(pool, is_banned).await? {
let post = Post::read(&conn, data.post_id)?;
if CommunityUserBanView::get(&conn, user_id, post.community_id).is_ok() {
return Err(APIError::err("community_ban").into());
}
// Check for a site ban
let user = blocking(pool, move |conn| User_::read(conn, user_id)).await??;
if user.banned {
if UserView::read(&conn, user_id)?.banned {
return Err(APIError::err("site_ban").into());
}
@ -446,60 +309,29 @@ impl Perform for Oper<CreatePostLike> {
};
// Remove any likes first
let like_form2 = like_form.clone();
blocking(pool, move |conn| PostLike::remove(conn, &like_form2)).await??;
PostLike::remove(&conn, &like_form)?;
// Only add the like if the score isnt 0
let do_add = like_form.score != 0 && (like_form.score == 1 || like_form.score == -1);
if do_add {
let like_form2 = like_form.clone();
let like = move |conn: &'_ _| PostLike::like(conn, &like_form2);
if blocking(pool, like).await?.is_err() {
return Err(APIError::err("couldnt_like_post").into());
}
if like_form.score == 1 {
post.send_like(&user, &self.client, pool).await?;
} else if like_form.score == -1 {
post.send_dislike(&user, &self.client, pool).await?;
}
} else {
post.send_undo_like(&user, &self.client, pool).await?;
let _inserted_like = match PostLike::like(&conn, &like_form) {
Ok(like) => like,
Err(_e) => return Err(APIError::err("couldnt_like_post").into()),
};
}
let post_id = data.post_id;
let post_view = match blocking(pool, move |conn| {
PostView::read(conn, post_id, Some(user_id))
})
.await?
{
let post_view = match PostView::read(&conn, data.post_id, Some(user_id)) {
Ok(post) => post,
Err(_e) => return Err(APIError::err("couldnt_find_post").into()),
};
let res = PostResponse { post: post_view };
if let Some(ws) = websocket_info {
ws.chatserver.do_send(SendPost {
op: UserOperation::CreatePostLike,
post: res.clone(),
my_id: ws.id,
});
}
Ok(res)
// just output the score
Ok(PostResponse { post: post_view })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<EditPost> {
type Response = PostResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<PostResponse, LemmyError> {
impl Perform<PostResponse> for Oper<EditPost> {
fn perform(&self, conn: &PgConnection) -> Result<PostResponse, Error> {
let data: &EditPost = &self.data;
if let Err(slurs) = slur_check(&data.name) {
@ -520,45 +352,33 @@ impl Perform for Oper<EditPost> {
let user_id = claims.id;
// Verify its the creator or a mod or admin
let community_id = data.community_id;
let mut editors: Vec<i32> = vec![data.creator_id];
editors.append(
&mut blocking(pool, move |conn| {
CommunityModeratorView::for_community(conn, community_id)
.map(|v| v.into_iter().map(|m| m.user_id).collect())
})
.await??,
);
editors.append(
&mut blocking(pool, move |conn| {
UserView::admins(conn).map(|v| v.into_iter().map(|a| a.id).collect())
})
.await??,
&mut CommunityModeratorView::for_community(&conn, data.community_id)?
.into_iter()
.map(|m| m.user_id)
.collect(),
);
editors.append(&mut UserView::admins(&conn)?.into_iter().map(|a| a.id).collect());
if !editors.contains(&user_id) {
return Err(APIError::err("no_post_edit_allowed").into());
}
// Check for a community ban
let community_id = data.community_id;
let is_banned =
move |conn: &'_ _| CommunityUserBanView::get(conn, user_id, community_id).is_ok();
if blocking(pool, is_banned).await? {
if CommunityUserBanView::get(&conn, user_id, data.community_id).is_ok() {
return Err(APIError::err("community_ban").into());
}
// Check for a site ban
let user = blocking(pool, move |conn| User_::read(conn, user_id)).await??;
if user.banned {
if UserView::read(&conn, user_id)?.banned {
return Err(APIError::err("site_ban").into());
}
// Fetch Iframely and Pictrs cached image
let (iframely_title, iframely_description, iframely_html, pictrs_thumbnail) =
fetch_iframely_and_pictrs_data(&self.client, data.url.to_owned()).await;
// Fetch Iframely and Pictshare cached image
let (iframely_title, iframely_description, iframely_html, pictshare_thumbnail) =
fetch_iframely_and_pictshare_data(data.url.to_owned());
let edit_id = data.edit_id;
let read_post = blocking(pool, move |conn| Post::read(conn, edit_id)).await??;
let read_post = Post::read(&conn, data.edit_id)?;
let post_form = PostForm {
name: data.name.to_owned(),
@ -575,15 +395,12 @@ impl Perform for Oper<EditPost> {
embed_title: iframely_title,
embed_description: iframely_description,
embed_html: iframely_html,
thumbnail_url: pictrs_thumbnail,
thumbnail_url: pictshare_thumbnail,
ap_id: read_post.ap_id,
local: read_post.local,
published: None,
};
let edit_id = data.edit_id;
let res = blocking(pool, move |conn| Post::update(conn, edit_id, &post_form)).await?;
let updated_post: Post = match res {
let _updated_post = match Post::update(&conn, data.edit_id, &post_form) {
Ok(post) => post,
Err(e) => {
let err_type = if e.to_string() == "value too long for type character varying(200)" {
@ -604,7 +421,7 @@ impl Perform for Oper<EditPost> {
removed: Some(removed),
reason: data.reason.to_owned(),
};
blocking(pool, move |conn| ModRemovePost::create(conn, &form)).await??;
ModRemovePost::create(&conn, &form)?;
}
if let Some(locked) = data.locked.to_owned() {
@ -613,7 +430,7 @@ impl Perform for Oper<EditPost> {
post_id: data.edit_id,
locked: Some(locked),
};
blocking(pool, move |conn| ModLockPost::create(conn, &form)).await??;
ModLockPost::create(&conn, &form)?;
}
if let Some(stickied) = data.stickied.to_owned() {
@ -622,58 +439,17 @@ impl Perform for Oper<EditPost> {
post_id: data.edit_id,
stickied: Some(stickied),
};
blocking(pool, move |conn| ModStickyPost::create(conn, &form)).await??;
ModStickyPost::create(&conn, &form)?;
}
if let Some(deleted) = data.deleted.to_owned() {
if deleted {
updated_post.send_delete(&user, &self.client, pool).await?;
} else {
updated_post
.send_undo_delete(&user, &self.client, pool)
.await?;
}
} else if let Some(removed) = data.removed.to_owned() {
if removed {
updated_post.send_remove(&user, &self.client, pool).await?;
} else {
updated_post
.send_undo_remove(&user, &self.client, pool)
.await?;
}
} else {
updated_post.send_update(&user, &self.client, pool).await?;
}
let post_view = PostView::read(&conn, data.edit_id, Some(user_id))?;
let edit_id = data.edit_id;
let post_view = blocking(pool, move |conn| {
PostView::read(conn, edit_id, Some(user_id))
})
.await??;
let res = PostResponse { post: post_view };
if let Some(ws) = websocket_info {
ws.chatserver.do_send(SendPost {
op: UserOperation::EditPost,
post: res.clone(),
my_id: ws.id,
});
}
Ok(res)
Ok(PostResponse { post: post_view })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<SavePost> {
type Response = PostResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<PostResponse, LemmyError> {
impl Perform<PostResponse> for Oper<SavePost> {
fn perform(&self, conn: &PgConnection) -> Result<PostResponse, Error> {
let data: &SavePost = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -689,22 +465,18 @@ impl Perform for Oper<SavePost> {
};
if data.save {
let save = move |conn: &'_ _| PostSaved::save(conn, &post_saved_form);
if blocking(pool, save).await?.is_err() {
return Err(APIError::err("couldnt_save_post").into());
}
match PostSaved::save(&conn, &post_saved_form) {
Ok(post) => post,
Err(_e) => return Err(APIError::err("couldnt_save_post").into()),
};
} else {
let unsave = move |conn: &'_ _| PostSaved::unsave(conn, &post_saved_form);
if blocking(pool, unsave).await?.is_err() {
return Err(APIError::err("couldnt_save_post").into());
}
match PostSaved::unsave(&conn, &post_saved_form) {
Ok(post) => post,
Err(_e) => return Err(APIError::err("couldnt_save_post").into()),
};
}
let post_id = data.post_id;
let post_view = blocking(pool, move |conn| {
PostView::read(conn, post_id, Some(user_id))
})
.await??;
let post_view = PostView::read(&conn, data.post_id, Some(user_id))?;
Ok(PostResponse { post: post_view })
}

590
server/src/api/site.rs
View File

@ -1,33 +1,9 @@
use super::user::Register;
use crate::{
api::{APIError, Oper, Perform},
apub::fetcher::search_by_apub_id,
blocking,
db::{
category::*,
comment_view::*,
community_view::*,
moderator::*,
moderator_views::*,
post_view::*,
site::*,
site_view::*,
user::*,
user_view::*,
Crud,
SearchType,
SortType,
},
naive_now,
settings::Settings,
slur_check,
slurs_vec_to_str,
websocket::{server::SendAllMessage, UserOperation, WebsocketInfo},
DbPool,
LemmyError,
};
use log::{debug, info};
use serde::{Deserialize, Serialize};
use super::*;
use crate::api::user::Register;
use crate::api::{Oper, Perform};
use crate::settings::Settings;
use diesel::PgConnection;
use log::info;
use std::str::FromStr;
#[derive(Serialize, Deserialize)]
@ -38,7 +14,7 @@ pub struct ListCategoriesResponse {
categories: Vec<Category>,
}
#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize)]
pub struct Search {
q: String,
type_: String,
@ -49,13 +25,13 @@ pub struct Search {
auth: Option<String>,
}
#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize)]
pub struct SearchResponse {
pub type_: String,
pub comments: Vec<CommentView>,
pub posts: Vec<PostView>,
pub communities: Vec<CommunityView>,
pub users: Vec<UserView>,
type_: String,
comments: Vec<CommentView>,
posts: Vec<PostView>,
communities: Vec<CommunityView>,
users: Vec<UserView>,
}
#[derive(Serialize, Deserialize)]
@ -102,7 +78,7 @@ pub struct EditSite {
#[derive(Serialize, Deserialize)]
pub struct GetSite {}
#[derive(Serialize, Deserialize, Clone)]
#[derive(Serialize, Deserialize)]
pub struct SiteResponse {
site: SiteView,
}
@ -121,95 +97,71 @@ pub struct TransferSite {
auth: String,
}
#[derive(Serialize, Deserialize)]
pub struct GetSiteConfig {
auth: String,
}
#[derive(Serialize, Deserialize)]
pub struct GetSiteConfigResponse {
config_hjson: String,
}
#[derive(Serialize, Deserialize)]
pub struct SaveSiteConfig {
config_hjson: String,
auth: String,
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<ListCategories> {
type Response = ListCategoriesResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<ListCategoriesResponse, LemmyError> {
impl Perform<ListCategoriesResponse> for Oper<ListCategories> {
fn perform(&self, conn: &PgConnection) -> Result<ListCategoriesResponse, Error> {
let _data: &ListCategories = &self.data;
let categories = blocking(pool, move |conn| Category::list_all(conn)).await??;
let categories: Vec<Category> = Category::list_all(&conn)?;
// Return the jwt
Ok(ListCategoriesResponse { categories })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<GetModlog> {
type Response = GetModlogResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<GetModlogResponse, LemmyError> {
impl Perform<GetModlogResponse> for Oper<GetModlog> {
fn perform(&self, conn: &PgConnection) -> Result<GetModlogResponse, Error> {
let data: &GetModlog = &self.data;
let community_id = data.community_id;
let mod_user_id = data.mod_user_id;
let page = data.page;
let limit = data.limit;
let removed_posts = blocking(pool, move |conn| {
ModRemovePostView::list(conn, community_id, mod_user_id, page, limit)
})
.await??;
let locked_posts = blocking(pool, move |conn| {
ModLockPostView::list(conn, community_id, mod_user_id, page, limit)
})
.await??;
let stickied_posts = blocking(pool, move |conn| {
ModStickyPostView::list(conn, community_id, mod_user_id, page, limit)
})
.await??;
let removed_comments = blocking(pool, move |conn| {
ModRemoveCommentView::list(conn, community_id, mod_user_id, page, limit)
})
.await??;
let banned_from_community = blocking(pool, move |conn| {
ModBanFromCommunityView::list(conn, community_id, mod_user_id, page, limit)
})
.await??;
let added_to_community = blocking(pool, move |conn| {
ModAddCommunityView::list(conn, community_id, mod_user_id, page, limit)
})
.await??;
let removed_posts = ModRemovePostView::list(
&conn,
data.community_id,
data.mod_user_id,
data.page,
data.limit,
)?;
let locked_posts = ModLockPostView::list(
&conn,
data.community_id,
data.mod_user_id,
data.page,
data.limit,
)?;
let stickied_posts = ModStickyPostView::list(
&conn,
data.community_id,
data.mod_user_id,
data.page,
data.limit,
)?;
let removed_comments = ModRemoveCommentView::list(
&conn,
data.community_id,
data.mod_user_id,
data.page,
data.limit,
)?;
let banned_from_community = ModBanFromCommunityView::list(
&conn,
data.community_id,
data.mod_user_id,
data.page,
data.limit,
)?;
let added_to_community = ModAddCommunityView::list(
&conn,
data.community_id,
data.mod_user_id,
data.page,
data.limit,
)?;
// These arrays are only for the full modlog, when a community isn't given
let (removed_communities, banned, added) = if data.community_id.is_none() {
blocking(pool, move |conn| {
Ok((
ModRemoveCommunityView::list(conn, mod_user_id, page, limit)?,
ModBanView::list(conn, mod_user_id, page, limit)?,
ModAddView::list(conn, mod_user_id, page, limit)?,
)) as Result<_, LemmyError>
})
.await??
(
ModRemoveCommunityView::list(&conn, data.mod_user_id, data.page, data.limit)?,
ModBanView::list(&conn, data.mod_user_id, data.page, data.limit)?,
ModAddView::list(&conn, data.mod_user_id, data.page, data.limit)?,
)
} else {
(Vec::new(), Vec::new(), Vec::new())
};
@ -229,15 +181,8 @@ impl Perform for Oper<GetModlog> {
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<CreateSite> {
type Response = SiteResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<SiteResponse, LemmyError> {
impl Perform<SiteResponse> for Oper<CreateSite> {
fn perform(&self, conn: &PgConnection) -> Result<SiteResponse, Error> {
let data: &CreateSite = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -258,8 +203,7 @@ impl Perform for Oper<CreateSite> {
let user_id = claims.id;
// Make sure user is an admin
let user = blocking(pool, move |conn| UserView::read(conn, user_id)).await??;
if !user.admin {
if !UserView::read(&conn, user_id)?.admin {
return Err(APIError::err("not_an_admin").into());
}
@ -273,25 +217,19 @@ impl Perform for Oper<CreateSite> {
updated: None,
};
let create_site = move |conn: &'_ _| Site::create(conn, &site_form);
if blocking(pool, create_site).await?.is_err() {
return Err(APIError::err("site_already_exists").into());
}
match Site::create(&conn, &site_form) {
Ok(site) => site,
Err(_e) => return Err(APIError::err("site_already_exists").into()),
};
let site_view = blocking(pool, move |conn| SiteView::read(conn)).await??;
let site_view = SiteView::read(&conn)?;
Ok(SiteResponse { site: site_view })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<EditSite> {
type Response = SiteResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<SiteResponse, LemmyError> {
impl Perform<SiteResponse> for Oper<EditSite> {
fn perform(&self, conn: &PgConnection) -> Result<SiteResponse, Error> {
let data: &EditSite = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -312,12 +250,11 @@ impl Perform for Oper<EditSite> {
let user_id = claims.id;
// Make sure user is an admin
let user = blocking(pool, move |conn| UserView::read(conn, user_id)).await??;
if !user.admin {
if !UserView::read(&conn, user_id)?.admin {
return Err(APIError::err("not_an_admin").into());
}
let found_site = blocking(pool, move |conn| Site::read(conn, 1)).await??;
let found_site = Site::read(&conn, 1)?;
let site_form = SiteForm {
name: data.name.to_owned(),
@ -329,42 +266,24 @@ impl Perform for Oper<EditSite> {
enable_nsfw: data.enable_nsfw,
};
let update_site = move |conn: &'_ _| Site::update(conn, 1, &site_form);
if blocking(pool, update_site).await?.is_err() {
return Err(APIError::err("couldnt_update_site").into());
}
match Site::update(&conn, 1, &site_form) {
Ok(site) => site,
Err(_e) => return Err(APIError::err("couldnt_update_site").into()),
};
let site_view = blocking(pool, move |conn| SiteView::read(conn)).await??;
let site_view = SiteView::read(&conn)?;
let res = SiteResponse { site: site_view };
if let Some(ws) = websocket_info {
ws.chatserver.do_send(SendAllMessage {
op: UserOperation::EditSite,
response: res.clone(),
my_id: ws.id,
});
}
Ok(res)
Ok(SiteResponse { site: site_view })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<GetSite> {
type Response = GetSiteResponse;
async fn perform(
&self,
pool: &DbPool,
websocket_info: Option<WebsocketInfo>,
) -> Result<GetSiteResponse, LemmyError> {
impl Perform<GetSiteResponse> for Oper<GetSite> {
fn perform(&self, conn: &PgConnection) -> Result<GetSiteResponse, Error> {
let _data: &GetSite = &self.data;
// TODO refactor this a little
let res = blocking(pool, move |conn| Site::read(conn, 1)).await?;
let site_view = if res.is_ok() {
Some(blocking(pool, move |conn| SiteView::read(conn)).await??)
let site = Site::read(&conn, 1);
let site_view = if site.is_ok() {
Some(SiteView::read(&conn)?)
} else if let Some(setup) = Settings::get().setup.as_ref() {
let register = Register {
username: setup.admin_username.to_owned(),
@ -374,81 +293,47 @@ impl Perform for Oper<GetSite> {
admin: true,
show_nsfw: true,
};
let login_response = Oper::new(register, self.client.clone())
.perform(pool, websocket_info.clone())
.await?;
let login_response = Oper::new(register).perform(&conn)?;
info!("Admin {} created", setup.admin_username);
let create_site = CreateSite {
name: setup.site_name.to_owned(),
description: None,
enable_downvotes: true,
open_registration: true,
enable_nsfw: true,
enable_downvotes: false,
open_registration: false,
enable_nsfw: false,
auth: login_response.jwt,
};
Oper::new(create_site, self.client.clone())
.perform(pool, websocket_info.clone())
.await?;
Oper::new(create_site).perform(&conn)?;
info!("Site {} created", setup.site_name);
Some(blocking(pool, move |conn| SiteView::read(conn)).await??)
Some(SiteView::read(&conn)?)
} else {
None
};
let mut admins = blocking(pool, move |conn| UserView::admins(conn)).await??;
// Make sure the site creator is the top admin
if let Some(site_view) = site_view.to_owned() {
let site_creator_id = site_view.creator_id;
// TODO investigate why this is sometimes coming back null
// Maybe user_.admin isn't being set to true?
if let Some(creator_index) = admins.iter().position(|r| r.id == site_creator_id) {
let creator_user = admins.remove(creator_index);
admins.insert(0, creator_user);
}
let mut admins = UserView::admins(&conn)?;
if site_view.is_some() {
let site_creator_id = site_view.to_owned().unwrap().creator_id;
let creator_index = admins.iter().position(|r| r.id == site_creator_id).unwrap();
let creator_user = admins.remove(creator_index);
admins.insert(0, creator_user);
}
let banned = blocking(pool, move |conn| UserView::banned(conn)).await??;
let online = if let Some(_ws) = websocket_info {
// TODO
1
// let fut = async {
// ws.chatserver.send(GetUsersOnline).await.unwrap()
// };
// Runtime::new().unwrap().block_on(fut)
} else {
0
};
let banned = UserView::banned(&conn)?;
Ok(GetSiteResponse {
site: site_view,
admins,
banned,
online,
online: 0,
})
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<Search> {
type Response = SearchResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<SearchResponse, LemmyError> {
impl Perform<SearchResponse> for Oper<Search> {
fn perform(&self, conn: &PgConnection) -> Result<SearchResponse, Error> {
let data: &Search = &self.data;
dbg!(&data);
match search_by_apub_id(&data.q, &self.client, pool).await {
Ok(r) => return Ok(r),
Err(e) => debug!("Failed to resolve search query as activitypub ID: {}", e),
}
let user_id: Option<i32> = match &data.auth {
Some(auth) => match Claims::decode(&auth) {
Ok(claims) => {
@ -460,6 +345,7 @@ impl Perform for Oper<Search> {
None => None,
};
let sort = SortType::from_str(&data.sort)?;
let type_ = SearchType::from_str(&data.type_)?;
let mut posts = Vec::new();
@ -469,126 +355,85 @@ impl Perform for Oper<Search> {
// TODO no clean / non-nsfw searching rn
let q = data.q.to_owned();
let page = data.page;
let limit = data.limit;
let sort = SortType::from_str(&data.sort)?;
let community_id = data.community_id;
match type_ {
SearchType::Posts => {
posts = blocking(pool, move |conn| {
PostQueryBuilder::create(conn)
.sort(&sort)
.show_nsfw(true)
.for_community_id(community_id)
.search_term(q)
.my_user_id(user_id)
.page(page)
.limit(limit)
.list()
})
.await??;
posts = PostQueryBuilder::create(&conn)
.sort(&sort)
.show_nsfw(true)
.for_community_id(data.community_id)
.search_term(data.q.to_owned())
.my_user_id(user_id)
.page(data.page)
.limit(data.limit)
.list()?;
}
SearchType::Comments => {
comments = blocking(pool, move |conn| {
CommentQueryBuilder::create(&conn)
.sort(&sort)
.search_term(q)
.my_user_id(user_id)
.page(page)
.limit(limit)
.list()
})
.await??;
comments = CommentQueryBuilder::create(&conn)
.sort(&sort)
.search_term(data.q.to_owned())
.my_user_id(user_id)
.page(data.page)
.limit(data.limit)
.list()?;
}
SearchType::Communities => {
communities = blocking(pool, move |conn| {
CommunityQueryBuilder::create(conn)
.sort(&sort)
.search_term(q)
.page(page)
.limit(limit)
.list()
})
.await??;
communities = CommunityQueryBuilder::create(&conn)
.sort(&sort)
.search_term(data.q.to_owned())
.page(data.page)
.limit(data.limit)
.list()?;
}
SearchType::Users => {
users = blocking(pool, move |conn| {
UserQueryBuilder::create(conn)
.sort(&sort)
.search_term(q)
.page(page)
.limit(limit)
.list()
})
.await??;
users = UserQueryBuilder::create(&conn)
.sort(&sort)
.search_term(data.q.to_owned())
.page(data.page)
.limit(data.limit)
.list()?;
}
SearchType::All => {
posts = blocking(pool, move |conn| {
PostQueryBuilder::create(conn)
.sort(&sort)
.show_nsfw(true)
.for_community_id(community_id)
.search_term(q)
.my_user_id(user_id)
.page(page)
.limit(limit)
.list()
})
.await??;
posts = PostQueryBuilder::create(&conn)
.sort(&sort)
.show_nsfw(true)
.for_community_id(data.community_id)
.search_term(data.q.to_owned())
.my_user_id(user_id)
.page(data.page)
.limit(data.limit)
.list()?;
let q = data.q.to_owned();
let sort = SortType::from_str(&data.sort)?;
comments = CommentQueryBuilder::create(&conn)
.sort(&sort)
.search_term(data.q.to_owned())
.my_user_id(user_id)
.page(data.page)
.limit(data.limit)
.list()?;
comments = blocking(pool, move |conn| {
CommentQueryBuilder::create(conn)
.sort(&sort)
.search_term(q)
.my_user_id(user_id)
.page(page)
.limit(limit)
.list()
})
.await??;
communities = CommunityQueryBuilder::create(&conn)
.sort(&sort)
.search_term(data.q.to_owned())
.page(data.page)
.limit(data.limit)
.list()?;
let q = data.q.to_owned();
let sort = SortType::from_str(&data.sort)?;
communities = blocking(pool, move |conn| {
CommunityQueryBuilder::create(conn)
.sort(&sort)
.search_term(q)
.page(page)
.limit(limit)
.list()
})
.await??;
let q = data.q.to_owned();
let sort = SortType::from_str(&data.sort)?;
users = blocking(pool, move |conn| {
UserQueryBuilder::create(conn)
.sort(&sort)
.search_term(q)
.page(page)
.limit(limit)
.list()
})
.await??;
users = UserQueryBuilder::create(&conn)
.sort(&sort)
.search_term(data.q.to_owned())
.page(data.page)
.limit(data.limit)
.list()?;
}
SearchType::Url => {
posts = blocking(pool, move |conn| {
PostQueryBuilder::create(conn)
.sort(&sort)
.show_nsfw(true)
.for_community_id(community_id)
.url_search(q)
.page(page)
.limit(limit)
.list()
})
.await??;
posts = PostQueryBuilder::create(&conn)
.sort(&sort)
.show_nsfw(true)
.for_community_id(data.community_id)
.url_search(data.q.to_owned())
.page(data.page)
.limit(data.limit)
.list()?;
}
};
@ -603,15 +448,8 @@ impl Perform for Oper<Search> {
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<TransferSite> {
type Response = GetSiteResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<GetSiteResponse, LemmyError> {
impl Perform<GetSiteResponse> for Oper<TransferSite> {
fn perform(&self, conn: &PgConnection) -> Result<GetSiteResponse, Error> {
let data: &TransferSite = &self.data;
let claims = match Claims::decode(&data.auth) {
@ -621,7 +459,7 @@ impl Perform for Oper<TransferSite> {
let user_id = claims.id;
let read_site = blocking(pool, move |conn| Site::read(conn, 1)).await??;
let read_site = Site::read(&conn, 1)?;
// Make sure user is the creator
if read_site.creator_id != user_id {
@ -638,9 +476,9 @@ impl Perform for Oper<TransferSite> {
enable_nsfw: read_site.enable_nsfw,
};
let update_site = move |conn: &'_ _| Site::update(conn, 1, &site_form);
if blocking(pool, update_site).await?.is_err() {
return Err(APIError::err("couldnt_update_site").into());
match Site::update(&conn, 1, &site_form) {
Ok(site) => site,
Err(_e) => return Err(APIError::err("couldnt_update_site").into()),
};
// Mod tables
@ -650,11 +488,11 @@ impl Perform for Oper<TransferSite> {
removed: Some(false),
};
blocking(pool, move |conn| ModAdd::create(conn, &form)).await??;
ModAdd::create(&conn, &form)?;
let site_view = blocking(pool, move |conn| SiteView::read(conn)).await??;
let site_view = SiteView::read(&conn)?;
let mut admins = blocking(pool, move |conn| UserView::admins(conn)).await??;
let mut admins = UserView::admins(&conn)?;
let creator_index = admins
.iter()
.position(|r| r.id == site_view.creator_id)
@ -662,7 +500,7 @@ impl Perform for Oper<TransferSite> {
let creator_user = admins.remove(creator_index);
admins.insert(0, creator_user);
let banned = blocking(pool, move |conn| UserView::banned(conn)).await??;
let banned = UserView::banned(&conn)?;
Ok(GetSiteResponse {
site: Some(site_view),
@ -672,71 +510,3 @@ impl Perform for Oper<TransferSite> {
})
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<GetSiteConfig> {
type Response = GetSiteConfigResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<GetSiteConfigResponse, LemmyError> {
let data: &GetSiteConfig = &self.data;
let claims = match Claims::decode(&data.auth) {
Ok(claims) => claims.claims,
Err(_e) => return Err(APIError::err("not_logged_in").into()),
};
let user_id = claims.id;
// Only let admins read this
let admins = blocking(pool, move |conn| UserView::admins(conn)).await??;
let admin_ids: Vec<i32> = admins.into_iter().map(|m| m.id).collect();
if !admin_ids.contains(&user_id) {
return Err(APIError::err("not_an_admin").into());
}
let config_hjson = Settings::read_config_file()?;
Ok(GetSiteConfigResponse { config_hjson })
}
}
#[async_trait::async_trait(?Send)]
impl Perform for Oper<SaveSiteConfig> {
type Response = GetSiteConfigResponse;
async fn perform(
&self,
pool: &DbPool,
_websocket_info: Option<WebsocketInfo>,
) -> Result<GetSiteConfigResponse, LemmyError> {
let data: &SaveSiteConfig = &self.data;
let claims = match Claims::decode(&data.auth) {
Ok(claims) => claims.claims,
Err(_e) => return Err(APIError::err("not_logged_in").into()),
};
let user_id = claims.id;
// Only let admins read this
let admins = blocking(pool, move |conn| UserView::admins(conn)).await??;
let admin_ids: Vec<i32> = admins.into_iter().map(|m| m.id).collect();
if !admin_ids.contains(&user_id) {
return Err(APIError::err("not_an_admin").into());
}
// Make sure docker doesn't have :ro at the end of the volume, so its not a read-only filesystem
let config_hjson = match Settings::save_config_file(&data.config_hjson) {
Ok(config_hjson) => config_hjson,
Err(_e) => return Err(APIError::err("couldnt_update_site").into()),
};
Ok(GetSiteConfigResponse { config_hjson })
}
}

801
server/src/api/user.rs
View File

File diff suppressed because it is too large Load Diff

88
server/src/apub/activities.rs
View File

@ -1,88 +0,0 @@
use crate::{
apub::{extensions::signatures::sign, is_apub_id_valid, ActorType},
db::{activity::insert_activity, community::Community, user::User_},
request::retry_custom,
DbPool,
LemmyError,
};
use activitystreams::{context, object::properties::ObjectProperties, public, Activity, Base};
use actix_web::client::Client;
use log::debug;
use serde::Serialize;
use std::fmt::Debug;
use url::Url;
pub fn populate_object_props(
props: &mut ObjectProperties,
addressed_ccs: Vec<String>,
object_id: &str,
) -> Result<(), LemmyError> {
props
.set_context_xsd_any_uri(context())?
// TODO: the activity needs a seperate id from the object
.set_id(object_id)?
// TODO: should to/cc go on the Create, or on the Post? or on both?
// TODO: handle privacy on the receiving side (at least ignore anything thats not public)
.set_to_xsd_any_uri(public())?
.set_many_cc_xsd_any_uris(addressed_ccs)?;
Ok(())
}
pub async fn send_activity_to_community<A>(
creator: &User_,
community: &Community,
to: Vec<String>,
activity: A,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>
where
A: Activity + Base + Serialize + Debug + Clone + Send + 'static,
{
insert_activity(creator.id, activity.clone(), true, pool).await?;
// if this is a local community, we need to do an announce from the community instead
if community.local {
Community::do_announce(activity, &community, creator, client, pool).await?;
} else {
send_activity(client, &activity, creator, to).await?;
}
Ok(())
}
/// Send an activity to a list of recipients, using the correct headers etc.
pub async fn send_activity<A>(
client: &Client,
activity: &A,
actor: &dyn ActorType,
to: Vec<String>,
) -> Result<(), LemmyError>
where
A: Serialize,
{
let activity = serde_json::to_string(&activity)?;
debug!("Sending activitypub activity {} to {:?}", activity, to);
for t in to {
let to_url = Url::parse(&t)?;
if !is_apub_id_valid(&to_url) {
debug!("Not sending activity to {} (invalid or blocklisted)", t);
continue;
}
let res = retry_custom(|| async {
let request = client.post(&t).header("Content-Type", "application/json");
match sign(request, actor, activity.clone()).await {
Ok(signed) => Ok(signed.send().await),
Err(e) => Err(e),
}
})
.await?;
debug!("Result for activity send: {:?}", res);
}
Ok(())
}

633
server/src/apub/comment.rs
View File

@ -1,633 +0,0 @@
use crate::{
apub::{
activities::{populate_object_props, send_activity_to_community},
create_apub_response,
create_apub_tombstone_response,
create_tombstone,
fetch_webfinger_url,
fetcher::{
get_or_fetch_and_insert_remote_comment,
get_or_fetch_and_insert_remote_post,
get_or_fetch_and_upsert_remote_user,
},
ActorType,
ApubLikeableType,
ApubObjectType,
FromApub,
ToApub,
},
blocking,
convert_datetime,
db::{
comment::{Comment, CommentForm},
community::Community,
post::Post,
user::User_,
Crud,
},
routes::DbPoolParam,
scrape_text_for_mentions,
DbPool,
LemmyError,
MentionData,
};
use activitystreams::{
activity::{Create, Delete, Dislike, Like, Remove, Undo, Update},
context,
link::Mention,
object::{kind::NoteType, properties::ObjectProperties, Note},
};
use activitystreams_new::object::Tombstone;
use actix_web::{body::Body, client::Client, web::Path, HttpResponse};
use itertools::Itertools;
use log::debug;
use serde::Deserialize;
#[derive(Deserialize)]
pub struct CommentQuery {
comment_id: String,
}
/// Return the post json over HTTP.
pub async fn get_apub_comment(
info: Path<CommentQuery>,
db: DbPoolParam,
) -> Result<HttpResponse<Body>, LemmyError> {
let id = info.comment_id.parse::<i32>()?;
let comment = blocking(&db, move |conn| Comment::read(conn, id)).await??;
if !comment.deleted {
Ok(create_apub_response(&comment.to_apub(&db).await?))
} else {
Ok(create_apub_tombstone_response(&comment.to_tombstone()?))
}
}
#[async_trait::async_trait(?Send)]
impl ToApub for Comment {
type Response = Note;
async fn to_apub(&self, pool: &DbPool) -> Result<Note, LemmyError> {
let mut comment = Note::default();
let oprops: &mut ObjectProperties = comment.as_mut();
let creator_id = self.creator_id;
let creator = blocking(pool, move |conn| User_::read(conn, creator_id)).await??;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
// Add a vector containing some important info to the "in_reply_to" field
// [post_ap_id, Option(parent_comment_ap_id)]
let mut in_reply_to_vec = vec![post.ap_id];
if let Some(parent_id) = self.parent_id {
let parent_comment = blocking(pool, move |conn| Comment::read(conn, parent_id)).await??;
in_reply_to_vec.push(parent_comment.ap_id);
}
oprops
// Not needed when the Post is embedded in a collection (like for community outbox)
.set_context_xsd_any_uri(context())?
.set_id(self.ap_id.to_owned())?
.set_published(convert_datetime(self.published))?
.set_to_xsd_any_uri(community.actor_id)?
.set_many_in_reply_to_xsd_any_uris(in_reply_to_vec)?
.set_content_xsd_string(self.content.to_owned())?
.set_attributed_to_xsd_any_uri(creator.actor_id)?;
if let Some(u) = self.updated {
oprops.set_updated(convert_datetime(u))?;
}
Ok(comment)
}
fn to_tombstone(&self) -> Result<Tombstone, LemmyError> {
create_tombstone(
self.deleted,
&self.ap_id,
self.updated,
NoteType.to_string(),
)
}
}
#[async_trait::async_trait(?Send)]
impl FromApub for CommentForm {
type ApubType = Note;
/// Parse an ActivityPub note received from another instance into a Lemmy comment
async fn from_apub(
note: &Note,
client: &Client,
pool: &DbPool,
) -> Result<CommentForm, LemmyError> {
let oprops = &note.object_props;
let creator_actor_id = &oprops.get_attributed_to_xsd_any_uri().unwrap().to_string();
let creator = get_or_fetch_and_upsert_remote_user(&creator_actor_id, client, pool).await?;
let mut in_reply_tos = oprops.get_many_in_reply_to_xsd_any_uris().unwrap();
let post_ap_id = in_reply_tos.next().unwrap().to_string();
// This post, or the parent comment might not yet exist on this server yet, fetch them.
let post = get_or_fetch_and_insert_remote_post(&post_ap_id, client, pool).await?;
// The 2nd item, if it exists, is the parent comment apub_id
// For deeply nested comments, FromApub automatically gets called recursively
let parent_id: Option<i32> = match in_reply_tos.next() {
Some(parent_comment_uri) => {
let parent_comment_ap_id = &parent_comment_uri.to_string();
let parent_comment =
get_or_fetch_and_insert_remote_comment(&parent_comment_ap_id, client, pool).await?;
Some(parent_comment.id)
}
None => None,
};
Ok(CommentForm {
creator_id: creator.id,
post_id: post.id,
parent_id,
content: oprops
.get_content_xsd_string()
.map(|c| c.to_string())
.unwrap(),
removed: None,
read: None,
published: oprops
.get_published()
.map(|u| u.as_ref().to_owned().naive_local()),
updated: oprops
.get_updated()
.map(|u| u.as_ref().to_owned().naive_local()),
deleted: None,
ap_id: oprops.get_id().unwrap().to_string(),
local: false,
})
}
}
#[async_trait::async_trait(?Send)]
impl ApubObjectType for Comment {
/// Send out information about a newly created comment, to the followers of the community.
async fn send_create(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let maa =
collect_non_local_mentions_and_addresses(&self.content, &community, client, pool).await?;
let id = format!("{}/create/{}", self.ap_id, uuid::Uuid::new_v4());
let mut create = Create::new();
populate_object_props(&mut create.object_props, maa.addressed_ccs, &id)?;
// Set the mention tags
create.object_props.set_many_tag_base_boxes(maa.tags)?;
create
.create_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
send_activity_to_community(&creator, &community, maa.inboxes, create, client, pool).await?;
Ok(())
}
/// Send out information about an edited post, to the followers of the community.
async fn send_update(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let maa =
collect_non_local_mentions_and_addresses(&self.content, &community, client, pool).await?;
let id = format!("{}/update/{}", self.ap_id, uuid::Uuid::new_v4());
let mut update = Update::new();
populate_object_props(&mut update.object_props, maa.addressed_ccs, &id)?;
// Set the mention tags
update.object_props.set_many_tag_base_boxes(maa.tags)?;
update
.update_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
send_activity_to_community(&creator, &community, maa.inboxes, update, client, pool).await?;
Ok(())
}
async fn send_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
let mut delete = Delete::default();
populate_object_props(
&mut delete.object_props,
vec![community.get_followers_url()],
&id,
)?;
delete
.delete_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
send_activity_to_community(
&creator,
&community,
vec![community.get_shared_inbox_url()],
delete,
client,
pool,
)
.await?;
Ok(())
}
async fn send_undo_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
// Generate a fake delete activity, with the correct object
let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
let mut delete = Delete::default();
populate_object_props(
&mut delete.object_props,
vec![community.get_followers_url()],
&id,
)?;
delete
.delete_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
// TODO
// Undo that fake activity
let undo_id = format!("{}/undo/delete/{}", self.ap_id, uuid::Uuid::new_v4());
let mut undo = Undo::default();
populate_object_props(
&mut undo.object_props,
vec![community.get_followers_url()],
&undo_id,
)?;
undo
.undo_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(delete)?;
send_activity_to_community(
&creator,
&community,
vec![community.get_shared_inbox_url()],
undo,
client,
pool,
)
.await?;
Ok(())
}
async fn send_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/remove/{}", self.ap_id, uuid::Uuid::new_v4());
let mut remove = Remove::default();
populate_object_props(
&mut remove.object_props,
vec![community.get_followers_url()],
&id,
)?;
remove
.remove_props
.set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
.set_object_base_box(note)?;
send_activity_to_community(
&mod_,
&community,
vec![community.get_shared_inbox_url()],
remove,
client,
pool,
)
.await?;
Ok(())
}
async fn send_undo_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
// Generate a fake delete activity, with the correct object
let id = format!("{}/remove/{}", self.ap_id, uuid::Uuid::new_v4());
let mut remove = Remove::default();
populate_object_props(
&mut remove.object_props,
vec![community.get_followers_url()],
&id,
)?;
remove
.remove_props
.set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
.set_object_base_box(note)?;
// Undo that fake activity
let undo_id = format!("{}/undo/remove/{}", self.ap_id, uuid::Uuid::new_v4());
let mut undo = Undo::default();
populate_object_props(
&mut undo.object_props,
vec![community.get_followers_url()],
&undo_id,
)?;
undo
.undo_props
.set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
.set_object_base_box(remove)?;
send_activity_to_community(
&mod_,
&community,
vec![community.get_shared_inbox_url()],
undo,
client,
pool,
)
.await?;
Ok(())
}
}
#[async_trait::async_trait(?Send)]
impl ApubLikeableType for Comment {
async fn send_like(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/like/{}", self.ap_id, uuid::Uuid::new_v4());
let mut like = Like::new();
populate_object_props(
&mut like.object_props,
vec![community.get_followers_url()],
&id,
)?;
like
.like_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
send_activity_to_community(
&creator,
&community,
vec![community.get_shared_inbox_url()],
like,
client,
pool,
)
.await?;
Ok(())
}
async fn send_dislike(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/dislike/{}", self.ap_id, uuid::Uuid::new_v4());
let mut dislike = Dislike::new();
populate_object_props(
&mut dislike.object_props,
vec![community.get_followers_url()],
&id,
)?;
dislike
.dislike_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
send_activity_to_community(
&creator,
&community,
vec![community.get_shared_inbox_url()],
dislike,
client,
pool,
)
.await?;
Ok(())
}
async fn send_undo_like(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let post_id = self.post_id;
let post = blocking(pool, move |conn| Post::read(conn, post_id)).await??;
let community_id = post.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/dislike/{}", self.ap_id, uuid::Uuid::new_v4());
let mut like = Like::new();
populate_object_props(
&mut like.object_props,
vec![community.get_followers_url()],
&id,
)?;
like
.like_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
// TODO
// Undo that fake activity
let undo_id = format!("{}/undo/like/{}", self.ap_id, uuid::Uuid::new_v4());
let mut undo = Undo::default();
populate_object_props(
&mut undo.object_props,
vec![community.get_followers_url()],
&undo_id,
)?;
undo
.undo_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(like)?;
send_activity_to_community(
&creator,
&community,
vec![community.get_shared_inbox_url()],
undo,
client,
pool,
)
.await?;
Ok(())
}
}
struct MentionsAndAddresses {
addressed_ccs: Vec<String>,
inboxes: Vec<String>,
tags: Vec<Mention>,
}
/// This takes a comment, and builds a list of to_addresses, inboxes,
/// and mention tags, so they know where to be sent to.
/// Addresses are the users / addresses that go in the cc field.
async fn collect_non_local_mentions_and_addresses(
content: &str,
community: &Community,
client: &Client,
pool: &DbPool,
) -> Result<MentionsAndAddresses, LemmyError> {
let mut addressed_ccs = vec![community.get_followers_url()];
// Add the mention tag
let mut tags = Vec::new();
// Get the inboxes for any mentions
let mentions = scrape_text_for_mentions(&content)
.into_iter()
// Filter only the non-local ones
.filter(|m| !m.is_local())
.collect::<Vec<MentionData>>();
let mut mention_inboxes = Vec::new();
for mention in &mentions {
// TODO should it be fetching it every time?
if let Ok(actor_id) = fetch_webfinger_url(mention, client).await {
debug!("mention actor_id: {}", actor_id);
addressed_ccs.push(actor_id.to_owned());
let mention_user = get_or_fetch_and_upsert_remote_user(&actor_id, client, pool).await?;
let shared_inbox = mention_user.get_shared_inbox_url();
mention_inboxes.push(shared_inbox);
let mut mention_tag = Mention::new();
mention_tag
.link_props
.set_href(actor_id)?
.set_name_xsd_string(mention.full_name())?;
tags.push(mention_tag);
}
}
let mut inboxes = vec![community.get_shared_inbox_url()];
inboxes.extend(mention_inboxes);
inboxes = inboxes.into_iter().unique().collect();
Ok(MentionsAndAddresses {
addressed_ccs,
inboxes,
tags,
})
}

556
server/src/apub/community.rs
View File

@ -1,488 +1,192 @@
use crate::{
apub::{
activities::{populate_object_props, send_activity},
create_apub_response,
create_apub_tombstone_response,
create_tombstone,
extensions::{group_extensions::GroupExtension, signatures::PublicKey},
fetcher::get_or_fetch_and_upsert_remote_user,
get_shared_inbox,
ActorType,
FromApub,
GroupExt,
ToApub,
},
blocking,
convert_datetime,
db::{
activity::insert_activity,
community::{Community, CommunityForm},
community_view::{CommunityFollowerView, CommunityModeratorView},
user::User_,
},
naive_now,
routes::DbPoolParam,
DbPool,
LemmyError,
};
use crate::apub::puller::fetch_remote_object;
use crate::apub::*;
use crate::convert_datetime;
use crate::db::community::Community;
use crate::db::community_view::{CommunityFollowerView, CommunityView};
use crate::db::establish_unpooled_connection;
use crate::db::post::Post;
use crate::settings::Settings;
use activitystreams::actor::properties::ApActorProperties;
use activitystreams::collection::OrderedCollection;
use activitystreams::{
activity::{Accept, Announce, Delete, Remove, Undo},
actor::{kind::GroupType, properties::ApActorProperties, Group},
collection::UnorderedCollection,
context,
endpoint::EndpointProperties,
actor::Group, collection::UnorderedCollection, context, ext::Extensible,
object::properties::ObjectProperties,
Activity,
Base,
BaseBox,
};
use activitystreams_ext::Ext3;
use activitystreams_new::{activity::Follow, object::Tombstone};
use actix_web::{body::Body, client::Client, web, HttpResponse};
use itertools::Itertools;
use serde::{Deserialize, Serialize};
use std::fmt::Debug;
use actix_web::body::Body;
use actix_web::web::Path;
use actix_web::HttpResponse;
use actix_web::{web, Result};
use diesel::r2d2::{ConnectionManager, Pool};
use diesel::PgConnection;
use failure::Error;
use serde::Deserialize;
#[derive(Deserialize)]
pub struct CommunityQuery {
community_name: String,
}
#[async_trait::async_trait(?Send)]
impl ToApub for Community {
type Response = GroupExt;
pub async fn get_apub_community_list(
db: web::Data<Pool<ConnectionManager<PgConnection>>>,
) -> Result<HttpResponse<Body>, Error> {
// TODO: implement pagination
let communities = Community::list(&db.get().unwrap())?
.iter()
.map(|c| c.as_group())
.collect::<Result<Vec<GroupExt>, Error>>()?;
let mut collection = UnorderedCollection::default();
let oprops: &mut ObjectProperties = collection.as_mut();
oprops.set_context_xsd_any_uri(context())?.set_id(format!(
"{}://{}/federation/communities",
get_apub_protocol_string(),
Settings::get().hostname
))?;
collection
.collection_props
.set_total_items(communities.len() as u64)?
.set_many_items_base_boxes(communities)?;
Ok(create_apub_response(&collection))
}
impl Community {
fn as_group(&self) -> Result<GroupExt, Error> {
let base_url = make_apub_endpoint(EndpointType::Community, &self.name);
// Turn a Lemmy Community into an ActivityPub group that can be sent out over the network.
async fn to_apub(&self, pool: &DbPool) -> Result<GroupExt, LemmyError> {
let mut group = Group::default();
let oprops: &mut ObjectProperties = group.as_mut();
// The attributed to, is an ordered vector with the creator actor_ids first,
// then the rest of the moderators
// TODO Technically the instance admins can mod the community, but lets
// ignore that for now
let id = self.id;
let moderators = blocking(pool, move |conn| {
CommunityModeratorView::for_community(&conn, id)
})
.await??;
let moderators = moderators.into_iter().map(|m| m.user_actor_id).collect();
oprops
.set_context_xsd_any_uri(context())?
.set_id(self.actor_id.to_owned())?
.set_id(base_url.to_owned())?
.set_name_xsd_string(self.name.to_owned())?
.set_published(convert_datetime(self.published))?
.set_many_attributed_to_xsd_any_uris(moderators)?;
.set_attributed_to_xsd_any_uri(make_apub_endpoint(
EndpointType::User,
&self.creator_id.to_string(),
))?;
if let Some(u) = self.updated.to_owned() {
oprops.set_updated(convert_datetime(u))?;
}
if let Some(d) = self.description.to_owned() {
// TODO: this should be html, also add source field with raw markdown
// -> same for post.content and others
oprops.set_content_xsd_string(d)?;
oprops.set_summary_xsd_string(d)?;
}
let mut endpoint_props = EndpointProperties::default();
endpoint_props.set_shared_inbox(self.get_shared_inbox_url())?;
let mut actor_props = ApActorProperties::default();
actor_props
.set_preferred_username(self.title.to_owned())?
.set_inbox(self.get_inbox_url())?
.set_outbox(self.get_outbox_url())?
.set_endpoints(endpoint_props)?
.set_followers(self.get_followers_url())?;
.set_inbox(format!("{}/inbox", &base_url))?
.set_outbox(format!("{}/outbox", &base_url))?
.set_followers(format!("{}/followers", &base_url))?;
let nsfw = self.nsfw;
let category_id = self.category_id;
let group_extension = blocking(pool, move |conn| {
GroupExtension::new(conn, category_id, nsfw)
})
.await??;
Ok(Ext3::new(
group,
group_extension,
actor_props,
self.get_public_key_ext(),
))
}
fn to_tombstone(&self) -> Result<Tombstone, LemmyError> {
create_tombstone(
self.deleted,
&self.actor_id,
self.updated,
GroupType.to_string(),
)
Ok(group.extend(actor_props))
}
}
#[async_trait::async_trait(?Send)]
impl ActorType for Community {
fn actor_id(&self) -> String {
self.actor_id.to_owned()
}
fn public_key(&self) -> String {
self.public_key.to_owned().unwrap()
}
fn private_key(&self) -> String {
self.private_key.to_owned().unwrap()
}
/// As a local community, accept the follow request from a remote user.
async fn send_accept_follow(
&self,
follow: &Follow,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let actor_uri = follow.actor.as_single_xsd_any_uri().unwrap().to_string();
let id = format!("{}/accept/{}", self.actor_id, uuid::Uuid::new_v4());
let mut accept = Accept::new();
accept
.object_props
.set_context_xsd_any_uri(context())?
.set_id(id)?;
accept
.accept_props
.set_actor_xsd_any_uri(self.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(follow.clone())?)?;
let to = format!("{}/inbox", actor_uri);
insert_activity(self.creator_id, accept.clone(), true, pool).await?;
send_activity(client, &accept, self, vec![to]).await?;
Ok(())
}
async fn send_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let group = self.to_apub(pool).await?;
let id = format!("{}/delete/{}", self.actor_id, uuid::Uuid::new_v4());
let mut delete = Delete::default();
populate_object_props(
&mut delete.object_props,
vec![self.get_followers_url()],
&id,
)?;
delete
.delete_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(group)?)?;
insert_activity(self.creator_id, delete.clone(), true, pool).await?;
let inboxes = self.get_follower_inboxes(pool).await?;
// Note: For an accept, since it was automatic, no one pushed a button,
// the community was the actor.
// But for delete, the creator is the actor, and does the signing
send_activity(client, &delete, creator, inboxes).await?;
Ok(())
}
async fn send_undo_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let group = self.to_apub(pool).await?;
let id = format!("{}/delete/{}", self.actor_id, uuid::Uuid::new_v4());
let mut delete = Delete::default();
populate_object_props(
&mut delete.object_props,
vec![self.get_followers_url()],
&id,
)?;
delete
.delete_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(group)?)?;
// TODO
// Undo that fake activity
let undo_id = format!("{}/undo/delete/{}", self.actor_id, uuid::Uuid::new_v4());
let mut undo = Undo::default();
populate_object_props(
&mut undo.object_props,
vec![self.get_followers_url()],
&undo_id,
)?;
undo
.undo_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(delete)?;
insert_activity(self.creator_id, undo.clone(), true, pool).await?;
let inboxes = self.get_follower_inboxes(pool).await?;
// Note: For an accept, since it was automatic, no one pushed a button,
// the community was the actor.
// But for delete, the creator is the actor, and does the signing
send_activity(client, &undo, creator, inboxes).await?;
Ok(())
}
async fn send_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let group = self.to_apub(pool).await?;
let id = format!("{}/remove/{}", self.actor_id, uuid::Uuid::new_v4());
let mut remove = Remove::default();
populate_object_props(
&mut remove.object_props,
vec![self.get_followers_url()],
&id,
)?;
remove
.remove_props
.set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(group)?)?;
insert_activity(mod_.id, remove.clone(), true, pool).await?;
let inboxes = self.get_follower_inboxes(pool).await?;
// Note: For an accept, since it was automatic, no one pushed a button,
// the community was the actor.
// But for delete, the creator is the actor, and does the signing
send_activity(client, &remove, mod_, inboxes).await?;
Ok(())
}
async fn send_undo_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let group = self.to_apub(pool).await?;
let id = format!("{}/remove/{}", self.actor_id, uuid::Uuid::new_v4());
let mut remove = Remove::default();
populate_object_props(
&mut remove.object_props,
vec![self.get_followers_url()],
&id,
)?;
remove
.remove_props
.set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(group)?)?;
// Undo that fake activity
let undo_id = format!("{}/undo/remove/{}", self.actor_id, uuid::Uuid::new_v4());
let mut undo = Undo::default();
populate_object_props(
&mut undo.object_props,
vec![self.get_followers_url()],
&undo_id,
)?;
undo
.undo_props
.set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
.set_object_base_box(remove)?;
insert_activity(mod_.id, undo.clone(), true, pool).await?;
let inboxes = self.get_follower_inboxes(pool).await?;
// Note: For an accept, since it was automatic, no one pushed a button,
// the community was the actor.
// But for remove , the creator is the actor, and does the signing
send_activity(client, &undo, mod_, inboxes).await?;
Ok(())
}
/// For a given community, returns the inboxes of all followers.
async fn get_follower_inboxes(&self, pool: &DbPool) -> Result<Vec<String>, LemmyError> {
let id = self.id;
let inboxes = blocking(pool, move |conn| {
CommunityFollowerView::for_community(conn, id)
})
.await??;
let inboxes = inboxes
.into_iter()
.map(|c| get_shared_inbox(&c.user_actor_id))
.filter(|s| !s.is_empty())
.unique()
.collect();
Ok(inboxes)
}
async fn send_follow(
&self,
_follow_actor_id: &str,
_client: &Client,
_pool: &DbPool,
) -> Result<(), LemmyError> {
unimplemented!()
}
async fn send_unfollow(
&self,
_follow_actor_id: &str,
_client: &Client,
_pool: &DbPool,
) -> Result<(), LemmyError> {
unimplemented!()
}
}
#[async_trait::async_trait(?Send)]
impl FromApub for CommunityForm {
type ApubType = GroupExt;
/// Parse an ActivityPub group received from another instance into a Lemmy community.
async fn from_apub(group: &GroupExt, client: &Client, pool: &DbPool) -> Result<Self, LemmyError> {
let group_extensions: &GroupExtension = &group.ext_one;
let oprops = &group.inner.object_props;
let aprops = &group.ext_two;
let public_key: &PublicKey = &group.ext_three.public_key;
let mut creator_and_moderator_uris = oprops.get_many_attributed_to_xsd_any_uris().unwrap();
let creator_uri = creator_and_moderator_uris.next().unwrap();
let creator = get_or_fetch_and_upsert_remote_user(creator_uri.as_str(), client, pool).await?;
Ok(CommunityForm {
name: oprops.get_name_xsd_string().unwrap().to_string(),
impl CommunityView {
pub fn from_group(group: &GroupExt, domain: &str) -> Result<CommunityView, Error> {
let followers_uri = &group.extension.get_followers().unwrap().to_string();
let outbox_uri = &group.extension.get_outbox().to_string();
let outbox = fetch_remote_object::<OrderedCollection>(outbox_uri)?;
let followers = fetch_remote_object::<UnorderedCollection>(followers_uri)?;
let oprops = &group.base.object_props;
let aprops = &group.extension;
Ok(CommunityView {
// TODO: we need to merge id and name into a single thing (stuff like @user@instance.com)
id: 1337, //community.object_props.get_id()
name: format_community_name(&oprops.get_name_xsd_string().unwrap().to_string(), domain),
title: aprops.get_preferred_username().unwrap().to_string(),
// TODO: should be parsed as html and tags like <script> removed (or use markdown source)
// -> same for post.content etc
description: oprops.get_content_xsd_string().map(|s| s.to_string()),
category_id: group_extensions.category.identifier.parse::<i32>()?,
creator_id: creator.id,
removed: None,
description: oprops.get_summary_xsd_string().map(|s| s.to_string()),
category_id: -1,
creator_id: -1, //community.object_props.get_attributed_to_xsd_any_uri()
removed: false,
published: oprops
.get_published()
.map(|u| u.as_ref().to_owned().naive_local()),
.unwrap()
.as_ref()
.naive_local()
.to_owned(),
updated: oprops
.get_updated()
.map(|u| u.as_ref().to_owned().naive_local()),
deleted: None,
nsfw: group_extensions.sensitive,
actor_id: oprops.get_id().unwrap().to_string(),
local: false,
private_key: None,
public_key: Some(public_key.to_owned().public_key_pem),
last_refreshed_at: Some(naive_now()),
deleted: false,
nsfw: false,
creator_name: "".to_string(),
creator_avatar: None,
category_name: "".to_string(),
number_of_subscribers: *followers
.collection_props
.get_total_items()
.unwrap()
.as_ref() as i64,
number_of_posts: *outbox.collection_props.get_total_items().unwrap().as_ref() as i64,
number_of_comments: -1,
hot_rank: -1,
user_id: None,
subscribed: None,
})
}
}
/// Return the community json over HTTP.
pub async fn get_apub_community_http(
info: web::Path<CommunityQuery>,
db: DbPoolParam,
) -> Result<HttpResponse<Body>, LemmyError> {
let community = blocking(&db, move |conn| {
Community::read_from_name(conn, &info.community_name)
})
.await??;
if !community.deleted {
let apub = community.to_apub(&db).await?;
Ok(create_apub_response(&apub))
} else {
Ok(create_apub_tombstone_response(&community.to_tombstone()?))
}
info: Path<CommunityQuery>,
db: web::Data<Pool<ConnectionManager<PgConnection>>>,
) -> Result<HttpResponse<Body>, Error> {
let community = Community::read_from_name(&&db.get()?, info.community_name.to_owned())?;
let c = community.as_group()?;
Ok(create_apub_response(&c))
}
/// Returns an empty followers collection, only populating the size (for privacy).
pub async fn get_apub_community_followers(
info: web::Path<CommunityQuery>,
db: DbPoolParam,
) -> Result<HttpResponse<Body>, LemmyError> {
let community = blocking(&db, move |conn| {
Community::read_from_name(&conn, &info.community_name)
})
.await??;
info: Path<CommunityQuery>,
db: web::Data<Pool<ConnectionManager<PgConnection>>>,
) -> Result<HttpResponse<Body>, Error> {
let community = Community::read_from_name(&&db.get()?, info.community_name.to_owned())?;
let base_url = make_apub_endpoint(EndpointType::Community, &community.name);
let community_id = community.id;
let community_followers = blocking(&db, move |conn| {
CommunityFollowerView::for_community(&conn, community_id)
})
.await??;
let connection = establish_unpooled_connection();
//As we are an object, we validated that the community id was valid
let community_followers =
CommunityFollowerView::for_community(&connection, community.id).unwrap();
let mut collection = UnorderedCollection::default();
let oprops: &mut ObjectProperties = collection.as_mut();
oprops
.set_context_xsd_any_uri(context())?
.set_id(community.actor_id)?;
.set_id(base_url)?;
collection
.collection_props
.set_total_items(community_followers.len() as u64)?;
Ok(create_apub_response(&collection))
}
impl Community {
pub async fn do_announce<A>(
activity: A,
community: &Community,
sender: &dyn ActorType,
client: &Client,
pool: &DbPool,
) -> Result<HttpResponse, LemmyError>
where
A: Activity + Base + Serialize + Debug,
{
let mut announce = Announce::default();
populate_object_props(
&mut announce.object_props,
vec![community.get_followers_url()],
&format!("{}/announce/{}", community.actor_id, uuid::Uuid::new_v4()),
)?;
announce
.announce_props
.set_actor_xsd_any_uri(community.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(activity)?)?;
pub async fn get_apub_community_outbox(
info: Path<CommunityQuery>,
db: web::Data<Pool<ConnectionManager<PgConnection>>>,
) -> Result<HttpResponse<Body>, Error> {
let community = Community::read_from_name(&&db.get()?, info.community_name.to_owned())?;
let base_url = make_apub_endpoint(EndpointType::Community, &community.name);
insert_activity(community.creator_id, announce.clone(), true, pool).await?;
let connection = establish_unpooled_connection();
//As we are an object, we validated that the community id was valid
let community_posts: Vec<Post> = Post::list_for_community(&connection, community.id)?;
// dont send to the instance where the activity originally came from, because that would result
// in a database error (same data inserted twice)
let mut to = community.get_follower_inboxes(pool).await?;
let mut collection = OrderedCollection::default();
let oprops: &mut ObjectProperties = collection.as_mut();
oprops
.set_context_xsd_any_uri(context())?
.set_id(base_url)?;
collection
.collection_props
.set_many_items_base_boxes(
community_posts
.iter()
.map(|c| c.as_page().unwrap())
.collect(),
)?
.set_total_items(community_posts.len() as u64)?;
// this seems to be the "easiest" stable alternative for remove_item()
to.retain(|x| *x != sender.get_shared_inbox_url());
send_activity(client, &announce, community, to).await?;
Ok(HttpResponse::Ok().finish())
}
Ok(create_apub_response(&collection))
}

136
server/src/apub/community_inbox.rs
View File

@ -1,136 +0,0 @@
use crate::{
apub::{
extensions::signatures::verify,
fetcher::{get_or_fetch_and_upsert_remote_community, get_or_fetch_and_upsert_remote_user},
ActorType,
},
blocking,
db::{
activity::insert_activity,
community::{Community, CommunityFollower, CommunityFollowerForm},
user::User_,
Followable,
},
routes::{ChatServerParam, DbPoolParam},
LemmyError,
};
use activitystreams::activity::Undo;
use activitystreams_new::activity::Follow;
use actix_web::{client::Client, web, HttpRequest, HttpResponse};
use log::debug;
use serde::Deserialize;
use std::fmt::Debug;
#[serde(untagged)]
#[derive(Deserialize, Debug)]
pub enum CommunityAcceptedObjects {
Follow(Follow),
Undo(Undo),
}
impl CommunityAcceptedObjects {
fn follow(&self) -> Result<Follow, LemmyError> {
match self {
CommunityAcceptedObjects::Follow(f) => Ok(f.to_owned()),
CommunityAcceptedObjects::Undo(u) => Ok(
u.undo_props
.get_object_base_box()
.to_owned()
.unwrap()
.to_owned()
.into_concrete::<Follow>()?,
),
}
}
}
/// Handler for all incoming activities to community inboxes.
pub async fn community_inbox(
request: HttpRequest,
input: web::Json<CommunityAcceptedObjects>,
path: web::Path<String>,
db: DbPoolParam,
client: web::Data<Client>,
_chat_server: ChatServerParam,
) -> Result<HttpResponse, LemmyError> {
let input = input.into_inner();
let path = path.into_inner();
let community = blocking(&db, move |conn| Community::read_from_name(&conn, &path)).await??;
if !community.local {
return Err(
format_err!(
"Received activity is addressed to remote community {}",
&community.actor_id
)
.into(),
);
}
debug!(
"Community {} received activity {:?}",
&community.name, &input
);
let follow = input.follow()?;
let user_uri = follow.actor.as_single_xsd_any_uri().unwrap().to_string();
let community_uri = follow.object.as_single_xsd_any_uri().unwrap().to_string();
let user = get_or_fetch_and_upsert_remote_user(&user_uri, &client, &db).await?;
let community = get_or_fetch_and_upsert_remote_community(&community_uri, &client, &db).await?;
verify(&request, &user)?;
match input {
CommunityAcceptedObjects::Follow(f) => handle_follow(f, user, community, &client, db).await,
CommunityAcceptedObjects::Undo(u) => handle_undo_follow(u, user, community, db).await,
}
}
/// Handle a follow request from a remote user, adding it to the local database and returning an
/// Accept activity.
async fn handle_follow(
follow: Follow,
user: User_,
community: Community,
client: &Client,
db: DbPoolParam,
) -> Result<HttpResponse, LemmyError> {
insert_activity(user.id, follow.clone(), false, &db).await?;
let community_follower_form = CommunityFollowerForm {
community_id: community.id,
user_id: user.id,
};
// This will fail if they're already a follower, but ignore the error.
blocking(&db, move |conn| {
CommunityFollower::follow(&conn, &community_follower_form).ok()
})
.await?;
community.send_accept_follow(&follow, &client, &db).await?;
Ok(HttpResponse::Ok().finish())
}
async fn handle_undo_follow(
undo: Undo,
user: User_,
community: Community,
db: DbPoolParam,
) -> Result<HttpResponse, LemmyError> {
insert_activity(user.id, undo, false, &db).await?;
let community_follower_form = CommunityFollowerForm {
community_id: community.id,
user_id: user.id,
};
// This will fail if they aren't a follower, but ignore the error.
blocking(&db, move |conn| {
CommunityFollower::unfollow(&conn, &community_follower_form).ok()
})
.await?;
Ok(HttpResponse::Ok().finish())
}

42
server/src/apub/extensions/group_extensions.rs
View File

@ -1,42 +0,0 @@
use crate::{
db::{category::Category, Crud},
LemmyError,
};
use activitystreams::{ext::Extension, Actor};
use diesel::PgConnection;
use serde::{Deserialize, Serialize};
#[derive(Clone, Debug, Default, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct GroupExtension {
pub category: GroupCategory,
pub sensitive: bool,
}
#[derive(Clone, Debug, Default, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct GroupCategory {
// Using a string because that's how Peertube does it.
pub identifier: String,
pub name: String,
}
impl GroupExtension {
pub fn new(
conn: &PgConnection,
category_id: i32,
sensitive: bool,
) -> Result<GroupExtension, LemmyError> {
let category = Category::read(conn, category_id)?;
let group_category = GroupCategory {
identifier: category_id.to_string(),
name: category.name,
};
Ok(GroupExtension {
category: group_category,
sensitive,
})
}
}
impl<T> Extension<T> for GroupExtension where T: Actor {}

3
server/src/apub/extensions/mod.rs
View File

@ -1,3 +0,0 @@
pub mod group_extensions;
pub mod page_extension;
pub mod signatures;

11
server/src/apub/extensions/page_extension.rs
View File

@ -1,11 +0,0 @@
use activitystreams::{ext::Extension, Base};
use serde::{Deserialize, Serialize};
#[derive(Clone, Debug, Default, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct PageExtension {
pub comments_enabled: bool,
pub sensitive: bool,
}
impl<T> Extension<T> for PageExtension where T: Base {}

119
server/src/apub/extensions/signatures.rs
View File

@ -1,119 +0,0 @@
use crate::{apub::ActorType, LemmyError};
use activitystreams::ext::Extension;
use actix_web::{client::ClientRequest, HttpRequest};
use http_signature_normalization_actix::{
digest::{DigestClient, SignExt},
Config,
};
use log::debug;
use openssl::{
hash::MessageDigest,
pkey::PKey,
rsa::Rsa,
sign::{Signer, Verifier},
};
use serde::{Deserialize, Serialize};
use sha2::{Digest, Sha256};
lazy_static! {
static ref HTTP_SIG_CONFIG: Config = Config::new();
}
pub struct Keypair {
pub private_key: String,
pub public_key: String,
}
/// Generate the asymmetric keypair for ActivityPub HTTP signatures.
pub fn generate_actor_keypair() -> Result<Keypair, LemmyError> {
let rsa = Rsa::generate(2048)?;
let pkey = PKey::from_rsa(rsa)?;
let public_key = pkey.public_key_to_pem()?;
let private_key = pkey.private_key_to_pem_pkcs8()?;
Ok(Keypair {
private_key: String::from_utf8(private_key)?,
public_key: String::from_utf8(public_key)?,
})
}
/// Signs request headers with the given keypair.
pub async fn sign(
request: ClientRequest,
actor: &dyn ActorType,
activity: String,
) -> Result<DigestClient<String>, LemmyError> {
let signing_key_id = format!("{}#main-key", actor.actor_id());
let private_key = actor.private_key();
let digest_client = request
.signature_with_digest(
HTTP_SIG_CONFIG.clone(),
signing_key_id,
Sha256::new(),
activity,
move |signing_string| {
let private_key = PKey::private_key_from_pem(private_key.as_bytes())?;
let mut signer = Signer::new(MessageDigest::sha256(), &private_key).unwrap();
signer.update(signing_string.as_bytes()).unwrap();
Ok(base64::encode(signer.sign_to_vec()?)) as Result<_, LemmyError>
},
)
.await?;
Ok(digest_client)
}
pub fn verify(request: &HttpRequest, actor: &dyn ActorType) -> Result<(), LemmyError> {
let verified = HTTP_SIG_CONFIG
.begin_verify(
request.method(),
request.uri().path_and_query(),
request.headers().clone(),
)?
.verify(|signature, signing_string| -> Result<bool, LemmyError> {
debug!(
"Verifying with key {}, message {}",
&actor.public_key(),
&signing_string
);
let public_key = PKey::public_key_from_pem(actor.public_key().as_bytes())?;
let mut verifier = Verifier::new(MessageDigest::sha256(), &public_key).unwrap();
verifier.update(&signing_string.as_bytes()).unwrap();
Ok(verifier.verify(&base64::decode(signature)?)?)
})?;
if verified {
debug!("verified signature for {}", &request.uri());
Ok(())
} else {
Err(format_err!("Invalid signature on request: {}", &request.uri()).into())
}
}
// The following is taken from here:
// https://docs.rs/activitystreams/0.5.0-alpha.17/activitystreams/ext/index.html
#[derive(Clone, Debug, Default, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct PublicKey {
pub id: String,
pub owner: String,
pub public_key_pem: String,
}
#[derive(Clone, Debug, Default, Deserialize, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct PublicKeyExtension {
pub public_key: PublicKey,
}
impl PublicKey {
pub fn to_ext(&self) -> PublicKeyExtension {
PublicKeyExtension {
public_key: self.to_owned(),
}
}
}
impl<T> Extension<T> for PublicKeyExtension where T: activitystreams::Actor {}

423
server/src/apub/fetcher.rs
View File

@ -1,423 +0,0 @@
use activitystreams::object::Note;
use actix_web::client::Client;
use diesel::{result::Error::NotFound, PgConnection};
use log::debug;
use serde::Deserialize;
use std::{fmt::Debug, time::Duration};
use url::Url;
use crate::{
api::site::SearchResponse,
blocking,
db::{
comment::{Comment, CommentForm},
comment_view::CommentView,
community::{Community, CommunityForm, CommunityModerator, CommunityModeratorForm},
community_view::CommunityView,
post::{Post, PostForm},
post_view::PostView,
user::{UserForm, User_},
Crud,
Joinable,
SearchType,
},
naive_now,
request::{retry, RecvError},
routes::nodeinfo::{NodeInfo, NodeInfoWellKnown},
DbPool,
LemmyError,
};
use crate::{
apub::{
get_apub_protocol_string,
is_apub_id_valid,
FromApub,
GroupExt,
PageExt,
PersonExt,
APUB_JSON_CONTENT_TYPE,
},
db::user_view::UserView,
};
use chrono::NaiveDateTime;
static ACTOR_REFETCH_INTERVAL_SECONDS: i64 = 24 * 60 * 60;
// Fetch nodeinfo metadata from a remote instance.
async fn _fetch_node_info(client: &Client, domain: &str) -> Result<NodeInfo, LemmyError> {
let well_known_uri = Url::parse(&format!(
"{}://{}/.well-known/nodeinfo",
get_apub_protocol_string(),
domain
))?;
let well_known = fetch_remote_object::<NodeInfoWellKnown>(client, &well_known_uri).await?;
let nodeinfo = fetch_remote_object::<NodeInfo>(client, &well_known.links.href).await?;
Ok(nodeinfo)
}
/// Fetch any type of ActivityPub object, handling things like HTTP headers, deserialisation,
/// timeouts etc.
pub async fn fetch_remote_object<Response>(
client: &Client,
url: &Url,
) -> Result<Response, LemmyError>
where
Response: for<'de> Deserialize<'de>,
{
if !is_apub_id_valid(&url) {
return Err(format_err!("Activitypub uri invalid or blocked: {}", url).into());
}
let timeout = Duration::from_secs(60);
let json = retry(|| {
client
.get(url.as_str())
.header("Accept", APUB_JSON_CONTENT_TYPE)
.timeout(timeout)
.send()
})
.await?
.json()
.await
.map_err(|e| {
debug!("Receive error, {}", e);
RecvError(e.to_string())
})?;
Ok(json)
}
/// The types of ActivityPub objects that can be fetched directly by searching for their ID.
#[serde(untagged)]
#[derive(serde::Deserialize, Debug)]
pub enum SearchAcceptedObjects {
Person(Box<PersonExt>),
Group(Box<GroupExt>),
Page(Box<PageExt>),
Comment(Box<Note>),
}
/// Attempt to parse the query as URL, and fetch an ActivityPub object from it.
///
/// Some working examples for use with the docker/federation/ setup:
/// http://lemmy_alpha:8540/c/main, or !main@lemmy_alpha:8540
/// http://lemmy_alpha:8540/u/lemmy_alpha, or @lemmy_alpha@lemmy_alpha:8540
/// http://lemmy_alpha:8540/post/3
/// http://lemmy_alpha:8540/comment/2
pub async fn search_by_apub_id(
query: &str,
client: &Client,
pool: &DbPool,
) -> Result<SearchResponse, LemmyError> {
// Parse the shorthand query url
let query_url = if query.contains('@') {
debug!("{}", query);
let split = query.split('@').collect::<Vec<&str>>();
// User type will look like ['', username, instance]
// Community will look like [!community, instance]
let (name, instance) = if split.len() == 3 {
(format!("/u/{}", split[1]), split[2])
} else if split.len() == 2 {
if split[0].contains('!') {
let split2 = split[0].split('!').collect::<Vec<&str>>();
(format!("/c/{}", split2[1]), split[1])
} else {
return Err(format_err!("Invalid search query: {}", query).into());
}
} else {
return Err(format_err!("Invalid search query: {}", query).into());
};
let url = format!("{}://{}{}", get_apub_protocol_string(), instance, name);
Url::parse(&url)?
} else {
Url::parse(&query)?
};
let mut response = SearchResponse {
type_: SearchType::All.to_string(),
comments: vec![],
posts: vec![],
communities: vec![],
users: vec![],
};
let response = match fetch_remote_object::<SearchAcceptedObjects>(client, &query_url).await? {
SearchAcceptedObjects::Person(p) => {
let user_uri = p.inner.object_props.get_id().unwrap().to_string();
let user = get_or_fetch_and_upsert_remote_user(&user_uri, client, pool).await?;
response.users = vec![blocking(pool, move |conn| UserView::read(conn, user.id)).await??];
response
}
SearchAcceptedObjects::Group(g) => {
let community_uri = g.inner.object_props.get_id().unwrap().to_string();
let community =
get_or_fetch_and_upsert_remote_community(&community_uri, client, pool).await?;
// TODO Maybe at some point in the future, fetch all the history of a community
// fetch_community_outbox(&c, conn)?;
response.communities = vec![
blocking(pool, move |conn| {
CommunityView::read(conn, community.id, None)
})
.await??,
];
response
}
SearchAcceptedObjects::Page(p) => {
let post_form = PostForm::from_apub(&p, client, pool).await?;
let p = blocking(pool, move |conn| upsert_post(&post_form, conn)).await??;
response.posts = vec![blocking(pool, move |conn| PostView::read(conn, p.id, None)).await??];
response
}
SearchAcceptedObjects::Comment(c) => {
let post_url = c
.object_props
.get_many_in_reply_to_xsd_any_uris()
.unwrap()
.next()
.unwrap()
.to_string();
// TODO: also fetch parent comments if any
let post = fetch_remote_object(client, &Url::parse(&post_url)?).await?;
let post_form = PostForm::from_apub(&post, client, pool).await?;
let comment_form = CommentForm::from_apub(&c, client, pool).await?;
blocking(pool, move |conn| upsert_post(&post_form, conn)).await??;
let c = blocking(pool, move |conn| upsert_comment(&comment_form, conn)).await??;
response.comments =
vec![blocking(pool, move |conn| CommentView::read(conn, c.id, None)).await??];
response
}
};
Ok(response)
}
/// Check if a remote user exists, create if not found, if its too old update it.Fetch a user, insert/update it in the database and return the user.
pub async fn get_or_fetch_and_upsert_remote_user(
apub_id: &str,
client: &Client,
pool: &DbPool,
) -> Result<User_, LemmyError> {
let apub_id_owned = apub_id.to_owned();
let user = blocking(pool, move |conn| {
User_::read_from_actor_id(conn, &apub_id_owned)
})
.await?;
match user {
// If its older than a day, re-fetch it
Ok(u) if !u.local && should_refetch_actor(u.last_refreshed_at) => {
debug!("Fetching and updating from remote user: {}", apub_id);
let person = fetch_remote_object::<PersonExt>(client, &Url::parse(apub_id)?).await?;
let mut uf = UserForm::from_apub(&person, client, pool).await?;
uf.last_refreshed_at = Some(naive_now());
let user = blocking(pool, move |conn| User_::update(conn, u.id, &uf)).await??;
Ok(user)
}
Ok(u) => Ok(u),
Err(NotFound {}) => {
debug!("Fetching and creating remote user: {}", apub_id);
let person = fetch_remote_object::<PersonExt>(client, &Url::parse(apub_id)?).await?;
let uf = UserForm::from_apub(&person, client, pool).await?;
let user = blocking(pool, move |conn| User_::create(conn, &uf)).await??;
Ok(user)
}
Err(e) => Err(e.into()),
}
}
/// Determines when a remote actor should be refetched from its instance. In release builds, this is
/// ACTOR_REFETCH_INTERVAL_SECONDS after the last refetch, in debug builds always.
///
/// TODO it won't pick up new avatars, summaries etc until a day after.
/// Actors need an "update" activity pushed to other servers to fix this.
fn should_refetch_actor(last_refreshed: NaiveDateTime) -> bool {
if cfg!(debug_assertions) {
true
} else {
let update_interval = chrono::Duration::seconds(ACTOR_REFETCH_INTERVAL_SECONDS);
last_refreshed.lt(&(naive_now() - update_interval))
}
}
/// Check if a remote community exists, create if not found, if its too old update it.Fetch a community, insert/update it in the database and return the community.
pub async fn get_or_fetch_and_upsert_remote_community(
apub_id: &str,
client: &Client,
pool: &DbPool,
) -> Result<Community, LemmyError> {
let apub_id_owned = apub_id.to_owned();
let community = blocking(pool, move |conn| {
Community::read_from_actor_id(conn, &apub_id_owned)
})
.await?;
match community {
Ok(c) if !c.local && should_refetch_actor(c.last_refreshed_at) => {
debug!("Fetching and updating from remote community: {}", apub_id);
let group = fetch_remote_object::<GroupExt>(client, &Url::parse(apub_id)?).await?;
let mut cf = CommunityForm::from_apub(&group, client, pool).await?;
cf.last_refreshed_at = Some(naive_now());
let community = blocking(pool, move |conn| Community::update(conn, c.id, &cf)).await??;
Ok(community)
}
Ok(c) => Ok(c),
Err(NotFound {}) => {
debug!("Fetching and creating remote community: {}", apub_id);
let group = fetch_remote_object::<GroupExt>(client, &Url::parse(apub_id)?).await?;
let cf = CommunityForm::from_apub(&group, client, pool).await?;
let community = blocking(pool, move |conn| Community::create(conn, &cf)).await??;
// Also add the community moderators too
let creator_and_moderator_uris = group
.inner
.object_props
.get_many_attributed_to_xsd_any_uris()
.unwrap();
let mut creator_and_moderators = Vec::new();
for uri in creator_and_moderator_uris {
let c_or_m = get_or_fetch_and_upsert_remote_user(uri.as_str(), client, pool).await?;
creator_and_moderators.push(c_or_m);
}
let community_id = community.id;
blocking(pool, move |conn| {
for mod_ in creator_and_moderators {
let community_moderator_form = CommunityModeratorForm {
community_id,
user_id: mod_.id,
};
CommunityModerator::join(conn, &community_moderator_form)?;
}
Ok(()) as Result<(), LemmyError>
})
.await??;
Ok(community)
}
Err(e) => Err(e.into()),
}
}
fn upsert_post(post_form: &PostForm, conn: &PgConnection) -> Result<Post, LemmyError> {
let existing = Post::read_from_apub_id(conn, &post_form.ap_id);
match existing {
Err(NotFound {}) => Ok(Post::create(conn, &post_form)?),
Ok(p) => Ok(Post::update(conn, p.id, &post_form)?),
Err(e) => Err(e.into()),
}
}
pub async fn get_or_fetch_and_insert_remote_post(
post_ap_id: &str,
client: &Client,
pool: &DbPool,
) -> Result<Post, LemmyError> {
let post_ap_id_owned = post_ap_id.to_owned();
let post = blocking(pool, move |conn| {
Post::read_from_apub_id(conn, &post_ap_id_owned)
})
.await?;
match post {
Ok(p) => Ok(p),
Err(NotFound {}) => {
debug!("Fetching and creating remote post: {}", post_ap_id);
let post = fetch_remote_object::<PageExt>(client, &Url::parse(post_ap_id)?).await?;
let post_form = PostForm::from_apub(&post, client, pool).await?;
let post = blocking(pool, move |conn| Post::create(conn, &post_form)).await??;
Ok(post)
}
Err(e) => Err(e.into()),
}
}
fn upsert_comment(comment_form: &CommentForm, conn: &PgConnection) -> Result<Comment, LemmyError> {
let existing = Comment::read_from_apub_id(conn, &comment_form.ap_id);
match existing {
Err(NotFound {}) => Ok(Comment::create(conn, &comment_form)?),
Ok(p) => Ok(Comment::update(conn, p.id, &comment_form)?),
Err(e) => Err(e.into()),
}
}
pub async fn get_or_fetch_and_insert_remote_comment(
comment_ap_id: &str,
client: &Client,
pool: &DbPool,
) -> Result<Comment, LemmyError> {
let comment_ap_id_owned = comment_ap_id.to_owned();
let comment = blocking(pool, move |conn| {
Comment::read_from_apub_id(conn, &comment_ap_id_owned)
})
.await?;
match comment {
Ok(p) => Ok(p),
Err(NotFound {}) => {
debug!(
"Fetching and creating remote comment and its parents: {}",
comment_ap_id
);
let comment = fetch_remote_object::<Note>(client, &Url::parse(comment_ap_id)?).await?;
let comment_form = CommentForm::from_apub(&comment, client, pool).await?;
let comment = blocking(pool, move |conn| Comment::create(conn, &comment_form)).await??;
Ok(comment)
}
Err(e) => Err(e.into()),
}
}
// TODO It should not be fetching data from a community outbox.
// All posts, comments, comment likes, etc should be posts to our community_inbox
// The only data we should be periodically fetching (if it hasn't been fetched in the last day
// maybe), is community and user actors
// and user actors
// Fetch all posts in the outbox of the given user, and insert them into the database.
// fn fetch_community_outbox(community: &Community, conn: &PgConnection) -> Result<Vec<Post>, LemmyError> {
// let outbox_url = Url::parse(&community.get_outbox_url())?;
// let outbox = fetch_remote_object::<OrderedCollection>(&outbox_url)?;
// let items = outbox.collection_props.get_many_items_base_boxes();
// Ok(
// items
// .unwrap()
// .map(|obox: &BaseBox| -> Result<PostForm, LemmyError> {
// let page = obox.clone().to_concrete::<Page>()?;
// PostForm::from_page(&page, conn)
// })
// .map(|pf| upsert_post(&pf?, conn))
// .collect::<Result<Vec<Post>, LemmyError>>()?,
// )
// }

390
server/src/apub/mod.rs
View File

@ -1,88 +1,47 @@
pub mod activities;
pub mod comment;
pub mod community;
pub mod community_inbox;
pub mod extensions;
pub mod fetcher;
pub mod post;
pub mod private_message;
pub mod shared_inbox;
pub mod puller;
pub mod user;
pub mod user_inbox;
use crate::Settings;
use openssl::{pkey::PKey, rsa::Rsa};
use crate::{
apub::extensions::{
group_extensions::GroupExtension,
page_extension::PageExtension,
signatures::{PublicKey, PublicKeyExtension},
},
convert_datetime,
db::user::User_,
request::{retry, RecvError},
routes::webfinger::WebFingerResponse,
DbPool,
LemmyError,
MentionData,
Settings,
};
use activitystreams::{
actor::{properties::ApActorProperties, Group, Person},
object::Page,
};
use activitystreams_ext::{Ext1, Ext2, Ext3};
use activitystreams_new::{activity::Follow, object::Tombstone, prelude::*};
use actix_web::{body::Body, client::Client, HttpResponse};
use chrono::NaiveDateTime;
use log::debug;
use serde::Serialize;
use activitystreams::actor::{properties::ApActorProperties, Group};
use activitystreams::ext::Ext;
use actix_web::body::Body;
use actix_web::HttpResponse;
use url::Url;
type GroupExt = Ext3<Group, GroupExtension, ApActorProperties, PublicKeyExtension>;
type PersonExt = Ext2<Person, ApActorProperties, PublicKeyExtension>;
type PageExt = Ext1<Page, PageExtension>;
type GroupExt = Ext<Group, ApActorProperties>;
pub static APUB_JSON_CONTENT_TYPE: &str = "application/activity+json";
fn create_apub_response<T>(json: &T) -> HttpResponse<Body>
where
T: serde::ser::Serialize,
{
HttpResponse::Ok()
.content_type("application/activity+json")
.json(json)
}
pub enum EndpointType {
Community,
User,
Post,
Comment,
PrivateMessage,
}
/// Convert the data to json and turn it into an HTTP Response with the correct ActivityPub
/// headers.
fn create_apub_response<T>(data: &T) -> HttpResponse<Body>
where
T: Serialize,
{
HttpResponse::Ok()
.content_type(APUB_JSON_CONTENT_TYPE)
.json(data)
}
fn create_apub_tombstone_response<T>(data: &T) -> HttpResponse<Body>
where
T: Serialize,
{
HttpResponse::Gone()
.content_type(APUB_JSON_CONTENT_TYPE)
.json(data)
}
/// Generates the ActivityPub ID for a given object type and ID.
// TODO: we will probably need to change apub endpoint urls so that html and activity+json content
// types are handled at the same endpoint, so that you can copy the url into mastodon search
// and have it fetch the object.
pub fn make_apub_endpoint(endpoint_type: EndpointType, name: &str) -> Url {
let point = match endpoint_type {
EndpointType::Community => "c",
EndpointType::User => "u",
EndpointType::Community => "community",
dessalines commented 2020-04-04 00:09:30 +00:00
Outdated
Review
Copy Link

@nutomic I changed these to their full names, they're urls so no reason to not be explicit, and /c/ could be either a community or a comment.

@nutomic I changed these to their full names, they're urls so no reason to not be explicit, and /c/ could be either a community or a comment.
nutomic commented 2020-04-07 11:11:37 +00:00
Outdated
Review
Copy Link

Not gonna work, you would have to change them in routes/federation.rs as well. Anyway I think we will have to change the URLs so that they are the same for html and activity+json content types, so that you can simply copy the URL into the Masto search and have it fetch the object.

Not gonna work, you would have to change them in routes/federation.rs as well. Anyway I think we will have to change the URLs so that they are the same for html and activity+json content types, so that you can simply copy the URL into the Masto search and have it fetch the object.
nutomic commented 2020-04-07 11:15:49 +00:00
Outdated
Review
Copy Link

Gonna revert this for now, anyway we are not federating comments yet.

Gonna revert this for now, anyway we are not federating comments yet.
EndpointType::User => "user",
EndpointType::Post => "post",
EndpointType::Comment => "comment",
EndpointType::PrivateMessage => "private_message",
};
Url::parse(&format!(
"{}://{}/{}/{}",
"{}://{}/federation/{}/{}",
get_apub_protocol_string(),
Settings::get().hostname,
point,
@ -99,276 +58,53 @@ pub fn get_apub_protocol_string() -> &'static str {
}
}
// Checks if the ID has a valid format, correct scheme, and is in the allowed instance list.
fn is_apub_id_valid(apub_id: &Url) -> bool {
debug!("Checking {}", apub_id);
if apub_id.scheme() != get_apub_protocol_string() {
debug!("invalid scheme: {:?}", apub_id.scheme());
return false;
}
let allowed_instances: Vec<String> = Settings::get()
.federation
.allowed_instances
.split(',')
.map(|d| d.to_string())
.collect();
match apub_id.domain() {
Some(d) => {
let contains = allowed_instances.contains(&d.to_owned());
if !contains {
debug!("{} not in {:?}", d, allowed_instances);
}
contains
}
None => {
debug!("missing domain");
false
}
}
}
#[async_trait::async_trait(?Send)]
pub trait ToApub {
type Response;
async fn to_apub(&self, pool: &DbPool) -> Result<Self::Response, LemmyError>;
fn to_tombstone(&self) -> Result<Tombstone, LemmyError>;
}
/// Updated is actually the deletion time
fn create_tombstone(
deleted: bool,
object_id: &str,
updated: Option<NaiveDateTime>,
former_type: String,
) -> Result<Tombstone, LemmyError> {
if deleted {
if let Some(updated) = updated {
let mut tombstone = Tombstone::new();
tombstone.set_id(object_id.parse()?);
tombstone.set_former_type(former_type);
tombstone.set_deleted(convert_datetime(updated).into());
Ok(tombstone)
} else {
Err(format_err!("Cant convert to tombstone because updated time was None.").into())
}
} else {
Err(format_err!("Cant convert object to tombstone if it wasnt deleted").into())
}
}
#[async_trait::async_trait(?Send)]
pub trait FromApub {
type ApubType;
async fn from_apub(
apub: &Self::ApubType,
client: &Client,
pool: &DbPool,
) -> Result<Self, LemmyError>
where
Self: Sized;
}
#[async_trait::async_trait(?Send)]
pub trait ApubObjectType {
async fn send_create(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_update(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_undo_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_undo_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
}
#[async_trait::async_trait(?Send)]
pub trait ApubLikeableType {
async fn send_like(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_dislike(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_undo_like(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
}
pub fn get_shared_inbox(actor_id: &str) -> String {
let url = Url::parse(actor_id).unwrap();
format!(
"{}://{}{}/inbox",
&url.scheme(),
&url.host_str().unwrap(),
if let Some(port) = url.port() {
format!(":{}", port)
} else {
"".to_string()
},
pub fn gen_keypair() -> (Vec<u8>, Vec<u8>) {
let rsa = Rsa::generate(2048).expect("sign::gen_keypair: key generation error");
let pkey = PKey::from_rsa(rsa).expect("sign::gen_keypair: parsing error");
(
pkey
.public_key_to_pem()
.expect("sign::gen_keypair: public key encoding error"),
pkey
.private_key_to_pem_pkcs8()
.expect("sign::gen_keypair: private key encoding error"),
)
}
#[async_trait::async_trait(?Send)]
pub trait ActorType {
fn actor_id(&self) -> String;
pub fn gen_keypair_str() -> (String, String) {
let (public_key, private_key) = gen_keypair();
(vec_bytes_to_str(public_key), vec_bytes_to_str(private_key))
}
fn public_key(&self) -> String;
fn private_key(&self) -> String;
fn vec_bytes_to_str(bytes: Vec<u8>) -> String {
String::from_utf8_lossy(&bytes).into_owned()
}
// These two have default impls, since currently a community can't follow anything,
// and a user can't be followed (yet)
#[allow(unused_variables)]
async fn send_follow(
&self,
follow_actor_id: &str,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_unfollow(
&self,
follow_actor_id: &str,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
#[allow(unused_variables)]
async fn send_accept_follow(
&self,
follow: &Follow,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_undo_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
async fn send_undo_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError>;
/// For a given community, returns the inboxes of all followers.
async fn get_follower_inboxes(&self, pool: &DbPool) -> Result<Vec<String>, LemmyError>;
// TODO move these to the db rows
fn get_inbox_url(&self) -> String {
format!("{}/inbox", &self.actor_id())
}
fn get_shared_inbox_url(&self) -> String {
get_shared_inbox(&self.actor_id())
}
fn get_outbox_url(&self) -> String {
format!("{}/outbox", &self.actor_id())
}
fn get_followers_url(&self) -> String {
format!("{}/followers", &self.actor_id())
}
fn get_following_url(&self) -> String {
format!("{}/following", &self.actor_id())
}
fn get_liked_url(&self) -> String {
format!("{}/liked", &self.actor_id())
}
fn get_public_key_ext(&self) -> PublicKeyExtension {
PublicKey {
id: format!("{}#main-key", self.actor_id()),
owner: self.actor_id(),
public_key_pem: self.public_key(),
}
.to_ext()
/// If community is on local instance, don't include the @instance part. This is only for displaying
/// to the user and should never be used otherwise.
pub fn format_community_name(name: &str, instance: &str) -> String {
if instance == Settings::get().hostname {
format!("!{}", name)
} else {
format!("!{}@{}", name, instance)
}
}
pub async fn fetch_webfinger_url(
mention: &MentionData,
client: &Client,
) -> Result<String, LemmyError> {
let fetch_url = format!(
"{}://{}/.well-known/webfinger?resource=acct:{}@{}",
get_apub_protocol_string(),
mention.domain,
mention.name,
mention.domain
);
debug!("Fetching webfinger url: {}", &fetch_url);
let mut response = retry(|| client.get(&fetch_url).send()).await?;
let res: WebFingerResponse = response
.json()
.await
.map_err(|e| RecvError(e.to_string()))?;
let link = res
.links
.iter()
.find(|l| l.type_.eq(&Some("application/activity+json".to_string())))
.ok_or_else(|| format_err!("No application/activity+json link found."))?;
link
.href
.to_owned()
.ok_or_else(|| format_err!("No href found.").into())
pub fn get_following_instances() -> Vec<&'static str> {
Settings::get()
.federation
.followed_instances
.split(',')
.collect()
}
/// Returns a tuple of (username, domain) from an identifier like "main@dev.lemmy.ml"
fn split_identifier(identifier: &str) -> (String, String) {
let x: Vec<&str> = identifier.split('@').collect();
(x[0].replace("!", ""), x[1].to_string())
}
fn get_remote_community_uri(identifier: &str) -> String {
let (name, domain) = split_identifier(identifier);
format!("http://{}/federation/c/{}", domain, name)
}

646
server/src/apub/post.rs
View File

@ -1,41 +1,14 @@
use crate::{
apub::{
activities::{populate_object_props, send_activity_to_community},
create_apub_response,
create_apub_tombstone_response,
create_tombstone,
extensions::page_extension::PageExtension,
fetcher::{get_or_fetch_and_upsert_remote_community, get_or_fetch_and_upsert_remote_user},
get_apub_protocol_string,
ActorType,
ApubLikeableType,
ApubObjectType,
FromApub,
PageExt,
ToApub,
},
blocking,
convert_datetime,
db::{
community::Community,
post::{Post, PostForm},
user::User_,
Crud,
},
routes::DbPoolParam,
DbPool,
LemmyError,
Settings,
};
use activitystreams::{
activity::{Create, Delete, Dislike, Like, Remove, Undo, Update},
context,
object::{kind::PageType, properties::ObjectProperties, AnyImage, Image, Page},
BaseBox,
};
use activitystreams_ext::Ext1;
use activitystreams_new::object::Tombstone;
use actix_web::{body::Body, client::Client, web, HttpResponse};
use crate::apub::{create_apub_response, make_apub_endpoint, EndpointType};
use crate::db::post::Post;
use crate::db::post_view::PostView;
use crate::{convert_datetime, naive_now};
use activitystreams::{object::properties::ObjectProperties, object::Page};
use actix_web::body::Body;
use actix_web::web::Path;
use actix_web::{web, HttpResponse};
use diesel::r2d2::{ConnectionManager, Pool};
use diesel::PgConnection;
use failure::Error;
use serde::Deserialize;
#[derive(Deserialize)]
@ -43,579 +16,98 @@ pub struct PostQuery {
post_id: String,
}
/// Return the post json over HTTP.
pub async fn get_apub_post(
info: web::Path<PostQuery>,
db: DbPoolParam,
) -> Result<HttpResponse<Body>, LemmyError> {
info: Path<PostQuery>,
db: web::Data<Pool<ConnectionManager<PgConnection>>>,
) -> Result<HttpResponse<Body>, Error> {
let id = info.post_id.parse::<i32>()?;
let post = blocking(&db, move |conn| Post::read(conn, id)).await??;
if !post.deleted {
Ok(create_apub_response(&post.to_apub(&db).await?))
} else {
Ok(create_apub_tombstone_response(&post.to_tombstone()?))
}
let post = Post::read(&&db.get()?, id)?;
Ok(create_apub_response(&post.as_page()?))
}
#[async_trait::async_trait(?Send)]
impl ToApub for Post {
type Response = PageExt;
// Turn a Lemmy post into an ActivityPub page that can be sent out over the network.
async fn to_apub(&self, pool: &DbPool) -> Result<PageExt, LemmyError> {
impl Post {
pub fn as_page(&self) -> Result<Page, Error> {
let base_url = make_apub_endpoint(EndpointType::Post, &self.id.to_string());
let mut page = Page::default();
let oprops: &mut ObjectProperties = page.as_mut();
let creator_id = self.creator_id;
let creator = blocking(pool, move |conn| User_::read(conn, creator_id)).await??;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
oprops
// Not needed when the Post is embedded in a collection (like for community outbox)
// TODO: need to set proper context defining sensitive/commentsEnabled fields
// https://git.asonix.dog/Aardwolf/activitystreams/issues/5
.set_context_xsd_any_uri(context())?
.set_id(self.ap_id.to_owned())?
// Use summary field to be consistent with mastodon content warning.
// https://mastodon.xyz/@Louisa/103987265222901387.json
.set_summary_xsd_string(self.name.to_owned())?
//.set_context_xsd_any_uri(context())?
.set_id(base_url)?
.set_name_xsd_string(self.name.to_owned())?
.set_published(convert_datetime(self.published))?
.set_to_xsd_any_uri(community.actor_id)?
.set_attributed_to_xsd_any_uri(creator.actor_id)?;
.set_attributed_to_xsd_any_uri(make_apub_endpoint(
EndpointType::User,
&self.creator_id.to_string(),
))?;
if let Some(body) = &self.body {
oprops.set_content_xsd_string(body.to_owned())?;
}
// TODO: hacky code because we get self.url == Some("")
// https://github.com/LemmyNet/lemmy/issues/602
// https://github.com/dessalines/lemmy/issues/602
let url = self.url.as_ref().filter(|u| !u.is_empty());
if let Some(u) = url {
oprops.set_url_xsd_any_uri(u.to_owned())?;
// Embeds
let mut page_preview = Page::new();
page_preview
.object_props
.set_url_xsd_any_uri(u.to_owned())?;
if let Some(embed_title) = &self.embed_title {
page_preview
.object_props
.set_name_xsd_string(embed_title.to_owned())?;
}
if let Some(embed_description) = &self.embed_description {
page_preview
.object_props
.set_summary_xsd_string(embed_description.to_owned())?;
}
if let Some(embed_html) = &self.embed_html {
page_preview
.object_props
.set_content_xsd_string(embed_html.to_owned())?;
}
oprops.set_preview_base_box(page_preview)?;
}
if let Some(thumbnail_url) = &self.thumbnail_url {
let full_url = format!(
"{}://{}/pictshare/{}",
get_apub_protocol_string(),
Settings::get().hostname,
thumbnail_url
);
let mut image = Image::new();
image.object_props.set_url_xsd_any_uri(full_url)?;
let any_image = AnyImage::from_concrete(image)?;
oprops.set_image_any_image(any_image)?;
}
if let Some(u) = self.updated {
oprops.set_updated(convert_datetime(u))?;
}
let ext = PageExtension {
comments_enabled: !self.locked,
sensitive: self.nsfw,
};
Ok(Ext1::new(page, ext))
}
fn to_tombstone(&self) -> Result<Tombstone, LemmyError> {
create_tombstone(
self.deleted,
&self.ap_id,
self.updated,
PageType.to_string(),
)
Ok(page)
}
}
#[async_trait::async_trait(?Send)]
impl FromApub for PostForm {
type ApubType = PageExt;
/// Parse an ActivityPub page received from another instance into a Lemmy post.
async fn from_apub(
page: &PageExt,
client: &Client,
pool: &DbPool,
) -> Result<PostForm, LemmyError> {
let ext = &page.ext_one;
let oprops = &page.inner.object_props;
let creator_actor_id = &oprops.get_attributed_to_xsd_any_uri().unwrap().to_string();
let creator = get_or_fetch_and_upsert_remote_user(&creator_actor_id, client, pool).await?;
let community_actor_id = &oprops.get_to_xsd_any_uri().unwrap().to_string();
let community =
get_or_fetch_and_upsert_remote_community(&community_actor_id, client, pool).await?;
let thumbnail_url = match oprops.get_image_any_image() {
Some(any_image) => any_image
.to_owned()
.into_concrete::<Image>()?
.object_props
.get_url_xsd_any_uri()
.map(|u| u.to_string()),
None => None,
};
let url = oprops.get_url_xsd_any_uri().map(|u| u.to_string());
let (embed_title, embed_description, embed_html) = match oprops.get_preview_base_box() {
Some(preview) => {
let preview_page = preview.to_owned().into_concrete::<Page>()?;
let name = preview_page
.object_props
.get_name_xsd_string()
.map(|n| n.to_string());
let summary = preview_page
.object_props
.get_summary_xsd_string()
.map(|s| s.to_string());
let content = preview_page
.object_props
.get_content_xsd_string()
.map(|c| c.to_string());
(name, summary, content)
}
None => (None, None, None),
};
Ok(PostForm {
name: oprops.get_summary_xsd_string().unwrap().to_string(),
url,
impl PostView {
pub fn from_page(page: &Page) -> Result<PostView, Error> {
let oprops = &page.object_props;
Ok(PostView {
id: -1,
name: oprops.get_name_xsd_string().unwrap().to_string(),
url: oprops.get_url_xsd_any_uri().map(|u| u.to_string()),
body: oprops.get_content_xsd_string().map(|c| c.to_string()),
creator_id: creator.id,
community_id: community.id,
removed: None,
locked: Some(!ext.comments_enabled),
creator_id: -1,
community_id: -1,
removed: false,
locked: false,
published: oprops
.get_published()
.map(|u| u.as_ref().to_owned().naive_local()),
.unwrap()
.as_ref()
.naive_local()
.to_owned(),
updated: oprops
.get_updated()
.map(|u| u.as_ref().to_owned().naive_local()),
deleted: None,
nsfw: ext.sensitive,
stickied: None, // -> put it in "featured" collection of the community
embed_title,
embed_description,
embed_html,
thumbnail_url,
ap_id: oprops.get_id().unwrap().to_string(),
local: false,
deleted: false,
nsfw: false,
stickied: false,
embed_title: None,
embed_description: None,
embed_html: None,
thumbnail_url: None,
banned: false,
banned_from_community: false,
creator_name: "".to_string(),
creator_avatar: None,
community_name: "".to_string(),
community_removed: false,
community_deleted: false,
community_nsfw: false,
number_of_comments: -1,
score: -1,
upvotes: -1,
downvotes: -1,
hot_rank: -1,
newest_activity_time: naive_now(),
user_id: None,
my_vote: None,
subscribed: None,
read: None,
saved: None,
})
}
}
#[async_trait::async_trait(?Send)]
impl ApubObjectType for Post {
/// Send out information about a newly created post, to the followers of the community.
async fn send_create(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let page = self.to_apub(pool).await?;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/create/{}", self.ap_id, uuid::Uuid::new_v4());
let mut create = Create::new();
populate_object_props(
&mut create.object_props,
vec![community.get_followers_url()],
&id,
)?;
create
.create_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(page)?)?;
send_activity_to_community(
creator,
&community,
vec![community.get_shared_inbox_url()],
create,
client,
pool,
)
.await?;
Ok(())
}
/// Send out information about an edited post, to the followers of the community.
async fn send_update(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let page = self.to_apub(pool).await?;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/update/{}", self.ap_id, uuid::Uuid::new_v4());
let mut update = Update::new();
populate_object_props(
&mut update.object_props,
vec![community.get_followers_url()],
&id,
)?;
update
.update_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(page)?)?;
send_activity_to_community(
creator,
&community,
vec![community.get_shared_inbox_url()],
update,
client,
pool,
)
.await?;
Ok(())
}
async fn send_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let page = self.to_apub(pool).await?;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
let mut delete = Delete::default();
populate_object_props(
&mut delete.object_props,
vec![community.get_followers_url()],
&id,
)?;
delete
.delete_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(page)?)?;
send_activity_to_community(
creator,
&community,
vec![community.get_shared_inbox_url()],
delete,
client,
pool,
)
.await?;
Ok(())
}
async fn send_undo_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let page = self.to_apub(pool).await?;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
let mut delete = Delete::default();
populate_object_props(
&mut delete.object_props,
vec![community.get_followers_url()],
&id,
)?;
delete
.delete_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(page)?)?;
// TODO
// Undo that fake activity
let undo_id = format!("{}/undo/delete/{}", self.ap_id, uuid::Uuid::new_v4());
let mut undo = Undo::default();
populate_object_props(
&mut undo.object_props,
vec![community.get_followers_url()],
&undo_id,
)?;
undo
.undo_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(delete)?;
send_activity_to_community(
creator,
&community,
vec![community.get_shared_inbox_url()],
undo,
client,
pool,
)
.await?;
Ok(())
}
async fn send_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let page = self.to_apub(pool).await?;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/remove/{}", self.ap_id, uuid::Uuid::new_v4());
let mut remove = Remove::default();
populate_object_props(
&mut remove.object_props,
vec![community.get_followers_url()],
&id,
)?;
remove
.remove_props
.set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(page)?)?;
send_activity_to_community(
mod_,
&community,
vec![community.get_shared_inbox_url()],
remove,
client,
pool,
)
.await?;
Ok(())
}
async fn send_undo_remove(
&self,
mod_: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let page = self.to_apub(pool).await?;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/remove/{}", self.ap_id, uuid::Uuid::new_v4());
let mut remove = Remove::default();
populate_object_props(
&mut remove.object_props,
vec![community.get_followers_url()],
&id,
)?;
remove
.remove_props
.set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(page)?)?;
// Undo that fake activity
let undo_id = format!("{}/undo/remove/{}", self.ap_id, uuid::Uuid::new_v4());
let mut undo = Undo::default();
populate_object_props(
&mut undo.object_props,
vec![community.get_followers_url()],
&undo_id,
)?;
undo
.undo_props
.set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
.set_object_base_box(remove)?;
send_activity_to_community(
mod_,
&community,
vec![community.get_shared_inbox_url()],
undo,
client,
pool,
)
.await?;
Ok(())
}
}
#[async_trait::async_trait(?Send)]
impl ApubLikeableType for Post {
async fn send_like(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let page = self.to_apub(pool).await?;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/like/{}", self.ap_id, uuid::Uuid::new_v4());
let mut like = Like::new();
populate_object_props(
&mut like.object_props,
vec![community.get_followers_url()],
&id,
)?;
like
.like_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(page)?)?;
send_activity_to_community(
&creator,
&community,
vec![community.get_shared_inbox_url()],
like,
client,
pool,
)
.await?;
Ok(())
}
async fn send_dislike(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let page = self.to_apub(pool).await?;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/dislike/{}", self.ap_id, uuid::Uuid::new_v4());
let mut dislike = Dislike::new();
populate_object_props(
&mut dislike.object_props,
vec![community.get_followers_url()],
&id,
)?;
dislike
.dislike_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(page)?)?;
send_activity_to_community(
&creator,
&community,
vec![community.get_shared_inbox_url()],
dislike,
client,
pool,
)
.await?;
Ok(())
}
async fn send_undo_like(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let page = self.to_apub(pool).await?;
let community_id = self.community_id;
let community = blocking(pool, move |conn| Community::read(conn, community_id)).await??;
let id = format!("{}/like/{}", self.ap_id, uuid::Uuid::new_v4());
let mut like = Like::new();
populate_object_props(
&mut like.object_props,
vec![community.get_followers_url()],
&id,
)?;
like
.like_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(BaseBox::from_concrete(page)?)?;
// TODO
// Undo that fake activity
let undo_id = format!("{}/undo/like/{}", self.ap_id, uuid::Uuid::new_v4());
let mut undo = Undo::default();
populate_object_props(
&mut undo.object_props,
vec![community.get_followers_url()],
&undo_id,
)?;
undo
.undo_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(like)?;
send_activity_to_community(
&creator,
&community,
vec![community.get_shared_inbox_url()],
undo,
client,
pool,
)
.await?;
Ok(())
}
}

264
server/src/apub/private_message.rs
View File

@ -1,264 +0,0 @@
use crate::{
apub::{
activities::send_activity,
create_tombstone,
fetcher::get_or_fetch_and_upsert_remote_user,
ApubObjectType,
FromApub,
ToApub,
},
blocking,
convert_datetime,
db::{
activity::insert_activity,
private_message::{PrivateMessage, PrivateMessageForm},
user::User_,
Crud,
},
DbPool,
LemmyError,
};
use activitystreams::{
activity::{Create, Delete, Undo, Update},
context,
object::{kind::NoteType, properties::ObjectProperties, Note},
};
use activitystreams_new::object::Tombstone;
use actix_web::client::Client;
#[async_trait::async_trait(?Send)]
impl ToApub for PrivateMessage {
type Response = Note;
async fn to_apub(&self, pool: &DbPool) -> Result<Note, LemmyError> {
let mut private_message = Note::default();
let oprops: &mut ObjectProperties = private_message.as_mut();
let creator_id = self.creator_id;
let creator = blocking(pool, move |conn| User_::read(conn, creator_id)).await??;
let recipient_id = self.recipient_id;
let recipient = blocking(pool, move |conn| User_::read(conn, recipient_id)).await??;
oprops
.set_context_xsd_any_uri(context())?
.set_id(self.ap_id.to_owned())?
.set_published(convert_datetime(self.published))?
.set_content_xsd_string(self.content.to_owned())?
.set_to_xsd_any_uri(recipient.actor_id)?
.set_attributed_to_xsd_any_uri(creator.actor_id)?;
if let Some(u) = self.updated {
oprops.set_updated(convert_datetime(u))?;
}
Ok(private_message)
}
fn to_tombstone(&self) -> Result<Tombstone, LemmyError> {
create_tombstone(
self.deleted,
&self.ap_id,
self.updated,
NoteType.to_string(),
)
}
}
#[async_trait::async_trait(?Send)]
impl FromApub for PrivateMessageForm {
type ApubType = Note;
/// Parse an ActivityPub note received from another instance into a Lemmy Private message
async fn from_apub(
note: &Note,
client: &Client,
pool: &DbPool,
) -> Result<PrivateMessageForm, LemmyError> {
let oprops = &note.object_props;
let creator_actor_id = &oprops.get_attributed_to_xsd_any_uri().unwrap().to_string();
let creator = get_or_fetch_and_upsert_remote_user(&creator_actor_id, client, pool).await?;
let recipient_actor_id = &oprops.get_to_xsd_any_uri().unwrap().to_string();
let recipient = get_or_fetch_and_upsert_remote_user(&recipient_actor_id, client, pool).await?;
Ok(PrivateMessageForm {
creator_id: creator.id,
recipient_id: recipient.id,
content: oprops
.get_content_xsd_string()
.map(|c| c.to_string())
.unwrap(),
published: oprops
.get_published()
.map(|u| u.as_ref().to_owned().naive_local()),
updated: oprops
.get_updated()
.map(|u| u.as_ref().to_owned().naive_local()),
deleted: None,
read: None,
ap_id: oprops.get_id().unwrap().to_string(),
local: false,
})
}
}
#[async_trait::async_trait(?Send)]
impl ApubObjectType for PrivateMessage {
/// Send out information about a newly created private message
async fn send_create(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let id = format!("{}/create/{}", self.ap_id, uuid::Uuid::new_v4());
let recipient_id = self.recipient_id;
let recipient = blocking(pool, move |conn| User_::read(conn, recipient_id)).await??;
let mut create = Create::new();
create
.object_props
.set_context_xsd_any_uri(context())?
.set_id(id)?;
let to = format!("{}/inbox", recipient.actor_id);
create
.create_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
insert_activity(creator.id, create.clone(), true, pool).await?;
send_activity(client, &create, creator, vec![to]).await?;
Ok(())
}
/// Send out information about an edited post, to the followers of the community.
async fn send_update(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let id = format!("{}/update/{}", self.ap_id, uuid::Uuid::new_v4());
let recipient_id = self.recipient_id;
let recipient = blocking(pool, move |conn| User_::read(conn, recipient_id)).await??;
let mut update = Update::new();
update
.object_props
.set_context_xsd_any_uri(context())?
.set_id(id)?;
let to = format!("{}/inbox", recipient.actor_id);
update
.update_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
insert_activity(creator.id, update.clone(), true, pool).await?;
send_activity(client, &update, creator, vec![to]).await?;
Ok(())
}
async fn send_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
let recipient_id = self.recipient_id;
let recipient = blocking(pool, move |conn| User_::read(conn, recipient_id)).await??;
let mut delete = Delete::new();
delete
.object_props
.set_context_xsd_any_uri(context())?
.set_id(id)?;
let to = format!("{}/inbox", recipient.actor_id);
delete
.delete_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
insert_activity(creator.id, delete.clone(), true, pool).await?;
send_activity(client, &delete, creator, vec![to]).await?;
Ok(())
}
async fn send_undo_delete(
&self,
creator: &User_,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let note = self.to_apub(pool).await?;
let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
let recipient_id = self.recipient_id;
let recipient = blocking(pool, move |conn| User_::read(conn, recipient_id)).await??;
let mut delete = Delete::new();
delete
.object_props
.set_context_xsd_any_uri(context())?
.set_id(id)?;
let to = format!("{}/inbox", recipient.actor_id);
delete
.delete_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(note)?;
// TODO
// Undo that fake activity
let undo_id = format!("{}/undo/delete/{}", self.ap_id, uuid::Uuid::new_v4());
let mut undo = Undo::default();
undo
.object_props
.set_context_xsd_any_uri(context())?
.set_id(undo_id)?;
undo
.undo_props
.set_actor_xsd_any_uri(creator.actor_id.to_owned())?
.set_object_base_box(delete)?;
insert_activity(creator.id, undo.clone(), true, pool).await?;
send_activity(client, &undo, creator, vec![to]).await?;
Ok(())
}
async fn send_remove(
&self,
_mod_: &User_,
_client: &Client,
_pool: &DbPool,
) -> Result<(), LemmyError> {
unimplemented!()
}
async fn send_undo_remove(
&self,
_mod_: &User_,
_client: &Client,
_pool: &DbPool,
) -> Result<(), LemmyError> {
unimplemented!()
}
}

104
server/src/apub/puller.rs Normal file
View File

@ -0,0 +1,104 @@
use crate::api::community::GetCommunityResponse;
use crate::api::post::GetPostsResponse;
use crate::apub::*;
use crate::db::community_view::CommunityView;
use crate::db::post_view::PostView;
use crate::routes::nodeinfo::{NodeInfo, NodeInfoWellKnown};
use crate::settings::Settings;
use activitystreams::collection::{OrderedCollection, UnorderedCollection};
use activitystreams::object::Page;
use activitystreams::BaseBox;
use failure::Error;
use isahc::prelude::*;
use log::warn;
use serde::Deserialize;
fn fetch_node_info(domain: &str) -> Result<NodeInfo, Error> {
let well_known_uri = format!(
"{}://{}/.well-known/nodeinfo",
get_apub_protocol_string(),
domain
);
let well_known = fetch_remote_object::<NodeInfoWellKnown>(&well_known_uri)?;
Ok(fetch_remote_object::<NodeInfo>(&well_known.links.href)?)
}
fn fetch_communities_from_instance(domain: &str) -> Result<Vec<CommunityView>, Error> {
let node_info = fetch_node_info(domain)?;
if let Some(community_list_url) = node_info.metadata.community_list_url {
let collection = fetch_remote_object::<UnorderedCollection>(&community_list_url)?;
let object_boxes = collection
.collection_props
.get_many_items_base_boxes()
.unwrap();
let communities: Result<Vec<CommunityView>, Error> = object_boxes
.map(|c| -> Result<CommunityView, Error> {
let group = c.to_owned().to_concrete::<GroupExt>()?;
CommunityView::from_group(&group, domain)
})
.collect();
Ok(communities?)
} else {
Err(format_err!(
"{} is not a Lemmy instance, federation is not supported",
domain
))
}
}
pub fn fetch_remote_object<Response>(uri: &str) -> Result<Response, Error>
where
Response: for<'de> Deserialize<'de>,
{
if Settings::get().federation.tls_enabled && !uri.starts_with("https://") {
return Err(format_err!("Activitypub uri is insecure: {}", uri));
}
// TODO: should cache responses here when we are in production
// TODO: this function should return a future
let text = isahc::get(uri)?.text()?;
let res: Response = serde_json::from_str(&text)?;
Ok(res)
}
pub fn fetch_remote_community_posts(identifier: &str) -> Result<GetPostsResponse, Error> {
let community = fetch_remote_object::<GroupExt>(&get_remote_community_uri(identifier))?;
let outbox_uri = &community.extension.get_outbox().to_string();
let outbox = fetch_remote_object::<OrderedCollection>(outbox_uri)?;
let items = outbox.collection_props.get_many_items_base_boxes();
let posts: Result<Vec<PostView>, Error> = items
.unwrap()
.map(|obox: &BaseBox| {
let page = obox.clone().to_concrete::<Page>().unwrap();
PostView::from_page(&page)
})
.collect();
Ok(GetPostsResponse { posts: posts? })
}
pub fn fetch_remote_community(identifier: &str) -> Result<GetCommunityResponse, failure::Error> {
let group = fetch_remote_object::<GroupExt>(&get_remote_community_uri(identifier))?;
// TODO: this is only for testing until we can call that function from GetPosts
// (once string ids are supported)
//dbg!(get_remote_community_posts(identifier)?);
let (_, domain) = split_identifier(identifier);
Ok(GetCommunityResponse {
moderators: vec![],
admins: vec![],
community: CommunityView::from_group(&group, &domain)?,
online: 0,
})
}
pub fn fetch_all_communities() -> Result<Vec<CommunityView>, Error> {
let mut communities_list: Vec<CommunityView> = vec![];
for instance in &get_following_instances() {
match fetch_communities_from_instance(instance) {
Ok(mut c) => communities_list.append(c.as_mut()),
Err(e) => warn!("Failed to fetch instance list from remote instance: {}", e),
};
}
Ok(communities_list)
}

1800
server/src/apub/shared_inbox.rs
View File

File diff suppressed because it is too large Load Diff

285
server/src/apub/user.rs
View File

@ -1,38 +1,19 @@
use crate::{
apub::{
activities::send_activity,
create_apub_response,
extensions::signatures::PublicKey,
ActorType,
FromApub,
PersonExt,
ToApub,
},
blocking,
convert_datetime,
db::{
activity::insert_activity,
user::{UserForm, User_},
},
naive_now,
routes::DbPoolParam,
DbPool,
LemmyError,
};
use crate::apub::{create_apub_response, make_apub_endpoint, EndpointType};
use crate::convert_datetime;
use crate::db::user::User_;
use activitystreams::{
actor::{properties::ApActorProperties, Person},
context,
endpoint::EndpointProperties,
object::{properties::ObjectProperties, AnyImage, Image},
primitives::XsdAnyUri,
ext::Extensible,
object::properties::ObjectProperties,
};
use activitystreams_ext::Ext2;
use activitystreams_new::{
activity::{Follow, Undo},
object::Tombstone,
prelude::*,
};
use actix_web::{body::Body, client::Client, web, HttpResponse};
use actix_web::body::Body;
use actix_web::web::Path;
use actix_web::HttpResponse;
use actix_web::{web, Result};
use diesel::r2d2::{ConnectionManager, Pool};
use diesel::PgConnection;
use failure::Error;
use serde::Deserialize;
#[derive(Deserialize)]
@ -40,223 +21,35 @@ pub struct UserQuery {
user_name: String,
}
#[async_trait::async_trait(?Send)]
impl ToApub for User_ {
type Response = PersonExt;
pub async fn get_apub_user(
info: Path<UserQuery>,
db: web::Data<Pool<ConnectionManager<PgConnection>>>,
) -> Result<HttpResponse<Body>, Error> {
let user = User_::find_by_email_or_username(&&db.get()?, &info.user_name)?;
let base_url = make_apub_endpoint(EndpointType::User, &user.name);
// Turn a Lemmy Community into an ActivityPub group that can be sent out over the network.
async fn to_apub(&self, _pool: &DbPool) -> Result<PersonExt, LemmyError> {
// TODO go through all these to_string and to_owned()
let mut person = Person::default();
let oprops: &mut ObjectProperties = person.as_mut();
oprops
.set_context_xsd_any_uri(context())?
.set_id(self.actor_id.to_string())?
.set_name_xsd_string(self.name.to_owned())?
.set_published(convert_datetime(self.published))?;
let mut person = Person::default();
let oprops: &mut ObjectProperties = person.as_mut();
oprops
.set_context_xsd_any_uri(context())?
.set_id(base_url.to_string())?
.set_published(convert_datetime(user.published))?;
if let Some(u) = self.updated {
oprops.set_updated(convert_datetime(u))?;
}
if let Some(i) = &self.preferred_username {
oprops.set_name_xsd_string(i.to_owned())?;
}
if let Some(avatar_url) = &self.avatar {
let mut image = Image::new();
image
.object_props
.set_url_xsd_any_uri(avatar_url.to_owned())?;
let any_image = AnyImage::from_concrete(image)?;
oprops.set_icon_any_image(any_image)?;
}
let mut endpoint_props = EndpointProperties::default();
endpoint_props.set_shared_inbox(self.get_shared_inbox_url())?;
let mut actor_props = ApActorProperties::default();
actor_props
.set_inbox(self.get_inbox_url())?
.set_outbox(self.get_outbox_url())?
.set_endpoints(endpoint_props)?
.set_followers(self.get_followers_url())?
.set_following(self.get_following_url())?
.set_liked(self.get_liked_url())?;
Ok(Ext2::new(person, actor_props, self.get_public_key_ext()))
if let Some(u) = user.updated {
oprops.set_updated(convert_datetime(u))?;
}
fn to_tombstone(&self) -> Result<Tombstone, LemmyError> {
unimplemented!()
if let Some(i) = &user.preferred_username {
oprops.set_name_xsd_string(i.to_owned())?;
}
}
#[async_trait::async_trait(?Send)]
impl ActorType for User_ {
fn actor_id(&self) -> String {
self.actor_id.to_owned()
}
fn public_key(&self) -> String {
self.public_key.to_owned().unwrap()
}
fn private_key(&self) -> String {
self.private_key.to_owned().unwrap()
}
/// As a given local user, send out a follow request to a remote community.
async fn send_follow(
&self,
follow_actor_id: &str,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let id = format!("{}/follow/{}", self.actor_id, uuid::Uuid::new_v4());
let mut follow = Follow::new(self.actor_id.to_owned(), follow_actor_id);
follow.set_context(context()).set_id(id.parse()?);
let to = format!("{}/inbox", follow_actor_id);
insert_activity(self.id, follow.clone(), true, pool).await?;
send_activity(client, &follow, self, vec![to]).await?;
Ok(())
}
async fn send_unfollow(
&self,
follow_actor_id: &str,
client: &Client,
pool: &DbPool,
) -> Result<(), LemmyError> {
let id = format!("{}/follow/{}", self.actor_id, uuid::Uuid::new_v4());
let mut follow = Follow::new(self.actor_id.to_owned(), follow_actor_id);
follow.set_context(context()).set_id(id.parse()?);
let to = format!("{}/inbox", follow_actor_id);
// TODO
// Undo that fake activity
let undo_id = format!("{}/undo/follow/{}", self.actor_id, uuid::Uuid::new_v4());
let mut undo = Undo::new(self.actor_id.parse::<XsdAnyUri>()?, follow.into_any_base()?);
undo.set_context(context()).set_id(undo_id.parse()?);
insert_activity(self.id, undo.clone(), true, pool).await?;
send_activity(client, &undo, self, vec![to]).await?;
Ok(())
}
async fn send_delete(
&self,
_creator: &User_,
_client: &Client,
_pool: &DbPool,
) -> Result<(), LemmyError> {
unimplemented!()
}
async fn send_undo_delete(
&self,
_creator: &User_,
_client: &Client,
_pool: &DbPool,
) -> Result<(), LemmyError> {
unimplemented!()
}
async fn send_remove(
&self,
_creator: &User_,
_client: &Client,
_pool: &DbPool,
) -> Result<(), LemmyError> {
unimplemented!()
}
async fn send_undo_remove(
&self,
_creator: &User_,
_client: &Client,
_pool: &DbPool,
) -> Result<(), LemmyError> {
unimplemented!()
}
async fn send_accept_follow(
&self,
_follow: &Follow,
_client: &Client,
_pool: &DbPool,
) -> Result<(), LemmyError> {
unimplemented!()
}
async fn get_follower_inboxes(&self, _pool: &DbPool) -> Result<Vec<String>, LemmyError> {
unimplemented!()
}
}
#[async_trait::async_trait(?Send)]
impl FromApub for UserForm {
type ApubType = PersonExt;
/// Parse an ActivityPub person received from another instance into a Lemmy user.
async fn from_apub(person: &PersonExt, _: &Client, _: &DbPool) -> Result<Self, LemmyError> {
let oprops = &person.inner.object_props;
let aprops = &person.ext_one;
let public_key: &PublicKey = &person.ext_two.public_key;
let avatar = match oprops.get_icon_any_image() {
Some(any_image) => any_image
.to_owned()
.into_concrete::<Image>()?
.object_props
.get_url_xsd_any_uri()
.map(|u| u.to_string()),
None => None,
};
Ok(UserForm {
name: oprops.get_name_xsd_string().unwrap().to_string(),
preferred_username: aprops.get_preferred_username().map(|u| u.to_string()),
password_encrypted: "".to_string(),
admin: false,
banned: false,
email: None,
avatar,
updated: oprops
.get_updated()
.map(|u| u.as_ref().to_owned().naive_local()),
show_nsfw: false,
theme: "".to_string(),
default_sort_type: 0,
default_listing_type: 0,
lang: "".to_string(),
show_avatars: false,
send_notifications_to_email: false,
matrix_user_id: None,
actor_id: oprops.get_id().unwrap().to_string(),
bio: oprops.get_summary_xsd_string().map(|s| s.to_string()),
local: false,
private_key: None,
public_key: Some(public_key.to_owned().public_key_pem),
last_refreshed_at: Some(naive_now()),
})
}
}
/// Return the user json over HTTP.
pub async fn get_apub_user_http(
info: web::Path<UserQuery>,
db: DbPoolParam,
) -> Result<HttpResponse<Body>, LemmyError> {
let user_name = info.into_inner().user_name;
let user = blocking(&db, move |conn| {
User_::find_by_email_or_username(conn, &user_name)
})
.await??;
let u = user.to_apub(&db).await?;
Ok(create_apub_response(&u))
let mut actor_props = ApActorProperties::default();
actor_props
.set_inbox(format!("{}/inbox", &base_url))?
.set_outbox(format!("{}/outbox", &base_url))?
.set_following(format!("{}/following", &base_url))?
.set_liked(format!("{}/liked", &base_url))?;
Ok(create_apub_response(&person.extend(actor_props)))
}

372
server/src/apub/user_inbox.rs
View File

@ -1,372 +0,0 @@
use crate::{
api::user::PrivateMessageResponse,
apub::{
extensions::signatures::verify,
fetcher::{get_or_fetch_and_upsert_remote_community, get_or_fetch_and_upsert_remote_user},
FromApub,
},
blocking,
db::{
activity::insert_activity,
community::{CommunityFollower, CommunityFollowerForm},
private_message::{PrivateMessage, PrivateMessageForm},
private_message_view::PrivateMessageView,
user::User_,
Crud,
Followable,
},
naive_now,
routes::{ChatServerParam, DbPoolParam},
websocket::{server::SendUserRoomMessage, UserOperation},
DbPool,
LemmyError,
};
use activitystreams::{
activity::{Accept, Create, Delete, Undo, Update},
object::Note,
};
use actix_web::{client::Client, web, HttpRequest, HttpResponse};
use log::debug;
use serde::Deserialize;
use std::fmt::Debug;
#[serde(untagged)]
#[derive(Deserialize, Debug)]
pub enum UserAcceptedObjects {
Accept(Box<Accept>),
Create(Box<Create>),
Update(Box<Update>),
Delete(Box<Delete>),
Undo(Box<Undo>),
}
/// Handler for all incoming activities to user inboxes.
pub async fn user_inbox(
request: HttpRequest,
input: web::Json<UserAcceptedObjects>,
path: web::Path<String>,
client: web::Data<Client>,
db: DbPoolParam,
chat_server: ChatServerParam,
) -> Result<HttpResponse, LemmyError> {
// TODO: would be nice if we could do the signature check here, but we cant access the actor property
let input = input.into_inner();
let username = path.into_inner();
debug!("User {} received activity: {:?}", &username, &input);
match input {
UserAcceptedObjects::Accept(a) => receive_accept(*a, &request, &username, &client, &db).await,
UserAcceptedObjects::Create(c) => {
receive_create_private_message(*c, &request, &client, &db, chat_server).await
}
UserAcceptedObjects::Update(u) => {
receive_update_private_message(*u, &request, &client, &db, chat_server).await
}
UserAcceptedObjects::Delete(d) => {
receive_delete_private_message(*d, &request, &client, &db, chat_server).await
}
UserAcceptedObjects::Undo(u) => {
receive_undo_delete_private_message(*u, &request, &client, &db, chat_server).await
}
}
}
/// Handle accepted follows.
async fn receive_accept(
accept: Accept,
request: &HttpRequest,
username: &str,
client: &Client,
pool: &DbPool,
) -> Result<HttpResponse, LemmyError> {
let community_uri = accept
.accept_props
.get_actor_xsd_any_uri()
.unwrap()
.to_string();
let community = get_or_fetch_and_upsert_remote_community(&community_uri, client, pool).await?;
verify(request, &community)?;
let username = username.to_owned();
let user = blocking(pool, move |conn| User_::read_from_name(conn, &username)).await??;
insert_activity(community.creator_id, accept, false, pool).await?;
// Now you need to add this to the community follower
let community_follower_form = CommunityFollowerForm {
community_id: community.id,
user_id: user.id,
};
// This will fail if they're already a follower
blocking(pool, move |conn| {
CommunityFollower::follow(conn, &community_follower_form)
})
.await??;
// TODO: make sure that we actually requested a follow
Ok(HttpResponse::Ok().finish())
}
async fn receive_create_private_message(
create: Create,
request: &HttpRequest,
client: &Client,
pool: &DbPool,
chat_server: ChatServerParam,
) -> Result<HttpResponse, LemmyError> {
let note = create
.create_props
.get_object_base_box()
.to_owned()
.unwrap()
.to_owned()
.into_concrete::<Note>()?;
let user_uri = create
.create_props
.get_actor_xsd_any_uri()
.unwrap()
.to_string();
let user = get_or_fetch_and_upsert_remote_user(&user_uri, client, pool).await?;
verify(request, &user)?;
insert_activity(user.id, create, false, pool).await?;
let private_message = PrivateMessageForm::from_apub(&note, client, pool).await?;
let inserted_private_message = blocking(pool, move |conn| {
PrivateMessage::create(conn, &private_message)
})
.await??;
let message = blocking(pool, move |conn| {
PrivateMessageView::read(conn, inserted_private_message.id)
})
.await??;
let res = PrivateMessageResponse { message };
let recipient_id = res.message.recipient_id;
chat_server.do_send(SendUserRoomMessage {
op: UserOperation::CreatePrivateMessage,
response: res,
recipient_id,
my_id: None,
});
Ok(HttpResponse::Ok().finish())
}
async fn receive_update_private_message(
update: Update,
request: &HttpRequest,
client: &Client,
pool: &DbPool,
chat_server: ChatServerParam,
) -> Result<HttpResponse, LemmyError> {
let note = update
.update_props
.get_object_base_box()
.to_owned()
.unwrap()
.to_owned()
.into_concrete::<Note>()?;
let user_uri = update
.update_props
.get_actor_xsd_any_uri()
.unwrap()
.to_string();
let user = get_or_fetch_and_upsert_remote_user(&user_uri, client, pool).await?;
verify(request, &user)?;
insert_activity(user.id, update, false, pool).await?;
let private_message_form = PrivateMessageForm::from_apub(&note, client, pool).await?;
let private_message_ap_id = private_message_form.ap_id.clone();
let private_message = blocking(pool, move |conn| {
PrivateMessage::read_from_apub_id(conn, &private_message_ap_id)
})
.await??;
let private_message_id = private_message.id;
blocking(pool, move |conn| {
PrivateMessage::update(conn, private_message_id, &private_message_form)
})
.await??;
let private_message_id = private_message.id;
let message = blocking(pool, move |conn| {
PrivateMessageView::read(conn, private_message_id)
})
.await??;
let res = PrivateMessageResponse { message };
let recipient_id = res.message.recipient_id;
chat_server.do_send(SendUserRoomMessage {
op: UserOperation::EditPrivateMessage,
response: res,
recipient_id,
my_id: None,
});
Ok(HttpResponse::Ok().finish())
}
async fn receive_delete_private_message(
delete: Delete,
request: &HttpRequest,
client: &Client,
pool: &DbPool,
chat_server: ChatServerParam,
) -> Result<HttpResponse, LemmyError> {
let note = delete
.delete_props
.get_object_base_box()
.to_owned()
.unwrap()
.to_owned()
.into_concrete::<Note>()?;
let user_uri = delete
.delete_props
.get_actor_xsd_any_uri()
.unwrap()
.to_string();
let user = get_or_fetch_and_upsert_remote_user(&user_uri, client, pool).await?;
verify(request, &user)?;
insert_activity(user.id, delete, false, pool).await?;
let private_message_form = PrivateMessageForm::from_apub(&note, client, pool).await?;
let private_message_ap_id = private_message_form.ap_id;
let private_message = blocking(pool, move |conn| {
PrivateMessage::read_from_apub_id(conn, &private_message_ap_id)
})
.await??;
let private_message_form = PrivateMessageForm {
content: private_message_form.content,
recipient_id: private_message.recipient_id,
creator_id: private_message.creator_id,
deleted: Some(true),
read: None,
ap_id: private_message.ap_id,
local: private_message.local,
published: None,
updated: Some(naive_now()),
};
let private_message_id = private_message.id;
blocking(pool, move |conn| {
PrivateMessage::update(conn, private_message_id, &private_message_form)
})
.await??;
let private_message_id = private_message.id;
let message = blocking(pool, move |conn| {
PrivateMessageView::read(&conn, private_message_id)
})
.await??;
let res = PrivateMessageResponse { message };
let recipient_id = res.message.recipient_id;
chat_server.do_send(SendUserRoomMessage {
op: UserOperation::EditPrivateMessage,
response: res,
recipient_id,
my_id: None,
});
Ok(HttpResponse::Ok().finish())
}
async fn receive_undo_delete_private_message(
undo: Undo,
request: &HttpRequest,
client: &Client,
pool: &DbPool,
chat_server: ChatServerParam,
) -> Result<HttpResponse, LemmyError> {
let delete = undo
.undo_props
.get_object_base_box()
.to_owned()
.unwrap()
.to_owned()
.into_concrete::<Delete>()?;
let note = delete
.delete_props
.get_object_base_box()
.to_owned()
.unwrap()
.to_owned()
.into_concrete::<Note>()?;
let user_uri = delete
.delete_props
.get_actor_xsd_any_uri()
.unwrap()
.to_string();
let user = get_or_fetch_and_upsert_remote_user(&user_uri, client, pool).await?;
verify(request, &user)?;
insert_activity(user.id, delete, false, pool).await?;
let private_message = PrivateMessageForm::from_apub(&note, client, pool).await?;
let private_message_ap_id = private_message.ap_id.clone();
let private_message_id = blocking(pool, move |conn| {
PrivateMessage::read_from_apub_id(conn, &private_message_ap_id).map(|pm| pm.id)
})
.await??;
let private_message_form = PrivateMessageForm {
content: private_message.content,
recipient_id: private_message.recipient_id,
creator_id: private_message.creator_id,
deleted: Some(false),
read: None,
ap_id: private_message.ap_id,
local: private_message.local,
published: None,
updated: Some(naive_now()),
};
blocking(pool, move |conn| {
PrivateMessage::update(conn, private_message_id, &private_message_form)
})
.await??;
let message = blocking(pool, move |conn| {
PrivateMessageView::read(&conn, private_message_id)
})
.await??;
let res = PrivateMessageResponse { message };
let recipient_id = res.message.recipient_id;
chat_server.do_send(SendUserRoomMessage {
op: UserOperation::EditPrivateMessage,
response: res,
recipient_id,
my_id: None,
});
Ok(HttpResponse::Ok().finish())
}

165
server/src/db/activity.rs
View File

@ -1,165 +0,0 @@
use crate::{blocking, db::Crud, schema::activity, DbPool, LemmyError};
use diesel::{dsl::*, result::Error, *};
use log::debug;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::fmt::Debug;
#[derive(Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
#[table_name = "activity"]
pub struct Activity {
pub id: i32,
pub user_id: i32,
pub data: Value,
pub local: bool,
pub published: chrono::NaiveDateTime,
pub updated: Option<chrono::NaiveDateTime>,
}
#[derive(Insertable, AsChangeset, Clone, Serialize, Deserialize)]
#[table_name = "activity"]
pub struct ActivityForm {
pub user_id: i32,
pub data: Value,
pub local: bool,
pub updated: Option<chrono::NaiveDateTime>,
}
impl Crud<ActivityForm> for Activity {
fn read(conn: &PgConnection, activity_id: i32) -> Result<Self, Error> {
use crate::schema::activity::dsl::*;
activity.find(activity_id).first::<Self>(conn)
}
fn delete(conn: &PgConnection, activity_id: i32) -> Result<usize, Error> {
use crate::schema::activity::dsl::*;
diesel::delete(activity.find(activity_id)).execute(conn)
}
fn create(conn: &PgConnection, new_activity: &ActivityForm) -> Result<Self, Error> {
use crate::schema::activity::dsl::*;
insert_into(activity)
.values(new_activity)
.get_result::<Self>(conn)
}
fn update(
conn: &PgConnection,
activity_id: i32,
new_activity: &ActivityForm,
) -> Result<Self, Error> {
use crate::schema::activity::dsl::*;
diesel::update(activity.find(activity_id))
.set(new_activity)
.get_result::<Self>(conn)
}
}
pub async fn insert_activity<T>(
user_id: i32,
data: T,
local: bool,
pool: &DbPool,
) -> Result<(), LemmyError>
where
T: Serialize + Debug + Send + 'static,
{
blocking(pool, move |conn| {
do_insert_activity(conn, user_id, &data, local)
})
.await??;
Ok(())
}
fn do_insert_activity<T>(
conn: &PgConnection,
user_id: i32,
data: &T,
local: bool,
) -> Result<(), LemmyError>
where
T: Serialize + Debug,
{
let activity_form = ActivityForm {
user_id,
data: serde_json::to_value(&data)?,
local,
updated: None,
};
debug!("inserting activity for user {}, data {:?}", user_id, data);
Activity::create(&conn, &activity_form)?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::{super::user::*, *};
use crate::db::{establish_unpooled_connection, Crud, ListingType, SortType};
#[test]
fn test_crud() {
let conn = establish_unpooled_connection();
let creator_form = UserForm {
name: "activity_creator_pm".into(),
preferred_username: None,
password_encrypted: "nope".into(),
email: None,
matrix_user_id: None,
avatar: None,
admin: false,
banned: false,
updated: None,
show_nsfw: false,
theme: "darkly".into(),
default_sort_type: SortType::Hot as i16,
default_listing_type: ListingType::Subscribed as i16,
lang: "browser".into(),
show_avatars: true,
send_notifications_to_email: false,
actor_id: "http://fake.com".into(),
bio: None,
local: true,
private_key: None,
public_key: None,
last_refreshed_at: None,
};
let inserted_creator = User_::create(&conn, &creator_form).unwrap();
let test_json: Value = serde_json::from_str(
r#"{
"street": "Article Circle Expressway 1",
"city": "North Pole",
"postcode": "99705",
"state": "Alaska"
}"#,
)
.unwrap();
let activity_form = ActivityForm {
user_id: inserted_creator.id,
data: test_json.to_owned(),
local: true,
updated: None,
};
let inserted_activity = Activity::create(&conn, &activity_form).unwrap();
let expected_activity = Activity {
id: inserted_activity.id,
user_id: inserted_creator.id,
data: test_json,
local: true,
published: inserted_activity.published,
updated: None,
};
let read_activity = Activity::read(&conn, inserted_activity.id).unwrap();
let num_deleted = Activity::delete(&conn, inserted_activity.id).unwrap();
User_::delete(&conn, inserted_creator.id).unwrap();
assert_eq!(expected_activity, read_activity);
assert_eq!(expected_activity, inserted_activity);
assert_eq!(1, num_deleted);
}
}

11
server/src/db/category.rs
View File

@ -1,9 +1,6 @@
use crate::{
db::Crud,
schema::{category, category::dsl::*},
};
use diesel::{dsl::*, result::Error, *};
use serde::{Deserialize, Serialize};
use super::*;
use crate::schema::category;
use crate::schema::category::dsl::*;
#[derive(Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
#[table_name = "category"]
@ -53,8 +50,6 @@ impl Category {
#[cfg(test)]
mod tests {
use super::*;
use crate::db::establish_unpooled_connection;
#[test]
fn test_crud() {
let conn = establish_unpooled_connection();

101
server/src/db/code_migrations.rs
View File

@ -1,45 +1,35 @@
// This is for db migrations that require code
use super::{
comment::Comment,
community::{Community, CommunityForm},
post::Post,
private_message::PrivateMessage,
user::{UserForm, User_},
};
use crate::{
apub::{extensions::signatures::generate_actor_keypair, make_apub_endpoint, EndpointType},
db::Crud,
naive_now,
LemmyError,
};
use diesel::*;
use super::comment::Comment;
use super::community::{Community, CommunityForm};
use super::post::Post;
use super::user::{UserForm, User_};
use super::*;
use crate::apub::{gen_keypair_str, make_apub_endpoint, EndpointType};
use crate::naive_now;
use log::info;
pub fn run_advanced_migrations(conn: &PgConnection) -> Result<(), LemmyError> {
user_updates_2020_04_02(&conn)?;
community_updates_2020_04_02(&conn)?;
post_updates_2020_04_03(&conn)?;
comment_updates_2020_04_03(&conn)?;
private_message_updates_2020_05_05(&conn)?;
pub fn run_advanced_migrations(conn: &PgConnection) -> Result<(), Error> {
user_updates_2020_04_02(conn)?;
community_updates_2020_04_02(conn)?;
post_updates_2020_04_03(conn)?;
comment_updates_2020_04_03(conn)?;
Ok(())
}
fn user_updates_2020_04_02(conn: &PgConnection) -> Result<(), LemmyError> {
fn user_updates_2020_04_02(conn: &PgConnection) -> Result<(), Error> {
use crate::schema::user_::dsl::*;
info!("Running user_updates_2020_04_02");
// Update the actor_id, private_key, and public_key, last_refreshed_at
let incorrect_users = user_
.filter(actor_id.eq("http://fake.com"))
.filter(actor_id.eq("changeme"))
.filter(local.eq(true))
.load::<User_>(conn)?;
sql_query("alter table user_ disable trigger refresh_user").execute(conn)?;
for cuser in &incorrect_users {
let keypair = generate_actor_keypair()?;
let (user_public_key, user_private_key) = gen_keypair_str();
let form = UserForm {
name: cuser.name.to_owned(),
@ -61,36 +51,32 @@ fn user_updates_2020_04_02(conn: &PgConnection) -> Result<(), LemmyError> {
actor_id: make_apub_endpoint(EndpointType::User, &cuser.name).to_string(),
bio: cuser.bio.to_owned(),
local: cuser.local,
private_key: Some(keypair.private_key),
public_key: Some(keypair.public_key),
private_key: Some(user_private_key),
public_key: Some(user_public_key),
last_refreshed_at: Some(naive_now()),
};
User_::update(&conn, cuser.id, &form)?;
}
sql_query("alter table user_ enable trigger refresh_user").execute(conn)?;
info!("{} user rows updated.", incorrect_users.len());
Ok(())
}
fn community_updates_2020_04_02(conn: &PgConnection) -> Result<(), LemmyError> {
fn community_updates_2020_04_02(conn: &PgConnection) -> Result<(), Error> {
use crate::schema::community::dsl::*;
info!("Running community_updates_2020_04_02");
// Update the actor_id, private_key, and public_key, last_refreshed_at
let incorrect_communities = community
.filter(actor_id.eq("http://fake.com"))
.filter(actor_id.eq("changeme"))
.filter(local.eq(true))
.load::<Community>(conn)?;
sql_query("alter table community disable trigger refresh_community").execute(conn)?;
for ccommunity in &incorrect_communities {
let keypair = generate_actor_keypair()?;
let (community_public_key, community_private_key) = gen_keypair_str();
let form = CommunityForm {
name: ccommunity.name.to_owned(),
@ -104,90 +90,55 @@ fn community_updates_2020_04_02(conn: &PgConnection) -> Result<(), LemmyError> {
updated: None,
actor_id: make_apub_endpoint(EndpointType::Community, &ccommunity.name).to_string(),
local: ccommunity.local,
private_key: Some(keypair.private_key),
public_key: Some(keypair.public_key),
private_key: Some(community_private_key),
public_key: Some(community_public_key),
last_refreshed_at: Some(naive_now()),
published: None,
};
Community::update(&conn, ccommunity.id, &form)?;
}
sql_query("alter table community enable trigger refresh_community").execute(conn)?;
info!("{} community rows updated.", incorrect_communities.len());
Ok(())
}
fn post_updates_2020_04_03(conn: &PgConnection) -> Result<(), LemmyError> {
fn post_updates_2020_04_03(conn: &PgConnection) -> Result<(), Error> {
use crate::schema::post::dsl::*;
info!("Running post_updates_2020_04_03");
// Update the ap_id
let incorrect_posts = post
.filter(ap_id.eq("http://fake.com"))
.filter(ap_id.eq("changeme"))
.filter(local.eq(true))
.load::<Post>(conn)?;
sql_query("alter table post disable trigger refresh_post").execute(conn)?;
for cpost in &incorrect_posts {
Post::update_ap_id(&conn, cpost.id)?;
}
info!("{} post rows updated.", incorrect_posts.len());
sql_query("alter table post enable trigger refresh_post").execute(conn)?;
Ok(())
}
fn comment_updates_2020_04_03(conn: &PgConnection) -> Result<(), LemmyError> {
fn comment_updates_2020_04_03(conn: &PgConnection) -> Result<(), Error> {
use crate::schema::comment::dsl::*;
info!("Running comment_updates_2020_04_03");
// Update the ap_id
let incorrect_comments = comment
.filter(ap_id.eq("http://fake.com"))
.filter(ap_id.eq("changeme"))
.filter(local.eq(true))
.load::<Comment>(conn)?;
sql_query("alter table comment disable trigger refresh_comment").execute(conn)?;
for ccomment in &incorrect_comments {
Comment::update_ap_id(&conn, ccomment.id)?;
}
sql_query("alter table comment enable trigger refresh_comment").execute(conn)?;
info!("{} comment rows updated.", incorrect_comments.len());
Ok(())
}
fn private_message_updates_2020_05_05(conn: &PgConnection) -> Result<(), LemmyError> {
use crate::schema::private_message::dsl::*;
info!("Running private_message_updates_2020_05_05");
// Update the ap_id
let incorrect_pms = private_message
.filter(ap_id.eq("http://fake.com"))
.filter(local.eq(true))
.load::<PrivateMessage>(conn)?;
sql_query("alter table private_message disable trigger refresh_private_message").execute(conn)?;
for cpm in &incorrect_pms {
PrivateMessage::update_ap_id(&conn, cpm.id)?;
}
sql_query("alter table private_message enable trigger refresh_private_message").execute(conn)?;
info!("{} private message rows updated.", incorrect_pms.len());
Ok(())
}

45
server/src/db/comment.rs
View File

@ -1,9 +1,8 @@
use super::{post::Post, *};
use crate::{
apub::{make_apub_endpoint, EndpointType},
naive_now,
schema::{comment, comment_like, comment_saved},
};
use super::post::Post;
use super::*;
use crate::apub::{make_apub_endpoint, EndpointType};
use crate::naive_now;
use crate::schema::{comment, comment_like, comment_saved};
// WITH RECURSIVE MyTree AS (
// SELECT * FROM comment WHERE parent_id IS NULL
@ -12,7 +11,7 @@ use crate::{
// )
// SELECT * FROM MyTree;
#[derive(Clone, Queryable, Associations, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
#[derive(Queryable, Associations, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
#[belongs_to(Post)]
#[table_name = "comment"]
pub struct Comment {
@ -22,7 +21,7 @@ pub struct Comment {
pub parent_id: Option<i32>,
pub content: String,
pub removed: bool,
pub read: bool, // Whether the recipient has read the comment or not
pub read: bool,
pub published: chrono::NaiveDateTime,
pub updated: Option<chrono::NaiveDateTime>,
pub deleted: bool,
@ -39,7 +38,6 @@ pub struct CommentForm {
pub content: String,
pub removed: Option<bool>,
pub read: Option<bool>,
pub published: Option<chrono::NaiveDateTime>,
pub updated: Option<chrono::NaiveDateTime>,
pub deleted: Option<bool>,
pub ap_id: String,
@ -86,11 +84,6 @@ impl Comment {
.get_result::<Self>(conn)
}
pub fn read_from_apub_id(conn: &PgConnection, object_id: &str) -> Result<Self, Error> {
use crate::schema::comment::dsl::*;
comment.filter(ap_id.eq(object_id)).first::<Self>(conn)
}
pub fn mark_as_read(conn: &PgConnection, comment_id: i32) -> Result<Self, Error> {
use crate::schema::comment::dsl::*;
@ -204,10 +197,10 @@ impl Saveable<CommentSavedForm> for CommentSaved {
#[cfg(test)]
mod tests {
use super::{
super::{community::*, post::*, user::*},
*,
};
use super::super::community::*;
use super::super::post::*;
use super::super::user::*;
use super::*;
#[test]
fn test_crud() {
let conn = establish_unpooled_connection();
@ -229,7 +222,7 @@ mod tests {
lang: "browser".into(),
show_avatars: true,
send_notifications_to_email: false,
actor_id: "http://fake.com".into(),
actor_id: "changeme".into(),
bio: None,
local: true,
private_key: None,
@ -249,12 +242,11 @@ mod tests {
deleted: None,
updated: None,
nsfw: false,
actor_id: "http://fake.com".into(),
actor_id: "changeme".into(),
local: true,
private_key: None,
public_key: None,
last_refreshed_at: None,
published: None,
};
let inserted_community = Community::create(&conn, &new_community).unwrap();
@ -275,9 +267,8 @@ mod tests {
embed_description: None,
embed_html: None,
thumbnail_url: None,
ap_id: "http://fake.com".into(),
ap_id: "changeme".into(),
local: true,
published: None,
};
let inserted_post = Post::create(&conn, &new_post).unwrap();
@ -290,9 +281,8 @@ mod tests {
deleted: None,
read: None,
parent_id: None,
published: None,
updated: None,
ap_id: "http://fake.com".into(),
ap_id: "changeme".into(),
local: true,
};
@ -309,7 +299,7 @@ mod tests {
parent_id: None,
published: inserted_comment.published,
updated: None,
ap_id: "http://fake.com".into(),
ap_id: "changeme".into(),
local: true,
};
@ -321,9 +311,8 @@ mod tests {
removed: None,
deleted: None,
read: None,
published: None,
updated: None,
ap_id: "http://fake.com".into(),
ap_id: "changeme".into(),
local: true,
};

69
server/src/db/comment_view.rs
View File

@ -1,6 +1,5 @@
use crate::db::{fuzzy_search, limit_and_offset, ListingType, MaybeOptional, SortType};
use diesel::{dsl::*, pg::Pg, result::Error, *};
use serde::{Deserialize, Serialize};
use super::*;
use diesel::pg::Pg;
// The faked schema since diesel doesn't do views
table! {
@ -15,16 +14,10 @@ table! {
published -> Timestamp,
updated -> Nullable<Timestamp>,
deleted -> Bool,
ap_id -> Text,
local -> Bool,
community_id -> Int4,
community_actor_id -> Text,
community_local -> Bool,
community_name -> Varchar,
banned -> Bool,
banned_from_community -> Bool,
creator_actor_id -> Text,
creator_local -> Bool,
creator_name -> Varchar,
creator_avatar -> Nullable<Text>,
score -> BigInt,
@ -50,16 +43,10 @@ table! {
published -> Timestamp,
updated -> Nullable<Timestamp>,
deleted -> Bool,
ap_id -> Text,
local -> Bool,
community_id -> Int4,
community_actor_id -> Text,
community_local -> Bool,
community_name -> Varchar,
banned -> Bool,
banned_from_community -> Bool,
creator_actor_id -> Text,
creator_local -> Bool,
creator_name -> Varchar,
creator_avatar -> Nullable<Text>,
score -> BigInt,
@ -88,16 +75,10 @@ pub struct CommentView {
pub published: chrono::NaiveDateTime,
pub updated: Option<chrono::NaiveDateTime>,
pub deleted: bool,
pub ap_id: String,
pub local: bool,
pub community_id: i32,
pub community_actor_id: String,
pub community_local: bool,
pub community_name: String,
pub banned: bool,
pub banned_from_community: bool,
pub creator_actor_id: String,
pub creator_local: bool,
pub creator_name: String,
pub creator_avatar: Option<String>,
pub score: i64,
@ -301,16 +282,10 @@ table! {
published -> Timestamp,
updated -> Nullable<Timestamp>,
deleted -> Bool,
ap_id -> Text,
local -> Bool,
community_id -> Int4,
community_actor_id -> Text,
community_local -> Bool,
community_name -> Varchar,
banned -> Bool,
banned_from_community -> Bool,
creator_actor_id -> Text,
creator_local -> Bool,
creator_name -> Varchar,
creator_avatar -> Nullable<Text>,
score -> BigInt,
@ -340,16 +315,10 @@ pub struct ReplyView {
pub published: chrono::NaiveDateTime,
pub updated: Option<chrono::NaiveDateTime>,
pub deleted: bool,
pub ap_id: String,
pub local: bool,
pub community_id: i32,
pub community_actor_id: String,
pub community_local: bool,
pub community_name: String,
pub banned: bool,
pub banned_from_community: bool,
pub creator_actor_id: String,
pub creator_local: bool,
pub creator_name: String,
pub creator_avatar: Option<String>,
pub score: i64,
@ -454,12 +423,11 @@ impl<'a> ReplyQueryBuilder<'a> {
#[cfg(test)]
mod tests {
use super::{
super::{comment::*, community::*, post::*, user::*},
*,
};
use crate::db::{establish_unpooled_connection, Crud, Likeable};
use super::super::comment::*;
use super::super::community::*;
use super::super::post::*;
use super::super::user::*;
use super::*;
#[test]
fn test_crud() {
let conn = establish_unpooled_connection();
@ -481,7 +449,7 @@ mod tests {
lang: "browser".into(),
show_avatars: true,
send_notifications_to_email: false,
actor_id: "http://fake.com".into(),
actor_id: "changeme".into(),
bio: None,
local: true,
private_key: None,
@ -501,12 +469,11 @@ mod tests {
deleted: None,
updated: None,
nsfw: false,
actor_id: "http://fake.com".into(),
actor_id: "changeme".into(),
local: true,
private_key: None,
public_key: None,
last_refreshed_at: None,
published: None,
};
let inserted_community = Community::create(&conn, &new_community).unwrap();
@ -527,9 +494,8 @@ mod tests {
embed_description: None,
embed_html: None,
thumbnail_url: None,
ap_id: "http://fake.com".into(),
ap_id: "changeme".into(),
local: true,
published: None,
};
let inserted_post = Post::create(&conn, &new_post).unwrap();
@ -542,9 +508,8 @@ mod tests {
removed: None,
deleted: None,
read: None,
published: None,
updated: None,
ap_id: "http://fake.com".into(),
ap_id: "changeme".into(),
local: true,
};
@ -584,12 +549,6 @@ mod tests {
my_vote: None,
subscribed: None,
saved: None,
ap_id: "http://fake.com".to_string(),
local: true,
community_actor_id: inserted_community.actor_id.to_owned(),
community_local: true,
creator_actor_id: inserted_user.actor_id.to_owned(),
creator_local: true,
};
let expected_comment_view_with_user = CommentView {
@ -617,12 +576,6 @@ mod tests {
my_vote: Some(1),
subscribed: None,
saved: None,
ap_id: "http://fake.com".to_string(),
local: true,
community_actor_id: inserted_community.actor_id.to_owned(),
community_local: true,
creator_actor_id: inserted_user.actor_id.to_owned(),
creator_local: true,
};
let mut read_comment_views_no_user = CommentQueryBuilder::create(&conn)

43
server/src/db/community.rs
View File

@ -1,11 +1,7 @@
use crate::{
db::{Bannable, Crud, Followable, Joinable},
schema::{community, community_follower, community_moderator, community_user_ban},
};
use diesel::{dsl::*, result::Error, *};
use serde::{Deserialize, Serialize};
use super::*;
use crate::schema::{community, community_follower, community_moderator, community_user_ban};
#[derive(Clone, Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
#[derive(Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
#[table_name = "community"]
pub struct Community {
pub id: i32,
@ -26,8 +22,7 @@ pub struct Community {
pub last_refreshed_at: chrono::NaiveDateTime,
}
// TODO add better delete, remove, lock actions here.
#[derive(Insertable, AsChangeset, Clone, Serialize, Deserialize, Debug)]
#[derive(Insertable, AsChangeset, Clone, Serialize, Deserialize)]
#[table_name = "community"]
pub struct CommunityForm {
pub name: String,
@ -36,7 +31,6 @@ pub struct CommunityForm {
pub category_id: i32,
pub creator_id: i32,
pub removed: Option<bool>,
pub published: Option<chrono::NaiveDateTime>,
pub updated: Option<chrono::NaiveDateTime>,
pub deleted: Option<bool>,
pub nsfw: bool,
@ -78,23 +72,20 @@ impl Crud<CommunityForm> for Community {
}
impl Community {
pub fn read_from_name(conn: &PgConnection, community_name: &str) -> Result<Self, Error> {
pub fn read_from_name(conn: &PgConnection, community_name: String) -> Result<Self, Error> {
use crate::schema::community::dsl::*;
community
.filter(name.eq(community_name))
.first::<Self>(conn)
}
pub fn read_from_actor_id(conn: &PgConnection, community_id: &str) -> Result<Self, Error> {
pub fn list(conn: &PgConnection) -> Result<Vec<Self>, Error> {
use crate::schema::community::dsl::*;
community
.filter(actor_id.eq(community_id))
.first::<Self>(conn)
community.load::<Community>(conn)
}
pub fn list_local(conn: &PgConnection) -> Result<Vec<Self>, Error> {
use crate::schema::community::dsl::*;
community.filter(local.eq(true)).load::<Community>(conn)
pub fn get_url(&self) -> String {
format!("https://{}/c/{}", Settings::get().hostname, self.name)
}
}
@ -216,7 +207,7 @@ impl Followable<CommunityFollowerForm> for CommunityFollower {
.values(community_follower_form)
.get_result::<Self>(conn)
}
fn unfollow(
fn ignore(
conn: &PgConnection,
community_follower_form: &CommunityFollowerForm,
) -> Result<usize, Error> {
@ -232,9 +223,8 @@ impl Followable<CommunityFollowerForm> for CommunityFollower {
#[cfg(test)]
mod tests {
use super::{super::user::*, *};
use crate::db::{establish_unpooled_connection, ListingType, SortType};
use super::super::user::*;
use super::*;
#[test]
fn test_crud() {
let conn = establish_unpooled_connection();
@ -256,7 +246,7 @@ mod tests {
lang: "browser".into(),
show_avatars: true,
send_notifications_to_email: false,
actor_id: "http://fake.com".into(),
actor_id: "changeme".into(),
bio: None,
local: true,
private_key: None,
@ -276,12 +266,11 @@ mod tests {
removed: None,
deleted: None,
updated: None,
actor_id: "http://fake.com".into(),
actor_id: "changeme".into(),
local: true,
private_key: None,
public_key: None,
last_refreshed_at: None,
published: None,
};
let inserted_community = Community::create(&conn, &new_community).unwrap();
@ -298,7 +287,7 @@ mod tests {
deleted: false,
published: inserted_community.published,
updated: None,
actor_id: "http://fake.com".into(),
actor_id: "changeme".into(),
local: true,
private_key: None,
public_key: None,
@ -352,7 +341,7 @@ mod tests {
let read_community = Community::read(&conn, inserted_community.id).unwrap();
let updated_community =
Community::update(&conn, inserted_community.id, &new_community).unwrap();
let ignored_community = CommunityFollower::unfollow(&conn, &community_follower_form).unwrap();
let ignored_community = CommunityFollower::ignore(&conn, &community_follower_form).unwrap();
let left_community = CommunityModerator::leave(&conn, &community_user_form).unwrap();
let unban = CommunityUserBan::unban(&conn, &community_user_ban_form).unwrap();
let num_deleted = Community::delete(&conn, inserted_community.id).unwrap();

Some files were not shown because too many files have changed in this diff Show More