.dockerignore

@@ -1,6 +1,12 @@
+# build folders and similar which are not needed for the docker build
 ui/node_modules
-ui/dist
 server/target
 docker/dev/volumes
 docker/federation/volumes
+docker/federation-test/volumes
 .git
+ansible
+
+# exceptions, needed for federation-test build
+
+!server/target/debug/lemmy_server

.gitignore

@@ -1,10 +1,18 @@
+# local ansible configuration
 ansible/inventory
 ansible/inventory_dev
 ansible/passwords/
+
+# docker build files
 docker/lemmy_mine.hjson
 docker/dev/env_deploy.sh
-build/
-.idea/
-ui/src/translations
-docker/dev/volumes
+docker/federation/volumes
 docker/federation-test/volumes
+docker/dev/volumes
+
+# local build files
+build/
+ui/src/translations
+
+# ide config
+.idea/

CODE_OF_CONDUCT.md

@@ -30,6 +30,6 @@ In the Lemmy community we strive to go the extra step to look out for each other
 
 And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could’ve communicated better — remember that it’s your responsibility to make others comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
 
-The enforcement policies listed above apply to all official Lemmy venues; including git repositories under [github.com/LemmyNet/lemmy](https://github.com/LemmyNet/lemmy) and [yerbamate.dev/LemmyNet/lemmy](https://yerbamate.dev/LemmyNet/lemmy), the [Matrix channel](https://matrix.to/#/!BZVTUuEiNmRcbFeLeI:matrix.org?via=matrix.org&via=privacytools.io&via=permaweb.io); and all instances under lemmy.ml. For other projects adopting the Rust Code of Conduct, please contact the maintainers of those projects for enforcement. If you wish to use this code of conduct for your own project, consider explicitly mentioning your moderation policy or making a copy with your own moderation policy so as to avoid confusion.
+The enforcement policies listed above apply to all official Lemmy venues; including git repositories under [github.com/LemmyNet/lemmy](https://github.com/LemmyNet/lemmy) and [yerbamate.dev/dessalines/lemmy](https://yerbamate.dev/dessalines/lemmy), the [Matrix channel](https://matrix.to/#/!BZVTUuEiNmRcbFeLeI:matrix.org?via=matrix.org&via=privacytools.io&via=permaweb.io); and all instances under lemmy.ml. For other projects adopting the Rust Code of Conduct, please contact the maintainers of those projects for enforcement. If you wish to use this code of conduct for your own project, consider explicitly mentioning your moderation policy or making a copy with your own moderation policy so as to avoid confusion.
 
 Adapted from the [Rust Code of Conduct](https://www.rust-lang.org/policies/code-of-conduct), which is based on the [Node.js Policy on Trolling](http://blog.izs.me/post/30036893703/policy-on-trolling) as well as the [Contributor Covenant v1.3.0](https://www.contributor-covenant.org/version/1/3/0/).

README.md

@@ -34,7 +34,7 @@
 
 Front Page|Post
 ---|---
-![main screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/main_screen.png)|![chat screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/chat_screen.png)
+![main screen](https://i.imgur.com/kZSRcRu.png)|![chat screen](https://i.imgur.com/4XghNh6.png)
 
 [Lemmy](https://github.com/LemmyNet/lemmy) is similar to sites like [Reddit](https://reddit.com), [Lobste.rs](https://lobste.rs), [Raddle](https://raddle.me), or [Hacker News](https://news.ycombinator.com/): you subscribe to forums you're interested in, post links and discussions, then vote, and comment on them. Behind the scenes, it is very different; anyone can easily run a server, and all these servers are federated (think email), and connected to the same universe, called the [Fediverse](https://en.wikipedia.org/wiki/Fediverse).
 
@@ -44,7 +44,7 @@ The overall goal is to create an easily self-hostable, decentralized alternative
 
 Each lemmy server can set its own moderation policy; appointing site-wide admins, and community moderators to keep out the trolls, and foster a healthy, non-toxic environment where all can feel comfortable contributing.
 
-*Note: Federation is still in active development and the WebSocket, as well as, HTTP API are currently unstable*
+*Note: Federation is still in active development*
 
 ### Why's it called Lemmy?
 
@@ -73,7 +73,7 @@ Each lemmy server can set its own moderation policy; appointing site-wide admins
   - Full vote scores `(+/-)` like old reddit.
   - Themes, including light, dark, and solarized.
   - Emojis with autocomplete support. Start typing `:`
-  - User tagging using `@`, Community tagging using `#`.
+  - User tagging using `@`, Community tagging using `!`.
   - Integrated image uploading in both posts and comments.
   - A post can consist of a title and any combination of self text, a URL, or nothing else.
   - Notifications, on comment replies and when you're tagged.
@@ -125,19 +125,16 @@ Lemmy is free, open-source software, meaning no advertising, monetizing, or vent
 - [Docker Development](https://dev.lemmy.ml/docs/contributing_docker_development.html)
 - [Local Development](https://dev.lemmy.ml/docs/contributing_local_development.html)
 
-### Translations
+### Translations 
 
 If you want to help with translating, take a look at [Weblate](https://weblate.yerbamate.dev/projects/lemmy/).
 
 ## Contact
 
-- [Mastodon](https://mastodon.social/@LemmyDev)
-- [Matrix](https://riot.im/app/#/room/#rust-reddit-fediverse:matrix.org)
-
-## Code Mirrors
-
+- [Mastodon](https://mastodon.social/@LemmyDev) - [![Mastodon Follow](https://img.shields.io/mastodon/follow/810572?domain=https%3A%2F%2Fmastodon.social&style=social)](https://mastodon.social/@LemmyDev)
+- [Matrix](https://riot.im/app/#/room/#rust-reddit-fediverse:matrix.org) - [![Matrix](https://img.shields.io/matrix/rust-reddit-fediverse:matrix.org.svg?label=matrix-chat)](https://riot.im/app/#/room/#rust-reddit-fediverse:matrix.org)
 - [GitHub](https://github.com/LemmyNet/lemmy)
-- [Gitea](https://yerbamate.dev/LemmyNet/lemmy)
+- [Gitea](https://yerbamate.dev/dessalines/lemmy)
 - [GitLab](https://gitlab.com/dessalines/lemmy)
 
 ## Credits

RELEASES.md

@@ -1,66 +1,3 @@
-# Lemmy v0.7.0 Release (2020-06-23)
-
-This release replaces [pictshare](https://github.com/HaschekSolutions/pictshare)
-with [pict-rs](https://git.asonix.dog/asonix/pict-rs), which improves performance
-and security.
-
-Overall, since our last major release in January (v0.6.0), we have closed over
-[100 issues!](https://github.com/LemmyNet/lemmy/milestone/16?closed=1)
-
-- Site-wide list of recent comments
-- Reconnecting websockets
-- Many more themes, including a default light one.
-- Expandable embeds for post links (and thumbnails), from
-[iframely](https://github.com/itteco/iframely)
-- Better icons
-- Emoji autocomplete to post and message bodies, and an Emoji Picker
-- Post body now searchable
-- Community title and description is now searchable
-- Simplified cross-posts
-- Better documentation
-- LOTS more languages
-- Lots of bugs squashed
-- And more ...
-
-## Upgrading
-
-Before starting the upgrade, make sure that you have a working backup of your
-database and image files. See our
-[documentation](https://dev.lemmy.ml/docs/administration_backup_and_restore.html)
-for backup instructions.
-
-**With Ansible:**
-
-```
-# deploy with ansible from your local lemmy git repo
-git pull
-cd ansible
-ansible-playbook lemmy.yml
-# connect via ssh to run the migration script
-ssh your-server
-cd /lemmy/
-wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/migrate-pictshare-to-pictrs.bash
-chmod +x migrate-pictshare-to-pictrs.bash
-sudo ./migrate-pictshare-to-pictrs.bash
-```
-
-**With manual Docker installation:**
-```
-# run these commands on your server
-cd /lemmy
-wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/ansible/templates/nginx.conf
-# Replace the {{ vars }}
-sudo mv nginx.conf /etc/nginx/sites-enabled/lemmy.conf
-sudo nginx -s reload
-wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/docker-compose.yml
-wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/migrate-pictshare-to-pictrs.bash
-chmod +x migrate-pictshare-to-pictrs.bash
-sudo bash migrate-pictshare-to-pictrs.bash
-```
-
-**Note:** After upgrading, all users need to reload the page, then logout and
-login again, so that images are loaded correctly.
-
 # Lemmy v0.6.0 Release (2020-01-16)
 
 `v0.6.0` is here, and we've closed [41 issues!](https://github.com/LemmyNet/lemmy/milestone/15?closed=1) 

ansible/VERSION

@@ -1 +1 @@
-v0.7.1
+v0.6.51

ansible/ansible.cfg

@@ -1,6 +1,5 @@
 [defaults]
 inventory=inventory
-interpreter_python=/usr/bin/python3
 
 [ssh_connection]
 pipelining = True

ansible/lemmy.yml

@@ -24,11 +24,10 @@
       creates: '/etc/letsencrypt/live/{{domain}}/privkey.pem'
 
   - name: create lemmy folder
-    file: path={{item.path}} {{item.owner}} state=directory
+    file: path={{item.path}} state=directory
     with_items:
-      - { path: '/lemmy/', owner: 'root' }
-      - { path: '/lemmy/volumes/', owner: 'root' }
-      - { path: '/lemmy/volumes/pictrs/', owner: '991' }
+      - { path: '/lemmy/' }
+      - { path: '/lemmy/volumes/' }
 
   - block:
     - name:  add template files
@@ -60,7 +59,6 @@
       project_src: /lemmy/
       state: present
       pull: yes
-      remove_orphans: yes
 
   - name: reload nginx with new config
     shell: nginx -s reload

ansible/lemmy_dev.yml

@@ -26,11 +26,10 @@
       creates: '/etc/letsencrypt/live/{{domain}}/privkey.pem'
 
   - name: create lemmy folder
-    file: path={{item.path}} owner={{item.owner}} state=directory
+    file: path={{item.path}} state=directory
     with_items:
-      - { path: '/lemmy/', owner: 'root' }
-      - { path: '/lemmy/volumes/', owner: 'root' }
-      - { path: '/lemmy/volumes/pictrs/', owner: '991' }
+      - { path: '/lemmy/' }
+      - { path: '/lemmy/volumes/' }
 
   - block:
     - name:  add template files
@@ -89,7 +88,6 @@
       project_src: /lemmy/
       state: present
       recreate: always
-      remove_orphans: yes
     ignore_errors: yes
 
   - name: reload nginx with new config

ansible/templates/docker-compose.yml

@@ -12,7 +12,7 @@ services:
       - ./lemmy.hjson:/config/config.hjson:ro
     depends_on:
       - postgres
-      - pictrs
+      - pictshare
       - iframely
 
   postgres:
@@ -25,13 +25,12 @@ services:
       - ./volumes/postgres:/var/lib/postgresql/data
     restart: always
 
-  pictrs:
-    image: asonix/pictrs:amd64-v0.1.0-r9
-    user: 991:991
+  pictshare:
+    image: shtripok/pictshare:latest
     ports:
-      - "127.0.0.1:8537:8080"
+      - "127.0.0.1:8537:80"
     volumes:
-      - ./volumes/pictrs:/mnt
+      - ./volumes/pictshare:/usr/share/nginx/html/data
     restart: always
 
   iframely:

ansible/templates/nginx.conf

@@ -36,7 +36,7 @@ server {
     # It might be nice to compress JSON, but leaving that out to protect against potential
     # compression+encryption information leak attacks like BREACH.
     gzip on;
-    gzip_types text/css application/javascript image/svg+xml;
+    gzip_types text/css application/javascript;
     gzip_vary on;
 
     # Only connect to this site via HTTPS for the two years
@@ -48,8 +48,8 @@ server {
     add_header X-Frame-Options "DENY";
     add_header X-XSS-Protection "1; mode=block";
 
-    # Upload limit for pictrs
-    client_max_body_size 20M;
+    # Upload limit for pictshare
+    client_max_body_size 50M;
 
     location / {
         proxy_pass http://0.0.0.0:8536;
@@ -70,21 +70,15 @@ server {
         proxy_cache_min_uses    5;
     }    
 
-    # Redirect pictshare images to pictrs
-    location ~ /pictshare/(.*)$ {
-      return 301 /pictrs/image/$1;
-    }
+    location /pictshare/ {
+      proxy_pass http://0.0.0.0:8537/;
+      proxy_set_header X-Real-IP $remote_addr;
+      proxy_set_header Host $host;
+      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 
-    # pict-rs images
-    location /pictrs {
-      location /pictrs/image {
-        proxy_pass http://0.0.0.0:8537/image;
-        proxy_set_header X-Real-IP $remote_addr;
-        proxy_set_header Host $host;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-      }
-      # Block the import
-      return 403;
+      if ($request_uri ~ \.(?:ico|gif|jpe?g|png|webp|bmp|mp4)$) {
+        add_header Cache-Control "public, max-age=31536000, immutable";
+      }   
     }
 
     location /iframely/ {

docker/dev/Dockerfile

@@ -21,13 +21,17 @@ COPY server/Cargo.toml server/Cargo.lock ./
 RUN sudo chown -R rust:rust .
 RUN mkdir -p ./src/bin \
   && echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs 
-RUN cargo build
+RUN cargo build --release
 RUN rm -f ./target/x86_64-unknown-linux-musl/release/deps/lemmy_server*
 COPY server/src ./src/
 COPY server/migrations ./migrations/
 
-# Build for debug
-RUN cargo build
+# Build for release
+RUN cargo build --frozen --release
+
+# Get diesel-cli on there just in case
+# RUN cargo install diesel_cli --no-default-features --features postgres
+
 
 FROM ekidd/rust-musl-builder:1.42.0-openssl11 as docs
 WORKDIR /app
@@ -35,14 +39,15 @@ COPY docs ./docs
 RUN sudo chown -R rust:rust .
 RUN mdbook build docs/
 
-FROM alpine:3.12
+
+FROM alpine:3.10
 
 # Install libpq for postgres
 RUN apk add libpq
 
 # Copy resources
 COPY server/config/defaults.hjson /config/defaults.hjson
-COPY --from=rust /app/server/target/x86_64-unknown-linux-musl/debug/lemmy_server /app/lemmy
+COPY --from=rust /app/server/target/x86_64-unknown-linux-musl/release/lemmy_server /app/lemmy
 COPY --from=docs /app/docs/book/ /app/dist/documentation/
 COPY --from=node /app/ui/dist /app/dist
 

docker/dev/Dockerfile.aarch64

@@ -0,0 +1,79 @@
+FROM node:10-jessie as node
+
+WORKDIR /app/ui
+
+# Cache deps
+COPY ui/package.json ui/yarn.lock ./
+RUN yarn install --pure-lockfile
+
+# Build
+COPY ui /app/ui
+RUN yarn build
+
+
+# contains qemu-*-static for cross-compilation
+FROM multiarch/qemu-user-static as qemu
+
+
+FROM arm64v8/rust:1.40-buster as rust
+
+COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
+#COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
+
+
+# Install musl
+#RUN apt-get update && apt-get install -y mc
+#RUN apt-get install -y musl-tools mc
+#libpq-dev mc
+#RUN rustup target add ${TARGET}
+
+# Cache deps
+WORKDIR /app
+RUN USER=root cargo new server
+WORKDIR /app/server
+COPY server/Cargo.toml server/Cargo.lock ./
+RUN  mkdir -p ./src/bin \
+  && echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
+RUN cargo build --release
+# RUN cargo build
+COPY server/src ./src/
+COPY server/migrations ./migrations/
+RUN rm -f ./target/release/deps/lemmy_server* ; rm -f ./target/debug/deps/lemmy_server*
+
+
+# build for release
+RUN cargo build --frozen --release
+# RUN cargo build --frozen
+
+# Get diesel-cli on there just in case
+# RUN cargo install diesel_cli --no-default-features --features postgres
+
+# RUN cp /app/server/target/debug/lemmy_server /app/server/ready
+RUN cp /app/server/target/release/lemmy_server /app/server/ready
+
+#FROM alpine:3.10
+# debian because build with dynamic linking with debian:buster
+FROM arm64v8/debian:buster-slim as lemmy
+
+#COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
+COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
+
+# Install libpq for postgres
+#RUN apk add libpq
+RUN apt-get update && apt-get install -y libpq5
+
+RUN addgroup --gid 1000 lemmy
+# for alpine
+#RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
+# for debian
+RUN adduser --disabled-password --shell /bin/sh --uid 1000 --ingroup lemmy lemmy
+
+# Copy resources
+COPY server/config/defaults.hjson /config/defaults.hjson
+COPY --from=rust /app/server/ready /app/lemmy
+COPY --from=node /app/ui/dist /app/dist
+
+RUN chown lemmy:lemmy /app/lemmy
+USER lemmy
+EXPOSE 8536
+CMD ["/app/lemmy"]

docker/dev/Dockerfile.armv7hf

@@ -0,0 +1,79 @@
+FROM node:10-jessie as node
+
+WORKDIR /app/ui
+
+# Cache deps
+COPY ui/package.json ui/yarn.lock ./
+RUN yarn install --pure-lockfile
+
+# Build
+COPY ui /app/ui
+RUN yarn build
+
+
+# contains qemu-*-static for cross-compilation
+FROM multiarch/qemu-user-static as qemu
+
+
+FROM arm32v7/rust:1.37-buster as rust
+
+#COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
+COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
+
+
+# Install musl
+#RUN apt-get update && apt-get install -y mc
+#RUN apt-get install -y musl-tools mc
+#libpq-dev mc
+#RUN rustup target add ${TARGET}
+
+# Cache deps
+WORKDIR /app
+RUN USER=root cargo new server
+WORKDIR /app/server
+COPY server/Cargo.toml server/Cargo.lock ./
+RUN  mkdir -p ./src/bin \
+  && echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
+#RUN cargo build --release
+# RUN cargo build
+  RUN RUSTFLAGS='-Ccodegen-units=1' cargo build
+COPY server/src ./src/
+COPY server/migrations ./migrations/
+RUN rm -f ./target/release/deps/lemmy_server* ; rm -f ./target/debug/deps/lemmy_server*
+
+
+# build for release
+#RUN cargo build --frozen --release
+RUN cargo build --frozen
+
+# Get diesel-cli on there just in case
+# RUN cargo install diesel_cli --no-default-features --features postgres
+
+RUN cp /app/server/target/debug/lemmy_server /app/server/ready
+#RUN cp /app/server/target/release/lemmy_server /app/server/ready
+
+#FROM alpine:3.10
+# debian because build with dynamic linking with debian:buster
+FROM arm32v7/debian:buster-slim as lemmy
+
+COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
+
+# Install libpq for postgres
+#RUN apk add libpq
+RUN apt-get update && apt-get install -y libpq5
+
+RUN addgroup --gid 1000 lemmy
+# for alpine
+#RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
+# for debian
+RUN adduser --disabled-password --shell /bin/sh --uid 1000 --ingroup lemmy lemmy
+
+# Copy resources
+COPY server/config/defaults.hjson /config/defaults.hjson
+COPY --from=rust /app/server/ready /app/lemmy
+COPY --from=node /app/ui/dist /app/dist
+
+RUN chown lemmy:lemmy /app/lemmy
+USER lemmy
+EXPOSE 8536
+CMD ["/app/lemmy"]

docker/dev/Dockerfile.libc

@@ -0,0 +1,88 @@
+# can be build on x64, arm32, arm64 platforms
+# to build on target platform run 
+# docker build -f Dockerfile.libc -t dessalines/lemmy:version ../..
+#
+# to use docker buildx run
+# docker buildx build --platform linux/amd64,linux/arm64 -f Dockerfile.libc -t YOURNAME/lemmy --push ../..
+
+FROM node:12-buster as node
+# use this if use docker buildx 
+#FROM --platform=$BUILDPLATFORM node:12-buster as node
+
+WORKDIR /app/ui
+
+# Cache deps
+COPY ui/package.json ui/yarn.lock ./
+RUN yarn install --pure-lockfile --network-timeout 100000
+
+# Build
+COPY ui /app/ui
+RUN yarn build
+
+
+FROM rust:1.42 as rust
+
+# Cache deps
+WORKDIR /app
+
+RUN USER=root cargo new server
+WORKDIR /app/server
+COPY server/Cargo.toml server/Cargo.lock ./
+RUN  mkdir -p ./src/bin \
+  && echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
+
+
+RUN cargo build --release
+#RUN cargo build && \
+#    rm -f ./target/release/deps/lemmy_server* ; rm -f ./target/debug/deps/lemmy_server*
+COPY server/src ./src/
+COPY server/migrations ./migrations/
+
+
+# build for release
+# workaround for https://github.com/rust-lang/rust/issues/62896
+#RUN RUSTFLAGS='-Ccodegen-units=1' cargo build --release
+RUN cargo build --release --frozen
+#RUN cargo build --frozen
+
+# Get diesel-cli on there just in case
+# RUN cargo install diesel_cli --no-default-features --features postgres
+
+# make result place always the same for lemmy container
+RUN cp /app/server/target/release/lemmy_server /app/server/ready
+#RUN cp /app/server/target/debug/lemmy_server /app/server/ready
+
+
+FROM rust:1.42 as docs
+
+WORKDIR /app
+
+# Build docs
+COPY docs ./docs
+RUN cargo install mdbook
+RUN mdbook build docs/
+
+
+#FROM alpine:3.10
+# debian because build with dynamic linking with debian:buster
+FROM debian:buster as lemmy
+
+# Install libpq for postgres
+#RUN apk add libpq
+RUN apt-get update && apt-get install -y libpq5
+RUN addgroup --gid 1000 lemmy
+# for alpine
+#RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
+# for debian
+RUN adduser --disabled-password --shell /bin/sh --uid 1000 --ingroup lemmy lemmy
+
+# Copy resources
+COPY server/config/defaults.hjson /config/defaults.hjson
+COPY --from=node /app/ui/dist /app/dist
+COPY --from=docs /app/docs/book/ /app/dist/documentation/
+COPY --from=rust /app/server/ready /app/lemmy
+
+RUN chown lemmy:lemmy /app/lemmy
+USER lemmy
+EXPOSE 8536
+CMD ["/app/lemmy"]

docker/dev/deploy.sh

@@ -1,5 +1,4 @@
 #!/bin/sh
-set -e
 git checkout master
 
 # Import translations
@@ -21,7 +20,7 @@ git add "server/src/version.rs"
 echo $new_tag > "ansible/VERSION"
 git add "ansible/VERSION"
 
-cd docker/prod || exit
+cd docker/dev || exit
 
 # Changing the docker-compose prod
 sed -i "s/dessalines\/lemmy:.*/dessalines\/lemmy:$new_tag/" ../prod/docker-compose.yml
@@ -33,24 +32,41 @@ git add ../../ansible/templates/docker-compose.yml
 git commit -m"Version $new_tag"
 git tag $new_tag
 
-export COMPOSE_DOCKER_CLI_BUILD=1
-export DOCKER_BUILDKIT=1
-
 # Rebuilding docker
+docker-compose build
+docker tag dev_lemmy:latest dessalines/lemmy:x64-$new_tag
+docker push dessalines/lemmy:x64-$new_tag
+
+# Build for Raspberry Pi / other archs
+
+# Arm currently not working
+# docker build -t lemmy:armv7hf -f Dockerfile.armv7hf ../../
+# docker tag lemmy:armv7hf dessalines/lemmy:armv7hf-$new_tag
+# docker push dessalines/lemmy:armv7hf-$new_tag
+
+# aarch64
+# Only do this on major releases (IE the third semver is 0)
 if [ $third_semver -eq 0 ]; then
-  # TODO get linux/arm/v7 build working
-  # Build for Raspberry Pi / other archs too
-  docker buildx build --platform linux/amd64,linux/arm64 ../../ \
-    --file Dockerfile \
-    --tag dessalines/lemmy:$new_tag \
-    --push
-else
-  docker buildx build --platform linux/amd64 ../../ \
-    --file Dockerfile \
-    --tag dessalines/lemmy:$new_tag \
-    --push
+  # Registering qemu binaries
+  docker run --rm --privileged multiarch/qemu-user-static:register --reset
+
+  docker build -t lemmy:aarch64 -f Dockerfile.aarch64 ../../
+  docker tag lemmy:aarch64 dessalines/lemmy:arm64-$new_tag
+  docker push dessalines/lemmy:arm64-$new_tag
 fi
 
+# Creating the manifest for the multi-arch build
+if [ $third_semver -eq 0 ]; then
+  docker manifest create dessalines/lemmy:$new_tag \
+  dessalines/lemmy:x64-$new_tag \
+  dessalines/lemmy:arm64-$new_tag
+else
+  docker manifest create dessalines/lemmy:$new_tag \
+  dessalines/lemmy:x64-$new_tag
+fi
+
+docker manifest push dessalines/lemmy:$new_tag
+
 # Push
 git push origin $new_tag
 git push

docker/dev/docker-compose.yml

@@ -1,6 +1,15 @@
 version: '3.3'
 
 services:
+  postgres:
+    image: postgres:12-alpine
+    environment:
+      - POSTGRES_USER=lemmy
+      - POSTGRES_PASSWORD=password
+      - POSTGRES_DB=lemmy
+    volumes:
+      - ./volumes/postgres:/var/lib/postgresql/data
+    restart: always
 
   lemmy:
     build: 
@@ -14,27 +23,16 @@ services:
     volumes:
       - ../lemmy.hjson:/config/config.hjson
     depends_on: 
-      - pictrs
       - postgres
+      - pictshare
       - iframely
 
-  postgres:
-    image: postgres:12-alpine
-    environment:
-      - POSTGRES_USER=lemmy
-      - POSTGRES_PASSWORD=password
-      - POSTGRES_DB=lemmy
+  pictshare:
+    image: shtripok/pictshare:latest
+    ports:
+      - "127.0.0.1:8537:80"
     volumes:
-      - ./volumes/postgres:/var/lib/postgresql/data
-    restart: always
-
-  pictrs:
-    image: asonix/pictrs:v0.1.13-r0
-    ports: 
-      - "127.0.0.1:8537:8080"
-    user: 991:991
-    volumes:
-      - ./volumes/pictrs:/mnt
+      - ./volumes/pictshare:/usr/share/nginx/html/data
     restart: always
 
   iframely:

docker/dev/docker_update.sh

@@ -1,6 +1,2 @@
 #!/bin/sh
-set -e
-
-export COMPOSE_DOCKER_CLI_BUILD=1
-export DOCKER_BUILDKIT=1
 docker-compose up -d --no-deps --build

docker/dev/test_deploy.sh

@@ -1,15 +1,16 @@
-#!/bin/sh
+#!/bin/bash
 set -e
 
-export COMPOSE_DOCKER_CLI_BUILD=1
-export DOCKER_BUILDKIT=1
+BRANCH=$1
+
+git checkout $BRANCH
+cd ../../
 
 # Rebuilding dev docker
-docker-compose build
-docker tag dev_lemmy:latest dessalines/lemmy:test
-docker push dessalines/lemmy:test
+sudo docker build . -f "docker/dev/Dockerfile" -t "dessalines/lemmy:$BRANCH"
+sudo docker push "dessalines/lemmy:$BRANCH"
 
 # Run the playbook
-pushd ../../../lemmy-ansible
-ansible-playbook -i test playbooks/site.yml --vault-password-file vault_pass
+pushd ../lemmy-ansible
+ansible-playbook -i test playbooks/site.yml
 popd

docker/federation-test/run-tests.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+set -e
+
+pushd ../../server/
+cargo build
+popd
+
+sudo docker build ../../ --file ../federation/Dockerfile --tag lemmy-federation:latest
+
+sudo docker-compose --file ../federation/docker-compose.yml --project-directory . up -d
+
+pushd ../../ui
+yarn
+echo "Waiting for Lemmy to start..."
+while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8540/api/v1/site')" != "200" ]]; do sleep 1; done
+while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8550/api/v1/site')" != "200" ]]; do sleep 1; done
+yarn api-test || true
+popd
+
+sudo docker-compose --file ../federation/docker-compose.yml --project-directory . down
+
+sudo rm -r volumes/

docker/federation/Dockerfile

@@ -0,0 +1,17 @@
+FROM ekidd/rust-musl-builder:1.38.0-openssl11
+
+USER root
+RUN mkdir /app/dist/documentation/ -p \
+ && addgroup --gid 1001 lemmy \
+ && adduser --disabled-password --shell /bin/sh -u 1001 --ingroup lemmy lemmy
+
+# Copy resources
+COPY server/config/defaults.hjson /app/config/defaults.hjson
+COPY ui/dist /app/dist
+COPY server/target/debug/lemmy_server /app/lemmy
+
+RUN chown lemmy:lemmy /app/ -R
+USER lemmy
+EXPOSE 8536
+WORKDIR /app
+CMD ["/app/lemmy"]

docker/federation/docker-compose.yml

@@ -0,0 +1,92 @@
+version: '3.3'
+
+services:
+  nginx:
+    image: nginx:1.17-alpine
+    ports:
+      - "8540:8540"
+      - "8550:8550"
+    volumes:
+      # Hack to make this work from both docker/federation/ and docker/federation-test/
+      - ../federation/nginx.conf:/etc/nginx/nginx.conf
+    depends_on:
+      - lemmy_alpha
+      - pictshare_alpha
+      - lemmy_beta
+      - pictshare_beta
+      - iframely
+    restart: "always"
+
+  lemmy_alpha:
+    image: lemmy-federation:latest
+    environment:
+      - LEMMY_HOSTNAME=lemmy_alpha:8540
+      - LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_alpha:5432/lemmy
+      - LEMMY_JWT_SECRET=changeme
+      - LEMMY_FRONT_END_DIR=/app/dist
+      - LEMMY_FEDERATION__ENABLED=true
+      - LEMMY_FEDERATION__TLS_ENABLED=false
+      - LEMMY_FEDERATION__INSTANCE_WHITELIST=lemmy_beta
+      - LEMMY_PORT=8540
+      - LEMMY_SETUP__ADMIN_USERNAME=lemmy_alpha
+      - LEMMY_SETUP__ADMIN_PASSWORD=lemmy
+      - LEMMY_SETUP__SITE_NAME=lemmy_alpha
+      - RUST_BACKTRACE=1
+      - RUST_LOG=debug
+    restart: always
+    depends_on:
+      - postgres_alpha
+  postgres_alpha:
+    image: postgres:12-alpine
+    environment:
+      - POSTGRES_USER=lemmy
+      - POSTGRES_PASSWORD=password
+      - POSTGRES_DB=lemmy
+    volumes:
+      - ./volumes/postgres_alpha:/var/lib/postgresql/data
+    restart: always
+  pictshare_alpha:
+    image: shtripok/pictshare:latest
+    volumes:
+      - ./volumes/pictshare_alpha:/usr/share/nginx/html/data
+    restart: always
+
+  lemmy_beta:
+    image: lemmy-federation:latest
+    environment:
+      - LEMMY_HOSTNAME=lemmy_beta:8550
+      - LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_beta:5432/lemmy
+      - LEMMY_JWT_SECRET=changeme
+      - LEMMY_FRONT_END_DIR=/app/dist
+      - LEMMY_FEDERATION__ENABLED=true
+      - LEMMY_FEDERATION__TLS_ENABLED=false
+      - LEMMY_FEDERATION__INSTANCE_WHITELIST=lemmy_alpha
+      - LEMMY_PORT=8550
+      - LEMMY_SETUP__ADMIN_USERNAME=lemmy_beta
+      - LEMMY_SETUP__ADMIN_PASSWORD=lemmy
+      - LEMMY_SETUP__SITE_NAME=lemmy_beta
+      - RUST_BACKTRACE=1
+      - RUST_LOG=debug
+    restart: always
+    depends_on:
+      - postgres_beta
+  postgres_beta:
+    image: postgres:12-alpine
+    environment:
+      - POSTGRES_USER=lemmy
+      - POSTGRES_PASSWORD=password
+      - POSTGRES_DB=lemmy
+    volumes:
+      - ./volumes/postgres_beta:/var/lib/postgresql/data
+    restart: always
+  pictshare_beta:
+    image: shtripok/pictshare:latest
+    volumes:
+      - ./volumes/pictshare_beta:/usr/share/nginx/html/data
+    restart: always
+
+  iframely:
+    image: dogbin/iframely:latest
+    volumes:
+      - ../iframely.config.local.js:/iframely/config.local.js:ro
+    restart: always

docker/federation/nginx.conf

@@ -0,0 +1,75 @@
+events {
+    worker_connections 1024;
+}
+
+http {
+    server {
+        listen 8540;
+        server_name 127.0.0.1;
+        access_log  off;
+
+        # Upload limit for pictshare
+        client_max_body_size 50M;
+
+        location / {
+            proxy_pass http://lemmy_alpha:8540;
+            proxy_set_header X-Real-IP $remote_addr;
+            proxy_set_header Host $host;
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+
+            # WebSocket support
+            proxy_http_version 1.1;
+            proxy_set_header Upgrade $http_upgrade;
+            proxy_set_header Connection "upgrade";
+        }
+
+        location /pictshare/ {
+            proxy_pass http://pictshare_alpha:80/;
+            proxy_set_header X-Real-IP $remote_addr;
+            proxy_set_header Host $host;
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        }
+
+        location /iframely/ {
+            proxy_pass http://iframely:80/;
+            proxy_set_header X-Real-IP $remote_addr;
+            proxy_set_header Host $host;
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        }
+    }
+
+    server {
+        listen 8550;
+        server_name 127.0.0.1;
+        access_log off;
+
+        # Upload limit for pictshare
+        client_max_body_size 50M;
+
+        location / {
+            proxy_pass http://lemmy_beta:8550;
+            proxy_set_header X-Real-IP $remote_addr;
+            proxy_set_header Host $host;
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+
+            # WebSocket support
+            proxy_http_version 1.1;
+            proxy_set_header Upgrade $http_upgrade;
+            proxy_set_header Connection "upgrade";
+        }
+
+        location /pictshare/ {
+            proxy_pass http://pictshare_beta:80/;
+            proxy_set_header X-Real-IP $remote_addr;
+            proxy_set_header Host $host;
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        }
+
+        location /iframely/ {
+            proxy_pass http://iframely:80/;
+            proxy_set_header X-Real-IP $remote_addr;
+            proxy_set_header Host $host;
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        }
+    }
+}

docker/federation/run-federation-test.bash

@@ -0,0 +1,17 @@
+#!/bin/bash
+set -e
+
+if [ "$1" = "-yarn" ]; then
+  pushd ../../ui/ || exit
+  yarn
+  yarn build
+  popd || exit
+fi
+
+pushd ../../server/ || exit
+cargo build
+popd || exit
+
+sudo docker build ../../ --file Dockerfile -t lemmy-federation:latest
+
+sudo docker-compose up

docker/lemmy.hjson

@@ -23,6 +23,9 @@
   jwt_secret: "changeme"
   # The dir for the front end
   front_end_dir: "/app/dist"
+  # whether to enable activitypub federation. this feature is in alpha, do not enable in production, as might
+  # cause problems like remote instances fetching and permanently storing bad data.
+  federation_enabled: false
   # rate limits for various user actions, by user ip
   rate_limit: {
     # maximum number of messages created in interval
@@ -57,7 +60,6 @@
 #    smtp_password: ""
 #    # address to send emails from, eg "info@your-instance.com"
 #    smtp_from_address: ""
-#    use_tls: true
 #  }
 }
 

docker/prod/Dockerfile

@@ -1,64 +0,0 @@
-ARG RUST_BUILDER_IMAGE=shtripok/rust-musl-builder:arm
-
-FROM $RUST_BUILDER_IMAGE as rust
-
-#ARG RUSTRELEASEDIR="debug"
-ARG RUSTRELEASEDIR="release"
-
-# Cache deps
-WORKDIR /app
-RUN sudo chown -R rust:rust .
-RUN USER=root cargo new server
-WORKDIR /app/server
-COPY --chown=rust:rust server/Cargo.toml server/Cargo.lock ./
-#RUN sudo chown -R rust:rust .
-RUN mkdir -p ./src/bin \
-   && echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
-RUN cargo build --release
-RUN rm -f ./target/$CARGO_BUILD_TARGET/$RUSTRELEASEDIR/deps/lemmy_server*
-COPY --chown=rust:rust server/src ./src/
-COPY --chown=rust:rust server/migrations ./migrations/
-
-# build for release
-# workaround for https://github.com/rust-lang/rust/issues/62896
-RUN cargo build --frozen --release
-
-# reduce binary size
-RUN strip ./target/$CARGO_BUILD_TARGET/$RUSTRELEASEDIR/lemmy_server
-
-RUN cp ./target/$CARGO_BUILD_TARGET/$RUSTRELEASEDIR/lemmy_server /app/server/
-
-FROM $RUST_BUILDER_IMAGE as docs
-WORKDIR /app
-COPY --chown=rust:rust docs ./docs
-RUN mdbook build docs/
-
-FROM node:12-buster as node
-
-WORKDIR /app/ui
-
-# Cache deps
-COPY ui/package.json ui/yarn.lock ./
-RUN yarn install --pure-lockfile --network-timeout 600000
-
-# Build
-COPY ui /app/ui
-RUN yarn build
-
-FROM alpine:3.12 as lemmy
-
-# Install libpq for postgres
-RUN apk add libpq
-RUN addgroup -g 1000 lemmy
-RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
-
-# Copy resources
-COPY --chown=lemmy:lemmy server/config/defaults.hjson /config/defaults.hjson
-COPY --chown=lemmy:lemmy --from=rust /app/server/lemmy_server /app/lemmy
-COPY --chown=lemmy:lemmy --from=docs /app/docs/book/ /app/dist/documentation/
-COPY --chown=lemmy:lemmy --from=node /app/ui/dist /app/dist
-
-RUN chown lemmy:lemmy /app/lemmy
-USER lemmy
-EXPOSE 8536
-CMD ["/app/lemmy"]

docker/prod/docker-compose.yml

@@ -1,4 +1,4 @@
-version: '2.2'
+version: '3.3'
 
 services:
   postgres:
@@ -12,7 +12,7 @@ services:
     restart: always
 
   lemmy:
-    image: dessalines/lemmy:v0.7.1
+    image: dessalines/lemmy:v0.6.51
     ports:
       - "127.0.0.1:8536:8536"
     restart: always
@@ -22,16 +22,15 @@ services:
       - ./lemmy.hjson:/config/config.hjson
     depends_on:
       - postgres
-      - pictrs
+      - pictshare
       - iframely
 
-  pictrs:
-    image: asonix/pictrs:v0.1.13-r0
-    ports: 
-      - "127.0.0.1:8537:8080"
-    user: 991:991
+  pictshare:
+    image: shtripok/pictshare:latest
+    ports:
+      - "127.0.0.1:8537:80"
     volumes:
-      - ./volumes/pictrs:/mnt
+      - ./volumes/pictshare:/usr/share/nginx/html/data
     restart: always
 
   iframely:
@@ -41,4 +40,3 @@ services:
     volumes:
       - ./iframely.config.local.js:/iframely/config.local.js:ro
     restart: always
-    mem_limit: 100m

docker/prod/migrate-pictshare-to-pictrs.bash

@@ -1,60 +0,0 @@
-#!/bin/bash
-set -e
-
-if [[ $(id -u) != 0 ]]; then 
-    echo "This migration needs to be run as root"
-    exit
-fi
-
-if [[ ! -f docker-compose.yml ]]; then 
-    echo "No docker-compose.yml found in current directory. Is this the right folder?"
-    exit
-fi
-
-# Fixing pictrs permissions
-mkdir -p volumes/pictrs
-sudo chown -R 991:991 volumes/pictrs
-
-echo "Restarting docker-compose, making sure that pictrs is started and pictshare is removed"
-docker-compose up -d --remove-orphans
-
-if [[ -z $(docker-compose ps | grep pictrs) ]]; then
-    echo "Pict-rs is not running, make sure you update Lemmy first"
-    exit
-fi
-
-# echo "Stopping Lemmy so that users dont upload new images during the migration"
-# docker-compose stop lemmy
-
-pushd volumes/pictshare/
-echo "Importing pictshare images to pict-rs..."
-IMAGE_NAMES=*
-for image in $IMAGE_NAMES; do
-    IMAGE_PATH="$(pwd)/$image/$image"
-    if [[ ! -f $IMAGE_PATH ]]; then
-        continue
-    fi
-    echo -e "\nImporting $IMAGE_PATH"
-    ret=0
-    curl --silent --fail -F "images[]=@$IMAGE_PATH" http://127.0.0.1:8537/import || ret=$?
-    if [[ $ret != 0 ]]; then
-      echo "Error for $IMAGE_PATH : $ret"
-    fi
-done
-
-echo "Fixing permissions on pictshare folder"
-find . -type d -exec chmod 755 {} \;
-find . -type f -exec chmod 644 {} \;
-
-popd
-
-echo "Rewrite image links in Lemmy database"
-docker-compose exec -u  postgres postgres psql -U lemmy -c "UPDATE user_ SET avatar = REPLACE(avatar, 'pictshare', 'pictrs/image') WHERE avatar is not null;"
-docker-compose exec -u  postgres postgres psql -U lemmy -c "UPDATE post SET url = REPLACE(url, 'pictshare', 'pictrs/image') WHERE url is not null;"
-
-echo "Moving pictshare data folder to pictshare_backup"
-mv volumes/pictshare volumes/pictshare_backup
-
-echo "Migration done, starting Lemmy again"
-echo "If everything went well, you can delete ./volumes/pictshare_backup/"
-docker-compose start lemmy

docs/src/about.md

@@ -2,7 +2,7 @@
 
 Front Page|Post
 ---|---
-![main screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/main_screen.png)|![chat screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/chat_screen.png)
+![main screen](https://i.imgur.com/kZSRcRu.png)|![chat screen](https://i.imgur.com/4XghNh6.png)
 
 [Lemmy](https://github.com/LemmyNet/lemmy) is similar to sites like [Reddit](https://reddit.com), [Lobste.rs](https://lobste.rs), [Raddle](https://raddle.me), or [Hacker News](https://news.ycombinator.com/): you subscribe to forums you're interested in, post links and discussions, then vote, and comment on them. Behind the scenes, it is very different; anyone can easily run a server, and all these servers are federated (think email), and connected to the same universe, called the [Fediverse](https://en.wikipedia.org/wiki/Fediverse).
 

docs/src/about_guide.md

@@ -3,7 +3,7 @@
 Start typing...
 
 - `@a_user_name` to get a list of usernames.
-- `#a_community` to get a list of communities.
+- `!a_community` to get a list of communities.
 - `:emoji` to get a list of emojis.
 
 ## Sorting

docs/src/about_ranking.md

@@ -26,4 +26,4 @@ Gravity = Decay gravity, 1.8 is default
 
 A plot of rank over 24 hours, of scores of 1, 5, 10, 100, 1000, with a scale factor of 10k.
 
-![](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/rank_algorithm.png)
+![](https://i.imgur.com/w8oBLlL.png)

docs/src/administration_install_docker.md

@@ -6,25 +6,19 @@ Make sure you have both docker and docker-compose(>=`1.24.0`) installed. On Ubun
 # create a folder for the lemmy files. the location doesnt matter, you can put this anywhere you want
 mkdir /lemmy
 cd /lemmy
-
 # download default config files
-wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/docker-compose.yml
-wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/lemmy.hjson
-wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/iframely.config.local.js
-
-# Set correct permissions for pictrs folder
-mkdir -p volumes/pictrs
-sudo chown -R 991:991 volumes/pictrs
+wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/prod/docker-compose.yml
+wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/lemmy.hjson
+wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/iframely.config.local.js
+docker-compose up -d
 ```
 
-After this, have a look at the [config file](administration_configuration.md) named `lemmy.hjson`, and adjust it, in particular the hostname, and possibly the db password. Then run:
+After this, have a look at the [config file](administration_configuration.md) named `lemmy.hjson`, and adjust it, in particular the hostname.
 
-`docker-compose up -d`
-
-To make Lemmy available outside the server, you need to setup a reverse proxy, like Nginx. [A sample nginx config](https://raw.githubusercontent.com/LemmyNet/lemmy/master/ansible/templates/nginx.conf), could be setup with:
+To make Lemmy available outside the server, you need to setup a reverse proxy, like Nginx. [A sample nginx config](/ansible/templates/nginx.conf), could be setup with:
 
 ```bash
-wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/ansible/templates/nginx.conf
+wget https://raw.githubusercontent.com/dessalines/lemmy/master/ansible/templates/nginx.conf
 # Replace the {{ vars }}
 sudo mv nginx.conf /etc/nginx/sites-enabled/lemmy.conf
 ```
@@ -36,6 +30,6 @@ You will also need to setup TLS, for example with [Let's Encrypt](https://letsen
 To update to the newest version, you can manually change the version in `docker-compose.yml`. Alternatively, fetch the latest version from our git repo:
 
 ```bash
-wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/docker-compose.yml
+wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/prod/docker-compose.yml
 docker-compose up -d
 ```

docs/src/contributing.md

@@ -4,9 +4,9 @@ Information about contributing to Lemmy, whether it is translating, testing, des
 
 ## Issue tracking / Repositories
 
-- [GitHub (for issues and pull requests)](https://github.com/LemmyNet/lemmy)
-- [Gitea (only for pull requests)](https://yerbamate.dev/LemmyNet/lemmy)
-- [GitLab (only code-mirror)](https://gitlab.com/dessalines/lemmy)
+- [GitHub (for issues)](https://github.com/LemmyNet/lemmy)
+- [Gitea](https://yerbamate.dev/dessalines/lemmy)
+- [GitLab](https://gitlab.com/dessalines/lemmy)
 
 ## Translating
 

docs/src/contributing_docker_development.md

@@ -3,22 +3,11 @@
 ## Running
 
 ```bash
-sudo apt install git docker-compose
 git clone https://github.com/LemmyNet/lemmy
 cd lemmy/docker/dev
-sudo docker-compose up --no-deps --build
+./docker_update.sh # This builds and runs it, updating for your changes
 ```
 
 and go to http://localhost:8536.
 
-To speed up the Docker compile, add the following to `/etc/docker/daemon.json` and restart Docker.
-```
-{
-  "features": {
-    "buildkit": true
-  }
-}
-```
-
-If the build is still too slow, you will have to use a
-[local build](contributing_local_development.md) instead.
+Note that compile times when changing `Cargo.toml` are relatively long with Docker, because builds can't be incrementally cached. If this is a problem for you, you should use [Local Development](contributing_local_development.md).

docs/src/contributing_federation_development.md

@@ -5,17 +5,17 @@
 If you don't have a local clone of the Lemmy repo yet, just run the following command:
 
 ```bash
-git clone https://github.com/LemmyNet/lemmy -b federation
+git clone https://yerbamate.dev/LemmyNet/lemmy.git -b federation
 ```
 
 If you already have the Lemmy repo cloned, you need to add a new remote:
 ```bash
-git remote add federation https://github.com/LemmyNet/lemmy
+git remote add federation https://yerbamate.dev/LemmyNet/lemmy.git
 git checkout federation
 git pull federation federation
 ```
 
-## Running
+## Running locally
 
 You need to have the following packages installed, the Docker service needs to be running.
 
@@ -31,7 +31,30 @@ cd dev/federation-test
 ```
 
 After the build is finished and the docker-compose setup is running, open [127.0.0.1:8540](http://127.0.0.1:8540) and
-[127.0.0.1:8541](http://127.0.0.1:8541) in your browser to use the test instances. You can login as admin with
-username `lemmy` and password `lemmy`, or create new accounts.
+[127.0.0.1:8550](http://127.0.0.1:8550) in your browser to use the test instances. You can login as admin with
+username `lemmy_alpha` and `lemmy_beta` respectively, with password `lemmy`.
 
-Please get in touch if you want to contribute to this, so we can coordinate things and avoid duplicate work.
+## Running on a server
+
+Note that federation is currently in alpha. Only use it for testing, not on any production server, and be aware
+that you might have to wipe the instance data at one point or another.
+
+Follow the normal installation instructions, either with [Ansible](administration_install_ansible.md) or
+[manually](administration_install_docker.md). Then replace the line `image: dessalines/lemmy:v0.x.x` in 
+`/lemmy/docker-compose.yml` with `image: dessalines/lemmy:federation`. Also add the following in
+`/lemmy/lemmy.hjson`:
+
+```
+    federation: {
+        enabled: true
+        instance_whitelist: example.com
+    }
+```
+
+Afterwards, and whenver you want to update to the latest version, run these commands on the server:
+
+```
+cd /lemmy/
+sudo docker-compose pull
+sudo docker-compose up -d
+```

docs/src/contributing_local_development.md

@@ -1,67 +1,31 @@
-### Ubuntu
+#### Requirements
 
+- [Rust](https://www.rust-lang.org/)
+- [Yarn](https://yarnpkg.com/en/)
+- [Postgres](https://www.postgresql.org/)
 
-#### Build requirements:
-```
-sudo apt install git cargo libssl-dev pkg-config libpq-dev yarn curl gnupg2 git
-# install yarn
-curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
-echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
-sudo apt update && sudo apt install yarn
-```
+#### Set up Postgres DB
 
-#### Get the source code
-```
-git clone https://github.com/LemmyNet/lemmy.git
-# or alternatively from gitea
-# git clone https://yerbamate.dev/LemmyNet/lemmy.git
-```
-
-All the following commands need to be run either in `lemmy/server` or `lemmy/ui`, as indicated
-by the `cd` command.
-
-#### Build the backend (Rust)
-```
+```bash
 cd server
-cargo build
-# for development, use `cargo check` instead)
+./db-init.sh
 ```
 
-#### Build the frontend (Typescript)
-```
-cd ui
-yarn
-yarn build
-```
+Or run the commands manually:
 
-#### Setup postgresql
-```
-sudo apt install postgresql
-sudo systemctl start postgresql
-# initialize postgres database
-sudo -u postgres psql -c "create user lemmy with password 'password' superuser;" -U postgres
-sudo -u postgres psql -c 'create database lemmy with owner lemmy;' -U postgres
+```bash
+psql -c "create user lemmy with password 'password' superuser;" -U postgres
+psql -c 'create database lemmy with owner lemmy;' -U postgres
 export LEMMY_DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
-# or execute server/db-init.sh
 ```
 
-#### Run a local development instance
-```
-# run each of these in a seperate terminal
-cd server && cargo run
-ui & yarn start
-```
+#### Running
 
-Then open [localhost:4444](http://localhost:4444) in your browser. It will auto-refresh if you edit
-any frontend files. For backend coding, you will have to rerun `cargo run`. You can use
-`cargo check` as a faster way to find compilation errors.
-
-To speed up incremental builds, you can add the following to `~/.cargo/config`:
+```bash
+git clone https://github.com/LemmyNet/lemmy
+cd lemmy
+./install.sh
+# For live coding, where both the front and back end, automagically reload on any save, do:
+# cd ui && yarn start
+# cd server && cargo watch -x run
 ```
-[target.x86_64-unknown-linux-gnu]
-rustflags = ["-Clink-arg=-fuse-ld=lld"]
-```
-
-Note that this setup doesn't include image uploads or link previews (provided by pict-rs and
-iframely respectively). If you want to test those, you should use the
-[Docker development](contributing_docker_development.md).

docs/src/contributing_websocket_http_api.md

@@ -1,6 +1,6 @@
 # Lemmy API
 
-*Note: this may lag behind the actual API endpoints [here](../server/src/api). The API should be considered unstable and may change any time.*
+*Note: this may lag behind the actual API endpoints [here](../server/src/api).*
 
 <!-- toc -->
 

docs/src/lemmy_council.md

@@ -49,7 +49,5 @@
 ## Member List / Contact Info
 General Contact [@LemmyDev Mastodon](https://mastodon.social/@LemmyDev)
 
-- [Dessalines](https://dev.lemmy.ml/u/dessalines)
-- [Nutomic](https://dev.lemmy.ml/u/nutomic)
-- [AgreeableLandscape](https://dev.lemmy.ml/u/AgreeableLandscape)
-- [fruechtchen](https://dev.lemmy.ml/u/fruechtchen)
+- Dessalines [Matrix](https://matrix.to/#/@happydooby:matrix.org)
+- Nutomic [Matrix](https://matrix.to/#/@nutomic:matrix.org), [Mastodon](https://radical.town/@felix)

install.sh

@@ -1,4 +1,4 @@
-#!/bin/sh
+#!/bin/bash
 set -e
 
 # Set the database variable to the default first.
@@ -10,55 +10,25 @@ export LEMMY_DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
 export JWT_SECRET=changeme
 export HOSTNAME=rrr
 
-yes_no_prompt_invalid() {
-  echo "Invalid input. Please enter either \"y\" or \"n\"." 1>&2
-}
-
-ask_to_init_db() {
-  init_db_valid=0
-  init_db_final=0
-  while [ "$init_db_valid" == 0 ]
-  do
-    read -p "Initialize database (y/n)? " init_db
-    case "$init_db" in
-      [yY]* ) init_db_valid=1; init_db_final=1;;
-      [nN]* ) init_db_valid=1; init_db_final=0;;
-      * ) yes_no_prompt_invalid;;
-    esac
-    echo
-  done
-  if [ "$init_db_final" = 1 ]
-  then
-    source ./server/db-init.sh
-    read -n 1 -s -r -p "Press ANY KEY to continue execution of this script, press CTRL+C to quit..."
-    echo
-  fi
-}
-
-ask_to_auto_reload() {
-  auto_reload_valid=0
-  auto_reload_final=0
-  while [ "$auto_reload_valid" == 0 ]
-  do
-    echo "Automagically reload the project when source files are changed?"
-    echo "ONLY ENABLE THIS FOR DEVELOPMENT!"
-    read -p "(y/n) " auto_reload
-    case "$auto_reload" in
-      [yY]* ) auto_reload_valid=1; auto_reload_final=1;;
-      [nN]* ) auto_reload_valid=1; auto_reload_final=0;;
-      * ) yes_no_prompt_invalid;;
-    esac
-    echo
-  done
-  if [ "$auto_reload_final" = 1 ]
-  then
-    cd ui && yarn start
-    cd server && cargo watch -x run
-  fi
-}
-
 # Optionally initialize the database
-ask_to_init_db
+init_db_valid=0
+init_db_final=0
+while [ "$init_db_valid" == 0 ]
+do
+  read -p "Initialize database (y/n)? " init_db
+  case "${init_db,,}" in
+    y|yes ) init_db_valid=1; init_db_final=1;;
+    n|no ) init_db_valid=1; init_db_final=0;;
+    * ) echo "Invalid input" 1>&2;;
+  esac
+  echo
+done
+if [ "$init_db_final" = 1 ]
+then
+  source ./server/db-init.sh
+  read -n 1 -s -r -p "Press ANY KEY to continue execution of this script, press CTRL+C to quit..."
+  echo
+fi
 
 # Build the web client
 cd ui
@@ -69,5 +39,6 @@ yarn build
 cd ../server
 RUST_LOG=debug cargo run
 
-# For live coding, where both the front and back end, automagically reload on any save
-ask_to_auto_reload
+# For live coding, where both the front and back end, automagically reload on any save, do:
+# cd ui && yarn start
+# cd server && cargo watch -x run

server/Cargo.toml

@@ -1,27 +1,24 @@
 [package]
 name = "lemmy_server"
 version = "0.0.1"
-authors = ["Dessalines <tyhou13@gmx.com>"]
+authors = ["Dessalines <happydooby@gmail.com>"]
 edition = "2018"
 
-[profile.release]
-lto = true
-
 [dependencies]
-diesel = { version = "1.4.2", features = ["postgres","chrono", "r2d2", "64-column-tables"] }
+diesel = { version = "1.4.4", features = ["postgres","chrono","r2d2","64-column-tables","serde_json"] }
 diesel_migrations = "1.4.0"
 dotenv = "0.15.0"
-bcrypt = "0.7.0"
-activitypub = "0.2.0"
-chrono = "0.4.7"
-failure = "0.1.8"
-serde_json = "1.0.52"
-serde = "1.0.105"
+activitystreams = "0.6.0"
+bcrypt = "0.6.2"
+chrono = { version = "0.4.7", features = ["serde"] }
+failure = "0.1.5"
+serde_json = { version = "1.0.48", features = ["preserve_order"]}
+serde = { version = "1.0.105", features = ["derive"] }
 actix = "0.9.0"
 actix-web = "2.0.0"
 actix-files = "0.2.1"
 actix-web-actors = "2.0.0"
-actix-rt = "1.1.1"
+actix-rt = "1.0.0"
 log = "0.4.0"
 env_logger = "0.7.1"
 rand = "0.7.3"
@@ -30,14 +27,22 @@ strum_macros = "0.18.0"
 jsonwebtoken = "7.0.1"
 regex = "1.3.5"
 lazy_static = "1.3.0"
-lettre = "0.9.3"
-lettre_email = "0.9.4"
+lettre = "0.9.2"
+lettre_email = "0.9.2"
 sha2 = "0.8.1"
 rss = "1.9.0"
 htmlescape = "0.3.1"
-config = {version = "0.10.1", default-features = false, features = ["hjson"] }
+config = "0.10.1"
+hjson = "0.8.2"
+url = { version = "2.1.1", features = ["serde"] }
 percent-encoding = "2.1.0"
-attohttpc = { version = "0.14.0", default-features = false, features = ["tls-rustls"] }
+isahc = "0.9"
 comrak = "0.7"
-tokio = "0.2.20"
+openssl = "0.10"
+http = "0.2.1"
+http-signature-normalization = "0.4.1"
+base64 = "0.12.0"
+tokio = "0.2.18"
 futures = "0.3.4"
+itertools = "0.9.0"
+uuid = { version = "0.8", features = ["serde", "v4"] }

server/config/config.hjson

@@ -0,0 +1,4 @@
+{
+  hostname: "localhost:8536"
+  federation_enabled: true
+}

server/config/defaults.hjson

@@ -26,7 +26,7 @@
     pool_size: 5
   }
   # the domain name of your instance (eg "dev.lemmy.ml")
-  hostname: "my_domain"
+  hostname: null
   # address where lemmy should listen for incoming requests
   bind: "0.0.0.0"
   # port where lemmy should listen for incoming requests
@@ -35,9 +35,6 @@
   jwt_secret: "changeme"
   # The dir for the front end
   front_end_dir: "../ui/dist"
-  # whether to enable activitypub federation. this feature is in alpha, do not enable in production, as might
-  # cause problems like remote instances fetching and permanently storing bad data.
-  federation_enabled: false
   # rate limits for various user actions, by user ip
   rate_limit: {
     # maximum number of messages created in interval
@@ -53,6 +50,15 @@
     # interval length for registration limit
     register_per_second: 3600
   }
+  # settings related to activitypub federation
+  federation: {
+    # whether to enable activitypub federation. this feature is in alpha, do not enable in production.
+    enabled: false
+    # whether tls is required for activitypub. only disable this for debugging, never for producion.
+    tls_enabled: true
+    # comma seperated list of instances with which federation is allowed
+    instance_whitelist: ""
+  }
 #  # email sending configuration
 #  email: {
 #    # hostname of the smtp server

server/db-init.sh

@@ -1,106 +1,43 @@
-#!/bin/sh
+#!/bin/bash
 
-# Default configurations
 username=lemmy
 dbname=lemmy
 port=5432
 
-yes_no_prompt_invalid() {
-  echo "Invalid input. Please enter either \"y\" or \"n\"." 1>&2
-}
+password=""
+password_confirm=""
+password_valid=0
 
-print_config() {
-  echo "  database name: $dbname"
-  echo "  username: $username"
-  echo "  port: $port"
-}
-
-ask_for_db_config() {
-  echo "The default database configuration is:"
-  print_config
+while [ "$password_valid" == 0 ]
+do
+  read -p "Enter database password: " -s password
   echo
 
-  default_config_final=0
-  default_config_valid=0
-  while [ "$default_config_valid" == 0 ]
-  do
-    read -p "Use this configuration (y/n)? " default_config
-    case "$default_config" in
-      [yY]* ) default_config_valid=1; default_config_final=1;;
-      [nN]* ) default_config_valid=1; default_config_final=0;;
-      * ) yes_no_prompt_invalid;;
-    esac
-    echo
-  done
+  read -p "Verify database password: " -s password_confirm
+  echo
+  echo
 
-  if [ "$default_config_final" == 0 ]
+  # Start the loop from the top if either check fails
+  if [ -z "$password" ]
   then
-    config_ok_final=0
-    while [ "$config_ok_final" == 0 ]
-    do
-      read -p "Database name:  " dbname
-      read -p "Username:  " username
-      read -p "Port:  " port
-      #echo
-      
-      #echo "The database configuration is:"
-      #print_config
-      #echo
-      
-      config_ok_valid=0
-      while [ "$config_ok_valid" == 0 ]
-      do
-        read -p "Use this configuration (y/n)? " config_ok
-        case "$config_ok" in
-          [yY]* ) config_ok_valid=1; config_ok_final=1;;
-          [nN]* ) config_ok_valid=1; config_ok_final=0;;
-          * ) yes_no_prompt_invalid;;
-        esac
-        echo
-      done
-    done
+    echo "Error: Password cannot be empty." 1>&2
+    echo
+    continue
   fi
-}
-
-ask_for_password() {
-  password=""
-  password_confirm=""
-  password_valid=0
-  while [ "$password_valid" == 0 ]
-  do
-    read -p "Enter database password: " -s password
+  if [ "$password" != "$password_confirm" ]
+  then
+    echo "Error: Passwords don't match." 1>&2
     echo
+    continue
+  fi
 
-    read -p "Verify database password: " -s password_confirm
-    echo
-    echo
+  # Set the password_valid variable to break out of the loop
+  password_valid=1
+done
 
-    # Start the loop from the top if either check fails
-    if [ -z "$password" ]
-    then
-      echo "Error: Password cannot be empty." 1>&2
-      echo
-      continue
-    fi
-    if [ "$password" != "$password_confirm" ]
-    then
-      echo "Error: Passwords don't match." 1>&2
-      echo
-      continue
-    fi
-
-    # Set the password_valid variable to break out of the loop
-    password_valid=1
-  done
-}
-
-ask_for_db_config
-
-ask_for_password
 
 psql -c "CREATE USER $username WITH PASSWORD '$password' SUPERUSER;" -U postgres
-psql -c "CREATE DATABASE $dbname WITH OWNER $username;" -U postgres
+psql -c 'CREATE DATABASE $dbname WITH OWNER $username;' -U postgres
 export LEMMY_DATABASE_URL=postgres://$username:$password@localhost:$port/$dbname
 
-echo "The database URL is $LEMMY_DATABASE_URL"
-
+echo $LEMMY_DATABASE_URL

server/migrations/2020-03-26-192410_add_activitypub_tables/down.sql

@@ -0,0 +1,16 @@
+drop table activity;
+
+alter table user_ 
+drop column actor_id, 
+drop column private_key,
+drop column public_key,
+drop column bio,
+drop column local,
+drop column last_refreshed_at;
+
+alter table community 
+drop column actor_id, 
+drop column private_key,
+drop column public_key,
+drop column local,
+drop column last_refreshed_at;

server/migrations/2020-03-26-192410_add_activitypub_tables/up.sql

@@ -0,0 +1,36 @@
+-- The Activitypub activity table
+-- All user actions must create a row here.
+create table activity (
+  id serial primary key,
+  user_id int references user_ on update cascade on delete cascade not null, -- Ensures that the user is set up here.
+  data jsonb not null,
+  local boolean not null default true,
+  published timestamp not null default now(),
+  updated timestamp
+);
+
+-- Making sure that id is unique
+create unique index idx_activity_unique_apid on activity ((data ->> 'id'::text));
+
+-- Add federation columns to the two actor tables
+alter table user_ 
+-- TODO uniqueness constraints should be added on these 3 columns later
+add column actor_id character varying(255) not null default 'changeme', -- This needs to be checked and updated in code, building from the site url if local
+add column bio text, -- not on community, already has description
+add column local boolean not null default true,
+add column private_key text, -- These need to be generated from code
+add column public_key text,
+add column last_refreshed_at timestamp not null default now() -- Used to re-fetch federated actor periodically
+;
+
+-- Community
+alter table community 
+add column actor_id character varying(255) not null default 'changeme', -- This needs to be checked and updated in code, building from the site url if local
+add column local boolean not null default true,
+add column private_key text, -- These need to be generated from code
+add column public_key text,
+add column last_refreshed_at timestamp not null default now() -- Used to re-fetch federated actor periodically
+;
+
+-- Don't worry about rebuilding the views right now.
+

server/migrations/2020-04-03-194936_add_activitypub_for_posts_and_comments/down.sql

@@ -0,0 +1,7 @@
+alter table post 
+drop column ap_id, 
+drop column local;
+
+alter table comment 
+drop column ap_id, 
+drop column local;

server/migrations/2020-04-03-194936_add_activitypub_for_posts_and_comments/up.sql

@@ -0,0 +1,14 @@
+-- Add federation columns to post, comment
+
+alter table post
+-- TODO uniqueness constraints should be added on these 3 columns later
+add column ap_id character varying(255) not null default 'changeme', -- This needs to be checked and updated in code, building from the site url if local
+add column local boolean not null default true
+;
+
+alter table comment
+-- TODO uniqueness constraints should be added on these 3 columns later
+add column ap_id character varying(255) not null default 'changeme', -- This needs to be checked and updated in code, building from the site url if local
+add column local boolean not null default true
+;
+

server/migrations/2020-04-07-135912_add_user_community_apub_constraints/down.sql

@@ -0,0 +1,36 @@
+-- User table
+drop view user_view cascade;
+
+alter table user_ 
+add column fedi_name varchar(40) not null default 'changeme';
+
+alter table user_
+add constraint user__name_fedi_name_key unique (name, fedi_name);
+
+-- Community
+alter table community
+add constraint community_name_key unique (name);
+
+
+create view user_view as 
+select 
+u.id,
+u.name,
+u.avatar,
+u.email,
+u.matrix_user_id,
+u.fedi_name,
+u.admin,
+u.banned,
+u.show_avatars,
+u.send_notifications_to_email,
+u.published,
+(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
+(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
+(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
+(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
+from user_ u;
+
+create materialized view user_mview as select * from user_view;
+
+create unique index idx_user_mview_id on user_mview (id);

server/migrations/2020-04-07-135912_add_user_community_apub_constraints/up.sql

@@ -0,0 +1,38 @@
+-- User table
+
+-- Need to regenerate user_view, user_mview
+drop view user_view cascade;
+
+-- Remove the fedi_name constraint, drop that useless column
+alter table user_ 
+drop constraint user__name_fedi_name_key;
+
+alter table user_
+drop column fedi_name;
+
+-- Community
+alter table community
+drop constraint community_name_key;
+
+create view user_view as 
+select 
+u.id,
+u.name,
+u.avatar,
+u.email,
+u.matrix_user_id,
+u.admin,
+u.banned,
+u.show_avatars,
+u.send_notifications_to_email,
+u.published,
+(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
+(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
+(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
+(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
+from user_ u;
+
+create materialized view user_mview as select * from user_view;
+
+create unique index idx_user_mview_id on user_mview (id);
+

server/migrations/2020-04-14-163701_update_views_for_activitypub/down.sql

@@ -0,0 +1,440 @@
+-- user_view
+drop view user_view cascade;
+
+create view user_view as 
+select 
+u.id,
+u.name,
+u.avatar,
+u.email,
+u.matrix_user_id,
+u.admin,
+u.banned,
+u.show_avatars,
+u.send_notifications_to_email,
+u.published,
+(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
+(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
+(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
+(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
+from user_ u;
+
+create materialized view user_mview as select * from user_view;
+
+create unique index idx_user_mview_id on user_mview (id);
+
+-- community_view
+drop view community_aggregates_view cascade;
+create view community_aggregates_view as
+select c.*,
+(select name from user_ u where c.creator_id = u.id) as creator_name,
+(select avatar from user_ u where c.creator_id = u.id) as creator_avatar,
+(select name from category ct where c.category_id = ct.id) as category_name,
+(select count(*) from community_follower cf where cf.community_id = c.id) as number_of_subscribers,
+(select count(*) from post p where p.community_id = c.id) as number_of_posts,
+(select count(*) from comment co, post p where c.id = p.community_id and p.id = co.post_id) as number_of_comments,
+hot_rank((select count(*) from community_follower cf where cf.community_id = c.id), c.published) as hot_rank
+from community c;
+
+create materialized view community_aggregates_mview as select * from community_aggregates_view;
+
+create unique index idx_community_aggregates_mview_id on community_aggregates_mview (id);
+
+create view community_view as
+with all_community as
+(
+  select
+  ca.*
+  from community_aggregates_view ca
+)
+
+select
+ac.*,
+u.id as user_id,
+(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
+from user_ u
+cross join all_community ac
+
+union all
+
+select 
+ac.*,
+null as user_id,
+null as subscribed
+from all_community ac
+;
+
+create view community_mview as
+with all_community as
+(
+  select
+  ca.*
+  from community_aggregates_mview ca
+)
+
+select
+ac.*,
+u.id as user_id,
+(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
+from user_ u
+cross join all_community ac
+
+union all
+
+select 
+ac.*,
+null as user_id,
+null as subscribed
+from all_community ac
+;
+
+-- community views
+drop view community_moderator_view;
+drop view community_follower_view;
+drop view community_user_ban_view;
+
+create view community_moderator_view as 
+select *,
+(select name from user_ u where cm.user_id = u.id) as user_name,
+(select avatar from user_ u where cm.user_id = u.id),
+(select name from community c where cm.community_id = c.id) as community_name
+from community_moderator cm;
+
+create view community_follower_view as 
+select *,
+(select name from user_ u where cf.user_id = u.id) as user_name,
+(select avatar from user_ u where cf.user_id = u.id),
+(select name from community c where cf.community_id = c.id) as community_name
+from community_follower cf;
+
+create view community_user_ban_view as 
+select *,
+(select name from user_ u where cm.user_id = u.id) as user_name,
+(select avatar from user_ u where cm.user_id = u.id),
+(select name from community c where cm.community_id = c.id) as community_name
+from community_user_ban cm;
+
+-- post_view
+drop view post_view;
+drop view post_mview;
+drop materialized view post_aggregates_mview;
+drop view post_aggregates_view;
+
+-- regen post view
+create view post_aggregates_view as
+select        
+p.*,
+(select u.banned from user_ u where p.creator_id = u.id) as banned,
+(select cb.id::bool from community_user_ban cb where p.creator_id = cb.user_id and p.community_id = cb.community_id) as banned_from_community,
+(select name from user_ where p.creator_id = user_.id) as creator_name,
+(select avatar from user_ where p.creator_id = user_.id) as creator_avatar,
+(select name from community where p.community_id = community.id) as community_name,
+(select removed from community c where p.community_id = c.id) as community_removed,
+(select deleted from community c where p.community_id = c.id) as community_deleted,
+(select nsfw from community c where p.community_id = c.id) as community_nsfw,
+(select count(*) from comment where comment.post_id = p.id) as number_of_comments,
+coalesce(sum(pl.score), 0) as score,
+count (case when pl.score = 1 then 1 else null end) as upvotes,
+count (case when pl.score = -1 then 1 else null end) as downvotes,
+hot_rank(coalesce(sum(pl.score) , 0), 
+  (
+    case when (p.published < ('now'::timestamp - '1 month'::interval)) then p.published -- Prevents necro-bumps
+    else greatest(c.recent_comment_time, p.published)
+    end
+  )
+) as hot_rank,
+(
+  case when (p.published < ('now'::timestamp - '1 month'::interval)) then p.published -- Prevents necro-bumps
+  else greatest(c.recent_comment_time, p.published)
+  end
+) as newest_activity_time
+from post p
+left join post_like pl on p.id = pl.post_id
+left join (
+  select post_id, 
+  max(published) as recent_comment_time
+  from comment
+  group by 1
+) c on p.id = c.post_id
+group by p.id, c.recent_comment_time;
+
+create materialized view post_aggregates_mview as select * from post_aggregates_view;
+
+create unique index idx_post_aggregates_mview_id on post_aggregates_mview (id);
+
+create view post_view as 
+with all_post as (
+  select
+  pa.*
+  from post_aggregates_view pa
+)
+select
+ap.*,
+u.id as user_id,
+coalesce(pl.score, 0) as my_vote,
+(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
+(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
+(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
+from user_ u
+cross join all_post ap
+left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
+
+union all
+
+select 
+ap.*,
+null as user_id,
+null as my_vote,
+null as subscribed,
+null as read,
+null as saved
+from all_post ap
+;
+
+create view post_mview as 
+with all_post as (
+  select
+  pa.*
+  from post_aggregates_mview pa
+)
+select
+ap.*,
+u.id as user_id,
+coalesce(pl.score, 0) as my_vote,
+(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
+(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
+(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
+from user_ u
+cross join all_post ap
+left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
+
+union all
+
+select 
+ap.*,
+null as user_id,
+null as my_vote,
+null as subscribed,
+null as read,
+null as saved
+from all_post ap
+;
+
+
+-- reply_view, comment_view, user_mention
+drop view reply_view;
+drop view user_mention_view;
+drop view user_mention_mview;
+drop view comment_view;
+drop view comment_mview;
+drop materialized view comment_aggregates_mview;
+drop view comment_aggregates_view;
+
+-- reply and comment view
+create view comment_aggregates_view as
+select        
+c.*,
+(select community_id from post p where p.id = c.post_id),
+(select co.name from post p, community co where p.id = c.post_id and p.community_id = co.id) as community_name,
+(select u.banned from user_ u where c.creator_id = u.id) as banned,
+(select cb.id::bool from community_user_ban cb, post p where c.creator_id = cb.user_id and p.id = c.post_id and p.community_id = cb.community_id) as banned_from_community,
+(select name from user_ where c.creator_id = user_.id) as creator_name,
+(select avatar from user_ where c.creator_id = user_.id) as creator_avatar,
+coalesce(sum(cl.score), 0) as score,
+count (case when cl.score = 1 then 1 else null end) as upvotes,
+count (case when cl.score = -1 then 1 else null end) as downvotes,
+hot_rank(coalesce(sum(cl.score) , 0), c.published) as hot_rank
+from comment c
+left join comment_like cl on c.id = cl.comment_id
+group by c.id;
+
+create materialized view comment_aggregates_mview as select * from comment_aggregates_view;
+
+create unique index idx_comment_aggregates_mview_id on comment_aggregates_mview (id);
+
+create view comment_view as
+with all_comment as
+(
+  select
+  ca.*
+  from comment_aggregates_view ca
+)
+
+select
+ac.*,
+u.id as user_id,
+coalesce(cl.score, 0) as my_vote,
+(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.community_id = cf.community_id) as subscribed,
+(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
+from user_ u
+cross join all_comment ac
+left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
+
+union all
+
+select 
+    ac.*,
+    null as user_id, 
+    null as my_vote,
+    null as subscribed,
+    null as saved
+from all_comment ac
+;
+
+create view comment_mview as
+with all_comment as
+(
+  select
+  ca.*
+  from comment_aggregates_mview ca
+)
+
+select
+ac.*,
+u.id as user_id,
+coalesce(cl.score, 0) as my_vote,
+(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.community_id = cf.community_id) as subscribed,
+(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
+from user_ u
+cross join all_comment ac
+left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
+
+union all
+
+select 
+    ac.*,
+    null as user_id, 
+    null as my_vote,
+    null as subscribed,
+    null as saved
+from all_comment ac
+;
+
+-- Do the reply_view referencing the comment_mview
+create view reply_view as 
+with closereply as (
+    select 
+    c2.id, 
+    c2.creator_id as sender_id, 
+    c.creator_id as recipient_id
+    from comment c
+    inner join comment c2 on c.id = c2.parent_id
+    where c2.creator_id != c.creator_id
+    -- Do union where post is null
+    union
+    select
+    c.id,
+    c.creator_id as sender_id,
+    p.creator_id as recipient_id
+    from comment c, post p
+    where c.post_id = p.id and c.parent_id is null and c.creator_id != p.creator_id
+)
+select cv.*,
+closereply.recipient_id
+from comment_mview cv, closereply
+where closereply.id = cv.id
+;
+
+-- user mention
+create view user_mention_view as
+select 
+    c.id,
+    um.id as user_mention_id,
+    c.creator_id,
+    c.post_id,
+    c.parent_id,
+    c.content,
+    c.removed,
+    um.read,
+    c.published,
+    c.updated,
+    c.deleted,
+    c.community_id,
+    c.community_name,
+    c.banned,
+    c.banned_from_community,
+    c.creator_name,
+    c.creator_avatar,
+    c.score,
+    c.upvotes,
+    c.downvotes,
+    c.hot_rank,
+    c.user_id,
+    c.my_vote,
+    c.saved,
+    um.recipient_id
+from user_mention um, comment_view c
+where um.comment_id = c.id;
+
+
+create view user_mention_mview as 
+with all_comment as
+(
+  select
+  ca.*
+  from comment_aggregates_mview ca
+)
+
+select
+    ac.id,
+    um.id as user_mention_id,
+    ac.creator_id,
+    ac.post_id,
+    ac.parent_id,
+    ac.content,
+    ac.removed,
+    um.read,
+    ac.published,
+    ac.updated,
+    ac.deleted,
+    ac.community_id,
+    ac.community_name,
+    ac.banned,
+    ac.banned_from_community,
+    ac.creator_name,
+    ac.creator_avatar,
+    ac.score,
+    ac.upvotes,
+    ac.downvotes,
+    ac.hot_rank,
+    u.id as user_id,
+    coalesce(cl.score, 0) as my_vote,
+    (select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved,
+    um.recipient_id
+from user_ u
+cross join all_comment ac
+left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
+left join user_mention um on um.comment_id = ac.id
+
+union all
+
+select 
+    ac.id,
+    um.id as user_mention_id,
+    ac.creator_id,
+    ac.post_id,
+    ac.parent_id,
+    ac.content,
+    ac.removed,
+    um.read,
+    ac.published,
+    ac.updated,
+    ac.deleted,
+    ac.community_id,
+    ac.community_name,
+    ac.banned,
+    ac.banned_from_community,
+    ac.creator_name,
+    ac.creator_avatar,
+    ac.score,
+    ac.upvotes,
+    ac.downvotes,
+    ac.hot_rank,
+    null as user_id, 
+    null as my_vote,
+    null as saved,
+    um.recipient_id
+from all_comment ac
+left join user_mention um on um.comment_id = ac.id
+;
+

server/migrations/2020-04-14-163701_update_views_for_activitypub/up.sql

@@ -0,0 +1,497 @@
+-- user_view
+drop view user_view cascade;
+
+create view user_view as 
+select 
+u.id,
+u.actor_id,
+u.name,
+u.avatar,
+u.email,
+u.matrix_user_id,
+u.bio,
+u.local,
+u.admin,
+u.banned,
+u.show_avatars,
+u.send_notifications_to_email,
+u.published,
+(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
+(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
+(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
+(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
+from user_ u;
+
+create materialized view user_mview as select * from user_view;
+
+create unique index idx_user_mview_id on user_mview (id);
+
+-- community_view
+drop view community_aggregates_view cascade;
+create view community_aggregates_view as
+-- Now that there's public and private keys, you have to be explicit here
+select c.id,
+c.name,
+c.title,
+c.description,
+c.category_id,
+c.creator_id,
+c.removed,
+c.published,
+c.updated,
+c.deleted,
+c.nsfw,
+c.actor_id,
+c.local,
+c.last_refreshed_at,
+(select actor_id from user_ u where c.creator_id = u.id) as creator_actor_id,
+(select local from user_ u where c.creator_id = u.id) as creator_local,
+(select name from user_ u where c.creator_id = u.id) as creator_name,
+(select avatar from user_ u where c.creator_id = u.id) as creator_avatar,
+(select name from category ct where c.category_id = ct.id) as category_name,
+(select count(*) from community_follower cf where cf.community_id = c.id) as number_of_subscribers,
+(select count(*) from post p where p.community_id = c.id) as number_of_posts,
+(select count(*) from comment co, post p where c.id = p.community_id and p.id = co.post_id) as number_of_comments,
+hot_rank((select count(*) from community_follower cf where cf.community_id = c.id), c.published) as hot_rank
+from community c;
+
+create materialized view community_aggregates_mview as select * from community_aggregates_view;
+
+create unique index idx_community_aggregates_mview_id on community_aggregates_mview (id);
+
+create view community_view as
+with all_community as
+(
+  select
+  ca.*
+  from community_aggregates_view ca
+)
+
+select
+ac.*,
+u.id as user_id,
+(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
+from user_ u
+cross join all_community ac
+
+union all
+
+select 
+ac.*,
+null as user_id,
+null as subscribed
+from all_community ac
+;
+
+create view community_mview as
+with all_community as
+(
+  select
+  ca.*
+  from community_aggregates_mview ca
+)
+
+select
+ac.*,
+u.id as user_id,
+(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
+from user_ u
+cross join all_community ac
+
+union all
+
+select 
+ac.*,
+null as user_id,
+null as subscribed
+from all_community ac
+;
+
+-- community views
+drop view community_moderator_view;
+drop view community_follower_view;
+drop view community_user_ban_view;
+
+create view community_moderator_view as 
+select *,
+(select actor_id from user_ u where cm.user_id = u.id) as user_actor_id,
+(select local from user_ u where cm.user_id = u.id) as user_local,
+(select name from user_ u where cm.user_id = u.id) as user_name,
+(select avatar from user_ u where cm.user_id = u.id),
+(select actor_id from community c where cm.community_id = c.id) as community_actor_id,
+(select local from community c where cm.community_id = c.id) as community_local,
+(select name from community c where cm.community_id = c.id) as community_name
+from community_moderator cm;
+
+create view community_follower_view as 
+select *,
+(select actor_id from user_ u where cf.user_id = u.id) as user_actor_id,
+(select local from user_ u where cf.user_id = u.id) as user_local,
+(select name from user_ u where cf.user_id = u.id) as user_name,
+(select avatar from user_ u where cf.user_id = u.id),
+(select actor_id from community c where cf.community_id = c.id) as community_actor_id,
+(select local from community c where cf.community_id = c.id) as community_local,
+(select name from community c where cf.community_id = c.id) as community_name
+from community_follower cf;
+
+create view community_user_ban_view as 
+select *,
+(select actor_id from user_ u where cm.user_id = u.id) as user_actor_id,
+(select local from user_ u where cm.user_id = u.id) as user_local,
+(select name from user_ u where cm.user_id = u.id) as user_name,
+(select avatar from user_ u where cm.user_id = u.id),
+(select actor_id from community c where cm.community_id = c.id) as community_actor_id,
+(select local from community c where cm.community_id = c.id) as community_local,
+(select name from community c where cm.community_id = c.id) as community_name
+from community_user_ban cm;
+
+-- post_view
+drop view post_view;
+drop view post_mview;
+drop materialized view post_aggregates_mview;
+drop view post_aggregates_view;
+
+-- regen post view
+create view post_aggregates_view as
+select        
+p.*,
+(select u.banned from user_ u where p.creator_id = u.id) as banned,
+(select cb.id::bool from community_user_ban cb where p.creator_id = cb.user_id and p.community_id = cb.community_id) as banned_from_community,
+(select actor_id from user_ where p.creator_id = user_.id) as creator_actor_id,
+(select local from user_ where p.creator_id = user_.id) as creator_local,
+(select name from user_ where p.creator_id = user_.id) as creator_name,
+(select avatar from user_ where p.creator_id = user_.id) as creator_avatar,
+(select actor_id from community where p.community_id = community.id) as community_actor_id,
+(select local from community where p.community_id = community.id) as community_local,
+(select name from community where p.community_id = community.id) as community_name,
+(select removed from community c where p.community_id = c.id) as community_removed,
+(select deleted from community c where p.community_id = c.id) as community_deleted,
+(select nsfw from community c where p.community_id = c.id) as community_nsfw,
+(select count(*) from comment where comment.post_id = p.id) as number_of_comments,
+coalesce(sum(pl.score), 0) as score,
+count (case when pl.score = 1 then 1 else null end) as upvotes,
+count (case when pl.score = -1 then 1 else null end) as downvotes,
+hot_rank(coalesce(sum(pl.score) , 0), 
+  (
+    case when (p.published < ('now'::timestamp - '1 month'::interval)) then p.published -- Prevents necro-bumps
+    else greatest(c.recent_comment_time, p.published)
+    end
+  )
+) as hot_rank,
+(
+  case when (p.published < ('now'::timestamp - '1 month'::interval)) then p.published -- Prevents necro-bumps
+  else greatest(c.recent_comment_time, p.published)
+  end
+) as newest_activity_time
+from post p
+left join post_like pl on p.id = pl.post_id
+left join (
+  select post_id, 
+  max(published) as recent_comment_time
+  from comment
+  group by 1
+) c on p.id = c.post_id
+group by p.id, c.recent_comment_time;
+
+create materialized view post_aggregates_mview as select * from post_aggregates_view;
+
+create unique index idx_post_aggregates_mview_id on post_aggregates_mview (id);
+
+create view post_view as 
+with all_post as (
+  select
+  pa.*
+  from post_aggregates_view pa
+)
+select
+ap.*,
+u.id as user_id,
+coalesce(pl.score, 0) as my_vote,
+(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
+(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
+(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
+from user_ u
+cross join all_post ap
+left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
+
+union all
+
+select 
+ap.*,
+null as user_id,
+null as my_vote,
+null as subscribed,
+null as read,
+null as saved
+from all_post ap
+;
+
+create view post_mview as 
+with all_post as (
+  select
+  pa.*
+  from post_aggregates_mview pa
+)
+select
+ap.*,
+u.id as user_id,
+coalesce(pl.score, 0) as my_vote,
+(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
+(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
+(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
+from user_ u
+cross join all_post ap
+left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
+
+union all
+
+select 
+ap.*,
+null as user_id,
+null as my_vote,
+null as subscribed,
+null as read,
+null as saved
+from all_post ap
+;
+
+
+-- reply_view, comment_view, user_mention
+drop view reply_view;
+drop view user_mention_view;
+drop view user_mention_mview;
+drop view comment_view;
+drop view comment_mview;
+drop materialized view comment_aggregates_mview;
+drop view comment_aggregates_view;
+
+-- reply and comment view
+create view comment_aggregates_view as
+select        
+c.*,
+(select community_id from post p where p.id = c.post_id),
+(select co.actor_id from post p, community co where p.id = c.post_id and p.community_id = co.id) as community_actor_id,
+(select co.local from post p, community co where p.id = c.post_id and p.community_id = co.id) as community_local,
+(select co.name from post p, community co where p.id = c.post_id and p.community_id = co.id) as community_name,
+(select u.banned from user_ u where c.creator_id = u.id) as banned,
+(select cb.id::bool from community_user_ban cb, post p where c.creator_id = cb.user_id and p.id = c.post_id and p.community_id = cb.community_id) as banned_from_community,
+(select actor_id from user_ where c.creator_id = user_.id) as creator_actor_id,
+(select local from user_ where c.creator_id = user_.id) as creator_local,
+(select name from user_ where c.creator_id = user_.id) as creator_name,
+(select avatar from user_ where c.creator_id = user_.id) as creator_avatar,
+coalesce(sum(cl.score), 0) as score,
+count (case when cl.score = 1 then 1 else null end) as upvotes,
+count (case when cl.score = -1 then 1 else null end) as downvotes,
+hot_rank(coalesce(sum(cl.score) , 0), c.published) as hot_rank
+from comment c
+left join comment_like cl on c.id = cl.comment_id
+group by c.id;
+
+create materialized view comment_aggregates_mview as select * from comment_aggregates_view;
+
+create unique index idx_comment_aggregates_mview_id on comment_aggregates_mview (id);
+
+create view comment_view as
+with all_comment as
+(
+  select
+  ca.*
+  from comment_aggregates_view ca
+)
+
+select
+ac.*,
+u.id as user_id,
+coalesce(cl.score, 0) as my_vote,
+(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.community_id = cf.community_id) as subscribed,
+(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
+from user_ u
+cross join all_comment ac
+left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
+
+union all
+
+select 
+    ac.*,
+    null as user_id, 
+    null as my_vote,
+    null as subscribed,
+    null as saved
+from all_comment ac
+;
+
+create view comment_mview as
+with all_comment as
+(
+  select
+  ca.*
+  from comment_aggregates_mview ca
+)
+
+select
+ac.*,
+u.id as user_id,
+coalesce(cl.score, 0) as my_vote,
+(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.community_id = cf.community_id) as subscribed,
+(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
+from user_ u
+cross join all_comment ac
+left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
+
+union all
+
+select 
+    ac.*,
+    null as user_id, 
+    null as my_vote,
+    null as subscribed,
+    null as saved
+from all_comment ac
+;
+
+-- Do the reply_view referencing the comment_mview
+create view reply_view as 
+with closereply as (
+    select 
+    c2.id, 
+    c2.creator_id as sender_id, 
+    c.creator_id as recipient_id
+    from comment c
+    inner join comment c2 on c.id = c2.parent_id
+    where c2.creator_id != c.creator_id
+    -- Do union where post is null
+    union
+    select
+    c.id,
+    c.creator_id as sender_id,
+    p.creator_id as recipient_id
+    from comment c, post p
+    where c.post_id = p.id and c.parent_id is null and c.creator_id != p.creator_id
+)
+select cv.*,
+closereply.recipient_id
+from comment_mview cv, closereply
+where closereply.id = cv.id
+;
+
+-- user mention
+create view user_mention_view as
+select 
+    c.id,
+    um.id as user_mention_id,
+    c.creator_id,
+    c.creator_actor_id,
+    c.creator_local,
+    c.post_id,
+    c.parent_id,
+    c.content,
+    c.removed,
+    um.read,
+    c.published,
+    c.updated,
+    c.deleted,
+    c.community_id,
+    c.community_actor_id,
+    c.community_local,
+    c.community_name,
+    c.banned,
+    c.banned_from_community,
+    c.creator_name,
+    c.creator_avatar,
+    c.score,
+    c.upvotes,
+    c.downvotes,
+    c.hot_rank,
+    c.user_id,
+    c.my_vote,
+    c.saved,
+    um.recipient_id,
+    (select actor_id from user_ u where u.id = um.recipient_id) as recipient_actor_id,
+    (select local from user_ u where u.id = um.recipient_id) as recipient_local
+from user_mention um, comment_view c
+where um.comment_id = c.id;
+
+
+create view user_mention_mview as 
+with all_comment as
+(
+  select
+  ca.*
+  from comment_aggregates_mview ca
+)
+
+select
+    ac.id,
+    um.id as user_mention_id,
+    ac.creator_id,
+    ac.creator_actor_id,
+    ac.creator_local,
+    ac.post_id,
+    ac.parent_id,
+    ac.content,
+    ac.removed,
+    um.read,
+    ac.published,
+    ac.updated,
+    ac.deleted,
+    ac.community_id,
+    ac.community_actor_id,
+    ac.community_local,
+    ac.community_name,
+    ac.banned,
+    ac.banned_from_community,
+    ac.creator_name,
+    ac.creator_avatar,
+    ac.score,
+    ac.upvotes,
+    ac.downvotes,
+    ac.hot_rank,
+    u.id as user_id,
+    coalesce(cl.score, 0) as my_vote,
+    (select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved,
+    um.recipient_id,
+    (select actor_id from user_ u where u.id = um.recipient_id) as recipient_actor_id,
+    (select local from user_ u where u.id = um.recipient_id) as recipient_local
+from user_ u
+cross join all_comment ac
+left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
+left join user_mention um on um.comment_id = ac.id
+
+union all
+
+select 
+    ac.id,
+    um.id as user_mention_id,
+    ac.creator_id,
+    ac.creator_actor_id,
+    ac.creator_local,
+    ac.post_id,
+    ac.parent_id,
+    ac.content,
+    ac.removed,
+    um.read,
+    ac.published,
+    ac.updated,
+    ac.deleted,
+    ac.community_id,
+    ac.community_actor_id,
+    ac.community_local,
+    ac.community_name,
+    ac.banned,
+    ac.banned_from_community,
+    ac.creator_name,
+    ac.creator_avatar,
+    ac.score,
+    ac.upvotes,
+    ac.downvotes,
+    ac.hot_rank,
+    null as user_id, 
+    null as my_vote,
+    null as saved,
+    um.recipient_id,
+    (select actor_id from user_ u where u.id = um.recipient_id) as recipient_actor_id,
+    (select local from user_ u where u.id = um.recipient_id) as recipient_local
+from all_comment ac
+left join user_mention um on um.comment_id = ac.id
+;
+

server/migrations/2020-04-21-123957_remove_unique_user_constraints/down.sql

@@ -0,0 +1,4 @@
+-- The username index
+drop index idx_user_name_lower_actor_id;
+create unique index idx_user_name_lower on user_ (lower(name));
+

server/migrations/2020-04-21-123957_remove_unique_user_constraints/up.sql

@@ -0,0 +1,2 @@
+drop index idx_user_name_lower;
+create unique index idx_user_name_lower_actor_id on user_ (lower(name), lower(actor_id));

server/migrations/2020-05-05-210233_add_activitypub_for_private_messages/down.sql

@@ -0,0 +1,21 @@
+drop materialized view private_message_mview;
+drop view private_message_view;
+
+alter table private_message 
+drop column ap_id, 
+drop column local;
+
+create view private_message_view as 
+select        
+pm.*,
+u.name as creator_name,
+u.avatar as creator_avatar,
+u2.name as recipient_name,
+u2.avatar as recipient_avatar
+from private_message pm
+inner join user_ u on u.id = pm.creator_id
+inner join user_ u2 on u2.id = pm.recipient_id;
+
+create materialized view private_message_mview as select * from private_message_view;
+
+create unique index idx_private_message_mview_id on private_message_mview (id);

server/migrations/2020-05-05-210233_add_activitypub_for_private_messages/up.sql

@@ -0,0 +1,25 @@
+alter table private_message
+add column ap_id character varying(255) not null default 'changeme', -- This needs to be checked and updated in code, building from the site url if local
+add column local boolean not null default true
+;
+
+drop materialized view private_message_mview;
+drop view private_message_view;
+create view private_message_view as 
+select        
+pm.*,
+u.name as creator_name,
+u.avatar as creator_avatar,
+u.actor_id as creator_actor_id,
+u.local as creator_local,
+u2.name as recipient_name,
+u2.avatar as recipient_avatar,
+u2.actor_id as recipient_actor_id,
+u2.local as recipient_local
+from private_message pm
+inner join user_ u on u.id = pm.creator_id
+inner join user_ u2 on u2.id = pm.recipient_id;
+
+create materialized view private_message_mview as select * from private_message_view;
+
+create unique index idx_private_message_mview_id on private_message_mview (id);

server/src/api/comment.rs

@@ -87,7 +87,8 @@ impl Perform for Oper<CreateComment> {
     }
 
     // Check for a site ban
-    if UserView::read(&conn, user_id)?.banned {
+    let user = User_::read(&conn, user_id)?;
+    if user.banned {
       return Err(APIError::err("site_ban").into());
     }
 
@@ -101,7 +102,10 @@ impl Perform for Oper<CreateComment> {
       removed: None,
       deleted: None,
       read: None,
+      published: None,
       updated: None,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     let inserted_comment = match Comment::create(&conn, &comment_form) {
@@ -109,13 +113,20 @@ impl Perform for Oper<CreateComment> {
       Err(_e) => return Err(APIError::err("couldnt_create_comment").into()),
     };
 
+    let updated_comment = match Comment::update_ap_id(&conn, inserted_comment.id) {
+      Ok(comment) => comment,
+      Err(_e) => return Err(APIError::err("couldnt_create_comment").into()),
+    };
+
+    updated_comment.send_create(&user, &conn)?;
+
     let mut recipient_ids = Vec::new();
 
     // Scan the comment for user mentions, add those rows
     let extracted_usernames = extract_usernames(&comment_form.content);
 
     for username_mention in &extracted_usernames {
-      if let Ok(mention_user) = User_::read_from_name(&conn, (*username_mention).to_string()) {
+      if let Ok(mention_user) = User_::read_from_name(&conn, username_mention) {
         // You can't mention yourself
         // At some point, make it so you can't tag the parent creator either
         // This can cause two notifications, one for reply and the other for mention
@@ -224,6 +235,8 @@ impl Perform for Oper<CreateComment> {
       Err(_e) => return Err(APIError::err("couldnt_like_comment").into()),
     };
 
+    updated_comment.send_like(&user, &conn)?;
+
     let comment_view = CommentView::read(&conn, inserted_comment.id, Some(user_id))?;
 
     let mut res = CommentResponse {
@@ -266,6 +279,8 @@ impl Perform for Oper<EditComment> {
 
     let conn = pool.get()?;
 
+    let user = User_::read(&conn, user_id)?;
+
     let orig_comment = CommentView::read(&conn, data.edit_id, None)?;
 
     // You are allowed to mark the comment as read even if you're banned.
@@ -290,13 +305,15 @@ impl Perform for Oper<EditComment> {
       }
 
       // Check for a site ban
-      if UserView::read(&conn, user_id)?.banned {
+      if user.banned {
         return Err(APIError::err("site_ban").into());
       }
     }
 
     let content_slurs_removed = remove_slurs(&data.content.to_owned());
 
+    let read_comment = Comment::read(&conn, data.edit_id)?;
+
     let comment_form = CommentForm {
       content: content_slurs_removed,
       parent_id: data.parent_id,
@@ -305,25 +322,44 @@ impl Perform for Oper<EditComment> {
       removed: data.removed.to_owned(),
       deleted: data.deleted.to_owned(),
       read: data.read.to_owned(),
+      published: None,
       updated: if data.read.is_some() {
         orig_comment.updated
       } else {
         Some(naive_now())
       },
+      ap_id: read_comment.ap_id,
+      local: read_comment.local,
     };
 
-    let _updated_comment = match Comment::update(&conn, data.edit_id, &comment_form) {
+    let updated_comment = match Comment::update(&conn, data.edit_id, &comment_form) {
       Ok(comment) => comment,
       Err(_e) => return Err(APIError::err("couldnt_update_comment").into()),
     };
 
+    if let Some(deleted) = data.deleted.to_owned() {
+      if deleted {
+        updated_comment.send_delete(&user, &conn)?;
+      } else {
+        updated_comment.send_undo_delete(&user, &conn)?;
+      }
+    } else if let Some(removed) = data.removed.to_owned() {
+      if removed {
+        updated_comment.send_remove(&user, &conn)?;
+      } else {
+        updated_comment.send_undo_remove(&user, &conn)?;
+      }
+    } else {
+      updated_comment.send_update(&user, &conn)?;
+    }
+
     let mut recipient_ids = Vec::new();
 
     // Scan the comment for user mentions, add those rows
     let extracted_usernames = extract_usernames(&comment_form.content);
 
     for username_mention in &extracted_usernames {
-      let mention_user = User_::read_from_name(&conn, (*username_mention).to_string());
+      let mention_user = User_::read_from_name(&conn, username_mention);
 
       if mention_user.is_ok() {
         let mention_user_id = mention_user?.id;
@@ -480,7 +516,8 @@ impl Perform for Oper<CreateCommentLike> {
     }
 
     // Check for a site ban
-    if UserView::read(&conn, user_id)?.banned {
+    let user = User_::read(&conn, user_id)?;
+    if user.banned {
       return Err(APIError::err("site_ban").into());
     }
 
@@ -517,6 +554,14 @@ impl Perform for Oper<CreateCommentLike> {
         Ok(like) => like,
         Err(_e) => return Err(APIError::err("couldnt_like_comment").into()),
       };
+
+      if like_form.score == 1 {
+        comment.send_like(&user, &conn)?;
+      } else if like_form.score == -1 {
+        comment.send_dislike(&user, &conn)?;
+      }
+    } else {
+      comment.send_undo_like(&user, &conn)?;
     }
 
     // Have to refetch the comment to get the current state

server/src/api/community.rs

@@ -1,18 +1,17 @@
 use super::*;
-use crate::is_valid_community_name;
 
 #[derive(Serialize, Deserialize)]
 pub struct GetCommunity {
   id: Option<i32>,
-  name: Option<String>,
+  pub name: Option<String>,
   auth: Option<String>,
 }
 
 #[derive(Serialize, Deserialize)]
 pub struct GetCommunityResponse {
   pub community: CommunityView,
-  moderators: Vec<CommunityModeratorView>,
-  admins: Vec<UserView>,
+  pub moderators: Vec<CommunityModeratorView>,
+  pub admins: Vec<UserView>,
   pub online: usize,
 }
 
@@ -31,17 +30,17 @@ pub struct CommunityResponse {
   pub community: CommunityView,
 }
 
-#[derive(Serialize, Deserialize)]
+#[derive(Serialize, Deserialize, Debug)]
 pub struct ListCommunities {
-  sort: String,
-  page: Option<i64>,
-  limit: Option<i64>,
-  auth: Option<String>,
+  pub sort: String,
+  pub page: Option<i64>,
+  pub limit: Option<i64>,
+  pub auth: Option<String>,
 }
 
-#[derive(Serialize, Deserialize)]
+#[derive(Serialize, Deserialize, Debug)]
 pub struct ListCommunitiesResponse {
-  communities: Vec<CommunityView>,
+  pub communities: Vec<CommunityView>,
 }
 
 #[derive(Serialize, Deserialize, Clone)]
@@ -135,25 +134,25 @@ impl Perform for Oper<GetCommunity> {
 
     let conn = pool.get()?;
 
-    let community_id = match data.id {
-      Some(id) => id,
+    let community = match data.id {
+      Some(id) => Community::read(&conn, id)?,
       None => {
         match Community::read_from_name(
           &conn,
-          data.name.to_owned().unwrap_or_else(|| "main".to_string()),
+          &data.name.to_owned().unwrap_or_else(|| "main".to_string()),
         ) {
-          Ok(community) => community.id,
+          Ok(community) => community,
           Err(_e) => return Err(APIError::err("couldnt_find_community").into()),
         }
       }
     };
 
-    let community_view = match CommunityView::read(&conn, community_id, user_id) {
+    let community_view = match CommunityView::read(&conn, community.id, user_id) {
       Ok(community) => community,
       Err(_e) => return Err(APIError::err("couldnt_find_community").into()),
     };
 
-    let moderators = match CommunityModeratorView::for_community(&conn, community_id) {
+    let moderators = match CommunityModeratorView::for_community(&conn, community.id) {
       Ok(moderators) => moderators,
       Err(_e) => return Err(APIError::err("couldnt_find_community").into()),
     };
@@ -166,8 +165,10 @@ impl Perform for Oper<GetCommunity> {
 
     let online = if let Some(ws) = websocket_info {
       if let Some(id) = ws.id {
-        ws.chatserver
-          .do_send(JoinCommunityRoom { community_id, id });
+        ws.chatserver.do_send(JoinCommunityRoom {
+          community_id: community.id,
+          id,
+        });
       }
 
       // TODO
@@ -221,10 +222,6 @@ impl Perform for Oper<CreateCommunity> {
       }
     }
 
-    if !is_valid_community_name(&data.name) {
-      return Err(APIError::err("invalid_community_name").into());
-    }
-
     let user_id = claims.id;
 
     let conn = pool.get()?;
@@ -235,6 +232,8 @@ impl Perform for Oper<CreateCommunity> {
     }
 
     // When you create a community, make sure the user becomes a moderator and a follower
+    let keypair = generate_actor_keypair()?;
+
     let community_form = CommunityForm {
       name: data.name.to_owned(),
       title: data.title.to_owned(),
@@ -245,6 +244,12 @@ impl Perform for Oper<CreateCommunity> {
       deleted: None,
       nsfw: data.nsfw,
       updated: None,
+      actor_id: make_apub_endpoint(EndpointType::Community, &data.name).to_string(),
+      local: true,
+      private_key: Some(keypair.private_key),
+      public_key: Some(keypair.public_key),
+      last_refreshed_at: None,
+      published: None,
     };
 
     let inserted_community = match Community::create(&conn, &community_form) {
@@ -311,16 +316,13 @@ impl Perform for Oper<EditCommunity> {
       Err(_e) => return Err(APIError::err("not_logged_in").into()),
     };
 
-    if !is_valid_community_name(&data.name) {
-      return Err(APIError::err("invalid_community_name").into());
-    }
-
     let user_id = claims.id;
 
     let conn = pool.get()?;
 
     // Check for a site ban
-    if UserView::read(&conn, user_id)?.banned {
+    let user = User_::read(&conn, user_id)?;
+    if user.banned {
       return Err(APIError::err("site_ban").into());
     }
 
@@ -337,6 +339,8 @@ impl Perform for Oper<EditCommunity> {
       return Err(APIError::err("no_community_edit_allowed").into());
     }
 
+    let read_community = Community::read(&conn, data.edit_id)?;
+
     let community_form = CommunityForm {
       name: data.name.to_owned(),
       title: data.title.to_owned(),
@@ -347,9 +351,15 @@ impl Perform for Oper<EditCommunity> {
       deleted: data.deleted.to_owned(),
       nsfw: data.nsfw,
       updated: Some(naive_now()),
+      actor_id: read_community.actor_id,
+      local: read_community.local,
+      private_key: read_community.private_key,
+      public_key: read_community.public_key,
+      last_refreshed_at: None,
+      published: None,
     };
 
-    let _updated_community = match Community::update(&conn, data.edit_id, &community_form) {
+    let updated_community = match Community::update(&conn, data.edit_id, &community_form) {
       Ok(community) => community,
       Err(_e) => return Err(APIError::err("couldnt_update_community").into()),
     };
@@ -370,6 +380,20 @@ impl Perform for Oper<EditCommunity> {
       ModRemoveCommunity::create(&conn, &form)?;
     }
 
+    if let Some(deleted) = data.deleted.to_owned() {
+      if deleted {
+        updated_community.send_delete(&user, &conn)?;
+      } else {
+        updated_community.send_undo_delete(&user, &conn)?;
+      }
+    } else if let Some(removed) = data.removed.to_owned() {
+      if removed {
+        updated_community.send_remove(&user, &conn)?;
+      } else {
+        updated_community.send_undo_remove(&user, &conn)?;
+      }
+    }
+
     let community_view = CommunityView::read(&conn, data.edit_id, Some(user_id))?;
 
     let res = CommunityResponse {
@@ -456,23 +480,41 @@ impl Perform for Oper<FollowCommunity> {
 
     let user_id = claims.id;
 
+    let conn = pool.get()?;
+
+    let community = Community::read(&conn, data.community_id)?;
     let community_follower_form = CommunityFollowerForm {
       community_id: data.community_id,
       user_id,
     };
 
-    let conn = pool.get()?;
-
-    if data.follow {
-      match CommunityFollower::follow(&conn, &community_follower_form) {
-        Ok(user) => user,
-        Err(_e) => return Err(APIError::err("community_follower_already_exists").into()),
-      };
+    if community.local {
+      if data.follow {
+        match CommunityFollower::follow(&conn, &community_follower_form) {
+          Ok(user) => user,
+          Err(_e) => return Err(APIError::err("community_follower_already_exists").into()),
+        };
+      } else {
+        match CommunityFollower::unfollow(&conn, &community_follower_form) {
+          Ok(user) => user,
+          Err(_e) => return Err(APIError::err("community_follower_already_exists").into()),
+        };
+      }
     } else {
-      match CommunityFollower::ignore(&conn, &community_follower_form) {
-        Ok(user) => user,
-        Err(_e) => return Err(APIError::err("community_follower_already_exists").into()),
-      };
+      let user = User_::read(&conn, user_id)?;
+
+      if data.follow {
+        // Dont actually add to the community followers here, because you need
+        // to wait for the accept
+        user.send_follow(&community.actor_id, &conn)?;
+      } else {
+        user.send_unfollow(&community.actor_id, &conn)?;
+        match CommunityFollower::unfollow(&conn, &community_follower_form) {
+          Ok(user) => user,
+          Err(_e) => return Err(APIError::err("community_follower_already_exists").into()),
+        };
+      }
+      // TODO: this needs to return a "pending" state, until Accept is received from the remote server
     }
 
     let community_view = CommunityView::read(&conn, data.community_id, Some(user_id))?;
@@ -684,11 +726,17 @@ impl Perform for Oper<TransferCommunity> {
       title: read_community.title,
       description: read_community.description,
       category_id: read_community.category_id,
-      creator_id: data.user_id,
+      creator_id: data.user_id, // This makes the new user the community creator
       removed: None,
       deleted: None,
       nsfw: read_community.nsfw,
       updated: Some(naive_now()),
+      actor_id: read_community.actor_id,
+      local: read_community.local,
+      private_key: read_community.private_key,
+      public_key: read_community.public_key,
+      last_refreshed_at: None,
+      published: None,
     };
 
     let _updated_community = match Community::update(&conn, data.community_id, &community_form) {

server/src/api/mod.rs

@@ -18,23 +18,27 @@ use crate::db::user_mention_view::*;
 use crate::db::user_view::*;
 use crate::db::*;
 use crate::{
-  extract_usernames, fetch_iframely_and_pictrs_data, generate_random_string, naive_from_unix,
+  extract_usernames, fetch_iframely_and_pictshare_data, generate_random_string, naive_from_unix,
   naive_now, remove_slurs, send_email, slur_check, slurs_vec_to_str,
 };
 
+use crate::apub::{
+  extensions::signatures::generate_actor_keypair,
+  fetcher::search_by_apub_id,
+  {make_apub_endpoint, ActorType, ApubLikeableType, ApubObjectType, EndpointType},
+};
 use crate::settings::Settings;
-use crate::websocket::UserOperation;
 use crate::websocket::{
   server::{
     JoinCommunityRoom, JoinPostRoom, JoinUserRoom, SendAllMessage, SendComment,
     SendCommunityRoomMessage, SendPost, SendUserRoomMessage,
   },
-  WebsocketInfo,
+  UserOperation, WebsocketInfo,
 };
 use diesel::r2d2::{ConnectionManager, Pool};
 use diesel::PgConnection;
 use failure::Error;
-use log::{error, info};
+use log::{debug, error, info};
 use serde::{Deserialize, Serialize};
 use std::str::FromStr;
 
@@ -62,8 +66,8 @@ pub struct Oper<T> {
   data: T,
 }
 
-impl<T> Oper<T> {
-  pub fn new(data: T) -> Oper<T> {
+impl<Data> Oper<Data> {
+  pub fn new(data: Data) -> Oper<Data> {
     Oper { data }
   }
 }

server/src/api/post.rs

@@ -1,6 +1,6 @@
 use super::*;
 
-#[derive(Serialize, Deserialize)]
+#[derive(Serialize, Deserialize, Debug)]
 pub struct CreatePost {
   name: String,
   url: Option<String>,
@@ -31,7 +31,7 @@ pub struct GetPostResponse {
   pub online: usize,
 }
 
-#[derive(Serialize, Deserialize)]
+#[derive(Serialize, Deserialize, Debug)]
 pub struct GetPosts {
   type_: String,
   sort: String,
@@ -41,9 +41,9 @@ pub struct GetPosts {
   auth: Option<String>,
 }
 
-#[derive(Serialize, Deserialize)]
+#[derive(Serialize, Deserialize, Debug)]
 pub struct GetPostsResponse {
-  posts: Vec<PostView>,
+  pub posts: Vec<PostView>,
 }
 
 #[derive(Serialize, Deserialize)]
@@ -112,13 +112,14 @@ impl Perform for Oper<CreatePost> {
     }
 
     // Check for a site ban
-    if UserView::read(&conn, user_id)?.banned {
+    let user = User_::read(&conn, user_id)?;
+    if user.banned {
       return Err(APIError::err("site_ban").into());
     }
 
-    // Fetch Iframely and pictrs cached image
-    let (iframely_title, iframely_description, iframely_html, pictrs_thumbnail) =
-      fetch_iframely_and_pictrs_data(data.url.to_owned());
+    // Fetch Iframely and Pictshare cached image
+    let (iframely_title, iframely_description, iframely_html, pictshare_thumbnail) =
+      fetch_iframely_and_pictshare_data(data.url.to_owned());
 
     let post_form = PostForm {
       name: data.name.to_owned(),
@@ -135,7 +136,10 @@ impl Perform for Oper<CreatePost> {
       embed_title: iframely_title,
       embed_description: iframely_description,
       embed_html: iframely_html,
-      thumbnail_url: pictrs_thumbnail,
+      thumbnail_url: pictshare_thumbnail,
+      ap_id: "changeme".into(),
+      local: true,
+      published: None,
     };
 
     let inserted_post = match Post::create(&conn, &post_form) {
@@ -151,6 +155,13 @@ impl Perform for Oper<CreatePost> {
       }
     };
 
+    let updated_post = match Post::update_ap_id(&conn, inserted_post.id) {
+      Ok(post) => post,
+      Err(_e) => return Err(APIError::err("couldnt_create_post").into()),
+    };
+
+    updated_post.send_create(&user, &conn)?;
+
     // They like their own post by default
     let like_form = PostLikeForm {
       post_id: inserted_post.id,
@@ -158,12 +169,13 @@ impl Perform for Oper<CreatePost> {
       score: 1,
     };
 
-    // Only add the like if the score isnt 0
     let _inserted_like = match PostLike::like(&conn, &like_form) {
       Ok(like) => like,
       Err(_e) => return Err(APIError::err("couldnt_like_post").into()),
     };
 
+    updated_post.send_like(&user, &conn)?;
+
     // Refetch the view
     let post_view = match PostView::read(&conn, inserted_post.id, Some(user_id)) {
       Ok(post) => post,
@@ -357,7 +369,8 @@ impl Perform for Oper<CreatePostLike> {
     }
 
     // Check for a site ban
-    if UserView::read(&conn, user_id)?.banned {
+    let user = User_::read(&conn, user_id)?;
+    if user.banned {
       return Err(APIError::err("site_ban").into());
     }
 
@@ -377,6 +390,14 @@ impl Perform for Oper<CreatePostLike> {
         Ok(like) => like,
         Err(_e) => return Err(APIError::err("couldnt_like_post").into()),
       };
+
+      if like_form.score == 1 {
+        post.send_like(&user, &conn)?;
+      } else if like_form.score == -1 {
+        post.send_dislike(&user, &conn)?;
+      }
+    } else {
+      post.send_undo_like(&user, &conn)?;
     }
 
     let post_view = match PostView::read(&conn, data.post_id, Some(user_id)) {
@@ -446,13 +467,16 @@ impl Perform for Oper<EditPost> {
     }
 
     // Check for a site ban
-    if UserView::read(&conn, user_id)?.banned {
+    let user = User_::read(&conn, user_id)?;
+    if user.banned {
       return Err(APIError::err("site_ban").into());
     }
 
-    // Fetch Iframely and Pictrs cached image
-    let (iframely_title, iframely_description, iframely_html, pictrs_thumbnail) =
-      fetch_iframely_and_pictrs_data(data.url.to_owned());
+    // Fetch Iframely and Pictshare cached image
+    let (iframely_title, iframely_description, iframely_html, pictshare_thumbnail) =
+      fetch_iframely_and_pictshare_data(data.url.to_owned());
+
+    let read_post = Post::read(&conn, data.edit_id)?;
 
     let post_form = PostForm {
       name: data.name.to_owned(),
@@ -469,10 +493,13 @@ impl Perform for Oper<EditPost> {
       embed_title: iframely_title,
       embed_description: iframely_description,
       embed_html: iframely_html,
-      thumbnail_url: pictrs_thumbnail,
+      thumbnail_url: pictshare_thumbnail,
+      ap_id: read_post.ap_id,
+      local: read_post.local,
+      published: None,
     };
 
-    let _updated_post = match Post::update(&conn, data.edit_id, &post_form) {
+    let updated_post = match Post::update(&conn, data.edit_id, &post_form) {
       Ok(post) => post,
       Err(e) => {
         let err_type = if e.to_string() == "value too long for type character varying(200)" {
@@ -514,6 +541,22 @@ impl Perform for Oper<EditPost> {
       ModStickyPost::create(&conn, &form)?;
     }
 
+    if let Some(deleted) = data.deleted.to_owned() {
+      if deleted {
+        updated_post.send_delete(&user, &conn)?;
+      } else {
+        updated_post.send_undo_delete(&user, &conn)?;
+      }
+    } else if let Some(removed) = data.removed.to_owned() {
+      if removed {
+        updated_post.send_remove(&user, &conn)?;
+      } else {
+        updated_post.send_undo_remove(&user, &conn)?;
+      }
+    } else {
+      updated_post.send_update(&user, &conn)?;
+    }
+
     let post_view = PostView::read(&conn, data.edit_id, Some(user_id))?;
 
     let res = PostResponse { post: post_view };

server/src/api/site.rs

@@ -9,7 +9,7 @@ pub struct ListCategoriesResponse {
   categories: Vec<Category>,
 }
 
-#[derive(Serialize, Deserialize)]
+#[derive(Serialize, Deserialize, Debug)]
 pub struct Search {
   q: String,
   type_: String,
@@ -20,13 +20,13 @@ pub struct Search {
   auth: Option<String>,
 }
 
-#[derive(Serialize, Deserialize)]
+#[derive(Serialize, Deserialize, Debug)]
 pub struct SearchResponse {
-  type_: String,
-  comments: Vec<CommentView>,
-  posts: Vec<PostView>,
-  communities: Vec<CommunityView>,
-  users: Vec<UserView>,
+  pub type_: String,
+  pub comments: Vec<CommentView>,
+  pub posts: Vec<PostView>,
+  pub communities: Vec<CommunityView>,
+  pub users: Vec<UserView>,
 }
 
 #[derive(Serialize, Deserialize)]
@@ -342,8 +342,7 @@ impl Perform for Oper<GetSite> {
     let conn = pool.get()?;
 
     // TODO refactor this a little
-    let site = Site::read(&conn, 1);
-    let site_view = if site.is_ok() {
+    let site_view = if let Ok(_site) = Site::read(&conn, 1) {
       Some(SiteView::read(&conn)?)
     } else if let Some(setup) = Settings::get().setup.as_ref() {
       let register = Register {
@@ -360,9 +359,9 @@ impl Perform for Oper<GetSite> {
       let create_site = CreateSite {
         name: setup.site_name.to_owned(),
         description: None,
-        enable_downvotes: false,
-        open_registration: false,
-        enable_nsfw: false,
+        enable_downvotes: true,
+        open_registration: true,
+        enable_nsfw: true,
         auth: login_response.jwt,
       };
       Oper::new(create_site).perform(pool, websocket_info.clone())?;
@@ -373,11 +372,16 @@ impl Perform for Oper<GetSite> {
     };
 
     let mut admins = UserView::admins(&conn)?;
-    if site_view.is_some() {
-      let site_creator_id = site_view.to_owned().unwrap().creator_id;
-      let creator_index = admins.iter().position(|r| r.id == site_creator_id).unwrap();
-      let creator_user = admins.remove(creator_index);
-      admins.insert(0, creator_user);
+
+    // Make sure the site creator is the top admin
+    if let Some(site_view) = site_view.to_owned() {
+      let site_creator_id = site_view.creator_id;
+      // TODO investigate why this is sometimes coming back null
+      // Maybe user_.admin isn't being set to true?
+      if let Some(creator_index) = admins.iter().position(|r| r.id == site_creator_id) {
+        let creator_user = admins.remove(creator_index);
+        admins.insert(0, creator_user);
+      }
     }
 
     let banned = UserView::banned(&conn)?;
@@ -412,6 +416,15 @@ impl Perform for Oper<Search> {
   ) -> Result<SearchResponse, Error> {
     let data: &Search = &self.data;
 
+    dbg!(&data);
+
+    let conn = pool.get()?;
+
+    match search_by_apub_id(&data.q, &conn) {
+      Ok(r) => return Ok(r),
+      Err(e) => debug!("Failed to resolve search query as activitypub ID: {}", e),
+    }
+
     let user_id: Option<i32> = match &data.auth {
       Some(auth) => match Claims::decode(&auth) {
         Ok(claims) => {
@@ -433,8 +446,6 @@ impl Perform for Oper<Search> {
 
     // TODO no clean / non-nsfw searching rn
 
-    let conn = pool.get()?;
-
     match type_ {
       SearchType::Posts => {
         posts = PostQueryBuilder::create(&conn)

server/src/api/user.rs

@@ -1,5 +1,4 @@
 use super::*;
-use crate::is_valid_username;
 use bcrypt::verify;
 
 #[derive(Serialize, Deserialize, Debug)]
@@ -187,7 +186,7 @@ pub struct PrivateMessagesResponse {
 
 #[derive(Serialize, Deserialize, Clone)]
 pub struct PrivateMessageResponse {
-  message: PrivateMessageView,
+  pub message: PrivateMessageView,
 }
 
 #[derive(Serialize, Deserialize, Debug)]
@@ -262,14 +261,11 @@ impl Perform for Oper<Register> {
       return Err(APIError::err("admin_already_created").into());
     }
 
-    if !is_valid_username(&data.username) {
-      return Err(APIError::err("invalid_username").into());
-    }
+    let user_keypair = generate_actor_keypair()?;
 
     // Register the new user
     let user_form = UserForm {
       name: data.username.to_owned(),
-      fedi_name: Settings::get().hostname,
       email: data.email.to_owned(),
       matrix_user_id: None,
       avatar: None,
@@ -285,6 +281,12 @@ impl Perform for Oper<Register> {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: make_apub_endpoint(EndpointType::User, &data.username).to_string(),
+      bio: None,
+      local: true,
+      private_key: Some(user_keypair.private_key),
+      public_key: Some(user_keypair.public_key),
+      last_refreshed_at: None,
     };
 
     // Create the user
@@ -303,12 +305,15 @@ impl Perform for Oper<Register> {
       }
     };
 
+    let main_community_keypair = generate_actor_keypair()?;
+
     // Create the main community if it doesn't exist
     let main_community: Community = match Community::read(&conn, 2) {
       Ok(c) => c,
       Err(_e) => {
+        let default_community_name = "main";
         let community_form = CommunityForm {
-          name: "main".to_string(),
+          name: default_community_name.to_string(),
           title: "The Default Community".to_string(),
           description: Some("The Default Community".to_string()),
           category_id: 1,
@@ -317,6 +322,12 @@ impl Perform for Oper<Register> {
           removed: None,
           deleted: None,
           updated: None,
+          actor_id: make_apub_endpoint(EndpointType::Community, default_community_name).to_string(),
+          local: true,
+          private_key: Some(main_community_keypair.private_key),
+          public_key: Some(main_community_keypair.public_key),
+          last_refreshed_at: None,
+          published: None,
         };
         Community::create(&conn, &community_form).unwrap()
       }
@@ -411,7 +422,6 @@ impl Perform for Oper<SaveUserSettings> {
 
     let user_form = UserForm {
       name: read_user.name,
-      fedi_name: read_user.fedi_name,
       email,
       matrix_user_id: data.matrix_user_id.to_owned(),
       avatar: data.avatar.to_owned(),
@@ -427,6 +437,12 @@ impl Perform for Oper<SaveUserSettings> {
       lang: data.lang.to_owned(),
       show_avatars: data.show_avatars,
       send_notifications_to_email: data.send_notifications_to_email,
+      actor_id: read_user.actor_id,
+      bio: read_user.bio,
+      local: read_user.local,
+      private_key: read_user.private_key,
+      public_key: read_user.public_key,
+      last_refreshed_at: None,
     };
 
     let updated_user = match User_::update(&conn, user_id, &user_form) {
@@ -488,7 +504,7 @@ impl Perform for Oper<GetUserDetails> {
       None => {
         match User_::read_from_name(
           &conn,
-          data
+          &data
             .username
             .to_owned()
             .unwrap_or_else(|| "admin".to_string()),
@@ -580,30 +596,7 @@ impl Perform for Oper<AddAdmin> {
       return Err(APIError::err("not_an_admin").into());
     }
 
-    let read_user = User_::read(&conn, data.user_id)?;
-
-    // TODO make addadmin easier
-    let user_form = UserForm {
-      name: read_user.name,
-      fedi_name: read_user.fedi_name,
-      email: read_user.email,
-      matrix_user_id: read_user.matrix_user_id,
-      avatar: read_user.avatar,
-      password_encrypted: read_user.password_encrypted,
-      preferred_username: read_user.preferred_username,
-      updated: Some(naive_now()),
-      admin: data.added,
-      banned: read_user.banned,
-      show_nsfw: read_user.show_nsfw,
-      theme: read_user.theme,
-      default_sort_type: read_user.default_sort_type,
-      default_listing_type: read_user.default_listing_type,
-      lang: read_user.lang,
-      show_avatars: read_user.show_avatars,
-      send_notifications_to_email: read_user.send_notifications_to_email,
-    };
-
-    match User_::update(&conn, data.user_id, &user_form) {
+    match User_::add_admin(&conn, user_id, data.added) {
       Ok(user) => user,
       Err(_e) => return Err(APIError::err("couldnt_update_user").into()),
     };
@@ -661,30 +654,7 @@ impl Perform for Oper<BanUser> {
       return Err(APIError::err("not_an_admin").into());
     }
 
-    let read_user = User_::read(&conn, data.user_id)?;
-
-    // TODO make bans and addadmins easier
-    let user_form = UserForm {
-      name: read_user.name,
-      fedi_name: read_user.fedi_name,
-      email: read_user.email,
-      matrix_user_id: read_user.matrix_user_id,
-      avatar: read_user.avatar,
-      password_encrypted: read_user.password_encrypted,
-      preferred_username: read_user.preferred_username,
-      updated: Some(naive_now()),
-      admin: read_user.admin,
-      banned: data.ban,
-      show_nsfw: read_user.show_nsfw,
-      theme: read_user.theme,
-      default_sort_type: read_user.default_sort_type,
-      default_listing_type: read_user.default_listing_type,
-      lang: read_user.lang,
-      show_avatars: read_user.show_avatars,
-      send_notifications_to_email: read_user.send_notifications_to_email,
-    };
-
-    match User_::update(&conn, data.user_id, &user_form) {
+    match User_::ban_user(&conn, user_id, data.ban) {
       Ok(user) => user,
       Err(_e) => return Err(APIError::err("couldnt_update_user").into()),
     };
@@ -855,18 +825,7 @@ impl Perform for Oper<MarkAllAsRead> {
       .list()?;
 
     for reply in &replies {
-      let comment_form = CommentForm {
-        content: reply.to_owned().content,
-        parent_id: reply.to_owned().parent_id,
-        post_id: reply.to_owned().post_id,
-        creator_id: reply.to_owned().creator_id,
-        removed: None,
-        deleted: None,
-        read: Some(true),
-        updated: reply.to_owned().updated,
-      };
-
-      let _updated_comment = match Comment::update(&conn, reply.id, &comment_form) {
+      match Comment::mark_as_read(&conn, reply.id) {
         Ok(comment) => comment,
         Err(_e) => return Err(APIError::err("couldnt_update_comment").into()),
       };
@@ -902,12 +861,15 @@ impl Perform for Oper<MarkAllAsRead> {
 
     for message in &messages {
       let private_message_form = PrivateMessageForm {
-        content: None,
+        content: message.to_owned().content,
         creator_id: message.to_owned().creator_id,
         recipient_id: message.to_owned().recipient_id,
         deleted: None,
         read: Some(true),
         updated: None,
+        ap_id: message.to_owned().ap_id,
+        local: message.local,
+        published: None,
       };
 
       let _updated_message = match PrivateMessage::update(&conn, message.id, &private_message_form)
@@ -955,18 +917,7 @@ impl Perform for Oper<DeleteAccount> {
       .list()?;
 
     for comment in &comments {
-      let comment_form = CommentForm {
-        content: "*Permananently Deleted*".to_string(),
-        parent_id: comment.to_owned().parent_id,
-        post_id: comment.to_owned().post_id,
-        creator_id: comment.to_owned().creator_id,
-        removed: None,
-        deleted: Some(true),
-        read: None,
-        updated: Some(naive_now()),
-      };
-
-      let _updated_comment = match Comment::update(&conn, comment.id, &comment_form) {
+      let _updated_comment = match Comment::permadelete(&conn, comment.id) {
         Ok(comment) => comment,
         Err(_e) => return Err(APIError::err("couldnt_update_comment").into()),
       };
@@ -980,25 +931,7 @@ impl Perform for Oper<DeleteAccount> {
       .list()?;
 
     for post in &posts {
-      let post_form = PostForm {
-        name: "*Permananently Deleted*".to_string(),
-        url: Some("https://deleted.com".to_string()),
-        body: Some("*Permananently Deleted*".to_string()),
-        creator_id: post.to_owned().creator_id,
-        community_id: post.to_owned().community_id,
-        removed: None,
-        deleted: Some(true),
-        nsfw: post.to_owned().nsfw,
-        locked: None,
-        stickied: None,
-        updated: Some(naive_now()),
-        embed_title: None,
-        embed_description: None,
-        embed_html: None,
-        thumbnail_url: None,
-      };
-
-      let _updated_post = match Post::update(&conn, post.id, &post_form) {
+      let _updated_post = match Post::permadelete(&conn, post.id) {
         Ok(post) => post,
         Err(_e) => return Err(APIError::err("couldnt_update_post").into()),
       };
@@ -1104,19 +1037,23 @@ impl Perform for Oper<CreatePrivateMessage> {
     let conn = pool.get()?;
 
     // Check for a site ban
-    if UserView::read(&conn, user_id)?.banned {
+    let user = User_::read(&conn, user_id)?;
+    if user.banned {
       return Err(APIError::err("site_ban").into());
     }
 
     let content_slurs_removed = remove_slurs(&data.content.to_owned());
 
     let private_message_form = PrivateMessageForm {
-      content: Some(content_slurs_removed.to_owned()),
+      content: content_slurs_removed.to_owned(),
       creator_id: user_id,
       recipient_id: data.recipient_id,
       deleted: None,
       read: None,
       updated: None,
+      ap_id: "changeme".into(),
+      local: true,
+      published: None,
     };
 
     let inserted_private_message = match PrivateMessage::create(&conn, &private_message_form) {
@@ -1126,6 +1063,14 @@ impl Perform for Oper<CreatePrivateMessage> {
       }
     };
 
+    let updated_private_message =
+      match PrivateMessage::update_ap_id(&conn, inserted_private_message.id) {
+        Ok(private_message) => private_message,
+        Err(_e) => return Err(APIError::err("couldnt_create_private_message").into()),
+      };
+
+    updated_private_message.send_create(&user, &conn)?;
+
     // Send notifications to the recipient
     let recipient_user = User_::read(&conn, data.recipient_id)?;
     if recipient_user.send_notifications_to_email {
@@ -1169,7 +1114,7 @@ impl Perform for Oper<EditPrivateMessage> {
   fn perform(
     &self,
     pool: Pool<ConnectionManager<PgConnection>>,
-    _websocket_info: Option<WebsocketInfo>,
+    websocket_info: Option<WebsocketInfo>,
   ) -> Result<PrivateMessageResponse, Error> {
     let data: &EditPrivateMessage = &self.data;
 
@@ -1185,7 +1130,8 @@ impl Perform for Oper<EditPrivateMessage> {
     let orig_private_message = PrivateMessage::read(&conn, data.edit_id)?;
 
     // Check for a site ban
-    if UserView::read(&conn, user_id)?.banned {
+    let user = User_::read(&conn, user_id)?;
+    if user.banned {
       return Err(APIError::err("site_ban").into());
     }
 
@@ -1197,8 +1143,8 @@ impl Perform for Oper<EditPrivateMessage> {
     }
 
     let content_slurs_removed = match &data.content {
-      Some(content) => Some(remove_slurs(content)),
-      None => None,
+      Some(content) => remove_slurs(content),
+      None => orig_private_message.content,
     };
 
     let private_message_form = PrivateMessageForm {
@@ -1212,17 +1158,41 @@ impl Perform for Oper<EditPrivateMessage> {
       } else {
         Some(naive_now())
       },
+      ap_id: orig_private_message.ap_id,
+      local: orig_private_message.local,
+      published: None,
     };
 
-    let _updated_private_message =
+    let updated_private_message =
       match PrivateMessage::update(&conn, data.edit_id, &private_message_form) {
         Ok(private_message) => private_message,
         Err(_e) => return Err(APIError::err("couldnt_update_private_message").into()),
       };
 
+    if let Some(deleted) = data.deleted.to_owned() {
+      if deleted {
+        updated_private_message.send_delete(&user, &conn)?;
+      } else {
+        updated_private_message.send_undo_delete(&user, &conn)?;
+      }
+    } else {
+      updated_private_message.send_update(&user, &conn)?;
+    }
+
     let message = PrivateMessageView::read(&conn, data.edit_id)?;
 
-    Ok(PrivateMessageResponse { message })
+    let res = PrivateMessageResponse { message };
+
+    if let Some(ws) = websocket_info {
+      ws.chatserver.do_send(SendUserRoomMessage {
+        op: UserOperation::EditPrivateMessage,
+        response: res.clone(),
+        recipient_id: orig_private_message.recipient_id,
+        my_id: ws.id,
+      });
+    }
+
+    Ok(res)
   }
 }
 

server/src/apub/activities.rs

@@ -0,0 +1,49 @@
+use super::*;
+
+pub fn populate_object_props(
+  props: &mut ObjectProperties,
+  addressed_to: &str,
+  object_id: &str,
+) -> Result<(), Error> {
+  props
+    .set_context_xsd_any_uri(context())?
+    // TODO: the activity needs a seperate id from the object
+    .set_id(object_id)?
+    // TODO: should to/cc go on the Create, or on the Post? or on both?
+    // TODO: handle privacy on the receiving side (at least ignore anything thats not public)
+    .set_to_xsd_any_uri(public())?
+    .set_cc_xsd_any_uri(addressed_to)?;
+  Ok(())
+}
+
+/// Send an activity to a list of recipients, using the correct headers etc.
+pub fn send_activity<A>(
+  activity: &A,
+  private_key: &str,
+  sender_id: &str,
+  to: Vec<String>,
+) -> Result<(), Error>
+where
+  A: Serialize + Debug,
+{
+  let json = serde_json::to_string(&activity)?;
+  debug!("Sending activitypub activity {} to {:?}", json, to);
+  // TODO it needs to expand, the to field needs to expand and dedup the followers urls
+  // The inbox is determined by first retrieving the target actor's JSON-LD representation and then looking up the inbox property. If a recipient is a Collection or OrderedCollection, then the server MUST dereference the collection (with the user's credentials) and discover inboxes for each item in the collection. Servers MUST limit the number of layers of indirections through collections which will be performed, which MAY be one.
+  for t in to {
+    let to_url = Url::parse(&t)?;
+    if !is_apub_id_valid(&to_url) {
+      debug!("Not sending activity to {} (invalid or blacklisted)", t);
+      continue;
+    }
+    let request = Request::post(t).header("Host", to_url.domain().unwrap());
+    let signature = sign(&request, private_key, sender_id)?;
+    let res = request
+      .header("Signature", signature)
+      .header("Content-Type", "application/json")
+      .body(json.to_owned())?
+      .send()?;
+    debug!("Result for activity send: {:?}", res);
+  }
+  Ok(())
+}

server/src/apub/comment.rs

@@ -0,0 +1,482 @@
+use super::*;
+
+#[derive(Deserialize)]
+pub struct CommentQuery {
+  comment_id: String,
+}
+
+/// Return the post json over HTTP.
+pub async fn get_apub_comment(
+  info: Path<CommentQuery>,
+  db: DbPoolParam,
+) -> Result<HttpResponse<Body>, Error> {
+  let id = info.comment_id.parse::<i32>()?;
+  let comment = Comment::read(&&db.get()?, id)?;
+  if !comment.deleted {
+    Ok(create_apub_response(&comment.to_apub(&db.get().unwrap())?))
+  } else {
+    Ok(create_apub_tombstone_response(&comment.to_tombstone()?))
+  }
+}
+
+impl ToApub for Comment {
+  type Response = Note;
+
+  fn to_apub(&self, conn: &PgConnection) -> Result<Note, Error> {
+    let mut comment = Note::default();
+    let oprops: &mut ObjectProperties = comment.as_mut();
+    let creator = User_::read(&conn, self.creator_id)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(&conn, post.community_id)?;
+
+    // Add a vector containing some important info to the "in_reply_to" field
+    // [post_ap_id, Option(parent_comment_ap_id)]
+    let mut in_reply_to_vec = vec![post.ap_id];
+
+    if let Some(parent_id) = self.parent_id {
+      let parent_comment = Comment::read(&conn, parent_id)?;
+      in_reply_to_vec.push(parent_comment.ap_id);
+    }
+
+    oprops
+      // Not needed when the Post is embedded in a collection (like for community outbox)
+      .set_context_xsd_any_uri(context())?
+      .set_id(self.ap_id.to_owned())?
+      .set_published(convert_datetime(self.published))?
+      .set_to_xsd_any_uri(community.actor_id)?
+      .set_many_in_reply_to_xsd_any_uris(in_reply_to_vec)?
+      .set_content_xsd_string(self.content.to_owned())?
+      .set_attributed_to_xsd_any_uri(creator.actor_id)?;
+
+    if let Some(u) = self.updated {
+      oprops.set_updated(convert_datetime(u))?;
+    }
+
+    Ok(comment)
+  }
+
+  fn to_tombstone(&self) -> Result<Tombstone, Error> {
+    create_tombstone(
+      self.deleted,
+      &self.ap_id,
+      self.updated,
+      NoteType.to_string(),
+    )
+  }
+}
+
+impl FromApub for CommentForm {
+  type ApubType = Note;
+
+  /// Parse an ActivityPub note received from another instance into a Lemmy comment
+  fn from_apub(note: &Note, conn: &PgConnection) -> Result<CommentForm, Error> {
+    let oprops = &note.object_props;
+    let creator_actor_id = &oprops.get_attributed_to_xsd_any_uri().unwrap().to_string();
+    let creator = get_or_fetch_and_upsert_remote_user(&creator_actor_id, &conn)?;
+
+    let mut in_reply_tos = oprops.get_many_in_reply_to_xsd_any_uris().unwrap();
+    let post_ap_id = in_reply_tos.next().unwrap().to_string();
+
+    // The 2nd item, if it exists, is the parent comment apub_id
+    let parent_id: Option<i32> = match in_reply_tos.next() {
+      Some(parent_comment_uri) => {
+        let parent_comment_uri_str = &parent_comment_uri.to_string();
+        let parent_comment = Comment::read_from_apub_id(&conn, &parent_comment_uri_str)?;
+
+        Some(parent_comment.id)
+      }
+      None => None,
+    };
+
+    let post = Post::read_from_apub_id(&conn, &post_ap_id)?;
+
+    Ok(CommentForm {
+      creator_id: creator.id,
+      post_id: post.id,
+      parent_id,
+      content: oprops
+        .get_content_xsd_string()
+        .map(|c| c.to_string())
+        .unwrap(),
+      removed: None,
+      read: None,
+      published: oprops
+        .get_published()
+        .map(|u| u.as_ref().to_owned().naive_local()),
+      updated: oprops
+        .get_updated()
+        .map(|u| u.as_ref().to_owned().naive_local()),
+      deleted: None,
+      ap_id: oprops.get_id().unwrap().to_string(),
+      local: false,
+    })
+  }
+}
+
+impl ApubObjectType for Comment {
+  /// Send out information about a newly created comment, to the followers of the community.
+  fn send_create(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(conn)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(conn, post.community_id)?;
+    let id = format!("{}/create/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut create = Create::new();
+    populate_object_props(
+      &mut create.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+    create
+      .create_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&create)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &create,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  /// Send out information about an edited post, to the followers of the community.
+  fn send_update(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(&conn)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(&conn, post.community_id)?;
+    let id = format!("{}/update/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut update = Update::new();
+    populate_object_props(
+      &mut update.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+    update
+      .update_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&update)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &update,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(&conn)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(&conn, post.community_id)?;
+    let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut delete = Delete::default();
+
+    populate_object_props(
+      &mut delete.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+
+    delete
+      .delete_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&delete)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &delete,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_undo_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(&conn)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(&conn, post.community_id)?;
+
+    // Generate a fake delete activity, with the correct object
+    let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut delete = Delete::default();
+
+    populate_object_props(
+      &mut delete.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+
+    delete
+      .delete_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // TODO
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/delete/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    populate_object_props(
+      &mut undo.object_props,
+      &community.get_followers_url(),
+      &undo_id,
+    )?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(delete)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &undo,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(&conn)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(&conn, post.community_id)?;
+    let id = format!("{}/remove/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut remove = Remove::default();
+
+    populate_object_props(
+      &mut remove.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+
+    remove
+      .remove_props
+      .set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: mod_.id,
+      data: serde_json::to_value(&remove)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &remove,
+      &mod_.private_key.as_ref().unwrap(),
+      &mod_.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_undo_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(&conn)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(&conn, post.community_id)?;
+
+    // Generate a fake delete activity, with the correct object
+    let id = format!("{}/remove/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut remove = Remove::default();
+
+    populate_object_props(
+      &mut remove.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+
+    remove
+      .remove_props
+      .set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/remove/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    populate_object_props(
+      &mut undo.object_props,
+      &community.get_followers_url(),
+      &undo_id,
+    )?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
+      .set_object_base_box(remove)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: mod_.id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &undo,
+      &mod_.private_key.as_ref().unwrap(),
+      &mod_.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+}
+
+impl ApubLikeableType for Comment {
+  fn send_like(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(&conn)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(&conn, post.community_id)?;
+    let id = format!("{}/like/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut like = Like::new();
+    populate_object_props(&mut like.object_props, &community.get_followers_url(), &id)?;
+    like
+      .like_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&like)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &like,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_dislike(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(&conn)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(&conn, post.community_id)?;
+    let id = format!("{}/dislike/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut dislike = Dislike::new();
+    populate_object_props(
+      &mut dislike.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+    dislike
+      .dislike_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&dislike)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &dislike,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_undo_like(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(&conn)?;
+    let post = Post::read(&conn, self.post_id)?;
+    let community = Community::read(&conn, post.community_id)?;
+    let id = format!("{}/dislike/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut like = Like::new();
+    populate_object_props(&mut like.object_props, &community.get_followers_url(), &id)?;
+    like
+      .like_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // TODO
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/like/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    populate_object_props(
+      &mut undo.object_props,
+      &community.get_followers_url(),
+      &undo_id,
+    )?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(like)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &undo,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+}

server/src/apub/community.rs

@@ -1,109 +1,424 @@
-use crate::apub::make_apub_endpoint;
-use crate::db::community::Community;
-use crate::db::community_view::CommunityFollowerView;
-use crate::db::establish_unpooled_connection;
-use crate::to_datetime_utc;
-use activitypub::{actor::Group, collection::UnorderedCollection, context};
-use actix_web::body::Body;
-use actix_web::web::Path;
-use actix_web::HttpResponse;
-use serde::Deserialize;
-
-impl Community {
-  pub fn as_group(&self) -> Group {
-    let base_url = make_apub_endpoint("c", &self.name);
-
-    let mut group = Group::default();
-
-    group.object_props.set_context_object(context()).ok();
-    group.object_props.set_id_string(base_url.to_string()).ok();
-    group
-      .object_props
-      .set_name_string(self.name.to_owned())
-      .ok();
-    group
-      .object_props
-      .set_published_utctime(to_datetime_utc(self.published))
-      .ok();
-    if let Some(updated) = self.updated {
-      group
-        .object_props
-        .set_updated_utctime(to_datetime_utc(updated))
-        .ok();
-    }
-
-    if let Some(description) = &self.description {
-      group
-        .object_props
-        .set_summary_string(description.to_string())
-        .ok();
-    }
-
-    group
-      .ap_actor_props
-      .set_inbox_string(format!("{}/inbox", &base_url))
-      .ok();
-    group
-      .ap_actor_props
-      .set_outbox_string(format!("{}/outbox", &base_url))
-      .ok();
-    group
-      .ap_actor_props
-      .set_followers_string(format!("{}/followers", &base_url))
-      .ok();
-
-    group
-  }
-
-  pub fn followers_as_collection(&self) -> UnorderedCollection {
-    let base_url = make_apub_endpoint("c", &self.name);
-
-    let mut collection = UnorderedCollection::default();
-    collection.object_props.set_context_object(context()).ok();
-    collection.object_props.set_id_string(base_url).ok();
-
-    let connection = establish_unpooled_connection();
-    //As we are an object, we validated that the community id was valid
-    let community_followers = CommunityFollowerView::for_community(&connection, self.id).unwrap();
-
-    let ap_followers = community_followers
-      .iter()
-      .map(|follower| make_apub_endpoint("u", &follower.user_name))
-      .collect();
-
-    collection
-      .collection_props
-      .set_items_string_vec(ap_followers)
-      .unwrap();
-    collection
-  }
-}
+use super::*;
 
 #[derive(Deserialize)]
 pub struct CommunityQuery {
   community_name: String,
 }
 
-pub async fn get_apub_community(info: Path<CommunityQuery>) -> HttpResponse<Body> {
-  let connection = establish_unpooled_connection();
+impl ToApub for Community {
+  type Response = GroupExt;
 
-  if let Ok(community) = Community::read_from_name(&connection, info.community_name.to_owned()) {
-    HttpResponse::Ok()
-      .content_type("application/activity+json")
-      .body(serde_json::to_string(&community.as_group()).unwrap())
-  } else {
-    HttpResponse::NotFound().finish()
+  // Turn a Lemmy Community into an ActivityPub group that can be sent out over the network.
+  fn to_apub(&self, conn: &PgConnection) -> Result<GroupExt, Error> {
+    let mut group = Group::default();
+    let oprops: &mut ObjectProperties = group.as_mut();
+
+    // The attributed to, is an ordered vector with the creator actor_ids first,
+    // then the rest of the moderators
+    // TODO Technically the instance admins can mod the community, but lets
+    // ignore that for now
+    let moderators = CommunityModeratorView::for_community(&conn, self.id)?
+      .into_iter()
+      .map(|m| m.user_actor_id)
+      .collect();
+
+    oprops
+      .set_context_xsd_any_uri(context())?
+      .set_id(self.actor_id.to_owned())?
+      .set_name_xsd_string(self.name.to_owned())?
+      .set_published(convert_datetime(self.published))?
+      .set_many_attributed_to_xsd_any_uris(moderators)?;
+
+    if let Some(u) = self.updated.to_owned() {
+      oprops.set_updated(convert_datetime(u))?;
+    }
+    if let Some(d) = self.description.to_owned() {
+      // TODO: this should be html, also add source field with raw markdown
+      //       -> same for post.content and others
+      oprops.set_summary_xsd_string(d)?;
+    }
+
+    let mut endpoint_props = EndpointProperties::default();
+
+    endpoint_props.set_shared_inbox(self.get_shared_inbox_url())?;
+
+    let mut actor_props = ApActorProperties::default();
+
+    actor_props
+      .set_preferred_username(self.title.to_owned())?
+      .set_inbox(self.get_inbox_url())?
+      .set_outbox(self.get_outbox_url())?
+      .set_endpoints(endpoint_props)?
+      .set_followers(self.get_followers_url())?;
+
+    let group_extension = GroupExtension::new(conn, self.category_id, self.nsfw)?;
+
+    Ok(
+      group
+        .extend(group_extension)
+        .extend(actor_props)
+        .extend(self.get_public_key_ext()),
+    )
+  }
+
+  fn to_tombstone(&self) -> Result<Tombstone, Error> {
+    create_tombstone(
+      self.deleted,
+      &self.actor_id,
+      self.updated,
+      GroupType.to_string(),
+    )
   }
 }
 
-pub async fn get_apub_community_followers(info: Path<CommunityQuery>) -> HttpResponse<Body> {
-  let connection = establish_unpooled_connection();
+impl ActorType for Community {
+  fn actor_id(&self) -> String {
+    self.actor_id.to_owned()
+  }
 
-  if let Ok(community) = Community::read_from_name(&connection, info.community_name.to_owned()) {
-    HttpResponse::Ok()
-      .content_type("application/activity+json")
-      .body(serde_json::to_string(&community.followers_as_collection()).unwrap())
-  } else {
-    HttpResponse::NotFound().finish()
+  fn public_key(&self) -> String {
+    self.public_key.to_owned().unwrap()
+  }
+
+  /// As a local community, accept the follow request from a remote user.
+  fn send_accept_follow(&self, follow: &Follow, conn: &PgConnection) -> Result<(), Error> {
+    let actor_uri = follow
+      .follow_props
+      .get_actor_xsd_any_uri()
+      .unwrap()
+      .to_string();
+    let id = format!("{}/accept/{}", self.actor_id, uuid::Uuid::new_v4());
+
+    let mut accept = Accept::new();
+    accept
+      .object_props
+      .set_context_xsd_any_uri(context())?
+      .set_id(id)?;
+    accept
+      .accept_props
+      .set_actor_xsd_any_uri(self.actor_id.to_owned())?
+      .set_object_base_box(BaseBox::from_concrete(follow.clone())?)?;
+    let to = format!("{}/inbox", actor_uri);
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: self.creator_id,
+      data: serde_json::to_value(&accept)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &accept,
+      &self.private_key.to_owned().unwrap(),
+      &self.actor_id,
+      vec![to],
+    )?;
+    Ok(())
+  }
+
+  fn send_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let group = self.to_apub(conn)?;
+    let id = format!("{}/delete/{}", self.actor_id, uuid::Uuid::new_v4());
+
+    let mut delete = Delete::default();
+    populate_object_props(&mut delete.object_props, &self.get_followers_url(), &id)?;
+
+    delete
+      .delete_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(group)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: self.creator_id,
+      data: serde_json::to_value(&delete)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    // Note: For an accept, since it was automatic, no one pushed a button,
+    // the community was the actor.
+    // But for delete, the creator is the actor, and does the signing
+    send_activity(
+      &delete,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      self.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_undo_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let group = self.to_apub(conn)?;
+    let id = format!("{}/delete/{}", self.actor_id, uuid::Uuid::new_v4());
+
+    let mut delete = Delete::default();
+    populate_object_props(&mut delete.object_props, &self.get_followers_url(), &id)?;
+
+    delete
+      .delete_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(group)?;
+
+    // TODO
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/delete/{}", self.actor_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    populate_object_props(&mut undo.object_props, &self.get_followers_url(), &undo_id)?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(delete)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: self.creator_id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    // Note: For an accept, since it was automatic, no one pushed a button,
+    // the community was the actor.
+    // But for delete, the creator is the actor, and does the signing
+    send_activity(
+      &undo,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      self.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let group = self.to_apub(conn)?;
+    let id = format!("{}/remove/{}", self.actor_id, uuid::Uuid::new_v4());
+
+    let mut remove = Remove::default();
+    populate_object_props(&mut remove.object_props, &self.get_followers_url(), &id)?;
+
+    remove
+      .remove_props
+      .set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
+      .set_object_base_box(group)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: mod_.id,
+      data: serde_json::to_value(&remove)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    // Note: For an accept, since it was automatic, no one pushed a button,
+    // the community was the actor.
+    // But for delete, the creator is the actor, and does the signing
+    send_activity(
+      &remove,
+      &mod_.private_key.as_ref().unwrap(),
+      &mod_.actor_id,
+      self.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_undo_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let group = self.to_apub(conn)?;
+    let id = format!("{}/remove/{}", self.actor_id, uuid::Uuid::new_v4());
+
+    let mut remove = Remove::default();
+    populate_object_props(&mut remove.object_props, &self.get_followers_url(), &id)?;
+
+    remove
+      .remove_props
+      .set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
+      .set_object_base_box(group)?;
+
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/remove/{}", self.actor_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    populate_object_props(&mut undo.object_props, &self.get_followers_url(), &undo_id)?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
+      .set_object_base_box(remove)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: mod_.id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    // Note: For an accept, since it was automatic, no one pushed a button,
+    // the community was the actor.
+    // But for remove , the creator is the actor, and does the signing
+    send_activity(
+      &undo,
+      &mod_.private_key.as_ref().unwrap(),
+      &mod_.actor_id,
+      self.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  /// For a given community, returns the inboxes of all followers.
+  fn get_follower_inboxes(&self, conn: &PgConnection) -> Result<Vec<String>, Error> {
+    Ok(
+      CommunityFollowerView::for_community(conn, self.id)?
+        .into_iter()
+        // TODO eventually this will have to use the inbox or shared_inbox column, meaning that view
+        // will have to change
+        .map(|c| {
+          // If the user is local, but the community isn't, get the community shared inbox
+          // and vice versa
+          if c.user_local && !c.community_local {
+            get_shared_inbox(&c.community_actor_id)
+          } else if !c.user_local && c.community_local {
+            get_shared_inbox(&c.user_actor_id)
+          } else {
+            "".to_string()
+          }
+        })
+        .filter(|s| !s.is_empty())
+        .unique()
+        .collect(),
+    )
+  }
+
+  fn send_follow(&self, _follow_actor_id: &str, _conn: &PgConnection) -> Result<(), Error> {
+    unimplemented!()
+  }
+
+  fn send_unfollow(&self, _follow_actor_id: &str, _conn: &PgConnection) -> Result<(), Error> {
+    unimplemented!()
   }
 }
+
+impl FromApub for CommunityForm {
+  type ApubType = GroupExt;
+
+  /// Parse an ActivityPub group received from another instance into a Lemmy community.
+  fn from_apub(group: &GroupExt, conn: &PgConnection) -> Result<Self, Error> {
+    let group_extensions: &GroupExtension = &group.base.base.extension;
+    let oprops = &group.base.base.base.object_props;
+    let aprops = &group.base.extension;
+    let public_key: &PublicKey = &group.extension.public_key;
+
+    let _followers_uri = Url::parse(&aprops.get_followers().unwrap().to_string())?;
+    let _outbox_uri = Url::parse(&aprops.get_outbox().to_string())?;
+    // TODO don't do extra fetching here
+    // let _outbox = fetch_remote_object::<OrderedCollection>(&outbox_uri)?;
+    // let _followers = fetch_remote_object::<UnorderedCollection>(&followers_uri)?;
+    let mut creator_and_moderator_uris = oprops.get_many_attributed_to_xsd_any_uris().unwrap();
+    let creator = creator_and_moderator_uris
+      .next()
+      .map(|c| get_or_fetch_and_upsert_remote_user(&c.to_string(), &conn).unwrap())
+      .unwrap();
+
+    Ok(CommunityForm {
+      name: oprops.get_name_xsd_string().unwrap().to_string(),
+      title: aprops.get_preferred_username().unwrap().to_string(),
+      // TODO: should be parsed as html and tags like <script> removed (or use markdown source)
+      //       -> same for post.content etc
+      description: oprops.get_content_xsd_string().map(|s| s.to_string()),
+      category_id: group_extensions.category.identifier.parse::<i32>()?,
+      creator_id: creator.id,
+      removed: None,
+      published: oprops
+        .get_published()
+        .map(|u| u.as_ref().to_owned().naive_local()),
+      updated: oprops
+        .get_updated()
+        .map(|u| u.as_ref().to_owned().naive_local()),
+      deleted: None,
+      nsfw: group_extensions.sensitive,
+      actor_id: oprops.get_id().unwrap().to_string(),
+      local: false,
+      private_key: None,
+      public_key: Some(public_key.to_owned().public_key_pem),
+      last_refreshed_at: Some(naive_now()),
+    })
+  }
+}
+
+/// Return the community json over HTTP.
+pub async fn get_apub_community_http(
+  info: Path<CommunityQuery>,
+  db: DbPoolParam,
+) -> Result<HttpResponse<Body>, Error> {
+  let community = Community::read_from_name(&&db.get()?, &info.community_name)?;
+  if !community.deleted {
+    Ok(create_apub_response(
+      &community.to_apub(&db.get().unwrap())?,
+    ))
+  } else {
+    Ok(create_apub_tombstone_response(&community.to_tombstone()?))
+  }
+}
+
+/// Returns an empty followers collection, only populating the siz (for privacy).
+// TODO this needs to return the actual followers, and the to: field needs this
+pub async fn get_apub_community_followers(
+  info: Path<CommunityQuery>,
+  db: DbPoolParam,
+) -> Result<HttpResponse<Body>, Error> {
+  let community = Community::read_from_name(&&db.get()?, &info.community_name)?;
+
+  let conn = db.get()?;
+
+  //As we are an object, we validated that the community id was valid
+  let community_followers = CommunityFollowerView::for_community(&conn, community.id).unwrap();
+
+  let mut collection = UnorderedCollection::default();
+  let oprops: &mut ObjectProperties = collection.as_mut();
+  oprops
+    .set_context_xsd_any_uri(context())?
+    .set_id(community.actor_id)?;
+  collection
+    .collection_props
+    .set_total_items(community_followers.len() as u64)?;
+  Ok(create_apub_response(&collection))
+}
+
+// TODO should not be doing this
+// Returns an UnorderedCollection with the latest posts from the community.
+//pub async fn get_apub_community_outbox(
+//  info: Path<CommunityQuery>,
+//  db: DbPoolParam,
+//  chat_server: ChatServerParam,
+//) -> Result<HttpResponse<Body>, Error> {
+//  let community = Community::read_from_name(&&db.get()?, &info.community_name)?;
+
+//  let conn = establish_unpooled_connection();
+//  //As we are an object, we validated that the community id was valid
+//  let community_posts: Vec<Post> = Post::list_for_community(&conn, community.id)?;
+
+//  let mut collection = OrderedCollection::default();
+//  let oprops: &mut ObjectProperties = collection.as_mut();
+//  oprops
+//    .set_context_xsd_any_uri(context())?
+//    .set_id(community.actor_id)?;
+//  collection
+//    .collection_props
+//    .set_many_items_base_boxes(
+//      community_posts
+//        .iter()
+//        .map(|c| c.as_page(&conn).unwrap())
+//        .collect(),
+//    )?
+//    .set_total_items(community_posts.len() as u64)?;
+
+//  Ok(create_apub_response(&collection))
+//}

server/src/apub/community_inbox.rs

@@ -0,0 +1,135 @@
+use super::*;
+
+#[serde(untagged)]
+#[derive(Deserialize, Debug)]
+pub enum CommunityAcceptedObjects {
+  Follow(Follow),
+  Undo(Undo),
+}
+
+// TODO Consolidate community and user inboxes into a single shared one
+/// Handler for all incoming activities to community inboxes.
+pub async fn community_inbox(
+  request: HttpRequest,
+  input: web::Json<CommunityAcceptedObjects>,
+  path: web::Path<String>,
+  db: DbPoolParam,
+  chat_server: ChatServerParam,
+) -> Result<HttpResponse, Error> {
+  let input = input.into_inner();
+  let community_name = path.into_inner();
+  debug!(
+    "Community {} received activity {:?}",
+    &community_name, &input
+  );
+  match input {
+    CommunityAcceptedObjects::Follow(f) => {
+      handle_follow(&f, &request, &community_name, db, chat_server)
+    }
+    CommunityAcceptedObjects::Undo(u) => {
+      handle_undo_follow(&u, &request, &community_name, db, chat_server)
+    }
+  }
+}
+
+/// Handle a follow request from a remote user, adding it to the local database and returning an
+/// Accept activity.
+fn handle_follow(
+  follow: &Follow,
+  request: &HttpRequest,
+  community_name: &str,
+  db: DbPoolParam,
+  _chat_server: ChatServerParam,
+) -> Result<HttpResponse, Error> {
+  let user_uri = follow
+    .follow_props
+    .get_actor_xsd_any_uri()
+    .unwrap()
+    .to_string();
+  let _community_uri = follow
+    .follow_props
+    .get_object_xsd_any_uri()
+    .unwrap()
+    .to_string();
+
+  let conn = db.get()?;
+
+  let user = get_or_fetch_and_upsert_remote_user(&user_uri, &conn)?;
+  let community = Community::read_from_name(&conn, &community_name)?;
+
+  verify(&request, &user.public_key.unwrap())?;
+
+  // Insert the received activity into the activity table
+  let activity_form = activity::ActivityForm {
+    user_id: user.id,
+    data: serde_json::to_value(&follow)?,
+    local: false,
+    updated: None,
+  };
+  activity::Activity::create(&conn, &activity_form)?;
+
+  let community_follower_form = CommunityFollowerForm {
+    community_id: community.id,
+    user_id: user.id,
+  };
+
+  // This will fail if they're already a follower, but ignore the error.
+  CommunityFollower::follow(&conn, &community_follower_form).ok();
+
+  community.send_accept_follow(&follow, &conn)?;
+
+  Ok(HttpResponse::Ok().finish())
+}
+
+fn handle_undo_follow(
+  undo: &Undo,
+  request: &HttpRequest,
+  community_name: &str,
+  db: DbPoolParam,
+  _chat_server: ChatServerParam,
+) -> Result<HttpResponse, Error> {
+  let follow = undo
+    .undo_props
+    .get_object_base_box()
+    .to_owned()
+    .unwrap()
+    .to_owned()
+    .into_concrete::<Follow>()?;
+
+  let user_uri = follow
+    .follow_props
+    .get_actor_xsd_any_uri()
+    .unwrap()
+    .to_string();
+
+  let _community_uri = follow
+    .follow_props
+    .get_object_xsd_any_uri()
+    .unwrap()
+    .to_string();
+
+  let conn = db.get()?;
+
+  let user = get_or_fetch_and_upsert_remote_user(&user_uri, &conn)?;
+  let community = Community::read_from_name(&conn, &community_name)?;
+
+  verify(&request, &user.public_key.unwrap())?;
+
+  // Insert the received activity into the activity table
+  let activity_form = activity::ActivityForm {
+    user_id: user.id,
+    data: serde_json::to_value(&follow)?,
+    local: false,
+    updated: None,
+  };
+  activity::Activity::create(&conn, &activity_form)?;
+
+  let community_follower_form = CommunityFollowerForm {
+    community_id: community.id,
+    user_id: user.id,
+  };
+
+  CommunityFollower::unfollow(&conn, &community_follower_form).ok();
+
+  Ok(HttpResponse::Ok().finish())
+}

server/src/apub/extensions/group_extensions.rs

@@ -0,0 +1,42 @@
+use crate::db::category::Category;
+use crate::db::Crud;
+use activitystreams::ext::Extension;
+use activitystreams::Actor;
+use diesel::PgConnection;
+use failure::Error;
+use serde::{Deserialize, Serialize};
+
+#[derive(Clone, Debug, Default, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct GroupExtension {
+  pub category: GroupCategory,
+  pub sensitive: bool,
+}
+
+#[derive(Clone, Debug, Default, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct GroupCategory {
+  // Using a string because that's how Peertube does it.
+  pub identifier: String,
+  pub name: String,
+}
+
+impl GroupExtension {
+  pub fn new(
+    conn: &PgConnection,
+    category_id: i32,
+    sensitive: bool,
+  ) -> Result<GroupExtension, Error> {
+    let category = Category::read(conn, category_id)?;
+    let group_category = GroupCategory {
+      identifier: category_id.to_string(),
+      name: category.name,
+    };
+    Ok(GroupExtension {
+      category: group_category,
+      sensitive,
+    })
+  }
+}
+
+impl<T> Extension<T> for GroupExtension where T: Actor {}

server/src/apub/extensions/mod.rs

@@ -0,0 +1,3 @@
+pub mod group_extensions;
+pub mod page_extension;
+pub mod signatures;

server/src/apub/extensions/page_extension.rs

@@ -0,0 +1,12 @@
+use activitystreams::ext::Extension;
+use activitystreams::Base;
+use serde::{Deserialize, Serialize};
+
+#[derive(Clone, Debug, Default, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct PageExtension {
+  pub comments_enabled: bool,
+  pub sensitive: bool,
+}
+
+impl<T> Extension<T> for PageExtension where T: Base {}

server/src/apub/extensions/signatures.rs

@@ -0,0 +1,133 @@
+use activitystreams::ext::Extension;
+use activitystreams::Actor;
+use actix_web::HttpRequest;
+use failure::Error;
+use http::request::Builder;
+use http_signature_normalization::Config;
+use log::debug;
+use openssl::hash::MessageDigest;
+use openssl::sign::{Signer, Verifier};
+use openssl::{pkey::PKey, rsa::Rsa};
+use serde::{Deserialize, Serialize};
+use std::collections::BTreeMap;
+
+lazy_static! {
+  static ref HTTP_SIG_CONFIG: Config = Config::new();
+}
+
+pub struct Keypair {
+  pub private_key: String,
+  pub public_key: String,
+}
+
+/// Generate the asymmetric keypair for ActivityPub HTTP signatures.
+pub fn generate_actor_keypair() -> Result<Keypair, Error> {
+  let rsa = Rsa::generate(2048)?;
+  let pkey = PKey::from_rsa(rsa)?;
+  let public_key = pkey.public_key_to_pem()?;
+  let private_key = pkey.private_key_to_pem_pkcs8()?;
+  Ok(Keypair {
+    private_key: String::from_utf8(private_key)?,
+    public_key: String::from_utf8(public_key)?,
+  })
+}
+
+/// Signs request headers with the given keypair.
+/// TODO: would be nice to pass the sending actor in, instead of raw privatekey/id strings
+pub fn sign(request: &Builder, private_key: &str, sender_id: &str) -> Result<String, Error> {
+  let signing_key_id = format!("{}#main-key", sender_id);
+
+  let headers = request
+    .headers_ref()
+    .unwrap()
+    .iter()
+    .map(|h| -> Result<(String, String), Error> {
+      Ok((h.0.as_str().to_owned(), h.1.to_str()?.to_owned()))
+    })
+    .collect::<Result<BTreeMap<String, String>, Error>>()?;
+
+  let signature_header_value = HTTP_SIG_CONFIG
+    .begin_sign(
+      request.method_ref().unwrap().as_str(),
+      request
+        .uri_ref()
+        .unwrap()
+        .path_and_query()
+        .unwrap()
+        .as_str(),
+      headers,
+    )
+    .sign(signing_key_id, |signing_string| {
+      let private_key = PKey::private_key_from_pem(private_key.as_bytes())?;
+      let mut signer = Signer::new(MessageDigest::sha256(), &private_key).unwrap();
+      signer.update(signing_string.as_bytes()).unwrap();
+      Ok(base64::encode(signer.sign_to_vec()?)) as Result<_, Error>
+    })?
+    .signature_header();
+
+  Ok(signature_header_value)
+}
+
+pub fn verify(request: &HttpRequest, public_key: &str) -> Result<(), Error> {
+  let headers = request
+    .headers()
+    .iter()
+    .map(|h| -> Result<(String, String), Error> {
+      Ok((h.0.as_str().to_owned(), h.1.to_str()?.to_owned()))
+    })
+    .collect::<Result<BTreeMap<String, String>, Error>>()?;
+
+  let verified = HTTP_SIG_CONFIG
+    .begin_verify(
+      request.method().as_str(),
+      request.uri().path_and_query().unwrap().as_str(),
+      headers,
+    )?
+    .verify(|signature, signing_string| -> Result<bool, Error> {
+      debug!(
+        "Verifying with key {}, message {}",
+        &public_key, &signing_string
+      );
+      let public_key = PKey::public_key_from_pem(public_key.as_bytes())?;
+      let mut verifier = Verifier::new(MessageDigest::sha256(), &public_key).unwrap();
+      verifier.update(&signing_string.as_bytes()).unwrap();
+      Ok(verifier.verify(&base64::decode(signature)?)?)
+    })?;
+
+  if verified {
+    debug!("verified signature for {}", &request.uri());
+    Ok(())
+  } else {
+    Err(format_err!(
+      "Invalid signature on request: {}",
+      &request.uri()
+    ))
+  }
+}
+
+// The following is taken from here:
+// https://docs.rs/activitystreams/0.5.0-alpha.17/activitystreams/ext/index.html
+
+#[derive(Clone, Debug, Default, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct PublicKey {
+  pub id: String,
+  pub owner: String,
+  pub public_key_pem: String,
+}
+
+#[derive(Clone, Debug, Default, Deserialize, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub struct PublicKeyExtension {
+  pub public_key: PublicKey,
+}
+
+impl PublicKey {
+  pub fn to_ext(&self) -> PublicKeyExtension {
+    PublicKeyExtension {
+      public_key: self.to_owned(),
+    }
+  }
+}
+
+impl<T> Extension<T> for PublicKeyExtension where T: Actor {}

server/src/apub/fetcher.rs

@@ -0,0 +1,247 @@
+use super::*;
+
+// Fetch nodeinfo metadata from a remote instance.
+fn _fetch_node_info(domain: &str) -> Result<NodeInfo, Error> {
+  let well_known_uri = Url::parse(&format!(
+    "{}://{}/.well-known/nodeinfo",
+    get_apub_protocol_string(),
+    domain
+  ))?;
+  let well_known = fetch_remote_object::<NodeInfoWellKnown>(&well_known_uri)?;
+  Ok(fetch_remote_object::<NodeInfo>(&well_known.links.href)?)
+}
+
+/// Fetch any type of ActivityPub object, handling things like HTTP headers, deserialisation,
+/// timeouts etc.
+pub fn fetch_remote_object<Response>(url: &Url) -> Result<Response, Error>
+where
+  Response: for<'de> Deserialize<'de>,
+{
+  if !is_apub_id_valid(&url) {
+    return Err(format_err!("Activitypub uri invalid or blocked: {}", url));
+  }
+  // TODO: this function should return a future
+  let timeout = Duration::from_secs(60);
+  let text = Request::get(url.as_str())
+    .header("Accept", APUB_JSON_CONTENT_TYPE)
+    .connect_timeout(timeout)
+    .timeout(timeout)
+    .body(())?
+    .send()?
+    .text()?;
+  let res: Response = serde_json::from_str(&text)?;
+  Ok(res)
+}
+
+/// The types of ActivityPub objects that can be fetched directly by searching for their ID.
+#[serde(untagged)]
+#[derive(serde::Deserialize, Debug)]
+pub enum SearchAcceptedObjects {
+  Person(Box<PersonExt>),
+  Group(Box<GroupExt>),
+  Page(Box<PageExt>),
+  Comment(Box<Note>),
+}
+
+/// Attempt to parse the query as URL, and fetch an ActivityPub object from it.
+///
+/// Some working examples for use with the docker/federation/ setup:
+/// http://lemmy_alpha:8540/c/main, or !main@lemmy_alpha:8540
+/// http://lemmy_alpha:8540/u/lemmy_alpha, or @lemmy_alpha@lemmy_alpha:8540
+/// http://lemmy_alpha:8540/post/3
+/// http://lemmy_alpha:8540/comment/2
+pub fn search_by_apub_id(query: &str, conn: &PgConnection) -> Result<SearchResponse, Error> {
+  // Parse the shorthand query url
+  let query_url = if query.contains('@') {
+    debug!("{}", query);
+    let split = query.split('@').collect::<Vec<&str>>();
+
+    // User type will look like ['', username, instance]
+    // Community will look like [!community, instance]
+    let (name, instance) = if split.len() == 3 {
+      (format!("/u/{}", split[1]), split[2])
+    } else if split.len() == 2 {
+      if split[0].contains('!') {
+        let split2 = split[0].split('!').collect::<Vec<&str>>();
+        (format!("/c/{}", split2[1]), split[1])
+      } else {
+        return Err(format_err!("Invalid search query: {}", query));
+      }
+    } else {
+      return Err(format_err!("Invalid search query: {}", query));
+    };
+
+    let url = format!("{}://{}{}", get_apub_protocol_string(), instance, name);
+    Url::parse(&url)?
+  } else {
+    Url::parse(&query)?
+  };
+
+  let mut response = SearchResponse {
+    type_: SearchType::All.to_string(),
+    comments: vec![],
+    posts: vec![],
+    communities: vec![],
+    users: vec![],
+  };
+  match fetch_remote_object::<SearchAcceptedObjects>(&query_url)? {
+    SearchAcceptedObjects::Person(p) => {
+      let user_uri = p.base.base.object_props.get_id().unwrap().to_string();
+      let user = get_or_fetch_and_upsert_remote_user(&user_uri, &conn)?;
+      response.users = vec![UserView::read(conn, user.id)?];
+    }
+    SearchAcceptedObjects::Group(g) => {
+      let community_uri = g.base.base.base.object_props.get_id().unwrap().to_string();
+      let community = get_or_fetch_and_upsert_remote_community(&community_uri, &conn)?;
+      // TODO Maybe at some point in the future, fetch all the history of a community
+      // fetch_community_outbox(&c, conn)?;
+      response.communities = vec![CommunityView::read(conn, community.id, None)?];
+    }
+    SearchAcceptedObjects::Page(p) => {
+      let p = upsert_post(&PostForm::from_apub(&p, conn)?, conn)?;
+      response.posts = vec![PostView::read(conn, p.id, None)?];
+    }
+    SearchAcceptedObjects::Comment(c) => {
+      let post_url = c
+        .object_props
+        .get_many_in_reply_to_xsd_any_uris()
+        .unwrap()
+        .next()
+        .unwrap()
+        .to_string();
+      // TODO: also fetch parent comments if any
+      let post = fetch_remote_object(&Url::parse(&post_url)?)?;
+      upsert_post(&PostForm::from_apub(&post, conn)?, conn)?;
+      let c = upsert_comment(&CommentForm::from_apub(&c, conn)?, conn)?;
+      response.comments = vec![CommentView::read(conn, c.id, None)?];
+    }
+  }
+  Ok(response)
+}
+
+/// Check if a remote user exists, create if not found, if its too old update it.Fetch a user, insert/update it in the database and return the user.
+pub fn get_or_fetch_and_upsert_remote_user(
+  apub_id: &str,
+  conn: &PgConnection,
+) -> Result<User_, Error> {
+  match User_::read_from_actor_id(&conn, &apub_id) {
+    Ok(u) => {
+      // If its older than a day, re-fetch it
+      if !u.local
+        && u
+          .last_refreshed_at
+          .lt(&(naive_now() - chrono::Duration::days(1)))
+      {
+        debug!("Fetching and updating from remote user: {}", apub_id);
+        let person = fetch_remote_object::<PersonExt>(&Url::parse(apub_id)?)?;
+        let mut uf = UserForm::from_apub(&person, &conn)?;
+        uf.last_refreshed_at = Some(naive_now());
+        Ok(User_::update(&conn, u.id, &uf)?)
+      } else {
+        Ok(u)
+      }
+    }
+    Err(NotFound {}) => {
+      debug!("Fetching and creating remote user: {}", apub_id);
+      let person = fetch_remote_object::<PersonExt>(&Url::parse(apub_id)?)?;
+      let uf = UserForm::from_apub(&person, &conn)?;
+      Ok(User_::create(conn, &uf)?)
+    }
+    Err(e) => Err(Error::from(e)),
+  }
+}
+
+/// Check if a remote community exists, create if not found, if its too old update it.Fetch a community, insert/update it in the database and return the community.
+pub fn get_or_fetch_and_upsert_remote_community(
+  apub_id: &str,
+  conn: &PgConnection,
+) -> Result<Community, Error> {
+  match Community::read_from_actor_id(&conn, &apub_id) {
+    Ok(c) => {
+      // If its older than a day, re-fetch it
+      if !c.local
+        && c
+          .last_refreshed_at
+          .lt(&(naive_now() - chrono::Duration::days(1)))
+      {
+        debug!("Fetching and updating from remote community: {}", apub_id);
+        let group = fetch_remote_object::<GroupExt>(&Url::parse(apub_id)?)?;
+        let mut cf = CommunityForm::from_apub(&group, conn)?;
+        cf.last_refreshed_at = Some(naive_now());
+        Ok(Community::update(&conn, c.id, &cf)?)
+      } else {
+        Ok(c)
+      }
+    }
+    Err(NotFound {}) => {
+      debug!("Fetching and creating remote community: {}", apub_id);
+      let group = fetch_remote_object::<GroupExt>(&Url::parse(apub_id)?)?;
+      let cf = CommunityForm::from_apub(&group, conn)?;
+      let community = Community::create(conn, &cf)?;
+
+      // Also add the community moderators too
+      let creator_and_moderator_uris = group
+        .base
+        .base
+        .base
+        .object_props
+        .get_many_attributed_to_xsd_any_uris()
+        .unwrap();
+      let creator_and_moderators = creator_and_moderator_uris
+        .map(|c| get_or_fetch_and_upsert_remote_user(&c.to_string(), &conn).unwrap())
+        .collect::<Vec<User_>>();
+
+      for mod_ in creator_and_moderators {
+        let community_moderator_form = CommunityModeratorForm {
+          community_id: community.id,
+          user_id: mod_.id,
+        };
+        CommunityModerator::join(&conn, &community_moderator_form)?;
+      }
+
+      Ok(community)
+    }
+    Err(e) => Err(Error::from(e)),
+  }
+}
+
+fn upsert_post(post_form: &PostForm, conn: &PgConnection) -> Result<Post, Error> {
+  let existing = Post::read_from_apub_id(conn, &post_form.ap_id);
+  match existing {
+    Err(NotFound {}) => Ok(Post::create(conn, &post_form)?),
+    Ok(p) => Ok(Post::update(conn, p.id, &post_form)?),
+    Err(e) => Err(Error::from(e)),
+  }
+}
+
+fn upsert_comment(comment_form: &CommentForm, conn: &PgConnection) -> Result<Comment, Error> {
+  let existing = Comment::read_from_apub_id(conn, &comment_form.ap_id);
+  match existing {
+    Err(NotFound {}) => Ok(Comment::create(conn, &comment_form)?),
+    Ok(p) => Ok(Comment::update(conn, p.id, &comment_form)?),
+    Err(e) => Err(Error::from(e)),
+  }
+}
+
+// TODO It should not be fetching data from a community outbox.
+// All posts, comments, comment likes, etc should be posts to our community_inbox
+// The only data we should be periodically fetching (if it hasn't been fetched in the last day
+// maybe), is community and user actors
+// and user actors
+// Fetch all posts in the outbox of the given user, and insert them into the database.
+// fn fetch_community_outbox(community: &Community, conn: &PgConnection) -> Result<Vec<Post>, Error> {
+//   let outbox_url = Url::parse(&community.get_outbox_url())?;
+//   let outbox = fetch_remote_object::<OrderedCollection>(&outbox_url)?;
+//   let items = outbox.collection_props.get_many_items_base_boxes();
+
+//   Ok(
+//     items
+//       .unwrap()
+//       .map(|obox: &BaseBox| -> Result<PostForm, Error> {
+//         let page = obox.clone().to_concrete::<Page>()?;
+//         PostForm::from_page(&page, conn)
+//       })
+//       .map(|pf| upsert_post(&pf?, conn))
+//       .collect::<Result<Vec<Post>, Error>>()?,
+//   )
+// }

server/src/apub/mod.rs

@@ -1,107 +1,287 @@
+pub mod activities;
+pub mod comment;
 pub mod community;
+pub mod community_inbox;
+pub mod extensions;
+pub mod fetcher;
 pub mod post;
+pub mod private_message;
+pub mod shared_inbox;
 pub mod user;
-use crate::Settings;
+pub mod user_inbox;
 
-use std::fmt::Display;
+use crate::api::community::CommunityResponse;
+use crate::websocket::server::SendCommunityRoomMessage;
+use activitystreams::object::kind::{NoteType, PageType};
+use activitystreams::{
+  activity::{Accept, Create, Delete, Dislike, Follow, Like, Remove, Undo, Update},
+  actor::{kind::GroupType, properties::ApActorProperties, Group, Person},
+  collection::UnorderedCollection,
+  context,
+  endpoint::EndpointProperties,
+  ext::{Ext, Extensible},
+  object::{properties::ObjectProperties, Note, Page, Tombstone},
+  public, BaseBox,
+};
+use actix_web::body::Body;
+use actix_web::web::Path;
+use actix_web::{web, HttpRequest, HttpResponse, Result};
+use diesel::result::Error::NotFound;
+use diesel::PgConnection;
+use failure::Error;
+use failure::_core::fmt::Debug;
+use isahc::prelude::*;
+use itertools::Itertools;
+use log::debug;
+use serde::{Deserialize, Serialize};
+use std::time::Duration;
+use url::Url;
 
-#[cfg(test)]
-mod tests {
-  use crate::db::community::Community;
-  use crate::db::post::Post;
-  use crate::db::user::User_;
-  use crate::db::{ListingType, SortType};
-  use crate::{naive_now, Settings};
+use crate::api::comment::CommentResponse;
+use crate::api::post::PostResponse;
+use crate::api::site::SearchResponse;
+use crate::api::user::PrivateMessageResponse;
+use crate::db::comment::{Comment, CommentForm, CommentLike, CommentLikeForm};
+use crate::db::comment_view::CommentView;
+use crate::db::community::{
+  Community, CommunityFollower, CommunityFollowerForm, CommunityForm, CommunityModerator,
+  CommunityModeratorForm,
+};
+use crate::db::community_view::{CommunityFollowerView, CommunityModeratorView, CommunityView};
+use crate::db::post::{Post, PostForm, PostLike, PostLikeForm};
+use crate::db::post_view::PostView;
+use crate::db::private_message::{PrivateMessage, PrivateMessageForm};
+use crate::db::private_message_view::PrivateMessageView;
+use crate::db::user::{UserForm, User_};
+use crate::db::user_view::UserView;
+use crate::db::{activity, Crud, Followable, Joinable, Likeable, SearchType};
+use crate::routes::nodeinfo::{NodeInfo, NodeInfoWellKnown};
+use crate::routes::{ChatServerParam, DbPoolParam};
+use crate::websocket::{
+  server::{SendComment, SendPost, SendUserRoomMessage},
+  UserOperation,
+};
+use crate::{convert_datetime, naive_now, Settings};
 
-  #[test]
-  fn test_person() {
-    let user = User_ {
-      id: 52,
-      name: "thom".into(),
-      fedi_name: "rrf".into(),
-      preferred_username: None,
-      password_encrypted: "here".into(),
-      email: None,
-      matrix_user_id: None,
-      avatar: None,
-      published: naive_now(),
-      admin: false,
-      banned: false,
-      updated: None,
-      show_nsfw: false,
-      theme: "darkly".into(),
-      default_sort_type: SortType::Hot as i16,
-      default_listing_type: ListingType::Subscribed as i16,
-      lang: "browser".into(),
-      show_avatars: true,
-      send_notifications_to_email: false,
-    };
+use crate::apub::extensions::group_extensions::GroupExtension;
+use crate::apub::extensions::page_extension::PageExtension;
+use activities::{populate_object_props, send_activity};
+use chrono::NaiveDateTime;
+use extensions::signatures::verify;
+use extensions::signatures::{sign, PublicKey, PublicKeyExtension};
+use fetcher::{get_or_fetch_and_upsert_remote_community, get_or_fetch_and_upsert_remote_user};
 
-    let person = user.as_person();
-    assert_eq!(
-      format!("https://{}/federation/u/thom", Settings::get().hostname),
-      person.object_props.id_string().unwrap()
-    );
-  }
+type GroupExt = Ext<Ext<Ext<Group, GroupExtension>, ApActorProperties>, PublicKeyExtension>;
+type PersonExt = Ext<Ext<Person, ApActorProperties>, PublicKeyExtension>;
+type PageExt = Ext<Page, PageExtension>;
 
-  #[test]
-  fn test_community() {
-    let community = Community {
-      id: 42,
-      name: "Test".into(),
-      title: "Test Title".into(),
-      description: Some("Test community".into()),
-      category_id: 32,
-      creator_id: 52,
-      removed: false,
-      published: naive_now(),
-      updated: Some(naive_now()),
-      deleted: false,
-      nsfw: false,
-    };
+pub static APUB_JSON_CONTENT_TYPE: &str = "application/activity+json";
 
-    let group = community.as_group();
-    assert_eq!(
-      format!("https://{}/federation/c/Test", Settings::get().hostname),
-      group.object_props.id_string().unwrap()
-    );
-  }
-
-  #[test]
-  fn test_post() {
-    let post = Post {
-      id: 62,
-      name: "A test post".into(),
-      url: None,
-      body: None,
-      creator_id: 52,
-      community_id: 42,
-      published: naive_now(),
-      removed: false,
-      locked: false,
-      stickied: false,
-      nsfw: false,
-      deleted: false,
-      updated: None,
-      embed_title: None,
-      embed_description: None,
-      embed_html: None,
-      thumbnail_url: None,
-    };
-
-    let page = post.as_page();
-    assert_eq!(
-      format!("https://{}/federation/post/62", Settings::get().hostname),
-      page.object_props.id_string().unwrap()
-    );
-  }
+pub enum EndpointType {
+  Community,
+  User,
+  Post,
+  Comment,
+  PrivateMessage,
 }
 
-pub fn make_apub_endpoint<S: Display, T: Display>(point: S, value: T) -> String {
-  format!(
-    "https://{}/federation/{}/{}",
+/// Convert the data to json and turn it into an HTTP Response with the correct ActivityPub
+/// headers.
+fn create_apub_response<T>(data: &T) -> HttpResponse<Body>
+where
+  T: Serialize,
+{
+  HttpResponse::Ok()
+    .content_type(APUB_JSON_CONTENT_TYPE)
+    .json(data)
+}
+
+fn create_apub_tombstone_response<T>(data: &T) -> HttpResponse<Body>
+where
+  T: Serialize,
+{
+  HttpResponse::Gone()
+    .content_type(APUB_JSON_CONTENT_TYPE)
+    .json(data)
+}
+
+/// Generates the ActivityPub ID for a given object type and name.
+///
+/// TODO: we will probably need to change apub endpoint urls so that html and activity+json content
+///       types are handled at the same endpoint, so that you can copy the url into mastodon search
+///       and have it fetch the object.
+pub fn make_apub_endpoint(endpoint_type: EndpointType, name: &str) -> Url {
+  let point = match endpoint_type {
+    EndpointType::Community => "c",
+    EndpointType::User => "u",
+    EndpointType::Post => "post",
+    // TODO I have to change this else my update advanced_migrations crashes the
+    // server if a comment exists.
+    EndpointType::Comment => "comment",
+    EndpointType::PrivateMessage => "private_message",
+  };
+
+  Url::parse(&format!(
+    "{}://{}/{}/{}",
+    get_apub_protocol_string(),
     Settings::get().hostname,
     point,
-    value
+    name
+  ))
+  .unwrap()
+}
+
+pub fn get_apub_protocol_string() -> &'static str {
+  if Settings::get().federation.tls_enabled {
+    "https"
+  } else {
+    "http"
+  }
+}
+
+// Checks if the ID has a valid format, correct scheme, and is in the whitelist.
+fn is_apub_id_valid(apub_id: &Url) -> bool {
+  if apub_id.scheme() != get_apub_protocol_string() {
+    return false;
+  }
+
+  let whitelist: Vec<String> = Settings::get()
+    .federation
+    .instance_whitelist
+    .split(',')
+    .map(|d| d.to_string())
+    .collect();
+  match apub_id.domain() {
+    Some(d) => whitelist.contains(&d.to_owned()),
+    None => false,
+  }
+}
+
+// TODO Not sure good names for these
+pub trait ToApub {
+  type Response;
+  fn to_apub(&self, conn: &PgConnection) -> Result<Self::Response, Error>;
+  fn to_tombstone(&self) -> Result<Tombstone, Error>;
+}
+
+/// Updated is actually the deletion time
+fn create_tombstone(
+  deleted: bool,
+  object_id: &str,
+  updated: Option<NaiveDateTime>,
+  former_type: String,
+) -> Result<Tombstone, Error> {
+  if deleted {
+    if let Some(updated) = updated {
+      let mut tombstone = Tombstone::default();
+      tombstone.object_props.set_id(object_id)?;
+      tombstone
+        .tombstone_props
+        .set_former_type_xsd_string(former_type)?
+        .set_deleted(convert_datetime(updated))?;
+      Ok(tombstone)
+    } else {
+      Err(format_err!(
+        "Cant convert to tombstone because updated time was None."
+      ))
+    }
+  } else {
+    Err(format_err!(
+      "Cant convert object to tombstone if it wasnt deleted"
+    ))
+  }
+}
+
+pub trait FromApub {
+  type ApubType;
+  fn from_apub(apub: &Self::ApubType, conn: &PgConnection) -> Result<Self, Error>
+  where
+    Self: Sized;
+}
+
+pub trait ApubObjectType {
+  fn send_create(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error>;
+  fn send_update(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error>;
+  fn send_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error>;
+  fn send_undo_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error>;
+  fn send_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error>;
+  fn send_undo_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error>;
+}
+
+pub trait ApubLikeableType {
+  fn send_like(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error>;
+  fn send_dislike(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error>;
+  fn send_undo_like(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error>;
+}
+
+pub fn get_shared_inbox(actor_id: &str) -> String {
+  let url = Url::parse(actor_id).unwrap();
+  format!(
+    "{}://{}{}/inbox",
+    &url.scheme(),
+    &url.host_str().unwrap(),
+    if let Some(port) = url.port() {
+      format!(":{}", port)
+    } else {
+      "".to_string()
+    },
   )
 }
+
+pub trait ActorType {
+  fn actor_id(&self) -> String;
+
+  fn public_key(&self) -> String;
+
+  // These two have default impls, since currently a community can't follow anything,
+  // and a user can't be followed (yet)
+  #[allow(unused_variables)]
+  fn send_follow(&self, follow_actor_id: &str, conn: &PgConnection) -> Result<(), Error>;
+  fn send_unfollow(&self, follow_actor_id: &str, conn: &PgConnection) -> Result<(), Error>;
+
+  #[allow(unused_variables)]
+  fn send_accept_follow(&self, follow: &Follow, conn: &PgConnection) -> Result<(), Error>;
+
+  fn send_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error>;
+  fn send_undo_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error>;
+
+  fn send_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error>;
+  fn send_undo_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error>;
+
+  /// For a given community, returns the inboxes of all followers.
+  fn get_follower_inboxes(&self, conn: &PgConnection) -> Result<Vec<String>, Error>;
+
+  // TODO move these to the db rows
+  fn get_inbox_url(&self) -> String {
+    format!("{}/inbox", &self.actor_id())
+  }
+
+  fn get_shared_inbox_url(&self) -> String {
+    get_shared_inbox(&self.actor_id())
+  }
+
+  fn get_outbox_url(&self) -> String {
+    format!("{}/outbox", &self.actor_id())
+  }
+
+  fn get_followers_url(&self) -> String {
+    format!("{}/followers", &self.actor_id())
+  }
+
+  fn get_following_url(&self) -> String {
+    format!("{}/following", &self.actor_id())
+  }
+
+  fn get_liked_url(&self) -> String {
+    format!("{}/liked", &self.actor_id())
+  }
+
+  fn get_public_key_ext(&self) -> PublicKeyExtension {
+    PublicKey {
+      id: format!("{}#main-key", self.actor_id()),
+      owner: self.actor_id(),
+      public_key_pem: self.public_key(),
+    }
+    .to_ext()
+  }
+}

server/src/apub/post.rs

@@ -1,38 +1,473 @@
-use crate::apub::make_apub_endpoint;
-use crate::db::post::Post;
-use crate::to_datetime_utc;
-use activitypub::{context, object::Page};
+use super::*;
 
-impl Post {
-  pub fn as_page(&self) -> Page {
-    let base_url = make_apub_endpoint("post", self.id);
-    let mut page = Page::default();
+#[derive(Deserialize)]
+pub struct PostQuery {
+  post_id: String,
+}
 
-    page.object_props.set_context_object(context()).ok();
-    page.object_props.set_id_string(base_url).ok();
-    page.object_props.set_name_string(self.name.to_owned()).ok();
-
-    if let Some(body) = &self.body {
-      page.object_props.set_content_string(body.to_owned()).ok();
-    }
-
-    if let Some(url) = &self.url {
-      page.object_props.set_url_string(url.to_owned()).ok();
-    }
-
-    //page.object_props.set_attributed_to_string
-
-    page
-      .object_props
-      .set_published_utctime(to_datetime_utc(self.published))
-      .ok();
-    if let Some(updated) = self.updated {
-      page
-        .object_props
-        .set_updated_utctime(to_datetime_utc(updated))
-        .ok();
-    }
-
-    page
+/// Return the post json over HTTP.
+pub async fn get_apub_post(
+  info: Path<PostQuery>,
+  db: DbPoolParam,
+) -> Result<HttpResponse<Body>, Error> {
+  let id = info.post_id.parse::<i32>()?;
+  let post = Post::read(&&db.get()?, id)?;
+  if !post.deleted {
+    Ok(create_apub_response(&post.to_apub(&db.get().unwrap())?))
+  } else {
+    Ok(create_apub_tombstone_response(&post.to_tombstone()?))
+  }
+}
+
+impl ToApub for Post {
+  type Response = PageExt;
+
+  // Turn a Lemmy post into an ActivityPub page that can be sent out over the network.
+  fn to_apub(&self, conn: &PgConnection) -> Result<PageExt, Error> {
+    let mut page = Page::default();
+    let oprops: &mut ObjectProperties = page.as_mut();
+    let creator = User_::read(conn, self.creator_id)?;
+    let community = Community::read(conn, self.community_id)?;
+
+    oprops
+      // Not needed when the Post is embedded in a collection (like for community outbox)
+      // TODO: need to set proper context defining sensitive/commentsEnabled fields
+      // https://git.asonix.dog/Aardwolf/activitystreams/issues/5
+      .set_context_xsd_any_uri(context())?
+      .set_id(self.ap_id.to_owned())?
+      // Use summary field to be consistent with mastodon content warning.
+      // https://mastodon.xyz/@Louisa/103987265222901387.json
+      .set_summary_xsd_string(self.name.to_owned())?
+      .set_published(convert_datetime(self.published))?
+      .set_to_xsd_any_uri(community.actor_id)?
+      .set_attributed_to_xsd_any_uri(creator.actor_id)?;
+
+    if let Some(body) = &self.body {
+      oprops.set_content_xsd_string(body.to_owned())?;
+    }
+
+    // TODO: hacky code because we get self.url == Some("")
+    // https://github.com/LemmyNet/lemmy/issues/602
+    let url = self.url.as_ref().filter(|u| !u.is_empty());
+    if let Some(u) = url {
+      oprops.set_url_xsd_any_uri(u.to_owned())?;
+    }
+
+    if let Some(u) = self.updated {
+      oprops.set_updated(convert_datetime(u))?;
+    }
+
+    let ext = PageExtension {
+      comments_enabled: !self.locked,
+      sensitive: self.nsfw,
+    };
+    Ok(page.extend(ext))
+  }
+
+  fn to_tombstone(&self) -> Result<Tombstone, Error> {
+    create_tombstone(
+      self.deleted,
+      &self.ap_id,
+      self.updated,
+      PageType.to_string(),
+    )
+  }
+}
+
+impl FromApub for PostForm {
+  type ApubType = PageExt;
+
+  /// Parse an ActivityPub page received from another instance into a Lemmy post.
+  fn from_apub(page: &PageExt, conn: &PgConnection) -> Result<PostForm, Error> {
+    let ext = &page.extension;
+    let page = &page.base;
+    let oprops = &page.object_props;
+    let creator_actor_id = &oprops.get_attributed_to_xsd_any_uri().unwrap().to_string();
+    let creator = get_or_fetch_and_upsert_remote_user(&creator_actor_id, &conn)?;
+    let community_actor_id = &oprops.get_to_xsd_any_uri().unwrap().to_string();
+    let community = get_or_fetch_and_upsert_remote_community(&community_actor_id, &conn)?;
+
+    Ok(PostForm {
+      name: oprops.get_summary_xsd_string().unwrap().to_string(),
+      url: oprops.get_url_xsd_any_uri().map(|u| u.to_string()),
+      body: oprops.get_content_xsd_string().map(|c| c.to_string()),
+      creator_id: creator.id,
+      community_id: community.id,
+      removed: None,
+      locked: Some(!ext.comments_enabled),
+      published: oprops
+        .get_published()
+        .map(|u| u.as_ref().to_owned().naive_local()),
+      updated: oprops
+        .get_updated()
+        .map(|u| u.as_ref().to_owned().naive_local()),
+      deleted: None,
+      nsfw: ext.sensitive,
+      stickied: None,    // -> put it in "featured" collection of the community
+      embed_title: None, // -> attachment? or fetch the embed locally
+      embed_description: None,
+      embed_html: None,
+      thumbnail_url: None,
+      ap_id: oprops.get_id().unwrap().to_string(),
+      local: false,
+    })
+  }
+}
+
+impl ApubObjectType for Post {
+  /// Send out information about a newly created post, to the followers of the community.
+  fn send_create(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let page = self.to_apub(conn)?;
+    let community = Community::read(conn, self.community_id)?;
+    let id = format!("{}/create/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut create = Create::new();
+    populate_object_props(
+      &mut create.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+    create
+      .create_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(page)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&create)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &create,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  /// Send out information about an edited post, to the followers of the community.
+  fn send_update(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let page = self.to_apub(conn)?;
+    let community = Community::read(conn, self.community_id)?;
+    let id = format!("{}/update/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut update = Update::new();
+    populate_object_props(
+      &mut update.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+    update
+      .update_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(page)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&update)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &update,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let page = self.to_apub(conn)?;
+    let community = Community::read(conn, self.community_id)?;
+    let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut delete = Delete::default();
+
+    populate_object_props(
+      &mut delete.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+
+    delete
+      .delete_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(page)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: self.creator_id,
+      data: serde_json::to_value(&delete)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    let community = Community::read(conn, self.community_id)?;
+    send_activity(
+      &delete,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_undo_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let page = self.to_apub(conn)?;
+    let community = Community::read(conn, self.community_id)?;
+    let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut delete = Delete::default();
+
+    populate_object_props(
+      &mut delete.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+
+    delete
+      .delete_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(page)?;
+
+    // TODO
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/delete/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    populate_object_props(
+      &mut undo.object_props,
+      &community.get_followers_url(),
+      &undo_id,
+    )?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(delete)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: self.creator_id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    let community = Community::read(conn, self.community_id)?;
+    send_activity(
+      &undo,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let page = self.to_apub(conn)?;
+    let community = Community::read(conn, self.community_id)?;
+    let id = format!("{}/remove/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut remove = Remove::default();
+
+    populate_object_props(
+      &mut remove.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+
+    remove
+      .remove_props
+      .set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
+      .set_object_base_box(page)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: mod_.id,
+      data: serde_json::to_value(&remove)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    let community = Community::read(conn, self.community_id)?;
+    send_activity(
+      &remove,
+      &mod_.private_key.as_ref().unwrap(),
+      &mod_.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+  fn send_undo_remove(&self, mod_: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let page = self.to_apub(conn)?;
+    let community = Community::read(conn, self.community_id)?;
+    let id = format!("{}/remove/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut remove = Remove::default();
+
+    populate_object_props(
+      &mut remove.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+
+    remove
+      .remove_props
+      .set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
+      .set_object_base_box(page)?;
+
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/remove/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    populate_object_props(
+      &mut undo.object_props,
+      &community.get_followers_url(),
+      &undo_id,
+    )?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(mod_.actor_id.to_owned())?
+      .set_object_base_box(remove)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: mod_.id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    let community = Community::read(conn, self.community_id)?;
+    send_activity(
+      &undo,
+      &mod_.private_key.as_ref().unwrap(),
+      &mod_.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+}
+
+impl ApubLikeableType for Post {
+  fn send_like(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let page = self.to_apub(conn)?;
+    let community = Community::read(conn, self.community_id)?;
+    let id = format!("{}/like/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut like = Like::new();
+    populate_object_props(&mut like.object_props, &community.get_followers_url(), &id)?;
+    like
+      .like_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(page)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&like)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &like,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_dislike(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let page = self.to_apub(conn)?;
+    let community = Community::read(conn, self.community_id)?;
+    let id = format!("{}/dislike/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut dislike = Dislike::new();
+    populate_object_props(
+      &mut dislike.object_props,
+      &community.get_followers_url(),
+      &id,
+    )?;
+    dislike
+      .dislike_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(page)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&dislike)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &dislike,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
+  }
+
+  fn send_undo_like(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let page = self.to_apub(conn)?;
+    let community = Community::read(conn, self.community_id)?;
+    let id = format!("{}/like/{}", self.ap_id, uuid::Uuid::new_v4());
+
+    let mut like = Like::new();
+    populate_object_props(&mut like.object_props, &community.get_followers_url(), &id)?;
+    like
+      .like_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(page)?;
+
+    // TODO
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/like/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    populate_object_props(
+      &mut undo.object_props,
+      &community.get_followers_url(),
+      &undo_id,
+    )?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(like)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &undo,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      community.get_follower_inboxes(&conn)?,
+    )?;
+    Ok(())
   }
 }

server/src/apub/private_message.rs

@@ -0,0 +1,234 @@
+use super::*;
+
+impl ToApub for PrivateMessage {
+  type Response = Note;
+
+  fn to_apub(&self, conn: &PgConnection) -> Result<Note, Error> {
+    let mut private_message = Note::default();
+    let oprops: &mut ObjectProperties = private_message.as_mut();
+    let creator = User_::read(&conn, self.creator_id)?;
+    let recipient = User_::read(&conn, self.recipient_id)?;
+
+    oprops
+      .set_context_xsd_any_uri(context())?
+      .set_id(self.ap_id.to_owned())?
+      .set_published(convert_datetime(self.published))?
+      .set_content_xsd_string(self.content.to_owned())?
+      .set_to_xsd_any_uri(recipient.actor_id)?
+      .set_attributed_to_xsd_any_uri(creator.actor_id)?;
+
+    if let Some(u) = self.updated {
+      oprops.set_updated(convert_datetime(u))?;
+    }
+
+    Ok(private_message)
+  }
+
+  fn to_tombstone(&self) -> Result<Tombstone, Error> {
+    create_tombstone(
+      self.deleted,
+      &self.ap_id,
+      self.updated,
+      NoteType.to_string(),
+    )
+  }
+}
+
+impl FromApub for PrivateMessageForm {
+  type ApubType = Note;
+
+  /// Parse an ActivityPub note received from another instance into a Lemmy Private message
+  fn from_apub(note: &Note, conn: &PgConnection) -> Result<PrivateMessageForm, Error> {
+    let oprops = &note.object_props;
+    let creator_actor_id = &oprops.get_attributed_to_xsd_any_uri().unwrap().to_string();
+    let creator = get_or_fetch_and_upsert_remote_user(&creator_actor_id, &conn)?;
+    let recipient_actor_id = &oprops.get_to_xsd_any_uri().unwrap().to_string();
+    let recipient = get_or_fetch_and_upsert_remote_user(&recipient_actor_id, &conn)?;
+
+    Ok(PrivateMessageForm {
+      creator_id: creator.id,
+      recipient_id: recipient.id,
+      content: oprops
+        .get_content_xsd_string()
+        .map(|c| c.to_string())
+        .unwrap(),
+      published: oprops
+        .get_published()
+        .map(|u| u.as_ref().to_owned().naive_local()),
+      updated: oprops
+        .get_updated()
+        .map(|u| u.as_ref().to_owned().naive_local()),
+      deleted: None,
+      read: None,
+      ap_id: oprops.get_id().unwrap().to_string(),
+      local: false,
+    })
+  }
+}
+
+impl ApubObjectType for PrivateMessage {
+  /// Send out information about a newly created private message
+  fn send_create(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(conn)?;
+    let id = format!("{}/create/{}", self.ap_id, uuid::Uuid::new_v4());
+    let recipient = User_::read(&conn, self.recipient_id)?;
+
+    let mut create = Create::new();
+    create
+      .object_props
+      .set_context_xsd_any_uri(context())?
+      .set_id(id)?;
+    let to = format!("{}/inbox", recipient.actor_id);
+
+    create
+      .create_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&create)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &create,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      vec![to],
+    )?;
+    Ok(())
+  }
+
+  /// Send out information about an edited post, to the followers of the community.
+  fn send_update(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(conn)?;
+    let id = format!("{}/update/{}", self.ap_id, uuid::Uuid::new_v4());
+    let recipient = User_::read(&conn, self.recipient_id)?;
+
+    let mut update = Update::new();
+    update
+      .object_props
+      .set_context_xsd_any_uri(context())?
+      .set_id(id)?;
+    let to = format!("{}/inbox", recipient.actor_id);
+
+    update
+      .update_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&update)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &update,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      vec![to],
+    )?;
+    Ok(())
+  }
+
+  fn send_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(conn)?;
+    let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
+    let recipient = User_::read(&conn, self.recipient_id)?;
+
+    let mut delete = Delete::new();
+    delete
+      .object_props
+      .set_context_xsd_any_uri(context())?
+      .set_id(id)?;
+    let to = format!("{}/inbox", recipient.actor_id);
+
+    delete
+      .delete_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&delete)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &delete,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      vec![to],
+    )?;
+    Ok(())
+  }
+
+  fn send_undo_delete(&self, creator: &User_, conn: &PgConnection) -> Result<(), Error> {
+    let note = self.to_apub(conn)?;
+    let id = format!("{}/delete/{}", self.ap_id, uuid::Uuid::new_v4());
+    let recipient = User_::read(&conn, self.recipient_id)?;
+
+    let mut delete = Delete::new();
+    delete
+      .object_props
+      .set_context_xsd_any_uri(context())?
+      .set_id(id)?;
+    let to = format!("{}/inbox", recipient.actor_id);
+
+    delete
+      .delete_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(note)?;
+
+    // TODO
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/delete/{}", self.ap_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    undo
+      .object_props
+      .set_context_xsd_any_uri(context())?
+      .set_id(undo_id)?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(creator.actor_id.to_owned())?
+      .set_object_base_box(delete)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: creator.id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &undo,
+      &creator.private_key.as_ref().unwrap(),
+      &creator.actor_id,
+      vec![to],
+    )?;
+    Ok(())
+  }
+
+  fn send_remove(&self, _mod_: &User_, _conn: &PgConnection) -> Result<(), Error> {
+    unimplemented!()
+  }
+
+  fn send_undo_remove(&self, _mod_: &User_, _conn: &PgConnection) -> Result<(), Error> {
+    unimplemented!()
+  }
+}

server/src/apub/user.rs

@@ -1,74 +1,212 @@
-use crate::apub::make_apub_endpoint;
-use crate::db::establish_unpooled_connection;
-use crate::db::user::User_;
-use crate::to_datetime_utc;
-use activitypub::{actor::Person, context};
-use actix_web::body::Body;
-use actix_web::web::Path;
-use actix_web::HttpResponse;
-use serde::Deserialize;
-
-impl User_ {
-  pub fn as_person(&self) -> Person {
-    let base_url = make_apub_endpoint("u", &self.name);
-    let mut person = Person::default();
-    person.object_props.set_context_object(context()).ok();
-    person.object_props.set_id_string(base_url.to_string()).ok();
-    person
-      .object_props
-      .set_name_string(self.name.to_owned())
-      .ok();
-    person
-      .object_props
-      .set_published_utctime(to_datetime_utc(self.published))
-      .ok();
-    if let Some(updated) = self.updated {
-      person
-        .object_props
-        .set_updated_utctime(to_datetime_utc(updated))
-        .ok();
-    }
-
-    person
-      .ap_actor_props
-      .set_inbox_string(format!("{}/inbox", &base_url))
-      .ok();
-    person
-      .ap_actor_props
-      .set_outbox_string(format!("{}/outbox", &base_url))
-      .ok();
-    person
-      .ap_actor_props
-      .set_following_string(format!("{}/following", &base_url))
-      .ok();
-    person
-      .ap_actor_props
-      .set_liked_string(format!("{}/liked", &base_url))
-      .ok();
-    if let Some(i) = &self.preferred_username {
-      person
-        .ap_actor_props
-        .set_preferred_username_string(i.to_string())
-        .ok();
-    }
-
-    person
-  }
-}
+use super::*;
 
 #[derive(Deserialize)]
 pub struct UserQuery {
   user_name: String,
 }
 
-pub async fn get_apub_user(info: Path<UserQuery>) -> HttpResponse<Body> {
-  let connection = establish_unpooled_connection();
+impl ToApub for User_ {
+  type Response = PersonExt;
 
-  if let Ok(user) = User_::find_by_email_or_username(&connection, &info.user_name) {
-    HttpResponse::Ok()
-      .content_type("application/activity+json")
-      .body(serde_json::to_string(&user.as_person()).unwrap())
-  } else {
-    HttpResponse::NotFound().finish()
+  // Turn a Lemmy Community into an ActivityPub group that can be sent out over the network.
+  fn to_apub(&self, _conn: &PgConnection) -> Result<PersonExt, Error> {
+    // TODO go through all these to_string and to_owned()
+    let mut person = Person::default();
+    let oprops: &mut ObjectProperties = person.as_mut();
+    oprops
+      .set_context_xsd_any_uri(context())?
+      .set_id(self.actor_id.to_string())?
+      .set_name_xsd_string(self.name.to_owned())?
+      .set_published(convert_datetime(self.published))?;
+
+    if let Some(u) = self.updated {
+      oprops.set_updated(convert_datetime(u))?;
+    }
+
+    if let Some(i) = &self.preferred_username {
+      oprops.set_name_xsd_string(i.to_owned())?;
+    }
+
+    let mut endpoint_props = EndpointProperties::default();
+
+    endpoint_props.set_shared_inbox(self.get_shared_inbox_url())?;
+
+    let mut actor_props = ApActorProperties::default();
+
+    actor_props
+      .set_inbox(self.get_inbox_url())?
+      .set_outbox(self.get_outbox_url())?
+      .set_endpoints(endpoint_props)?
+      .set_followers(self.get_followers_url())?
+      .set_following(self.get_following_url())?
+      .set_liked(self.get_liked_url())?;
+
+    Ok(person.extend(actor_props).extend(self.get_public_key_ext()))
+  }
+  fn to_tombstone(&self) -> Result<Tombstone, Error> {
+    unimplemented!()
   }
 }
+
+impl ActorType for User_ {
+  fn actor_id(&self) -> String {
+    self.actor_id.to_owned()
+  }
+
+  fn public_key(&self) -> String {
+    self.public_key.to_owned().unwrap()
+  }
+
+  /// As a given local user, send out a follow request to a remote community.
+  fn send_follow(&self, follow_actor_id: &str, conn: &PgConnection) -> Result<(), Error> {
+    let mut follow = Follow::new();
+
+    let id = format!("{}/follow/{}", self.actor_id, uuid::Uuid::new_v4());
+
+    follow
+      .object_props
+      .set_context_xsd_any_uri(context())?
+      .set_id(id)?;
+    follow
+      .follow_props
+      .set_actor_xsd_any_uri(self.actor_id.to_owned())?
+      .set_object_xsd_any_uri(follow_actor_id)?;
+    let to = format!("{}/inbox", follow_actor_id);
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: self.id,
+      data: serde_json::to_value(&follow)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &follow,
+      &self.private_key.as_ref().unwrap(),
+      &follow_actor_id,
+      vec![to],
+    )?;
+    Ok(())
+  }
+
+  fn send_unfollow(&self, follow_actor_id: &str, conn: &PgConnection) -> Result<(), Error> {
+    let mut follow = Follow::new();
+
+    let id = format!("{}/follow/{}", self.actor_id, uuid::Uuid::new_v4());
+
+    follow
+      .object_props
+      .set_context_xsd_any_uri(context())?
+      .set_id(id)?;
+    follow
+      .follow_props
+      .set_actor_xsd_any_uri(self.actor_id.to_owned())?
+      .set_object_xsd_any_uri(follow_actor_id)?;
+    let to = format!("{}/inbox", follow_actor_id);
+
+    // TODO
+    // Undo that fake activity
+    let undo_id = format!("{}/undo/follow/{}", self.actor_id, uuid::Uuid::new_v4());
+    let mut undo = Undo::default();
+
+    undo
+      .object_props
+      .set_context_xsd_any_uri(context())?
+      .set_id(undo_id)?;
+
+    undo
+      .undo_props
+      .set_actor_xsd_any_uri(self.actor_id.to_owned())?
+      .set_object_base_box(follow)?;
+
+    // Insert the sent activity into the activity table
+    let activity_form = activity::ActivityForm {
+      user_id: self.id,
+      data: serde_json::to_value(&undo)?,
+      local: true,
+      updated: None,
+    };
+    activity::Activity::create(&conn, &activity_form)?;
+
+    send_activity(
+      &undo,
+      &self.private_key.as_ref().unwrap(),
+      &follow_actor_id,
+      vec![to],
+    )?;
+    Ok(())
+  }
+
+  fn send_delete(&self, _creator: &User_, _conn: &PgConnection) -> Result<(), Error> {
+    unimplemented!()
+  }
+
+  fn send_undo_delete(&self, _creator: &User_, _conn: &PgConnection) -> Result<(), Error> {
+    unimplemented!()
+  }
+
+  fn send_remove(&self, _creator: &User_, _conn: &PgConnection) -> Result<(), Error> {
+    unimplemented!()
+  }
+
+  fn send_undo_remove(&self, _creator: &User_, _conn: &PgConnection) -> Result<(), Error> {
+    unimplemented!()
+  }
+
+  fn send_accept_follow(&self, _follow: &Follow, _conn: &PgConnection) -> Result<(), Error> {
+    unimplemented!()
+  }
+
+  fn get_follower_inboxes(&self, _conn: &PgConnection) -> Result<Vec<String>, Error> {
+    unimplemented!()
+  }
+}
+
+impl FromApub for UserForm {
+  type ApubType = PersonExt;
+  /// Parse an ActivityPub person received from another instance into a Lemmy user.
+  fn from_apub(person: &PersonExt, _conn: &PgConnection) -> Result<Self, Error> {
+    let oprops = &person.base.base.object_props;
+    let aprops = &person.base.extension;
+    let public_key: &PublicKey = &person.extension.public_key;
+
+    Ok(UserForm {
+      name: oprops.get_name_xsd_string().unwrap().to_string(),
+      preferred_username: aprops.get_preferred_username().map(|u| u.to_string()),
+      password_encrypted: "".to_string(),
+      admin: false,
+      banned: false,
+      email: None,
+      avatar: None, // -> icon, image
+      updated: oprops
+        .get_updated()
+        .map(|u| u.as_ref().to_owned().naive_local()),
+      show_nsfw: false,
+      theme: "".to_string(),
+      default_sort_type: 0,
+      default_listing_type: 0,
+      lang: "".to_string(),
+      show_avatars: false,
+      send_notifications_to_email: false,
+      matrix_user_id: None,
+      actor_id: oprops.get_id().unwrap().to_string(),
+      bio: oprops.get_summary_xsd_string().map(|s| s.to_string()),
+      local: false,
+      private_key: None,
+      public_key: Some(public_key.to_owned().public_key_pem),
+      last_refreshed_at: Some(naive_now()),
+    })
+  }
+}
+
+/// Return the user json over HTTP.
+pub async fn get_apub_user_http(
+  info: Path<UserQuery>,
+  db: DbPoolParam,
+) -> Result<HttpResponse<Body>, Error> {
+  let user = User_::find_by_email_or_username(&&db.get()?, &info.user_name)?;
+  let u = user.to_apub(&db.get().unwrap())?;
+  Ok(create_apub_response(&u))
+}

server/src/apub/user_inbox.rs

@@ -0,0 +1,320 @@
+use super::*;
+
+#[serde(untagged)]
+#[derive(Deserialize, Debug)]
+pub enum UserAcceptedObjects {
+  Accept(Box<Accept>),
+  Create(Box<Create>),
+  Update(Box<Update>),
+  Delete(Box<Delete>),
+  Undo(Box<Undo>),
+}
+
+/// Handler for all incoming activities to user inboxes.
+pub async fn user_inbox(
+  request: HttpRequest,
+  input: web::Json<UserAcceptedObjects>,
+  path: web::Path<String>,
+  db: DbPoolParam,
+  chat_server: ChatServerParam,
+) -> Result<HttpResponse, Error> {
+  // TODO: would be nice if we could do the signature check here, but we cant access the actor property
+  let input = input.into_inner();
+  let conn = &db.get().unwrap();
+  let username = path.into_inner();
+  debug!("User {} received activity: {:?}", &username, &input);
+
+  match input {
+    UserAcceptedObjects::Accept(a) => receive_accept(&a, &request, &username, &conn),
+    UserAcceptedObjects::Create(c) => {
+      receive_create_private_message(&c, &request, &conn, chat_server)
+    }
+    UserAcceptedObjects::Update(u) => {
+      receive_update_private_message(&u, &request, &conn, chat_server)
+    }
+    UserAcceptedObjects::Delete(d) => {
+      receive_delete_private_message(&d, &request, &conn, chat_server)
+    }
+    UserAcceptedObjects::Undo(u) => {
+      receive_undo_delete_private_message(&u, &request, &conn, chat_server)
+    }
+  }
+}
+
+/// Handle accepted follows.
+fn receive_accept(
+  accept: &Accept,
+  request: &HttpRequest,
+  username: &str,
+  conn: &PgConnection,
+) -> Result<HttpResponse, Error> {
+  let community_uri = accept
+    .accept_props
+    .get_actor_xsd_any_uri()
+    .unwrap()
+    .to_string();
+
+  let community = get_or_fetch_and_upsert_remote_community(&community_uri, conn)?;
+  verify(request, &community.public_key.unwrap())?;
+
+  let user = User_::read_from_name(&conn, username)?;
+
+  // Insert the received activity into the activity table
+  let activity_form = activity::ActivityForm {
+    user_id: community.creator_id,
+    data: serde_json::to_value(&accept)?,
+    local: false,
+    updated: None,
+  };
+  activity::Activity::create(&conn, &activity_form)?;
+
+  // Now you need to add this to the community follower
+  let community_follower_form = CommunityFollowerForm {
+    community_id: community.id,
+    user_id: user.id,
+  };
+
+  // This will fail if they're already a follower
+  CommunityFollower::follow(&conn, &community_follower_form)?;
+
+  // TODO: make sure that we actually requested a follow
+  // TODO: at this point, indicate to the user that they are following the community
+  Ok(HttpResponse::Ok().finish())
+}
+
+fn receive_create_private_message(
+  create: &Create,
+  request: &HttpRequest,
+  conn: &PgConnection,
+  chat_server: ChatServerParam,
+) -> Result<HttpResponse, Error> {
+  let note = create
+    .create_props
+    .get_object_base_box()
+    .to_owned()
+    .unwrap()
+    .to_owned()
+    .into_concrete::<Note>()?;
+
+  let user_uri = create
+    .create_props
+    .get_actor_xsd_any_uri()
+    .unwrap()
+    .to_string();
+
+  let user = get_or_fetch_and_upsert_remote_user(&user_uri, &conn)?;
+  verify(request, &user.public_key.unwrap())?;
+
+  // Insert the received activity into the activity table
+  let activity_form = activity::ActivityForm {
+    user_id: user.id,
+    data: serde_json::to_value(&create)?,
+    local: false,
+    updated: None,
+  };
+  activity::Activity::create(&conn, &activity_form)?;
+
+  let private_message = PrivateMessageForm::from_apub(&note, &conn)?;
+  let inserted_private_message = PrivateMessage::create(&conn, &private_message)?;
+
+  let message = PrivateMessageView::read(&conn, inserted_private_message.id)?;
+
+  let res = PrivateMessageResponse {
+    message: message.to_owned(),
+  };
+
+  chat_server.do_send(SendUserRoomMessage {
+    op: UserOperation::CreatePrivateMessage,
+    response: res,
+    recipient_id: message.recipient_id,
+    my_id: None,
+  });
+
+  Ok(HttpResponse::Ok().finish())
+}
+
+fn receive_update_private_message(
+  update: &Update,
+  request: &HttpRequest,
+  conn: &PgConnection,
+  chat_server: ChatServerParam,
+) -> Result<HttpResponse, Error> {
+  let note = update
+    .update_props
+    .get_object_base_box()
+    .to_owned()
+    .unwrap()
+    .to_owned()
+    .into_concrete::<Note>()?;
+
+  let user_uri = update
+    .update_props
+    .get_actor_xsd_any_uri()
+    .unwrap()
+    .to_string();
+
+  let user = get_or_fetch_and_upsert_remote_user(&user_uri, &conn)?;
+  verify(request, &user.public_key.unwrap())?;
+
+  // Insert the received activity into the activity table
+  let activity_form = activity::ActivityForm {
+    user_id: user.id,
+    data: serde_json::to_value(&update)?,
+    local: false,
+    updated: None,
+  };
+  activity::Activity::create(&conn, &activity_form)?;
+
+  let private_message = PrivateMessageForm::from_apub(&note, &conn)?;
+  let private_message_id = PrivateMessage::read_from_apub_id(&conn, &private_message.ap_id)?.id;
+  PrivateMessage::update(conn, private_message_id, &private_message)?;
+
+  let message = PrivateMessageView::read(&conn, private_message_id)?;
+
+  let res = PrivateMessageResponse {
+    message: message.to_owned(),
+  };
+
+  chat_server.do_send(SendUserRoomMessage {
+    op: UserOperation::EditPrivateMessage,
+    response: res,
+    recipient_id: message.recipient_id,
+    my_id: None,
+  });
+
+  Ok(HttpResponse::Ok().finish())
+}
+
+fn receive_delete_private_message(
+  delete: &Delete,
+  request: &HttpRequest,
+  conn: &PgConnection,
+  chat_server: ChatServerParam,
+) -> Result<HttpResponse, Error> {
+  let note = delete
+    .delete_props
+    .get_object_base_box()
+    .to_owned()
+    .unwrap()
+    .to_owned()
+    .into_concrete::<Note>()?;
+
+  let user_uri = delete
+    .delete_props
+    .get_actor_xsd_any_uri()
+    .unwrap()
+    .to_string();
+
+  let user = get_or_fetch_and_upsert_remote_user(&user_uri, &conn)?;
+  verify(request, &user.public_key.unwrap())?;
+
+  // Insert the received activity into the activity table
+  let activity_form = activity::ActivityForm {
+    user_id: user.id,
+    data: serde_json::to_value(&delete)?,
+    local: false,
+    updated: None,
+  };
+  activity::Activity::create(&conn, &activity_form)?;
+
+  let private_message = PrivateMessageForm::from_apub(&note, &conn)?;
+  let private_message_id = PrivateMessage::read_from_apub_id(&conn, &private_message.ap_id)?.id;
+  let private_message_form = PrivateMessageForm {
+    content: private_message.content,
+    recipient_id: private_message.recipient_id,
+    creator_id: private_message.creator_id,
+    deleted: Some(true),
+    read: None,
+    ap_id: private_message.ap_id,
+    local: private_message.local,
+    published: None,
+    updated: Some(naive_now()),
+  };
+  PrivateMessage::update(conn, private_message_id, &private_message_form)?;
+
+  let message = PrivateMessageView::read(&conn, private_message_id)?;
+
+  let res = PrivateMessageResponse {
+    message: message.to_owned(),
+  };
+
+  chat_server.do_send(SendUserRoomMessage {
+    op: UserOperation::EditPrivateMessage,
+    response: res,
+    recipient_id: message.recipient_id,
+    my_id: None,
+  });
+
+  Ok(HttpResponse::Ok().finish())
+}
+
+fn receive_undo_delete_private_message(
+  undo: &Undo,
+  request: &HttpRequest,
+  conn: &PgConnection,
+  chat_server: ChatServerParam,
+) -> Result<HttpResponse, Error> {
+  let delete = undo
+    .undo_props
+    .get_object_base_box()
+    .to_owned()
+    .unwrap()
+    .to_owned()
+    .into_concrete::<Delete>()?;
+
+  let note = delete
+    .delete_props
+    .get_object_base_box()
+    .to_owned()
+    .unwrap()
+    .to_owned()
+    .into_concrete::<Note>()?;
+
+  let user_uri = delete
+    .delete_props
+    .get_actor_xsd_any_uri()
+    .unwrap()
+    .to_string();
+
+  let user = get_or_fetch_and_upsert_remote_user(&user_uri, &conn)?;
+  verify(request, &user.public_key.unwrap())?;
+
+  // Insert the received activity into the activity table
+  let activity_form = activity::ActivityForm {
+    user_id: user.id,
+    data: serde_json::to_value(&delete)?,
+    local: false,
+    updated: None,
+  };
+  activity::Activity::create(&conn, &activity_form)?;
+
+  let private_message = PrivateMessageForm::from_apub(&note, &conn)?;
+  let private_message_id = PrivateMessage::read_from_apub_id(&conn, &private_message.ap_id)?.id;
+  let private_message_form = PrivateMessageForm {
+    content: private_message.content,
+    recipient_id: private_message.recipient_id,
+    creator_id: private_message.creator_id,
+    deleted: Some(false),
+    read: None,
+    ap_id: private_message.ap_id,
+    local: private_message.local,
+    published: None,
+    updated: Some(naive_now()),
+  };
+  PrivateMessage::update(conn, private_message_id, &private_message_form)?;
+
+  let message = PrivateMessageView::read(&conn, private_message_id)?;
+
+  let res = PrivateMessageResponse {
+    message: message.to_owned(),
+  };
+
+  chat_server.do_send(SendUserRoomMessage {
+    op: UserOperation::EditPrivateMessage,
+    response: res,
+    recipient_id: message.recipient_id,
+    my_id: None,
+  });
+
+  Ok(HttpResponse::Ok().finish())
+}

server/src/db/activity.rs

@@ -0,0 +1,123 @@
+use super::*;
+use crate::schema::activity;
+use crate::schema::activity::dsl::*;
+use serde_json::Value;
+
+#[derive(Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
+#[table_name = "activity"]
+pub struct Activity {
+  pub id: i32,
+  pub user_id: i32,
+  pub data: Value,
+  pub local: bool,
+  pub published: chrono::NaiveDateTime,
+  pub updated: Option<chrono::NaiveDateTime>,
+}
+
+#[derive(Insertable, AsChangeset, Clone, Serialize, Deserialize)]
+#[table_name = "activity"]
+pub struct ActivityForm {
+  pub user_id: i32,
+  pub data: Value,
+  pub local: bool,
+  pub updated: Option<chrono::NaiveDateTime>,
+}
+
+impl Crud<ActivityForm> for Activity {
+  fn read(conn: &PgConnection, activity_id: i32) -> Result<Self, Error> {
+    activity.find(activity_id).first::<Self>(conn)
+  }
+
+  fn delete(conn: &PgConnection, activity_id: i32) -> Result<usize, Error> {
+    diesel::delete(activity.find(activity_id)).execute(conn)
+  }
+
+  fn create(conn: &PgConnection, new_activity: &ActivityForm) -> Result<Self, Error> {
+    insert_into(activity)
+      .values(new_activity)
+      .get_result::<Self>(conn)
+  }
+
+  fn update(
+    conn: &PgConnection,
+    activity_id: i32,
+    new_activity: &ActivityForm,
+  ) -> Result<Self, Error> {
+    diesel::update(activity.find(activity_id))
+      .set(new_activity)
+      .get_result::<Self>(conn)
+  }
+}
+
+#[cfg(test)]
+mod tests {
+  use super::super::user::*;
+  use super::*;
+
+  #[test]
+  fn test_crud() {
+    let conn = establish_unpooled_connection();
+
+    let creator_form = UserForm {
+      name: "activity_creator_pm".into(),
+      preferred_username: None,
+      password_encrypted: "nope".into(),
+      email: None,
+      matrix_user_id: None,
+      avatar: None,
+      admin: false,
+      banned: false,
+      updated: None,
+      show_nsfw: false,
+      theme: "darkly".into(),
+      default_sort_type: SortType::Hot as i16,
+      default_listing_type: ListingType::Subscribed as i16,
+      lang: "browser".into(),
+      show_avatars: true,
+      send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
+    };
+
+    let inserted_creator = User_::create(&conn, &creator_form).unwrap();
+
+    let test_json: Value = serde_json::from_str(
+      r#"{
+    "street": "Article Circle Expressway 1",
+    "city": "North Pole",
+    "postcode": "99705",
+    "state": "Alaska"
+}"#,
+    )
+    .unwrap();
+    let activity_form = ActivityForm {
+      user_id: inserted_creator.id,
+      data: test_json.to_owned(),
+      local: true,
+      updated: None,
+    };
+
+    let inserted_activity = Activity::create(&conn, &activity_form).unwrap();
+
+    let expected_activity = Activity {
+      id: inserted_activity.id,
+      user_id: inserted_creator.id,
+      data: test_json,
+      local: true,
+      published: inserted_activity.published,
+      updated: None,
+    };
+
+    let read_activity = Activity::read(&conn, inserted_activity.id).unwrap();
+    let num_deleted = Activity::delete(&conn, inserted_activity.id).unwrap();
+    User_::delete(&conn, inserted_creator.id).unwrap();
+
+    assert_eq!(expected_activity, read_activity);
+    assert_eq!(expected_activity, inserted_activity);
+    assert_eq!(1, num_deleted);
+  }
+}

server/src/db/code_migrations.rs

@@ -0,0 +1,169 @@
+// This is for db migrations that require code
+use super::comment::Comment;
+use super::community::{Community, CommunityForm};
+use super::post::Post;
+use super::private_message::PrivateMessage;
+use super::user::{UserForm, User_};
+use super::*;
+use crate::apub::extensions::signatures::generate_actor_keypair;
+use crate::apub::{make_apub_endpoint, EndpointType};
+use crate::naive_now;
+use failure::Error;
+use log::info;
+
+pub fn run_advanced_migrations(conn: &PgConnection) -> Result<(), Error> {
+  user_updates_2020_04_02(conn)?;
+  community_updates_2020_04_02(conn)?;
+  post_updates_2020_04_03(conn)?;
+  comment_updates_2020_04_03(conn)?;
+  private_message_updates_2020_05_05(conn)?;
+
+  Ok(())
+}
+
+fn user_updates_2020_04_02(conn: &PgConnection) -> Result<(), Error> {
+  use crate::schema::user_::dsl::*;
+
+  info!("Running user_updates_2020_04_02");
+
+  // Update the actor_id, private_key, and public_key, last_refreshed_at
+  let incorrect_users = user_
+    .filter(actor_id.eq("changeme"))
+    .filter(local.eq(true))
+    .load::<User_>(conn)?;
+
+  for cuser in &incorrect_users {
+    let keypair = generate_actor_keypair()?;
+
+    let form = UserForm {
+      name: cuser.name.to_owned(),
+      email: cuser.email.to_owned(),
+      matrix_user_id: cuser.matrix_user_id.to_owned(),
+      avatar: cuser.avatar.to_owned(),
+      password_encrypted: cuser.password_encrypted.to_owned(),
+      preferred_username: cuser.preferred_username.to_owned(),
+      updated: None,
+      admin: cuser.admin,
+      banned: cuser.banned,
+      show_nsfw: cuser.show_nsfw,
+      theme: cuser.theme.to_owned(),
+      default_sort_type: cuser.default_sort_type,
+      default_listing_type: cuser.default_listing_type,
+      lang: cuser.lang.to_owned(),
+      show_avatars: cuser.show_avatars,
+      send_notifications_to_email: cuser.send_notifications_to_email,
+      actor_id: make_apub_endpoint(EndpointType::User, &cuser.name).to_string(),
+      bio: cuser.bio.to_owned(),
+      local: cuser.local,
+      private_key: Some(keypair.private_key),
+      public_key: Some(keypair.public_key),
+      last_refreshed_at: Some(naive_now()),
+    };
+
+    User_::update(&conn, cuser.id, &form)?;
+  }
+
+  info!("{} user rows updated.", incorrect_users.len());
+
+  Ok(())
+}
+
+fn community_updates_2020_04_02(conn: &PgConnection) -> Result<(), Error> {
+  use crate::schema::community::dsl::*;
+
+  info!("Running community_updates_2020_04_02");
+
+  // Update the actor_id, private_key, and public_key, last_refreshed_at
+  let incorrect_communities = community
+    .filter(actor_id.eq("changeme"))
+    .filter(local.eq(true))
+    .load::<Community>(conn)?;
+
+  for ccommunity in &incorrect_communities {
+    let keypair = generate_actor_keypair()?;
+
+    let form = CommunityForm {
+      name: ccommunity.name.to_owned(),
+      title: ccommunity.title.to_owned(),
+      description: ccommunity.description.to_owned(),
+      category_id: ccommunity.category_id,
+      creator_id: ccommunity.creator_id,
+      removed: None,
+      deleted: None,
+      nsfw: ccommunity.nsfw,
+      updated: None,
+      actor_id: make_apub_endpoint(EndpointType::Community, &ccommunity.name).to_string(),
+      local: ccommunity.local,
+      private_key: Some(keypair.private_key),
+      public_key: Some(keypair.public_key),
+      last_refreshed_at: Some(naive_now()),
+      published: None,
+    };
+
+    Community::update(&conn, ccommunity.id, &form)?;
+  }
+
+  info!("{} community rows updated.", incorrect_communities.len());
+
+  Ok(())
+}
+
+fn post_updates_2020_04_03(conn: &PgConnection) -> Result<(), Error> {
+  use crate::schema::post::dsl::*;
+
+  info!("Running post_updates_2020_04_03");
+
+  // Update the ap_id
+  let incorrect_posts = post
+    .filter(ap_id.eq("changeme"))
+    .filter(local.eq(true))
+    .load::<Post>(conn)?;
+
+  for cpost in &incorrect_posts {
+    Post::update_ap_id(&conn, cpost.id)?;
+  }
+
+  info!("{} post rows updated.", incorrect_posts.len());
+
+  Ok(())
+}
+
+fn comment_updates_2020_04_03(conn: &PgConnection) -> Result<(), Error> {
+  use crate::schema::comment::dsl::*;
+
+  info!("Running comment_updates_2020_04_03");
+
+  // Update the ap_id
+  let incorrect_comments = comment
+    .filter(ap_id.eq("changeme"))
+    .filter(local.eq(true))
+    .load::<Comment>(conn)?;
+
+  for ccomment in &incorrect_comments {
+    Comment::update_ap_id(&conn, ccomment.id)?;
+  }
+
+  info!("{} comment rows updated.", incorrect_comments.len());
+
+  Ok(())
+}
+
+fn private_message_updates_2020_05_05(conn: &PgConnection) -> Result<(), Error> {
+  use crate::schema::private_message::dsl::*;
+
+  info!("Running private_message_updates_2020_05_05");
+
+  // Update the ap_id
+  let incorrect_pms = private_message
+    .filter(ap_id.eq("changeme"))
+    .filter(local.eq(true))
+    .load::<PrivateMessage>(conn)?;
+
+  for cpm in &incorrect_pms {
+    PrivateMessage::update_ap_id(&conn, cpm.id)?;
+  }
+
+  info!("{} private message rows updated.", incorrect_pms.len());
+
+  Ok(())
+}

server/src/db/comment.rs

@@ -1,5 +1,7 @@
 use super::post::Post;
 use super::*;
+use crate::apub::{make_apub_endpoint, EndpointType};
+use crate::naive_now;
 use crate::schema::{comment, comment_like, comment_saved};
 
 // WITH RECURSIVE MyTree AS (
@@ -19,10 +21,12 @@ pub struct Comment {
   pub parent_id: Option<i32>,
   pub content: String,
   pub removed: bool,
-  pub read: bool,
+  pub read: bool, // Whether the recipient has read the comment or not
   pub published: chrono::NaiveDateTime,
   pub updated: Option<chrono::NaiveDateTime>,
   pub deleted: bool,
+  pub ap_id: String,
+  pub local: bool,
 }
 
 #[derive(Insertable, AsChangeset, Clone)]
@@ -34,8 +38,11 @@ pub struct CommentForm {
   pub content: String,
   pub removed: Option<bool>,
   pub read: Option<bool>,
+  pub published: Option<chrono::NaiveDateTime>,
   pub updated: Option<chrono::NaiveDateTime>,
   pub deleted: Option<bool>,
+  pub ap_id: String,
+  pub local: bool,
 }
 
 impl Crud<CommentForm> for Comment {
@@ -68,6 +75,42 @@ impl Crud<CommentForm> for Comment {
   }
 }
 
+impl Comment {
+  pub fn update_ap_id(conn: &PgConnection, comment_id: i32) -> Result<Self, Error> {
+    use crate::schema::comment::dsl::*;
+
+    let apid = make_apub_endpoint(EndpointType::Comment, &comment_id.to_string()).to_string();
+    diesel::update(comment.find(comment_id))
+      .set(ap_id.eq(apid))
+      .get_result::<Self>(conn)
+  }
+
+  pub fn read_from_apub_id(conn: &PgConnection, object_id: &str) -> Result<Self, Error> {
+    use crate::schema::comment::dsl::*;
+    comment.filter(ap_id.eq(object_id)).first::<Self>(conn)
+  }
+
+  pub fn mark_as_read(conn: &PgConnection, comment_id: i32) -> Result<Self, Error> {
+    use crate::schema::comment::dsl::*;
+
+    diesel::update(comment.find(comment_id))
+      .set(read.eq(true))
+      .get_result::<Self>(conn)
+  }
+
+  pub fn permadelete(conn: &PgConnection, comment_id: i32) -> Result<Self, Error> {
+    use crate::schema::comment::dsl::*;
+
+    diesel::update(comment.find(comment_id))
+      .set((
+        content.eq("*Permananently Deleted*"),
+        deleted.eq(true),
+        updated.eq(naive_now()),
+      ))
+      .get_result::<Self>(conn)
+  }
+}
+
 #[derive(Identifiable, Queryable, Associations, PartialEq, Debug, Clone)]
 #[belongs_to(Comment)]
 #[table_name = "comment_like"]
@@ -170,7 +213,6 @@ mod tests {
 
     let new_user = UserForm {
       name: "terry".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -186,6 +228,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_user = User_::create(&conn, &new_user).unwrap();
@@ -200,6 +248,12 @@ mod tests {
       deleted: None,
       updated: None,
       nsfw: false,
+      actor_id: "changeme".into(),
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
+      published: None,
     };
 
     let inserted_community = Community::create(&conn, &new_community).unwrap();
@@ -220,6 +274,9 @@ mod tests {
       embed_description: None,
       embed_html: None,
       thumbnail_url: None,
+      ap_id: "changeme".into(),
+      local: true,
+      published: None,
     };
 
     let inserted_post = Post::create(&conn, &new_post).unwrap();
@@ -232,7 +289,10 @@ mod tests {
       deleted: None,
       read: None,
       parent_id: None,
+      published: None,
       updated: None,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     let inserted_comment = Comment::create(&conn, &comment_form).unwrap();
@@ -248,6 +308,8 @@ mod tests {
       parent_id: None,
       published: inserted_comment.published,
       updated: None,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     let child_comment_form = CommentForm {
@@ -258,7 +320,10 @@ mod tests {
       removed: None,
       deleted: None,
       read: None,
+      published: None,
       updated: None,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     let inserted_child_comment = Comment::create(&conn, &child_comment_form).unwrap();

server/src/db/comment_view.rs

@@ -14,10 +14,16 @@ table! {
     published -> Timestamp,
     updated -> Nullable<Timestamp>,
     deleted -> Bool,
+    ap_id -> Text,
+    local -> Bool,
     community_id -> Int4,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
     banned -> Bool,
     banned_from_community -> Bool,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     creator_name -> Varchar,
     creator_avatar -> Nullable<Text>,
     score -> BigInt,
@@ -43,10 +49,16 @@ table! {
     published -> Timestamp,
     updated -> Nullable<Timestamp>,
     deleted -> Bool,
+    ap_id -> Text,
+    local -> Bool,
     community_id -> Int4,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
     banned -> Bool,
     banned_from_community -> Bool,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     creator_name -> Varchar,
     creator_avatar -> Nullable<Text>,
     score -> BigInt,
@@ -75,10 +87,16 @@ pub struct CommentView {
   pub published: chrono::NaiveDateTime,
   pub updated: Option<chrono::NaiveDateTime>,
   pub deleted: bool,
+  pub ap_id: String,
+  pub local: bool,
   pub community_id: i32,
+  pub community_actor_id: String,
+  pub community_local: bool,
   pub community_name: String,
   pub banned: bool,
   pub banned_from_community: bool,
+  pub creator_actor_id: String,
+  pub creator_local: bool,
   pub creator_name: String,
   pub creator_avatar: Option<String>,
   pub score: i64,
@@ -282,10 +300,16 @@ table! {
     published -> Timestamp,
     updated -> Nullable<Timestamp>,
     deleted -> Bool,
+    ap_id -> Text,
+    local -> Bool,
     community_id -> Int4,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
     banned -> Bool,
     banned_from_community -> Bool,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     creator_name -> Varchar,
     creator_avatar -> Nullable<Text>,
     score -> BigInt,
@@ -315,10 +339,16 @@ pub struct ReplyView {
   pub published: chrono::NaiveDateTime,
   pub updated: Option<chrono::NaiveDateTime>,
   pub deleted: bool,
+  pub ap_id: String,
+  pub local: bool,
   pub community_id: i32,
+  pub community_actor_id: String,
+  pub community_local: bool,
   pub community_name: String,
   pub banned: bool,
   pub banned_from_community: bool,
+  pub creator_actor_id: String,
+  pub creator_local: bool,
   pub creator_name: String,
   pub creator_avatar: Option<String>,
   pub score: i64,
@@ -434,7 +464,6 @@ mod tests {
 
     let new_user = UserForm {
       name: "timmy".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -450,6 +479,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_user = User_::create(&conn, &new_user).unwrap();
@@ -464,6 +499,12 @@ mod tests {
       deleted: None,
       updated: None,
       nsfw: false,
+      actor_id: "changeme".into(),
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
+      published: None,
     };
 
     let inserted_community = Community::create(&conn, &new_community).unwrap();
@@ -484,6 +525,9 @@ mod tests {
       embed_description: None,
       embed_html: None,
       thumbnail_url: None,
+      ap_id: "changeme".into(),
+      local: true,
+      published: None,
     };
 
     let inserted_post = Post::create(&conn, &new_post).unwrap();
@@ -496,7 +540,10 @@ mod tests {
       removed: None,
       deleted: None,
       read: None,
+      published: None,
       updated: None,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     let inserted_comment = Comment::create(&conn, &comment_form).unwrap();
@@ -535,6 +582,12 @@ mod tests {
       my_vote: None,
       subscribed: None,
       saved: None,
+      ap_id: "changeme".to_string(),
+      local: true,
+      community_actor_id: inserted_community.actor_id.to_owned(),
+      community_local: true,
+      creator_actor_id: inserted_user.actor_id.to_owned(),
+      creator_local: true,
     };
 
     let expected_comment_view_with_user = CommentView {
@@ -562,6 +615,12 @@ mod tests {
       my_vote: Some(1),
       subscribed: None,
       saved: None,
+      ap_id: "changeme".to_string(),
+      local: true,
+      community_actor_id: inserted_community.actor_id.to_owned(),
+      community_local: true,
+      creator_actor_id: inserted_user.actor_id.to_owned(),
+      creator_local: true,
     };
 
     let mut read_comment_views_no_user = CommentQueryBuilder::create(&conn)

server/src/db/community.rs

@@ -15,9 +15,15 @@ pub struct Community {
   pub updated: Option<chrono::NaiveDateTime>,
   pub deleted: bool,
   pub nsfw: bool,
+  pub actor_id: String,
+  pub local: bool,
+  pub private_key: Option<String>,
+  pub public_key: Option<String>,
+  pub last_refreshed_at: chrono::NaiveDateTime,
 }
 
-#[derive(Insertable, AsChangeset, Clone, Serialize, Deserialize)]
+// TODO add better delete, remove, lock actions here.
+#[derive(Insertable, AsChangeset, Clone, Serialize, Deserialize, Debug)]
 #[table_name = "community"]
 pub struct CommunityForm {
   pub name: String,
@@ -26,9 +32,15 @@ pub struct CommunityForm {
   pub category_id: i32,
   pub creator_id: i32,
   pub removed: Option<bool>,
+  pub published: Option<chrono::NaiveDateTime>,
   pub updated: Option<chrono::NaiveDateTime>,
   pub deleted: Option<bool>,
   pub nsfw: bool,
+  pub actor_id: String,
+  pub local: bool,
+  pub private_key: Option<String>,
+  pub public_key: Option<String>,
+  pub last_refreshed_at: Option<chrono::NaiveDateTime>,
 }
 
 impl Crud<CommunityForm> for Community {
@@ -62,13 +74,25 @@ impl Crud<CommunityForm> for Community {
 }
 
 impl Community {
-  pub fn read_from_name(conn: &PgConnection, community_name: String) -> Result<Self, Error> {
+  pub fn read_from_name(conn: &PgConnection, community_name: &str) -> Result<Self, Error> {
     use crate::schema::community::dsl::*;
     community
       .filter(name.eq(community_name))
       .first::<Self>(conn)
   }
 
+  pub fn read_from_actor_id(conn: &PgConnection, community_id: &str) -> Result<Self, Error> {
+    use crate::schema::community::dsl::*;
+    community
+      .filter(actor_id.eq(community_id))
+      .first::<Self>(conn)
+  }
+
+  pub fn list_local(conn: &PgConnection) -> Result<Vec<Self>, Error> {
+    use crate::schema::community::dsl::*;
+    community.filter(local.eq(true)).load::<Community>(conn)
+  }
+
   pub fn get_url(&self) -> String {
     format!("https://{}/c/{}", Settings::get().hostname, self.name)
   }
@@ -192,7 +216,7 @@ impl Followable<CommunityFollowerForm> for CommunityFollower {
       .values(community_follower_form)
       .get_result::<Self>(conn)
   }
-  fn ignore(
+  fn unfollow(
     conn: &PgConnection,
     community_follower_form: &CommunityFollowerForm,
   ) -> Result<usize, Error> {
@@ -216,7 +240,6 @@ mod tests {
 
     let new_user = UserForm {
       name: "bobbee".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -232,6 +255,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_user = User_::create(&conn, &new_user).unwrap();
@@ -246,6 +275,12 @@ mod tests {
       removed: None,
       deleted: None,
       updated: None,
+      actor_id: "changeme".into(),
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
+      published: None,
     };
 
     let inserted_community = Community::create(&conn, &new_community).unwrap();
@@ -262,6 +297,11 @@ mod tests {
       deleted: false,
       published: inserted_community.published,
       updated: None,
+      actor_id: "changeme".into(),
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: inserted_community.published,
     };
 
     let community_follower_form = CommunityFollowerForm {
@@ -311,7 +351,7 @@ mod tests {
     let read_community = Community::read(&conn, inserted_community.id).unwrap();
     let updated_community =
       Community::update(&conn, inserted_community.id, &new_community).unwrap();
-    let ignored_community = CommunityFollower::ignore(&conn, &community_follower_form).unwrap();
+    let ignored_community = CommunityFollower::unfollow(&conn, &community_follower_form).unwrap();
     let left_community = CommunityModerator::leave(&conn, &community_user_form).unwrap();
     let unban = CommunityUserBan::unban(&conn, &community_user_ban_form).unwrap();
     let num_deleted = Community::delete(&conn, inserted_community.id).unwrap();

server/src/db/community_view.rs

@@ -15,6 +15,11 @@ table! {
     updated -> Nullable<Timestamp>,
     deleted -> Bool,
     nsfw -> Bool,
+    actor_id -> Text,
+    local -> Bool,
+    last_refreshed_at -> Timestamp,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     creator_name -> Varchar,
     creator_avatar -> Nullable<Text>,
     category_name -> Varchar,
@@ -40,6 +45,11 @@ table! {
     updated -> Nullable<Timestamp>,
     deleted -> Bool,
     nsfw -> Bool,
+    actor_id -> Text,
+    local -> Bool,
+    last_refreshed_at -> Timestamp,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     creator_name -> Varchar,
     creator_avatar -> Nullable<Text>,
     category_name -> Varchar,
@@ -58,8 +68,12 @@ table! {
     community_id -> Int4,
     user_id -> Int4,
     published -> Timestamp,
+    user_actor_id -> Text,
+    user_local -> Bool,
     user_name -> Varchar,
     avatar -> Nullable<Text>,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
   }
 }
@@ -70,8 +84,12 @@ table! {
     community_id -> Int4,
     user_id -> Int4,
     published -> Timestamp,
+    user_actor_id -> Text,
+    user_local -> Bool,
     user_name -> Varchar,
     avatar -> Nullable<Text>,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
   }
 }
@@ -82,8 +100,12 @@ table! {
     community_id -> Int4,
     user_id -> Int4,
     published -> Timestamp,
+    user_actor_id -> Text,
+    user_local -> Bool,
     user_name -> Varchar,
     avatar -> Nullable<Text>,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
   }
 }
@@ -104,6 +126,11 @@ pub struct CommunityView {
   pub updated: Option<chrono::NaiveDateTime>,
   pub deleted: bool,
   pub nsfw: bool,
+  pub actor_id: String,
+  pub local: bool,
+  pub last_refreshed_at: chrono::NaiveDateTime,
+  pub creator_actor_id: String,
+  pub creator_local: bool,
   pub creator_name: String,
   pub creator_avatar: Option<String>,
   pub category_name: String,
@@ -257,8 +284,12 @@ pub struct CommunityModeratorView {
   pub community_id: i32,
   pub user_id: i32,
   pub published: chrono::NaiveDateTime,
+  pub user_actor_id: String,
+  pub user_local: bool,
   pub user_name: String,
   pub avatar: Option<String>,
+  pub community_actor_id: String,
+  pub community_local: bool,
   pub community_name: String,
 }
 
@@ -287,8 +318,12 @@ pub struct CommunityFollowerView {
   pub community_id: i32,
   pub user_id: i32,
   pub published: chrono::NaiveDateTime,
+  pub user_actor_id: String,
+  pub user_local: bool,
   pub user_name: String,
   pub avatar: Option<String>,
+  pub community_actor_id: String,
+  pub community_local: bool,
   pub community_name: String,
 }
 
@@ -317,8 +352,12 @@ pub struct CommunityUserBanView {
   pub community_id: i32,
   pub user_id: i32,
   pub published: chrono::NaiveDateTime,
+  pub user_actor_id: String,
+  pub user_local: bool,
   pub user_name: String,
   pub avatar: Option<String>,
+  pub community_actor_id: String,
+  pub community_local: bool,
   pub community_name: String,
 }
 

server/src/db/mod.rs

@@ -4,7 +4,9 @@ use diesel::result::Error;
 use diesel::*;
 use serde::{Deserialize, Serialize};
 
+pub mod activity;
 pub mod category;
+pub mod code_migrations;
 pub mod comment;
 pub mod comment_view;
 pub mod community;
@@ -42,7 +44,7 @@ pub trait Followable<T> {
   fn follow(conn: &PgConnection, form: &T) -> Result<Self, Error>
   where
     Self: Sized;
-  fn ignore(conn: &PgConnection, form: &T) -> Result<usize, Error>
+  fn unfollow(conn: &PgConnection, form: &T) -> Result<usize, Error>
   where
     Self: Sized;
 }

server/src/db/moderator.rs

@@ -438,7 +438,6 @@ mod tests {
 
     let new_mod = UserForm {
       name: "the mod".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -454,13 +453,18 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_mod = User_::create(&conn, &new_mod).unwrap();
 
     let new_user = UserForm {
       name: "jim2".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -476,6 +480,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_user = User_::create(&conn, &new_user).unwrap();
@@ -490,6 +500,12 @@ mod tests {
       deleted: None,
       updated: None,
       nsfw: false,
+      actor_id: "changeme".into(),
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
+      published: None,
     };
 
     let inserted_community = Community::create(&conn, &new_community).unwrap();
@@ -510,6 +526,9 @@ mod tests {
       embed_description: None,
       embed_html: None,
       thumbnail_url: None,
+      ap_id: "changeme".into(),
+      local: true,
+      published: None,
     };
 
     let inserted_post = Post::create(&conn, &new_post).unwrap();
@@ -522,7 +541,10 @@ mod tests {
       deleted: None,
       read: None,
       parent_id: None,
+      published: None,
       updated: None,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     let inserted_comment = Comment::create(&conn, &comment_form).unwrap();

server/src/db/password_reset_request.rs

@@ -88,7 +88,6 @@ mod tests {
 
     let new_user = UserForm {
       name: "thommy prw".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -104,6 +103,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_user = User_::create(&conn, &new_user).unwrap();

server/src/db/post.rs

@@ -1,4 +1,6 @@
 use super::*;
+use crate::apub::{make_apub_endpoint, EndpointType};
+use crate::naive_now;
 use crate::schema::{post, post_like, post_read, post_saved};
 
 #[derive(Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
@@ -21,9 +23,11 @@ pub struct Post {
   pub embed_description: Option<String>,
   pub embed_html: Option<String>,
   pub thumbnail_url: Option<String>,
+  pub ap_id: String,
+  pub local: bool,
 }
 
-#[derive(Insertable, AsChangeset, Clone)]
+#[derive(Insertable, AsChangeset, Clone, Debug)]
 #[table_name = "post"]
 pub struct PostForm {
   pub name: String,
@@ -33,6 +37,7 @@ pub struct PostForm {
   pub community_id: i32,
   pub removed: Option<bool>,
   pub locked: Option<bool>,
+  pub published: Option<chrono::NaiveDateTime>,
   pub updated: Option<chrono::NaiveDateTime>,
   pub deleted: Option<bool>,
   pub nsfw: bool,
@@ -41,6 +46,56 @@ pub struct PostForm {
   pub embed_description: Option<String>,
   pub embed_html: Option<String>,
   pub thumbnail_url: Option<String>,
+  pub ap_id: String,
+  pub local: bool,
+}
+
+impl Post {
+  pub fn read(conn: &PgConnection, post_id: i32) -> Result<Self, Error> {
+    use crate::schema::post::dsl::*;
+    post.filter(id.eq(post_id)).first::<Self>(conn)
+  }
+
+  pub fn list_for_community(
+    conn: &PgConnection,
+    the_community_id: i32,
+  ) -> Result<Vec<Self>, Error> {
+    use crate::schema::post::dsl::*;
+    post
+      .filter(community_id.eq(the_community_id))
+      .load::<Self>(conn)
+  }
+
+  pub fn read_from_apub_id(conn: &PgConnection, object_id: &str) -> Result<Self, Error> {
+    use crate::schema::post::dsl::*;
+    post.filter(ap_id.eq(object_id)).first::<Self>(conn)
+  }
+
+  pub fn update_ap_id(conn: &PgConnection, post_id: i32) -> Result<Self, Error> {
+    use crate::schema::post::dsl::*;
+
+    let apid = make_apub_endpoint(EndpointType::Post, &post_id.to_string()).to_string();
+    diesel::update(post.find(post_id))
+      .set(ap_id.eq(apid))
+      .get_result::<Self>(conn)
+  }
+
+  pub fn permadelete(conn: &PgConnection, post_id: i32) -> Result<Self, Error> {
+    use crate::schema::post::dsl::*;
+
+    let perma_deleted = "*Permananently Deleted*";
+    let perma_deleted_url = "https://deleted.com";
+
+    diesel::update(post.find(post_id))
+      .set((
+        name.eq(perma_deleted),
+        url.eq(perma_deleted_url),
+        body.eq(perma_deleted),
+        deleted.eq(true),
+        updated.eq(naive_now()),
+      ))
+      .get_result::<Self>(conn)
+  }
 }
 
 impl Crud<PostForm> for Post {
@@ -191,7 +246,6 @@ mod tests {
 
     let new_user = UserForm {
       name: "jim".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -207,6 +261,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_user = User_::create(&conn, &new_user).unwrap();
@@ -221,6 +281,12 @@ mod tests {
       deleted: None,
       updated: None,
       nsfw: false,
+      actor_id: "changeme".into(),
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
+      published: None,
     };
 
     let inserted_community = Community::create(&conn, &new_community).unwrap();
@@ -241,6 +307,9 @@ mod tests {
       embed_description: None,
       embed_html: None,
       thumbnail_url: None,
+      ap_id: "changeme".into(),
+      local: true,
+      published: None,
     };
 
     let inserted_post = Post::create(&conn, &new_post).unwrap();
@@ -263,6 +332,8 @@ mod tests {
       embed_description: None,
       embed_html: None,
       thumbnail_url: None,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     // Post Like

server/src/db/post_view.rs

@@ -22,10 +22,16 @@ table! {
     embed_description -> Nullable<Text>,
     embed_html -> Nullable<Text>,
     thumbnail_url -> Nullable<Text>,
+    ap_id -> Text,
+    local -> Bool,
     banned -> Bool,
     banned_from_community -> Bool,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     creator_name -> Varchar,
     creator_avatar -> Nullable<Text>,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
     community_removed -> Bool,
     community_deleted -> Bool,
@@ -63,10 +69,16 @@ table! {
     embed_description -> Nullable<Text>,
     embed_html -> Nullable<Text>,
     thumbnail_url -> Nullable<Text>,
+    ap_id -> Text,
+    local -> Bool,
     banned -> Bool,
     banned_from_community -> Bool,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     creator_name -> Varchar,
     creator_avatar -> Nullable<Text>,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
     community_removed -> Bool,
     community_deleted -> Bool,
@@ -107,10 +119,16 @@ pub struct PostView {
   pub embed_description: Option<String>,
   pub embed_html: Option<String>,
   pub thumbnail_url: Option<String>,
+  pub ap_id: String,
+  pub local: bool,
   pub banned: bool,
   pub banned_from_community: bool,
+  pub creator_actor_id: String,
+  pub creator_local: bool,
   pub creator_name: String,
   pub creator_avatar: Option<String>,
+  pub community_actor_id: String,
+  pub community_local: bool,
   pub community_name: String,
   pub community_removed: bool,
   pub community_deleted: bool,
@@ -359,7 +377,6 @@ mod tests {
 
     let new_user = UserForm {
       name: user_name.to_owned(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -375,6 +392,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_user = User_::create(&conn, &new_user).unwrap();
@@ -389,6 +412,12 @@ mod tests {
       deleted: None,
       updated: None,
       nsfw: false,
+      actor_id: "changeme".into(),
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
+      published: None,
     };
 
     let inserted_community = Community::create(&conn, &new_community).unwrap();
@@ -409,6 +438,9 @@ mod tests {
       embed_description: None,
       embed_html: None,
       thumbnail_url: None,
+      ap_id: "changeme".into(),
+      local: true,
+      published: None,
     };
 
     let inserted_post = Post::create(&conn, &new_post).unwrap();
@@ -473,6 +505,12 @@ mod tests {
       embed_description: None,
       embed_html: None,
       thumbnail_url: None,
+      ap_id: "changeme".to_string(),
+      local: true,
+      creator_actor_id: inserted_user.actor_id.to_owned(),
+      creator_local: true,
+      community_actor_id: inserted_community.actor_id.to_owned(),
+      community_local: true,
     };
 
     let expected_post_listing_with_user = PostView {
@@ -512,6 +550,12 @@ mod tests {
       embed_description: None,
       embed_html: None,
       thumbnail_url: None,
+      ap_id: "changeme".to_string(),
+      local: true,
+      creator_actor_id: inserted_user.actor_id.to_owned(),
+      creator_local: true,
+      community_actor_id: inserted_community.actor_id.to_owned(),
+      community_local: true,
     };
 
     let read_post_listings_with_user = PostQueryBuilder::create(&conn)

server/src/db/private_message.rs

@@ -1,4 +1,5 @@
 use super::*;
+use crate::apub::{make_apub_endpoint, EndpointType};
 use crate::schema::private_message;
 
 #[derive(Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
@@ -12,6 +13,8 @@ pub struct PrivateMessage {
   pub read: bool,
   pub published: chrono::NaiveDateTime,
   pub updated: Option<chrono::NaiveDateTime>,
+  pub ap_id: String,
+  pub local: bool,
 }
 
 #[derive(Insertable, AsChangeset, Clone)]
@@ -19,10 +22,13 @@ pub struct PrivateMessage {
 pub struct PrivateMessageForm {
   pub creator_id: i32,
   pub recipient_id: i32,
-  pub content: Option<String>,
+  pub content: String,
   pub deleted: Option<bool>,
   pub read: Option<bool>,
+  pub published: Option<chrono::NaiveDateTime>,
   pub updated: Option<chrono::NaiveDateTime>,
+  pub ap_id: String,
+  pub local: bool,
 }
 
 impl Crud<PrivateMessageForm> for PrivateMessage {
@@ -55,6 +61,28 @@ impl Crud<PrivateMessageForm> for PrivateMessage {
   }
 }
 
+impl PrivateMessage {
+  pub fn update_ap_id(conn: &PgConnection, private_message_id: i32) -> Result<Self, Error> {
+    use crate::schema::private_message::dsl::*;
+
+    let apid = make_apub_endpoint(
+      EndpointType::PrivateMessage,
+      &private_message_id.to_string(),
+    )
+    .to_string();
+    diesel::update(private_message.find(private_message_id))
+      .set(ap_id.eq(apid))
+      .get_result::<Self>(conn)
+  }
+
+  pub fn read_from_apub_id(conn: &PgConnection, object_id: &str) -> Result<Self, Error> {
+    use crate::schema::private_message::dsl::*;
+    private_message
+      .filter(ap_id.eq(object_id))
+      .first::<Self>(conn)
+  }
+}
+
 #[cfg(test)]
 mod tests {
   use super::super::user::*;
@@ -65,7 +93,6 @@ mod tests {
 
     let creator_form = UserForm {
       name: "creator_pm".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -81,13 +108,18 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_creator = User_::create(&conn, &creator_form).unwrap();
 
     let recipient_form = UserForm {
       name: "recipient_pm".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -103,17 +135,26 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_recipient = User_::create(&conn, &recipient_form).unwrap();
 
     let private_message_form = PrivateMessageForm {
-      content: Some("A test private message".into()),
+      content: "A test private message".into(),
       creator_id: inserted_creator.id,
       recipient_id: inserted_recipient.id,
       deleted: None,
       read: None,
+      published: None,
       updated: None,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     let inserted_private_message = PrivateMessage::create(&conn, &private_message_form).unwrap();
@@ -127,6 +168,8 @@ mod tests {
       read: false,
       updated: None,
       published: inserted_private_message.published,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     let read_private_message = PrivateMessage::read(&conn, inserted_private_message.id).unwrap();

server/src/db/private_message_view.rs

@@ -12,10 +12,16 @@ table! {
     read -> Bool,
     published -> Timestamp,
     updated -> Nullable<Timestamp>,
+    ap_id -> Text,
+    local -> Bool,
     creator_name -> Varchar,
     creator_avatar -> Nullable<Text>,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     recipient_name -> Varchar,
     recipient_avatar -> Nullable<Text>,
+    recipient_actor_id -> Text,
+    recipient_local -> Bool,
   }
 }
 
@@ -29,10 +35,16 @@ table! {
     read -> Bool,
     published -> Timestamp,
     updated -> Nullable<Timestamp>,
+    ap_id -> Text,
+    local -> Bool,
     creator_name -> Varchar,
     creator_avatar -> Nullable<Text>,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     recipient_name -> Varchar,
     recipient_avatar -> Nullable<Text>,
+    recipient_actor_id -> Text,
+    recipient_local -> Bool,
   }
 }
 
@@ -49,10 +61,16 @@ pub struct PrivateMessageView {
   pub read: bool,
   pub published: chrono::NaiveDateTime,
   pub updated: Option<chrono::NaiveDateTime>,
+  pub ap_id: String,
+  pub local: bool,
   pub creator_name: String,
   pub creator_avatar: Option<String>,
+  pub creator_actor_id: String,
+  pub creator_local: bool,
   pub recipient_name: String,
   pub recipient_avatar: Option<String>,
+  pub recipient_actor_id: String,
+  pub recipient_local: bool,
 }
 
 pub struct PrivateMessageQueryBuilder<'a> {

server/src/db/user.rs

@@ -1,7 +1,7 @@
 use super::*;
 use crate::schema::user_;
 use crate::schema::user_::dsl::*;
-use crate::{is_email_regex, Settings};
+use crate::{is_email_regex, naive_now, Settings};
 use bcrypt::{hash, DEFAULT_COST};
 use jsonwebtoken::{decode, encode, DecodingKey, EncodingKey, Header, TokenData, Validation};
 
@@ -10,7 +10,6 @@ use jsonwebtoken::{decode, encode, DecodingKey, EncodingKey, Header, TokenData,
 pub struct User_ {
   pub id: i32,
   pub name: String,
-  pub fedi_name: String,
   pub preferred_username: Option<String>,
   pub password_encrypted: String,
   pub email: Option<String>,
@@ -27,13 +26,18 @@ pub struct User_ {
   pub show_avatars: bool,
   pub send_notifications_to_email: bool,
   pub matrix_user_id: Option<String>,
+  pub actor_id: String,
+  pub bio: Option<String>,
+  pub local: bool,
+  pub private_key: Option<String>,
+  pub public_key: Option<String>,
+  pub last_refreshed_at: chrono::NaiveDateTime,
 }
 
-#[derive(Insertable, AsChangeset, Clone)]
+#[derive(Insertable, AsChangeset, Clone, Debug)]
 #[table_name = "user_"]
 pub struct UserForm {
   pub name: String,
-  pub fedi_name: String,
   pub preferred_username: Option<String>,
   pub password_encrypted: String,
   pub admin: bool,
@@ -49,6 +53,12 @@ pub struct UserForm {
   pub show_avatars: bool,
   pub send_notifications_to_email: bool,
   pub matrix_user_id: Option<String>,
+  pub actor_id: String,
+  pub bio: Option<String>,
+  pub local: bool,
+  pub private_key: Option<String>,
+  pub public_key: Option<String>,
+  pub last_refreshed_at: Option<chrono::NaiveDateTime>,
 }
 
 impl Crud<UserForm> for User_ {
@@ -78,6 +88,7 @@ impl User_ {
     Self::create(&conn, &edited_user)
   }
 
+  // TODO do more individual updates like these
   pub fn update_password(
     conn: &PgConnection,
     user_id: i32,
@@ -86,13 +97,33 @@ impl User_ {
     let password_hash = hash(new_password, DEFAULT_COST).expect("Couldn't hash password");
 
     diesel::update(user_.find(user_id))
-      .set(password_encrypted.eq(password_hash))
+      .set((
+        password_encrypted.eq(password_hash),
+        updated.eq(naive_now()),
+      ))
       .get_result::<Self>(conn)
   }
 
-  pub fn read_from_name(conn: &PgConnection, from_user_name: String) -> Result<Self, Error> {
+  pub fn read_from_name(conn: &PgConnection, from_user_name: &str) -> Result<Self, Error> {
     user_.filter(name.eq(from_user_name)).first::<Self>(conn)
   }
+
+  pub fn add_admin(conn: &PgConnection, user_id: i32, added: bool) -> Result<Self, Error> {
+    diesel::update(user_.find(user_id))
+      .set(admin.eq(added))
+      .get_result::<Self>(conn)
+  }
+
+  pub fn ban_user(conn: &PgConnection, user_id: i32, ban: bool) -> Result<Self, Error> {
+    diesel::update(user_.find(user_id))
+      .set(banned.eq(ban))
+      .get_result::<Self>(conn)
+  }
+
+  pub fn read_from_actor_id(conn: &PgConnection, object_id: &str) -> Result<Self, Error> {
+    use crate::schema::user_::dsl::*;
+    user_.filter(actor_id.eq(object_id)).first::<Self>(conn)
+  }
 }
 
 #[derive(Debug, Serialize, Deserialize)]
@@ -129,7 +160,7 @@ impl User_ {
     let my_claims = Claims {
       id: self.id,
       username: self.name.to_owned(),
-      iss: self.fedi_name.to_owned(),
+      iss: Settings::get().hostname,
       show_nsfw: self.show_nsfw,
       theme: self.theme.to_owned(),
       default_sort_type: self.default_sort_type,
@@ -186,7 +217,6 @@ mod tests {
 
     let new_user = UserForm {
       name: "thommy".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -202,6 +232,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_user = User_::create(&conn, &new_user).unwrap();
@@ -209,7 +245,6 @@ mod tests {
     let expected_user = User_ {
       id: inserted_user.id,
       name: "thommy".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -226,6 +261,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: inserted_user.published,
     };
 
     let read_user = User_::read(&conn, inserted_user.id).unwrap();

server/src/db/user_mention.rs

@@ -64,7 +64,6 @@ mod tests {
 
     let new_user = UserForm {
       name: "terrylake".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -80,13 +79,18 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_user = User_::create(&conn, &new_user).unwrap();
 
     let recipient_form = UserForm {
       name: "terrylakes recipient".into(),
-      fedi_name: "rrf".into(),
       preferred_username: None,
       password_encrypted: "nope".into(),
       email: None,
@@ -102,6 +106,12 @@ mod tests {
       lang: "browser".into(),
       show_avatars: true,
       send_notifications_to_email: false,
+      actor_id: "changeme".into(),
+      bio: None,
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
     };
 
     let inserted_recipient = User_::create(&conn, &recipient_form).unwrap();
@@ -116,6 +126,12 @@ mod tests {
       deleted: None,
       updated: None,
       nsfw: false,
+      actor_id: "changeme".into(),
+      local: true,
+      private_key: None,
+      public_key: None,
+      last_refreshed_at: None,
+      published: None,
     };
 
     let inserted_community = Community::create(&conn, &new_community).unwrap();
@@ -136,6 +152,9 @@ mod tests {
       embed_description: None,
       embed_html: None,
       thumbnail_url: None,
+      ap_id: "changeme".into(),
+      local: true,
+      published: None,
     };
 
     let inserted_post = Post::create(&conn, &new_post).unwrap();
@@ -148,7 +167,10 @@ mod tests {
       deleted: None,
       read: None,
       parent_id: None,
+      published: None,
       updated: None,
+      ap_id: "changeme".into(),
+      local: true,
     };
 
     let inserted_comment = Comment::create(&conn, &comment_form).unwrap();

server/src/db/user_mention_view.rs

@@ -7,6 +7,8 @@ table! {
     id -> Int4,
     user_mention_id -> Int4,
     creator_id -> Int4,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     post_id -> Int4,
     parent_id -> Nullable<Int4>,
     content -> Text,
@@ -16,6 +18,8 @@ table! {
     updated -> Nullable<Timestamp>,
     deleted -> Bool,
     community_id -> Int4,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
     banned -> Bool,
     banned_from_community -> Bool,
@@ -29,6 +33,8 @@ table! {
     my_vote -> Nullable<Int4>,
     saved -> Nullable<Bool>,
     recipient_id -> Int4,
+    recipient_actor_id -> Text,
+    recipient_local -> Bool,
   }
 }
 
@@ -37,6 +43,8 @@ table! {
     id -> Int4,
     user_mention_id -> Int4,
     creator_id -> Int4,
+    creator_actor_id -> Text,
+    creator_local -> Bool,
     post_id -> Int4,
     parent_id -> Nullable<Int4>,
     content -> Text,
@@ -46,6 +54,8 @@ table! {
     updated -> Nullable<Timestamp>,
     deleted -> Bool,
     community_id -> Int4,
+    community_actor_id -> Text,
+    community_local -> Bool,
     community_name -> Varchar,
     banned -> Bool,
     banned_from_community -> Bool,
@@ -59,6 +69,8 @@ table! {
     my_vote -> Nullable<Int4>,
     saved -> Nullable<Bool>,
     recipient_id -> Int4,
+    recipient_actor_id -> Text,
+    recipient_local -> Bool,
   }
 }
 
@@ -70,6 +82,8 @@ pub struct UserMentionView {
   pub id: i32,
   pub user_mention_id: i32,
   pub creator_id: i32,
+  pub creator_actor_id: String,
+  pub creator_local: bool,
   pub post_id: i32,
   pub parent_id: Option<i32>,
   pub content: String,
@@ -79,6 +93,8 @@ pub struct UserMentionView {
   pub updated: Option<chrono::NaiveDateTime>,
   pub deleted: bool,
   pub community_id: i32,
+  pub community_actor_id: String,
+  pub community_local: bool,
   pub community_name: String,
   pub banned: bool,
   pub banned_from_community: bool,
@@ -92,6 +108,8 @@ pub struct UserMentionView {
   pub my_vote: Option<i32>,
   pub saved: Option<bool>,
   pub recipient_id: i32,
+  pub recipient_actor_id: String,
+  pub recipient_local: bool,
 }
 
 pub struct UserMentionQueryBuilder<'a> {

server/src/db/user_view.rs

@@ -5,11 +5,13 @@ use diesel::pg::Pg;
 table! {
   user_view (id) {
     id -> Int4,
+    actor_id -> Text,
     name -> Varchar,
     avatar -> Nullable<Text>,
     email -> Nullable<Text>,
     matrix_user_id -> Nullable<Text>,
-    fedi_name -> Varchar,
+    bio -> Nullable<Text>,
+    local -> Bool,
     admin -> Bool,
     banned -> Bool,
     show_avatars -> Bool,
@@ -25,11 +27,13 @@ table! {
 table! {
   user_mview (id) {
     id -> Int4,
+    actor_id -> Text,
     name -> Varchar,
     avatar -> Nullable<Text>,
     email -> Nullable<Text>,
     matrix_user_id -> Nullable<Text>,
-    fedi_name -> Varchar,
+    bio -> Nullable<Text>,
+    local -> Bool,
     admin -> Bool,
     banned -> Bool,
     show_avatars -> Bool,
@@ -48,11 +52,13 @@ table! {
 #[table_name = "user_view"]
 pub struct UserView {
   pub id: i32,
+  pub actor_id: String,
   pub name: String,
   pub avatar: Option<String>,
   pub email: Option<String>,
   pub matrix_user_id: Option<String>,
-  pub fedi_name: String,
+  pub bio: Option<String>,
+  pub local: bool,
   pub admin: bool,
   pub banned: bool,
   pub show_avatars: bool,

server/src/lib.rs

@@ -16,6 +16,8 @@ pub extern crate dotenv;
 pub extern crate jsonwebtoken;
 pub extern crate lettre;
 pub extern crate lettre_email;
+extern crate log;
+pub extern crate openssl;
 pub extern crate rand;
 pub extern crate regex;
 pub extern crate rss;
@@ -34,8 +36,10 @@ pub mod settings;
 pub mod version;
 pub mod websocket;
 
+use crate::settings::Settings;
 use actix_web::dev::ConnectionInfo;
-use chrono::{DateTime, NaiveDateTime, Utc};
+use chrono::{DateTime, FixedOffset, Local, NaiveDateTime, Utc};
+use isahc::prelude::*;
 use lettre::smtp::authentication::{Credentials, Mechanism};
 use lettre::smtp::extension::ClientId;
 use lettre::smtp::ConnectionReuseParameters;
@@ -48,8 +52,6 @@ use rand::{thread_rng, Rng};
 use regex::{Regex, RegexBuilder};
 use serde::Deserialize;
 
-use crate::settings::Settings;
-
 pub type ConnectionId = usize;
 pub type PostId = i32;
 pub type CommunityId = i32;
@@ -68,23 +70,13 @@ pub fn naive_from_unix(time: i64) -> NaiveDateTime {
   NaiveDateTime::from_timestamp(time, 0)
 }
 
-pub fn is_email_regex(test: &str) -> bool {
-  EMAIL_REGEX.is_match(test)
+pub fn convert_datetime(datetime: NaiveDateTime) -> DateTime<FixedOffset> {
+  let now = Local::now();
+  DateTime::<FixedOffset>::from_utc(datetime, *now.offset())
 }
 
-pub fn is_image_content_type(test: &str) -> Result<(), failure::Error> {
-  if attohttpc::get(test)
-    .send()?
-    .headers()
-    .get("Content-Type")
-    .ok_or_else(|| format_err!("No Content-Type header"))?
-    .to_str()?
-    .starts_with("image/")
-  {
-    Ok(())
-  } else {
-    Err(format_err!("Not an image type."))
-  }
+pub fn is_email_regex(test: &str) -> bool {
+  EMAIL_REGEX.is_match(test)
 }
 
 pub fn remove_slurs(test: &str) -> String {
@@ -182,40 +174,28 @@ pub struct IframelyResponse {
 
 pub fn fetch_iframely(url: &str) -> Result<IframelyResponse, failure::Error> {
   let fetch_url = format!("http://iframely/oembed?url={}", url);
-  let text: String = attohttpc::get(&fetch_url).send()?.text()?;
+  let text = isahc::get(&fetch_url)?.text()?;
   let res: IframelyResponse = serde_json::from_str(&text)?;
   Ok(res)
 }
 
-#[derive(Deserialize, Debug, Clone)]
-pub struct PictrsResponse {
-  files: Vec<PictrsFile>,
-  msg: String,
+#[derive(Deserialize, Debug)]
+pub struct PictshareResponse {
+  status: String,
+  url: String,
 }
 
-#[derive(Deserialize, Debug, Clone)]
-pub struct PictrsFile {
-  file: String,
-  delete_token: String,
-}
-
-pub fn fetch_pictrs(image_url: &str) -> Result<PictrsResponse, failure::Error> {
-  is_image_content_type(image_url)?;
-
+pub fn fetch_pictshare(image_url: &str) -> Result<PictshareResponse, failure::Error> {
   let fetch_url = format!(
-    "http://pictrs:8080/image/download?url={}",
-    utf8_percent_encode(image_url, NON_ALPHANUMERIC) // TODO this might not be needed
+    "http://pictshare/api/geturl.php?url={}",
+    utf8_percent_encode(image_url, NON_ALPHANUMERIC)
   );
-  let text = attohttpc::get(&fetch_url).send()?.text()?;
-  let res: PictrsResponse = serde_json::from_str(&text)?;
-  if res.msg == "ok" {
-    Ok(res)
-  } else {
-    Err(format_err!("{}", &res.msg))
-  }
+  let text = isahc::get(&fetch_url)?.text()?;
+  let res: PictshareResponse = serde_json::from_str(&text)?;
+  Ok(res)
 }
 
-fn fetch_iframely_and_pictrs_data(
+fn fetch_iframely_and_pictshare_data(
   url: Option<String>,
 ) -> (
   Option<String>,
@@ -223,46 +203,36 @@ fn fetch_iframely_and_pictrs_data(
   Option<String>,
   Option<String>,
 ) {
-  match &url {
-    Some(url) => {
-      // Fetch iframely data
-      let (iframely_title, iframely_description, iframely_thumbnail_url, iframely_html) =
-        match fetch_iframely(url) {
-          Ok(res) => (res.title, res.description, res.thumbnail_url, res.html),
-          Err(e) => {
-            error!("iframely err: {}", e);
-            (None, None, None, None)
-          }
-        };
-
-      // Fetch pictrs thumbnail
-      let pictrs_thumbnail = match iframely_thumbnail_url {
-        Some(iframely_thumbnail_url) => match fetch_pictrs(&iframely_thumbnail_url) {
-          Ok(res) => Some(res.files[0].file.to_owned()),
-          Err(e) => {
-            error!("pictrs err: {}", e);
-            None
-          }
-        },
-        // Try to generate a small thumbnail if iframely is not supported
-        None => match fetch_pictrs(&url) {
-          Ok(res) => Some(res.files[0].file.to_owned()),
-          Err(e) => {
-            error!("pictrs err: {}", e);
-            None
-          }
-        },
-      };
-
-      (
-        iframely_title,
-        iframely_description,
-        iframely_html,
-        pictrs_thumbnail,
-      )
-    }
+  // Fetch iframely data
+  let (iframely_title, iframely_description, iframely_thumbnail_url, iframely_html) = match url {
+    Some(url) => match fetch_iframely(&url) {
+      Ok(res) => (res.title, res.description, res.thumbnail_url, res.html),
+      Err(e) => {
+        error!("iframely err: {}", e);
+        (None, None, None, None)
+      }
+    },
     None => (None, None, None, None),
-  }
+  };
+
+  // Fetch pictshare thumbnail
+  let pictshare_thumbnail = match iframely_thumbnail_url {
+    Some(iframely_thumbnail_url) => match fetch_pictshare(&iframely_thumbnail_url) {
+      Ok(res) => Some(res.url),
+      Err(e) => {
+        error!("pictshare err: {}", e);
+        None
+      }
+    },
+    None => None,
+  };
+
+  (
+    iframely_title,
+    iframely_description,
+    iframely_html,
+    pictshare_thumbnail,
+  )
 }
 
 pub fn markdown_to_html(text: &str) -> String {
@@ -279,29 +249,9 @@ pub fn get_ip(conn_info: &ConnectionInfo) -> String {
     .to_string()
 }
 
-pub fn is_valid_username(name: &str) -> bool {
-  VALID_USERNAME_REGEX.is_match(name)
-}
-
-pub fn is_valid_community_name(name: &str) -> bool {
-  VALID_COMMUNITY_NAME_REGEX.is_match(name)
-}
-
 #[cfg(test)]
 mod tests {
-  use crate::{
-    extract_usernames, is_email_regex, is_image_content_type, is_valid_community_name,
-    is_valid_username, remove_slurs, slur_check, slurs_vec_to_str,
-  };
-
-  #[test]
-  fn test_image() {
-    assert!(is_image_content_type("https://1734811051.rsc.cdn77.org/data/images/full/365645/as-virus-kills-navajos-in-their-homes-tribal-women-provide-lifeline.jpg?w=600?w=650").is_ok());
-    assert!(is_image_content_type(
-      "https://twitter.com/BenjaminNorton/status/1259922424272957440?s=20"
-    )
-    .is_err());
-  }
+  use crate::{extract_usernames, is_email_regex, remove_slurs, slur_check, slurs_vec_to_str};
 
   #[test]
   fn test_email() {
@@ -309,24 +259,6 @@ mod tests {
     assert!(!is_email_regex("nada_neutho"));
   }
 
-  #[test]
-  fn test_valid_register_username() {
-    assert!(is_valid_username("Hello_98"));
-    assert!(is_valid_username("ten"));
-    assert!(!is_valid_username("Hello-98"));
-    assert!(!is_valid_username("a"));
-    assert!(!is_valid_username(""));
-  }
-
-  #[test]
-  fn test_valid_community_name() {
-    assert!(is_valid_community_name("example"));
-    assert!(is_valid_community_name("example_community"));
-    assert!(!is_valid_community_name("Example"));
-    assert!(!is_valid_community_name("Ex"));
-    assert!(!is_valid_community_name(""));
-  }
-
   #[test]
   fn test_slur_filter() {
     let test =
@@ -388,6 +320,4 @@ lazy_static! {
   static ref EMAIL_REGEX: Regex = Regex::new(r"^[a-zA-Z0-9.!#$%&’*+/=?^_`{|}~-]+@[a-zA-Z0-9-]+(?:\.[a-zA-Z0-9-]+)*$").unwrap();
   static ref SLUR_REGEX: Regex = RegexBuilder::new(r"(fag(g|got|tard)?|maricos?|cock\s?sucker(s|ing)?|nig(\b|g?(a|er)?(s|z)?)\b|dindu(s?)|mudslime?s?|kikes?|mongoloids?|towel\s*heads?|\bspi(c|k)s?\b|\bchinks?|niglets?|beaners?|\bnips?\b|\bcoons?\b|jungle\s*bunn(y|ies?)|jigg?aboo?s?|\bpakis?\b|rag\s*heads?|gooks?|cunts?|bitch(es|ing|y)?|puss(y|ies?)|twats?|feminazis?|whor(es?|ing)|\bslut(s|t?y)?|\btrann?(y|ies?)|ladyboy(s?)|\b(b|re|r)tard(ed)?s?)").case_insensitive(true).build().unwrap();
   static ref USERNAME_MATCHES_REGEX: Regex = Regex::new(r"/u/[a-zA-Z][0-9a-zA-Z_]*").unwrap();
-  static ref VALID_USERNAME_REGEX: Regex = Regex::new(r"^[a-zA-Z0-9_]{3,20}$").unwrap();
-  static ref VALID_COMMUNITY_NAME_REGEX: Regex = Regex::new(r"^[a-z0-9_]{3,20}$").unwrap();
 }

server/src/main.rs

@@ -1,40 +1,26 @@
 extern crate lemmy_server;
 #[macro_use]
 extern crate diesel_migrations;
-#[macro_use]
-pub extern crate lazy_static;
 
-use crate::lemmy_server::actix_web::dev::Service;
 use actix::prelude::*;
-use actix_web::body::Body;
-use actix_web::dev::{ServiceRequest, ServiceResponse};
-use actix_web::http::header::CONTENT_TYPE;
-use actix_web::http::{header::CACHE_CONTROL, HeaderValue};
 use actix_web::*;
 use diesel::r2d2::{ConnectionManager, Pool};
 use diesel::PgConnection;
+use failure::Error;
 use lemmy_server::{
+  db::code_migrations::run_advanced_migrations,
   rate_limit::{rate_limiter::RateLimiter, RateLimit},
   routes::{api, federation, feeds, index, nodeinfo, webfinger},
   settings::Settings,
   websocket::server::*,
 };
-use regex::Regex;
-use std::{io, sync::Arc};
+use std::sync::Arc;
 use tokio::sync::Mutex;
 
-lazy_static! {
-  static ref CACHE_CONTROL_REGEX: Regex =
-    Regex::new("^((text|image)/.+|application/javascript)$").unwrap();
-  // static ref CACHE_CONTROL_VALUE: String = format!("public, max-age={}", 365 * 24 * 60 * 60);
-  // Test out 1 hour here, this is breaking some things
-  static ref CACHE_CONTROL_VALUE: String = format!("public, max-age={}", 60 * 60);
-}
-
 embed_migrations!();
 
 #[actix_rt::main]
-async fn main() -> io::Result<()> {
+async fn main() -> Result<(), Error> {
   env_logger::init();
   let settings = Settings::get();
 
@@ -48,6 +34,7 @@ async fn main() -> io::Result<()> {
   // Run the migrations from code
   let conn = pool.get().unwrap();
   embedded_migrations::run(&conn).unwrap();
+  run_advanced_migrations(&conn).unwrap();
 
   // Set up the rate limiter
   let rate_limiter = RateLimit {
@@ -63,52 +50,33 @@ async fn main() -> io::Result<()> {
   );
 
   // Create Http server with websocket support
-  HttpServer::new(move || {
-    let settings = Settings::get();
-    let rate_limiter = rate_limiter.clone();
-    App::new()
-      .wrap_fn(add_cache_headers)
-      .wrap(middleware::Logger::default())
-      .data(pool.clone())
-      .data(server.clone())
-      // The routes
-      .configure(move |cfg| api::config(cfg, &rate_limiter))
-      .configure(federation::config)
-      .configure(feeds::config)
-      .configure(index::config)
-      .configure(nodeinfo::config)
-      .configure(webfinger::config)
-      // static files
-      .service(actix_files::Files::new(
-        "/static",
-        settings.front_end_dir.to_owned(),
-      ))
-      .service(actix_files::Files::new(
-        "/docs",
-        settings.front_end_dir + "/documentation",
-      ))
-  })
-  .bind((settings.bind, settings.port))?
-  .run()
-  .await
-}
-
-fn add_cache_headers<S>(
-  req: ServiceRequest,
-  srv: &mut S,
-) -> impl Future<Output = Result<ServiceResponse, Error>>
-where
-  S: Service<Request = ServiceRequest, Response = ServiceResponse<Body>, Error = Error>,
-{
-  let fut = srv.call(req);
-  async move {
-    let mut res = fut.await?;
-    if let Some(content_type) = res.headers().get(CONTENT_TYPE) {
-      if CACHE_CONTROL_REGEX.is_match(content_type.to_str().unwrap()) {
-        let header_val = HeaderValue::from_static(&CACHE_CONTROL_VALUE);
-        res.headers_mut().insert(CACHE_CONTROL, header_val);
-      }
-    }
-    Ok(res)
-  }
+  Ok(
+    HttpServer::new(move || {
+      let settings = Settings::get();
+      let rate_limiter = rate_limiter.clone();
+      App::new()
+        .wrap(middleware::Logger::default())
+        .data(pool.clone())
+        .data(server.clone())
+        // The routes
+        .configure(move |cfg| api::config(cfg, &rate_limiter))
+        .configure(federation::config)
+        .configure(feeds::config)
+        .configure(index::config)
+        .configure(nodeinfo::config)
+        .configure(webfinger::config)
+        // static files
+        .service(actix_files::Files::new(
+          "/static",
+          settings.front_end_dir.to_owned(),
+        ))
+        .service(actix_files::Files::new(
+          "/docs",
+          settings.front_end_dir + "/documentation",
+        ))
+    })
+    .bind((settings.bind, settings.port))?
+    .run()
+    .await?,
+  )
 }

server/src/rate_limit/mod.rs

@@ -92,7 +92,7 @@ impl RateLimited {
         }
         RateLimitType::Post => {
           limiter.check_rate_limit_full(
-            self.type_,
+            self.type_.clone(),
             &ip_addr,
             rate_limit.post,
             rate_limit.post_per_second,