LemmyNet
/
lemmy
3
2
Fork 3
Code Pull Requests 1 Releases Wiki Activity

Implement webfinger (fixes #149) #4

Closed
nutomic wants to merge 0 commits from nutomic:webfinger into master
pull from: nutomic:webfinger
merge into: LemmyNet:master
Conversation 1 Commits - Files Changed 308 +12554 -46311
308 changed files with 12554 additions and 46311 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
.dockerignore vendored
View File

@ -1,12 +1,5 @@
# build folders and similar which are not needed for the docker build
ui/node_modules
ui/dist
server/target
docker/dev/volumes
docker/federation/volumes
docker/federation-test/volumes
docs
.git
ansible
# exceptions, needed for federation-test build
!server/target/debug/lemmy_server

1
.github/FUNDING.yml vendored
View File

@ -1,4 +1,3 @@
# These are supported funding model platforms
patreon: dessalines
liberapay: Lemmy

27
.github/ISSUE_TEMPLATE/BUG_REPORT.md vendored
View File

@ -1,27 +0,0 @@
---
name: "\U0001F41E Bug Report"
about: Create a report to help us improve Lemmy
title: ''
labels: bug
assignees: ''
---
Found a bug? Please fill out the sections below. 👍
### Issue Summary
A summary of the bug.
### Steps to Reproduce
1. (for example) I clicked login, and an endless spinner show up.
2. I tried to install lemmy via this guide, and I'm getting this error.
3. ...
### Technical details
* Please post your log: `sudo docker-compose logs > lemmy_log.out`.
* What OS are you trying to install lemmy on?
* Any browser console errors?

42
.github/ISSUE_TEMPLATE/FEATURE_REQUEST.md vendored
View File

@ -1,42 +0,0 @@
---
name: "\U0001F680 Feature request"
about: Suggest an idea for improving Lemmy
title: ''
labels: enhancement
assignees: ''
---
### Is your proposal related to a problem?
<!--
Provide a clear and concise description of what the problem is.
For example, "I'm always frustrated when..."
-->
(Write your answer here.)
### Describe the solution you'd like
<!--
Provide a clear and concise description of what you want to happen.
-->
(Describe your proposed solution here.)
### Describe alternatives you've considered
<!--
Let us know about other solutions you've tried or researched.
-->
(Write your answer here.)
### Additional context
<!--
Is there anything else you can add about the proposal?
You might want to link to related issues here, if you haven't already.
-->
(Write your answer here.)

10
.github/ISSUE_TEMPLATE/QUESTION.md vendored
View File

@ -1,10 +0,0 @@
---
name: "? Question"
about: General questions about Lemmy
title: ''
labels: question
assignees: ''
---
What's the question you have about lemmy?

15
.gitignore vendored
View File

@ -1,18 +1,5 @@
# local ansible configuration
ansible/inventory
ansible/inventory_dev
ansible/passwords/
# docker build files
docker/lemmy_mine.hjson
docker/dev/env_deploy.sh
docker/federation/volumes
docker/federation-test/volumes
docker/dev/volumes
# local build files
build/
ui/src/translations
# ide config
.idea/
docker/dev/config/config.hjson

28
.travis.yml vendored
View File

@ -5,31 +5,21 @@ matrix:
allow_failures:
- rust: nightly
fast_finish: true
cache: cargo
cache:
directories:
- /home/travis/.cargo
before_cache:
- rm -rfv target/debug/incremental/lemmy_server-*
- rm -rfv target/debug/.fingerprint/lemmy_server-*
- rm -rfv target/debug/build/lemmy_server-*
- rm -rfv target/debug/deps/lemmy_server-*
- rm -rfv target/debug/lemmy_server.d
- rm -rf /home/travis/.cargo/registry
before_script:
- psql -c "create user lemmy with password 'password' superuser;" -U postgres
- psql -c 'create database lemmy with owner lemmy;' -U postgres
- rustup component add clippy --toolchain stable-x86_64-unknown-linux-gnu
- psql -c "create user rrr with password 'rrr' superuser;" -U postgres
- psql -c 'create database rrr with owner rrr;' -U postgres
before_install:
- cd server
script:
# Default checks, but fail if anything is detected
- cargo build
- cargo clippy -- -D clippy::style -D clippy::correctness -D clippy::complexity -D clippy::perf
- cargo install diesel_cli --no-default-features --features postgres --force
- diesel migration run
- cargo test --workspace
- cargo build
- cargo test
env:
global:
- DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
- LEMMY_DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
- RUST_TEST_THREADS=1
- DATABASE_URL=postgres://rrr:rrr@localhost/rrr
addons:
postgresql: "9.4"

35
CODE_OF_CONDUCT.md vendored
View File

@ -1,35 +0,0 @@
# Code of Conduct
- We are committed to providing a friendly, safe and welcoming environment for all, regardless of level of experience, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, religion, nationality, or other similar characteristic.
- Please avoid using overtly sexual aliases or other nicknames that might detract from a friendly, safe and welcoming environment for all.
- Please be kind and courteous. There’s no need to be mean or rude.
- Respect that people have differences of opinion and that every design or implementation choice carries a trade-off and numerous costs. There is seldom a right answer.
- Please keep unstructured critique to a minimum. If you have solid ideas you want to experiment with, make a fork and see how it works.
- We will exclude you from interaction if you insult, demean or harass anyone. That is not welcome behavior. We interpret the term “harassment” as including the definition in the Citizen Code of Conduct; if you have any lack of clarity about what might be included in that concept, please read their definition. In particular, we don’t tolerate behavior that excludes people in socially marginalized groups.
- Private harassment is also unacceptable. No matter who you are, if you feel you have been or are being harassed or made uncomfortable by a community member, please contact one of the channel ops or any of the Lemmy moderation team immediately. Whether you’re a regular contributor or a newcomer, we care about making this community a safe place for you and we’ve got your back.
- Likewise any spamming, trolling, flaming, baiting or other attention-stealing behavior is not welcome.
[**Message the Moderation Team on Mastodon**](https://mastodon.social/@LemmyDev)
[**Email The Moderation Team**](mailto:contact@lemmy.ml)
## Moderation
These are the policies for upholding our community’s standards of conduct. If you feel that a thread needs moderation, please contact the Lemmy moderation team .
1. Remarks that violate the Lemmy standards of conduct, including hateful, hurtful, oppressive, or exclusionary remarks, are not allowed. (Cursing is allowed, but never targeting another user, and never in a hateful manner.)
2. Remarks that moderators find inappropriate, whether listed in the code of conduct or not, are also not allowed.
3. Moderators will first respond to such remarks with a warning, at the same time the offending content will likely be removed whenever possible.
4. If the warning is unheeded, the user will be “kicked,” i.e., kicked out of the communication channel to cool off.
5. If the user comes back and continues to make trouble, they will be banned, i.e., indefinitely excluded.
6. Moderators may choose at their discretion to un-ban the user if it was a first offense and they offer the offended party a genuine apology.
7. If a moderator bans someone and you think it was unjustified, please take it up with that moderator, or with a different moderator, in private. Complaints about bans in-channel are not allowed.
8. Moderators are held to a higher standard than other community members. If a moderator creates an inappropriate situation, they should expect less leeway than others.
In the Lemmy community we strive to go the extra step to look out for each other. Don’t just aim to be technically unimpeachable, try to be your best self. In particular, avoid flirting with offensive or sensitive issues, particularly if they’re off-topic; this all too often leads to unnecessary fights, hurt feelings, and damaged trust; worse, it can drive people away from the community entirely.
And if someone takes issue with something you said or did, resist the urge to be defensive. Just stop doing what it was they complained about and apologize. Even if you feel you were misinterpreted or unfairly accused, chances are good there was something you could’ve communicated better — remember that it’s your responsibility to make others comfortable. Everyone wants to get along and we are all here first and foremost because we want to talk about cool technology. You will find that people will be eager to assume good intent and forgive as long as you earn their trust.
The enforcement policies listed above apply to all official Lemmy venues; including git repositories under [github.com/LemmyNet/lemmy](https://github.com/LemmyNet/lemmy) and [yerbamate.dev/LemmyNet/lemmy](https://yerbamate.dev/LemmyNet/lemmy), the [Matrix channel](https://matrix.to/#/!BZVTUuEiNmRcbFeLeI:matrix.org?via=matrix.org&via=privacytools.io&via=permaweb.io); and all instances under lemmy.ml. For other projects adopting the Rust Code of Conduct, please contact the maintainers of those projects for enforcement. If you wish to use this code of conduct for your own project, consider explicitly mentioning your moderation policy or making a copy with your own moderation policy so as to avoid confusion.
Adapted from the [Rust Code of Conduct](https://www.rust-lang.org/policies/code-of-conduct), which is based on the [Node.js Policy on Trolling](http://blog.izs.me/post/30036893703/policy-on-trolling) as well as the [Contributor Covenant v1.3.0](https://www.contributor-covenant.org/version/1/3/0/).

4
CONTRIBUTING.md vendored
View File

@ -1,4 +0,0 @@
# Contributing
See [here](https://dev.lemmy.ml/docs/contributing.html) for contributing Instructions.

341
README.md vendored
View File

@ -1,50 +1,105 @@
<div align="center">
![GitHub tag (latest SemVer)](https://img.shields.io/github/tag/LemmyNet/lemmy.svg)
[![Build Status](https://travis-ci.org/LemmyNet/lemmy.svg?branch=master)](https://travis-ci.org/LemmyNet/lemmy)
[![GitHub issues](https://img.shields.io/github/issues-raw/LemmyNet/lemmy.svg)](https://github.com/LemmyNet/lemmy/issues)
[![Docker Pulls](https://img.shields.io/docker/pulls/dessalines/lemmy.svg)](https://cloud.docker.com/repository/docker/dessalines/lemmy/)
[![Translation status](http://weblate.yerbamate.dev/widgets/lemmy/-/lemmy/svg-badge.svg)](http://weblate.yerbamate.dev/engage/lemmy/)
[![License](https://img.shields.io/github/license/LemmyNet/lemmy.svg)](LICENSE)
![GitHub stars](https://img.shields.io/github/stars/LemmyNet/lemmy?style=social)
</div>
<p align="center">
<a href="https://dev.lemmy.ml/" rel="noopener">
<a href="" rel="noopener">
<img width=200px height=200px src="ui/assets/favicon.svg"></a>
<h3 align="center"><a href="https://dev.lemmy.ml">Lemmy</a></h3>
<p align="center">
A link aggregator / Reddit clone for the fediverse.
<br />
<br />
<a href="https://dev.lemmy.ml">View Site</a>
·
<a href="https://dev.lemmy.ml/docs/index.html">Documentation</a>
·
<a href="https://github.com/LemmyNet/lemmy/issues">Report Bug</a>
·
<a href="https://github.com/LemmyNet/lemmy/issues">Request Feature</a>
·
<a href="https://github.com/LemmyNet/lemmy/blob/master/RELEASES.md">Releases</a>
</p>
</p>
## About The Project
<h3 align="center">Lemmy</h3>
<div align="center">
[![Github](https://img.shields.io/badge/-Github-blue)](https://github.com/dessalines/lemmy)
[![Gitlab](https://img.shields.io/badge/-Gitlab-yellowgreen)](https://gitlab.com/dessalines/lemmy)
![Mastodon Follow](https://img.shields.io/mastodon/follow/810572?domain=https%3A%2F%2Fmastodon.social&style=social)
![GitHub stars](https://img.shields.io/github/stars/dessalines/lemmy?style=social)
[![Matrix](https://img.shields.io/matrix/rust-reddit-fediverse:matrix.org.svg?label=matrix-chat)](https://riot.im/app/#/room/#rust-reddit-fediverse:matrix.org)
![GitHub tag (latest SemVer)](https://img.shields.io/github/tag/dessalines/lemmy.svg)
[![Build Status](https://travis-ci.org/dessalines/lemmy.svg?branch=master)](https://travis-ci.org/dessalines/lemmy)
[![GitHub issues](https://img.shields.io/github/issues-raw/dessalines/lemmy.svg)](https://github.com/dessalines/lemmy/issues)
[![Docker Pulls](https://img.shields.io/docker/pulls/dessalines/lemmy.svg)](https://cloud.docker.com/repository/docker/dessalines/lemmy/)
![GitHub commit activity](https://img.shields.io/github/commit-activity/m/dessalines/lemmy.svg)
![GitHub repo size](https://img.shields.io/github/repo-size/dessalines/lemmy.svg)
[![License](https://img.shields.io/github/license/dessalines/lemmy.svg)](LICENSE)
[![Patreon](https://img.shields.io/badge/-Support%20on%20Patreon-blueviolet.svg)](https://www.patreon.com/dessalines)
</div>
---
<p align="center">A link aggregator / reddit clone for the fediverse.
<br>
</p>
[Lemmy Dev instance](https://dev.lemmy.ml) *for testing purposes only*
This is a **very early beta version**, and a lot of features are currently broken or in active development, such as federation.
Front Page|Post
---|---
![main screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/main_screen.png)|![chat screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/chat_screen.png)
![main screen](https://i.imgur.com/kZSRcRu.png)|![chat screen](https://i.imgur.com/4XghNh6.png)
[Lemmy](https://github.com/LemmyNet/lemmy) is similar to sites like [Reddit](https://reddit.com), [Lobste.rs](https://lobste.rs), [Raddle](https://raddle.me), or [Hacker News](https://news.ycombinator.com/): you subscribe to forums you're interested in, post links and discussions, then vote, and comment on them. Behind the scenes, it is very different; anyone can easily run a server, and all these servers are federated (think email), and connected to the same universe, called the [Fediverse](https://en.wikipedia.org/wiki/Fediverse).
## 📝 Table of Contents
<!-- toc -->
- [Features](#features)
- [About](#about)
* [Why's it called Lemmy?](#whys-it-called-lemmy)
- [Install](#install)
* [Docker](#docker)
+ [Updating](#updating)
* [Ansible](#ansible)
* [Kubernetes](#kubernetes)
- [Develop](#develop)
* [Docker Development](#docker-development)
* [Local Development](#local-development)
+ [Requirements](#requirements)
+ [Set up Postgres DB](#set-up-postgres-db)
+ [Running](#running)
- [Configuration](#configuration)
- [Documentation](#documentation)
- [Support](#support)
- [Translations](#translations)
- [Credits](#credits)
<!-- tocstop -->
## Features
- Open source, [AGPL License](/LICENSE).
- Self hostable, easy to deploy.
- Comes with [Docker](#docker), [Ansible](#ansible), [Kubernetes](#kubernetes).
- Clean, mobile-friendly interface.
- Live-updating Comment threads.
- Full vote scores `(+/-)` like old reddit.
- Themes, including light, dark, and solarized.
- Emojis with autocomplete support. Start typing `:`
- User tagging using `@`, Community tagging using `#`.
- Notifications, on comment replies and when you're tagged.
- i18n / internationalization support.
- RSS / Atom feeds for `All`, `Subscribed`, `Inbox`, `User`, and `Community`.
- Cross-posting support.
- A *similar post search* when creating new posts. Great for question / answer communities.
- Moderation abilities.
- Public Moderation Logs.
- Both site admins, and community moderators, who can appoint other moderators.
- Can lock, remove, and restore posts and comments.
- Can ban and unban users from communities and the site.
- Can transfer site and communities to others.
- Can fully erase your data, replacing all posts and comments.
- NSFW post / community support.
- High performance.
- Server is written in rust.
- Front end is `~80kB` gzipped.
- Supports arm64 / Raspberry Pi.
## About
[Lemmy](https://github.com/dessalines/lemmy) is similar to sites like [Reddit](https://reddit.com), [Lobste.rs](https://lobste.rs), [Raddle](https://raddle.me), or [Hacker News](https://news.ycombinator.com/): you subscribe to forums you're interested in, post links and discussions, then vote, and comment on them. Behind the scenes, it is very different; anyone can easily run a server, and all these servers are federated (think email), and connected to the same universe, called the [Fediverse](https://en.wikipedia.org/wiki/Fediverse).
For a link aggregator, this means a user registered on one server can subscribe to forums on any other server, and can have discussions with users registered elsewhere.
The overall goal is to create an easily self-hostable, decentralized alternative to Reddit and other link aggregators, outside of their corporate control and meddling.
The overall goal is to create an easily self-hostable, decentralized alternative to reddit and other link aggregators, outside of their corporate control and meddling.
Each Lemmy server can set its own moderation policy; appointing site-wide admins, and community moderators to keep out the trolls, and foster a healthy, non-toxic environment where all can feel comfortable contributing.
*Note: Federation is still in active development and the WebSocket, as well as, HTTP API are currently unstable*
Each lemmy server can set its own moderation policy; appointing site-wide admins, and community moderators to keep out the trolls, and foster a healthy, non-toxic environment where all can feel comfortable contributing.
### Why's it called Lemmy?
@ -53,92 +108,176 @@ Each Lemmy server can set its own moderation policy; appointing site-wide admins
- The [Koopa from Super Mario](https://www.mariowiki.com/Lemmy_Koopa).
- The [furry rodents](http://sunchild.fpwc.org/lemming-the-little-giant-of-the-north/).
### Built With
Made with [Rust](https://www.rust-lang.org), [Actix](https://actix.rs/), [Inferno](https://www.infernojs.org), [Typescript](https://www.typescriptlang.org/) and [Diesel](http://diesel.rs/).
- [Rust](https://www.rust-lang.org)
- [Actix](https://actix.rs/)
- [Diesel](http://diesel.rs/)
- [Inferno](https://infernojs.org)
- [Typescript](https://www.typescriptlang.org/)
## Install
## Features
### Docker
- Open source, [AGPL License](/LICENSE).
- Self hostable, easy to deploy.
- Comes with [Docker](#docker), [Ansible](#ansible), [Kubernetes](#kubernetes).
- Clean, mobile-friendly interface.
- Only a minimum of a username and password is required to sign up!
- User avatar support.
- Live-updating Comment threads.
- Full vote scores `(+/-)` like old Reddit.
- Themes, including light, dark, and solarized.
- Emojis with autocomplete support. Start typing `:`
- User tagging using `@`, Community tagging using `!`.
- Integrated image uploading in both posts and comments.
- A post can consist of a title and any combination of self text, a URL, or nothing else.
- Notifications, on comment replies and when you're tagged.
- Notifications can be sent via email.
- Private messaging support.
- i18n / internationalization support.
- RSS / Atom feeds for `All`, `Subscribed`, `Inbox`, `User`, and `Community`.
- Cross-posting support.
- A *similar post search* when creating new posts. Great for question / answer communities.
- Moderation abilities.
- Public Moderation Logs.
- Can sticky posts to the top of communities.
- Both site admins, and community moderators, who can appoint other moderators.
- Can lock, remove, and restore posts and comments.
- Can ban and unban users from communities and the site.
- Can transfer site and communities to others.
- Can fully erase your data, replacing all posts and comments.
- NSFW post / community support.
- OEmbed support via Iframely.
- High performance.
- Server is written in rust.
- Front end is `~80kB` gzipped.
- Supports arm64 / Raspberry Pi.
Make sure you have both docker and docker-compose(>=`1.24.0`) installed:
## Installation
```bash
mkdir lemmy/
cd lemmy/
wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/prod/docker-compose.yml
wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/prod/.env
# Edit the .env if you want custom passwords
docker-compose up -d
```
- [Docker](https://dev.lemmy.ml/docs/administration_install_docker.html)
- [Ansible](https://dev.lemmy.ml/docs/administration_install_ansible.html)
- [Kubernetes](https://dev.lemmy.ml/docs/administration_install_kubernetes.html)
and go to http://localhost:8536.
## Support / Donate
[A sample nginx config](/ansible/templates/nginx.conf), could be setup with:
```bash
wget https://raw.githubusercontent.com/dessalines/lemmy/master/ansible/templates/nginx.conf
# Replace the {{ vars }}
sudo mv nginx.conf /etc/nginx/sites-enabled/lemmy.conf
```
#### Updating
To update to the newest version, run:
```bash
wget https://raw.githubusercontent.com/dessalines/lemmy/master/docker/prod/docker-compose.yml
docker-compose up -d
```
### Ansible
First, you need to [install Ansible on your local computer](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) (e.g. using `sudo apt install ansible`) or the equivalent for you platform.
Then run the following commands on your local computer:
```bash
git clone https://github.com/dessalines/lemmy.git
cd lemmy/ansible/
cp inventory.example inventory
nano inventory # enter your server, domain, contact email
ansible-playbook lemmy.yml --become
```
### Kubernetes
You'll need to have an existing Kubernetes cluster and [storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/).
Setting this up will vary depending on your provider.
To try it locally, you can use [MicroK8s](https://microk8s.io/) or [Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/).
Once you have a working cluster, edit the environment variables and volume sizes in `docker/k8s/*.yml`.
You may also want to change the service types to use `LoadBalancer`s depending on where you're running your cluster (add `type: LoadBalancer` to `ports)`, or `NodePort`s.
By default they will use `ClusterIP`s, which will allow access only within the cluster. See the [docs](https://kubernetes.io/docs/concepts/services-networking/service/) for more on networking in Kubernetes.
**Important** Running a database in Kubernetes will work, but is generally not recommended.
If you're deploying on any of the common cloud providers, you should consider using their managed database service instead (RDS, Cloud SQL, Azure Databse, etc.).
Now you can deploy:
```bash
# Add `-n foo` if you want to deploy into a specific namespace `foo`;
# otherwise your resources will be created in the `default` namespace.
kubectl apply -f docker/k8s/db.yml
kubectl apply -f docker/k8s/pictshare.yml
kubectl apply -f docker/k8s/lemmy.yml
```
If you used a `LoadBalancer`, you should see it in your cloud provider's console.
## Develop
### Docker Development
Run:
```bash
git clone https://github.com/dessalines/lemmy
cd lemmy/docker/dev
./docker_update.sh # This builds and runs it, updating for your changes
```
and go to http://localhost:8536.
### Local Development
#### Requirements
- [Rust](https://www.rust-lang.org/)
- [Yarn](https://yarnpkg.com/en/)
- [Postgres](https://www.postgresql.org/)
#### Set up Postgres DB
```bash
psql -c "create user lemmy with password 'password' superuser;" -U postgres
psql -c 'create database lemmy with owner lemmy;' -U postgres
export DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
```
#### Running
```bash
git clone https://github.com/dessalines/lemmy
cd lemmy
./install.sh
# For live coding, where both the front and back end, automagically reload on any save, do:
# cd ui && yarn start
# cd server && cargo watch -x run
```
## Configuration
The configuration is based on the file [defaults.hjson](server/config/defaults.hjson). This file also contains
documentation for all the available options. To override the defaults, you can copy the options you want to change
into your local `config.hjson` file.
Additionally, you can override any config files with environment variables. These have the same name as the config
options, and are prefixed with `LEMMY_`. For example, you can override the `database.password` with
`LEMMY_DATABASE_PASSWORD=my_password`.
An additional option `LEMMY_DATABASE_URL` is available, which can be used with a PostgreSQL connection string like
`postgres://lemmy:password@lemmy_db:5432/lemmy`, passing all connection details at once.
## Documentation
- [Websocket API for App developers](docs/api.md)
- [ActivityPub API.md](docs/apub_api_outline.md)
- [Goals](docs/goals.md)
- [Ranking Algorithm](docs/ranking.md)
## Support
Lemmy is free, open-source software, meaning no advertising, monetizing, or venture capital, ever. Your donations directly support full-time development of the project.
- [Support on Liberapay](https://liberapay.com/Lemmy).
- [Support on Patreon](https://www.patreon.com/dessalines).
- [Support on OpenCollective](https://opencollective.com/lemmy).
- [List of Sponsors](https://dev.lemmy.ml/sponsors).
### Crypto
- [Sponsor List](https://dev.lemmy.ml/sponsors).
- bitcoin: `1Hefs7miXS5ff5Ck5xvmjKjXf5242KzRtK`
- ethereum: `0x400c96c96acbC6E7B3B43B1dc1BB446540a88A01`
- monero: `41taVyY6e1xApqKyMVDRVxJ76sPkfZhALLTjRvVKpaAh2pBd4wv9RgYj1tSPrx8wc6iE1uWUfjtQdTmTy2FGMeChGVKPQuV`
## Contributing
## Translations
- [Contributing instructions](https://dev.lemmy.ml/docs/contributing.html)
- [Docker Development](https://dev.lemmy.ml/docs/contributing_docker_development.html)
- [Local Development](https://dev.lemmy.ml/docs/contributing_local_development.html)
If you'd like to add translations, take a look a look at the [English translation file](ui/src/translations/en.ts).
### Translations
- Languages supported: English (`en`), Chinese (`zh`), Dutch (`nl`), Esperanto (`eo`), French (`fr`), Spanish (`es`), Swedish (`sv`), German (`de`), Russian (`ru`), Italian (`it`).
If you want to help with translating, take a look at [Weblate](https://weblate.yerbamate.dev/projects/lemmy/).
lang | done | missing
--- | --- | ---
de | 100% |
eo | 86% | number_of_communities,preview,upload_image,formatting_help,view_source,sticky,unsticky,archive_link,stickied,delete_account,delete_account_confirm,banned,creator,number_online,replies,mentions,forgot_password,reset_password_mail_sent,password_change,new_password,no_email_setup,language,browser_default,theme,are_you_sure,yes,no
es | 95% | archive_link,replies,mentions,forgot_password,reset_password_mail_sent,password_change,new_password,no_email_setup,language,browser_default
fr | 95% | archive_link,replies,mentions,forgot_password,reset_password_mail_sent,password_change,new_password,no_email_setup,language,browser_default
it | 96% | archive_link,forgot_password,reset_password_mail_sent,password_change,new_password,no_email_setup,language,browser_default
nl | 88% | preview,upload_image,formatting_help,view_source,sticky,unsticky,archive_link,stickied,delete_account,delete_account_confirm,banned,creator,number_online,replies,mentions,forgot_password,reset_password_mail_sent,password_change,new_password,no_email_setup,language,browser_default,theme
ru | 82% | cross_posts,cross_post,number_of_communities,preview,upload_image,formatting_help,view_source,sticky,unsticky,archive_link,stickied,delete_account,delete_account_confirm,banned,creator,number_online,replies,mentions,forgot_password,reset_password_mail_sent,password_change,new_password,no_email_setup,language,browser_default,recent_comments,theme,monero,by,to,transfer_community,transfer_site,are_you_sure,yes,no
sv | 95% | archive_link,replies,mentions,forgot_password,reset_password_mail_sent,password_change,new_password,no_email_setup,language,browser_default
zh | 80% | cross_posts,cross_post,users,number_of_communities,preview,upload_image,formatting_help,view_source,sticky,unsticky,archive_link,settings,stickied,delete_account,delete_account_confirm,banned,creator,number_online,replies,mentions,forgot_password,reset_password_mail_sent,password_change,new_password,no_email_setup,language,browser_default,recent_comments,nsfw,show_nsfw,theme,monero,by,to,transfer_community,transfer_site,are_you_sure,yes,no
## Contact
- [Mastodon](https://mastodon.social/@LemmyDev)
- [Matrix](https://riot.im/app/#/room/#rust-reddit-fediverse:matrix.org)
If you'd like to update this report, run:
## Code Mirrors
- [GitHub](https://github.com/LemmyNet/lemmy)
- [Gitea](https://yerbamate.dev/LemmyNet/lemmy)
- [GitLab](https://gitlab.com/dessalines/lemmy)
```bash
cd ui
ts-node translation_report.ts > tmp # And replace the text above.
```
## Credits

85
RELEASES.md vendored
View File

@ -1,85 +0,0 @@
# Lemmy v0.7.0 Release (2020-06-23)
This release replaces [pictshare](https://github.com/HaschekSolutions/pictshare)
with [pict-rs](https://git.asonix.dog/asonix/pict-rs), which improves performance
and security.
Overall, since our last major release in January (v0.6.0), we have closed over
[100 issues!](https://github.com/LemmyNet/lemmy/milestone/16?closed=1)
- Site-wide list of recent comments
- Reconnecting websockets
- Many more themes, including a default light one.
- Expandable embeds for post links (and thumbnails), from
[iframely](https://github.com/itteco/iframely)
- Better icons
- Emoji autocomplete to post and message bodies, and an Emoji Picker
- Post body now searchable
- Community title and description is now searchable
- Simplified cross-posts
- Better documentation
- LOTS more languages
- Lots of bugs squashed
- And more ...
## Upgrading
Before starting the upgrade, make sure that you have a working backup of your
database and image files. See our
[documentation](https://dev.lemmy.ml/docs/administration_backup_and_restore.html)
for backup instructions.
**With Ansible:**
```
# deploy with ansible from your local lemmy git repo
git pull
cd ansible
ansible-playbook lemmy.yml
# connect via ssh to run the migration script
ssh your-server
cd /lemmy/
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/migrate-pictshare-to-pictrs.bash
chmod +x migrate-pictshare-to-pictrs.bash
sudo ./migrate-pictshare-to-pictrs.bash
```
**With manual Docker installation:**
```
# run these commands on your server
cd /lemmy
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/ansible/templates/nginx.conf
# Replace the {{ vars }}
sudo mv nginx.conf /etc/nginx/sites-enabled/lemmy.conf
sudo nginx -s reload
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/docker-compose.yml
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/migrate-pictshare-to-pictrs.bash
chmod +x migrate-pictshare-to-pictrs.bash
sudo bash migrate-pictshare-to-pictrs.bash
```
**Note:** After upgrading, all users need to reload the page, then logout and
login again, so that images are loaded correctly.
# Lemmy v0.6.0 Release (2020-01-16)
`v0.6.0` is here, and we've closed [41 issues!](https://github.com/LemmyNet/lemmy/milestone/15?closed=1)
This is the biggest release by far:
- Avatars!
- Optional Email notifications for username mentions, post and comment replies.
- Ability to change your password and email address.
- Can set a custom language.
- Lemmy-wide settings to disable downvotes, and close registration.
- A better documentation system, hosted in lemmy itself.
- [Huge DB performance gains](https://github.com/LemmyNet/lemmy/issues/411) (everthing down to < `30ms`) by using materialized views.
- Fixed major issue with similar post URL and title searching.
- Upgraded to Actix `2.0`
- Faster comment / post voting.
- Better small screen support.
- Lots of bug fixes, refactoring of back end code.
Another major announcement is that Lemmy now has another lead developer besides me, [@felix@radical.town](https://radical.town/@felix). Theyve created a better documentation system, implemented RSS feeds, simplified docker and project configs, upgraded actix, working on federation, a whole lot else.
https://dev.lemmy.ml

1
ansible/VERSION vendored
View File

@ -1 +0,0 @@
v0.7.21

3
ansible/ansible.cfg vendored
View File

@ -1,6 +1,5 @@
[defaults]
inventory = inventory
interpreter_python = /usr/bin/python3
inventory=inventory
[ssh_connection]
pipelining = True

10
ansible/inventory.example vendored
View File

@ -1,12 +1,6 @@
[lemmy]
# to get started, copy this file to `inventory` and adjust the values below.
# - `myuser@example.com`: replace with the destination you use to connect to your server via ssh
# - `domain=example.com`: replace `example.com` with your lemmy domain
# - `letsencrypt_contact_email=your@email.com` replace `your@email.com` with your email address,
# to get notifications if your ssl cert expires
# - `lemmy_base_dir=/srv/lemmy`: the location on the server where lemmy can be installed, can be any folder
# if you are upgrading from a previous version, set this to `/lemmy`
myuser@example.com domain=example.com letsencrypt_contact_email=your@email.com lemmy_base_dir=/srv/lemmy
# define the username and hostname that you use for ssh connection, and specify the domain
myuser@example.com domain=example.com letsencrypt_contact_email=your@email.com
[all:vars]
ansible_connection=ssh

96
ansible/lemmy.yml vendored
View File

@ -5,41 +5,18 @@
# https://www.josharcher.uk/code/ansible-python-connection-failure-ubuntu-server-1604/
gather_facts: False
pre_tasks:
- name: check lemmy_base_dir
fail:
msg: "`lemmy_base_dir` is unset. if you are upgrading from an older version, add `lemmy_base_dir=/lemmy` to your inventory file."
when: lemmy_base_dir is not defined
- name: install python for Ansible
# python2-minimal instead of python-minimal for ubuntu 20.04 and up
raw: test -e /usr/bin/python || (apt -y update && apt install -y python-minimal python-setuptools)
args:
executable: /bin/bash
register: output
changed_when: output.stdout != ''
changed_when: output.stdout != ""
- setup: # gather facts
tasks:
- name: install dependencies
apt:
pkg:
- 'nginx'
- 'docker-compose'
- 'docker.io'
- 'certbot'
- name: install certbot-nginx on ubuntu < 20
apt:
pkg:
- 'python-certbot-nginx'
when: ansible_distribution == 'Ubuntu' and ansible_distribution_version is version('20.04', '<')
- name: install certbot-nginx on ubuntu > 20
apt:
pkg:
- 'python3-certbot-nginx'
when: ansible_distribution == 'Ubuntu' and ansible_distribution_version is version('20.04', '>=')
pkg: ['nginx', 'docker-compose', 'docker.io', 'certbot', 'python-certbot-nginx']
- name: request initial letsencrypt certificate
command: certbot certonly --nginx --agree-tos -d '{{ domain }}' -m '{{ letsencrypt_contact_email }}'
@ -47,52 +24,30 @@
creates: '/etc/letsencrypt/live/{{domain}}/privkey.pem'
- name: create lemmy folder
file:
path: '{{item.path}}'
owner: '{{item.owner}}'
state: directory
file: path={{item.path}} state=directory
with_items:
- path: '{{lemmy_base_dir}}'
owner: 'root'
- path: '{{lemmy_base_dir}}/volumes/'
owner: 'root'
- path: '{{lemmy_base_dir}}/volumes/pictrs/'
owner: '991'
- { path: '/lemmy/' }
- { path: '/lemmy/volumes/' }
- block:
- name: add template files
template:
src: '{{item.src}}'
dest: '{{item.dest}}'
mode: '{{item.mode}}'
with_items:
- src: 'templates/docker-compose.yml'
dest: '{{lemmy_base_dir}}/docker-compose.yml'
mode: '0600'
- src: 'templates/nginx.conf'
dest: '/etc/nginx/sites-enabled/lemmy.conf'
mode: '0644'
- src: '../docker/iframely.config.local.js'
dest: '{{lemmy_base_dir}}/iframely.config.local.js'
mode: '0600'
vars:
lemmy_docker_image: "dessalines/lemmy:{{ lookup('file', 'VERSION') }}"
lemmy_port: "8536"
pictshare_port: "8537"
iframely_port: "8538"
- name: add config file (only during initial setup)
template:
src: 'templates/config.hjson'
dest: '{{lemmy_base_dir}}/lemmy.hjson'
mode: '0600'
force: false
owner: '1000'
group: '1000'
- name: add all template files
template: src={{item.src}} dest={{item.dest}}
with_items:
- { src: 'templates/env', dest: '/lemmy/.env' }
- { src: 'templates/config.hjson', dest: '/lemmy/config.hjson' }
- { src: '../docker/prod/docker-compose.yml', dest: '/lemmy/docker-compose.yml' }
- { src: 'templates/nginx.conf', dest: '/etc/nginx/sites-enabled/lemmy.conf' }
vars:
postgres_password: "{{ lookup('password', 'passwords/{{ inventory_hostname }}/postgres chars=ascii_letters,digits') }}"
jwt_password: "{{ lookup('password', 'passwords/{{ inventory_hostname }}/jwt chars=ascii_letters,digits') }}"
- name: set env file permissions
file:
path: "/lemmy/.env"
state: touch
mode: 0600
access_time: preserve
modification_time: preserve
- name: enable and start docker service
systemd:
name: docker
@ -101,17 +56,16 @@
- name: start docker-compose
docker_compose:
project_src: '{{lemmy_base_dir}}'
project_src: /lemmy/
state: present
pull: yes
remove_orphans: yes
- name: reload nginx with new config
shell: nginx -s reload
- name: certbot renewal cronjob
cron:
special_time: daily
name: certbot-renew-lemmy
user: root
job: "certbot certonly --nginx -d '{{ domain }}' --deploy-hook 'nginx -s reload'"
special_time=daily
name=certbot-renew-lemmy
user=root
job="certbot certonly --nginx -d '{{ domain }}' --deploy-hook 'docker-compose -f /peertube/docker-compose.yml exec nginx nginx -s reload'"

141
ansible/lemmy_dev.yml vendored
View File

@ -1,141 +0,0 @@
---
- hosts: all
vars:
lemmy_docker_image: 'lemmy:dev'
# Install python if required
# https://www.josharcher.uk/code/ansible-python-connection-failure-ubuntu-server-1604/
gather_facts: False
pre_tasks:
- name: check lemmy_base_dir
fail:
msg: "`lemmy_base_dir` is unset. if you are upgrading from an older version, add `lemmy_base_dir=/lemmy` to your inventory file."
when: lemmy_base_dir is not defined
- name: install python for Ansible
raw: test -e /usr/bin/python || (apt -y update && apt install -y python-minimal python-setuptools)
args:
executable: /bin/bash
register: output
changed_when: output.stdout != ''
- setup: # gather facts
tasks:
- name: install dependencies
apt:
pkg:
- 'nginx'
- 'docker-compose'
- 'docker.io'
- 'certbot'
- 'python-certbot-nginx'
- name: request initial letsencrypt certificate
command: certbot certonly --nginx --agree-tos -d '{{ domain }}' -m '{{ letsencrypt_contact_email }}'
args:
creates: '/etc/letsencrypt/live/{{domain}}/privkey.pem'
- name: create lemmy folder
file:
path: '{{item.path}}'
owner: '{{item.owner}}'
state: directory
with_items:
- path: '{{lemmy_base_dir}}/lemmy/'
owner: 'root'
- path: '{{lemmy_base_dir}}/volumes/'
owner: 'root'
- path: '{{lemmy_base_dir}}/volumes/pictrs/'
owner: '991'
- block:
- name: add template files
template:
src: '{{item.src}}'
dest: '{{item.dest}}'
mode: '{{item.mode}}'
with_items:
- src: 'templates/docker-compose.yml'
dest: '{{lemmy_base_dir}}/docker-compose.yml'
mode: '0600'
- src: 'templates/nginx.conf'
dest: '/etc/nginx/sites-enabled/lemmy.conf'
mode: '0644'
- src: '../docker/iframely.config.local.js'
dest: '{{lemmy_base_dir}}/iframely.config.local.js'
mode: '0600'
- name: add config file (only during initial setup)
template:
src: 'templates/config.hjson'
dest: '{{lemmy_base_dir}}/lemmy.hjson'
mode: '0600'
force: false
owner: '1000'
group: '1000'
vars:
postgres_password: "{{ lookup('password', 'passwords/{{ inventory_hostname }}/postgres chars=ascii_letters,digits') }}"
jwt_password: "{{ lookup('password', 'passwords/{{ inventory_hostname }}/jwt chars=ascii_letters,digits') }}"
- name: build the dev docker image
local_action: shell cd .. && sudo docker build . -f docker/dev/Dockerfile -t lemmy:dev
register: image_build
- name: find hash of the new docker image
set_fact:
image_hash: "{{ image_build.stdout | regex_search('(?<=Successfully built )[0-9a-f]{12}') }}"
# this does not use become so that the output file is written as non-root user and is easy to delete later
- name: save dev docker image to file
local_action: shell sudo docker save lemmy:dev > lemmy-dev.tar
- name: copy dev docker image to server
copy:
src: lemmy-dev.tar
dest: '{{lemmy_base_dir}}/lemmy-dev.tar'
- name: import docker image
docker_image:
name: lemmy
tag: dev
load_path: '{{lemmy_base_dir}}/lemmy-dev.tar'
source: load
force_source: yes
register: image_import
- name: delete remote image file
file:
path: '{{lemmy_base_dir}}/lemmy-dev.tar'
state: absent
- name: delete local image file
local_action:
module: file
path: lemmy-dev.tar
state: absent
- name: enable and start docker service
systemd:
name: docker
enabled: yes
state: started
# cant pull here because that fails due to lemmy:dev (without dessalines/) not being on docker hub, but that shouldnt
# be a problem for testing
- name: start docker-compose
docker_compose:
project_src: '{{lemmy_base_dir}}'
state: present
recreate: always
remove_orphans: yes
ignore_errors: yes
- name: reload nginx with new config
shell: nginx -s reload
- name: certbot renewal cronjob
cron:
special_time: daily
name: certbot-renew-lemmy
user: root
job: "certbot certonly --nginx -d '{{ domain }}' --deploy-hook 'nginx -s reload'"

21
ansible/templates/config.hjson vendored
View File

@ -1,26 +1,13 @@
{
# for more info about the config, check out the documentation
# https://dev.lemmy.ml/docs/administration_configuration.html
# settings related to the postgresql database
database: {
# password to connect to postgres
password: "{{ postgres_password }}"
# host where postgres is running
host: "postgres"
}
# the domain name of your instance (eg "dev.lemmy.ml")
hostname: "{{ domain }}"
# json web token for authorization between server and client
jwt_secret: "{{ jwt_password }}"
# The location of the frontend
front_end_dir: "/app/dist"
# email sending configuration
email: {
# hostname of the smtp server
smtp_server: "postfix:25"
# address to send emails from, eg "noreply@your-instance.com"
smtp_from_address: "noreply@{{ domain }}"
use_tls: false
smtp_server: "{{ smtp_server }}"
smtp_login: "{{ smtp_login }}"
smtp_password: "{{ smtp_password }}"
smtp_from_address: "{{ smtp_from_address }}"
}
}

49
ansible/templates/docker-compose.yml vendored
View File

@ -1,49 +0,0 @@
version: '3.3'
services:
lemmy:
image: {{ lemmy_docker_image }}
ports:
- "127.0.0.1:8536:8536"
restart: always
environment:
- RUST_LOG=error
volumes:
- ./lemmy.hjson:/config/config.hjson:ro
depends_on:
- postgres
- pictrs
- iframely
postgres:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD={{ postgres_password }}
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres:/var/lib/postgresql/data
restart: always
pictrs:
image: asonix/pictrs:amd64-v0.1.0-r9
user: 991:991
ports:
- "127.0.0.1:8537:8080"
volumes:
- ./volumes/pictrs:/mnt
restart: always
iframely:
image: dogbin/iframely:latest
ports:
- "127.0.0.1:8061:80"
volumes:
- ./iframely.config.local.js:/iframely/config.local.js:ro
restart: always
postfix:
image: mwader/postfix-relay
environment:
- POSTFIX_myhostname={{ domain }}
restart: "always"

2
ansible/templates/env vendored Normal file
View File

@ -0,0 +1,2 @@
DATABASE_PASSWORD={{ postgres_password }}
LEMMY_FRONT_END_DIR=/app/dist

47
ansible/templates/nginx.conf vendored
View File

@ -1,6 +1,3 @@
proxy_cache_path /var/cache/lemmy_frontend levels=1:2 keys_zone=lemmy_frontend_cache:10m max_size=100m use_temp_path=off;
limit_req_zone $binary_remote_addr zone=lemmy_ratelimit:10m rate=1r/s;
server {
listen 80;
server_name {{ domain }};
@ -37,7 +34,7 @@ server {
# It might be nice to compress JSON, but leaving that out to protect against potential
# compression+encryption information leak attacks like BREACH.
gzip on;
gzip_types text/css application/javascript image/svg+xml;
gzip_types text/css application/javascript;
gzip_vary on;
# Only connect to this site via HTTPS for the two years
@ -49,11 +46,8 @@ server {
add_header X-Frame-Options "DENY";
add_header X-XSS-Protection "1; mode=block";
# Upload limit for pictrs
client_max_body_size 20M;
# Rate limit
limit_req zone=lemmy_ratelimit burst=30 nodelay;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://0.0.0.0:8536;
@ -65,37 +59,18 @@ server {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Proxy Cache
proxy_cache lemmy_frontend_cache;
proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
proxy_cache_revalidate on;
proxy_cache_lock on;
proxy_cache_min_uses 5;
}
# Redirect pictshare images to pictrs
location ~ /pictshare/(.*)$ {
return 301 /pictrs/image/$1;
}
# pict-rs images
location /pictrs {
location /pictrs/image {
proxy_pass http://0.0.0.0:8537/image;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Block the import
return 403;
}
location /iframely/ {
proxy_pass http://0.0.0.0:8061/;
location /pictshare/ {
proxy_pass http://0.0.0.0:8537/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
if ($request_uri ~ \.(?:ico|gif|jpe?g|png|webp|bmp|mp4)$) {
add_header Cache-Control "public";
expires max;
}
}
}
@ -110,4 +85,4 @@ map $remote_addr $remote_addr_anon {
}
log_format main '$remote_addr_anon - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" "$http_user_agent"';
access_log /var/log/nginx/access.log main;
access_log /dev/stdout main;

54
ansible/uninstall.yml vendored
View File

@ -1,54 +0,0 @@
---
- hosts: all
vars_prompt:
- name: confirm_uninstall
prompt: "Do you really want to uninstall Lemmy? This will delete all data and can not be reverted [yes/no]"
private: no
- name: delete_certs
prompt: "Delete certificates? Select 'no' if you want to reinstall Lemmy [yes/no]"
private: no
tasks:
- name: end play if no confirmation was given
debug:
msg: "Uninstall cancelled, doing nothing"
when: not confirm_uninstall|bool
- meta: end_play
when: not confirm_uninstall|bool
- name: stop docker-compose
docker_compose:
project_src: '{{lemmy_base_dir}}'
state: absent
- name: delete data
file:
path: '{{item.path}}'
state: absent
with_items:
- path: '{{lemmy_base_dir}}'
- path: '/etc/nginx/sites-enabled/lemmy.conf'
- name: Remove a volume
docker_volume:
name: '{{item.name}}'
state: absent
with_items:
- name: 'lemmy_lemmy_db'
- name: 'lemmy_lemmy_pictshare'
- name: delete entire ecloud folder
file:
path: '/mnt/repo-base/'
state: absent
when: delete_certs|bool
- name: remove certbot cronjob
cron:
name: certbot-renew-lemmy
state: absent

5
docker/dev/.env vendored Normal file
View File

@ -0,0 +1,5 @@
LEMMY_DOMAIN=my_domain
LEMMY_DATABASE_PASSWORD=password
LEMMY_DATABASE_URL=postgres://lemmy:password@lemmy_db:5432/lemmy
LEMMY_JWT_SECRET=changeme
LEMMY_FRONT_END_DIR=/app/dist

29
docker/dev/Dockerfile vendored
View File

@ -10,7 +10,7 @@ RUN yarn install --pure-lockfile
COPY ui /app/ui
RUN yarn build
FROM ekidd/rust-musl-builder:1.42.0-openssl11 as rust
FROM ekidd/rust-musl-builder:1.38.0-openssl11 as rust
# Cache deps
WORKDIR /app
@ -18,33 +18,28 @@ RUN sudo chown -R rust:rust .
RUN USER=root cargo new server
WORKDIR /app/server
COPY server/Cargo.toml server/Cargo.lock ./
COPY server/lemmy_db ./lemmy_db
COPY server/lemmy_utils ./lemmy_utils
RUN sudo chown -R rust:rust .
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
RUN cargo build
RUN find target/debug -type f -name "$(echo "lemmy_server" | tr '-' '_')*" -exec touch -t 200001010000 {} +
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
RUN cargo build --release
RUN rm -f ./target/x86_64-unknown-linux-musl/release/deps/lemmy_server*
COPY server/src ./src/
COPY server/migrations ./migrations/
# Build for debug
RUN cargo build
# Build for release
RUN cargo build --frozen --release
FROM ekidd/rust-musl-builder:1.42.0-openssl11 as docs
WORKDIR /app
COPY docs ./docs
RUN sudo chown -R rust:rust .
RUN mdbook build docs/
# Get diesel-cli on there just in case
# RUN cargo install diesel_cli --no-default-features --features postgres
FROM alpine:3.12
FROM alpine:3.10
# Install libpq for postgres
RUN apk add libpq
# Copy resources
COPY server/config/defaults.hjson /config/defaults.hjson
COPY --from=rust /app/server/target/x86_64-unknown-linux-musl/debug/lemmy_server /app/lemmy
COPY --from=docs /app/docs/book/ /app/dist/documentation/
COPY server/config /config
COPY --from=rust /app/server/target/x86_64-unknown-linux-musl/release/lemmy_server /app/lemmy
COPY --from=node /app/ui/dist /app/dist
RUN addgroup -g 1000 lemmy

79
docker/dev/Dockerfile.aarch64 vendored Normal file
View File

@ -0,0 +1,79 @@
FROM node:10-jessie as node
WORKDIR /app/ui
# Cache deps
COPY ui/package.json ui/yarn.lock ./
RUN yarn install --pure-lockfile
# Build
COPY ui /app/ui
RUN yarn build
# contains qemu-*-static for cross-compilation
FROM multiarch/qemu-user-static as qemu
FROM arm64v8/rust:1.37-buster as rust
COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
#COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
# Install musl
#RUN apt-get update && apt-get install -y mc
#RUN apt-get install -y musl-tools mc
#libpq-dev mc
#RUN rustup target add ${TARGET}
# Cache deps
WORKDIR /app
RUN USER=root cargo new server
WORKDIR /app/server
COPY server/Cargo.toml server/Cargo.lock ./
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
RUN cargo build --release
# RUN cargo build
COPY server/src ./src/
COPY server/migrations ./migrations/
RUN rm -f ./target/release/deps/lemmy_server* ; rm -f ./target/debug/deps/lemmy_server*
# build for release
RUN cargo build --frozen --release
# RUN cargo build --frozen
# Get diesel-cli on there just in case
# RUN cargo install diesel_cli --no-default-features --features postgres
# RUN cp /app/server/target/debug/lemmy_server /app/server/ready
RUN cp /app/server/target/release/lemmy_server /app/server/ready
#FROM alpine:3.10
# debian because build with dynamic linking with debian:buster
FROM arm64v8/debian:buster-slim as lemmy
#COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
# Install libpq for postgres
#RUN apk add libpq
RUN apt-get update && apt-get install -y libpq5
RUN addgroup --gid 1000 lemmy
# for alpine
#RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
# for debian
RUN adduser --disabled-password --shell /bin/sh --uid 1000 --ingroup lemmy lemmy
# Copy resources
COPY server/config /app/config
COPY --from=rust /app/server/ready /app/lemmy
COPY --from=node /app/ui/dist /app/dist
RUN chown lemmy:lemmy /app/lemmy
USER lemmy
EXPOSE 8536
CMD ["/app/lemmy"]

79
docker/dev/Dockerfile.armv7hf vendored Normal file
View File

@ -0,0 +1,79 @@
FROM node:10-jessie as node
WORKDIR /app/ui
# Cache deps
COPY ui/package.json ui/yarn.lock ./
RUN yarn install --pure-lockfile
# Build
COPY ui /app/ui
RUN yarn build
# contains qemu-*-static for cross-compilation
FROM multiarch/qemu-user-static as qemu
FROM arm32v7/rust:1.37-buster as rust
#COPY --from=qemu /usr/bin/qemu-aarch64-static /usr/bin
COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
# Install musl
#RUN apt-get update && apt-get install -y mc
#RUN apt-get install -y musl-tools mc
#libpq-dev mc
#RUN rustup target add ${TARGET}
# Cache deps
WORKDIR /app
RUN USER=root cargo new server
WORKDIR /app/server
COPY server/Cargo.toml server/Cargo.lock ./
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
#RUN cargo build --release
# RUN cargo build
RUN RUSTFLAGS='-Ccodegen-units=1' cargo build
COPY server/src ./src/
COPY server/migrations ./migrations/
RUN rm -f ./target/release/deps/lemmy_server* ; rm -f ./target/debug/deps/lemmy_server*
# build for release
#RUN cargo build --frozen --release
RUN cargo build --frozen
# Get diesel-cli on there just in case
# RUN cargo install diesel_cli --no-default-features --features postgres
RUN cp /app/server/target/debug/lemmy_server /app/server/ready
#RUN cp /app/server/target/release/lemmy_server /app/server/ready
#FROM alpine:3.10
# debian because build with dynamic linking with debian:buster
FROM arm32v7/debian:buster-slim as lemmy
COPY --from=qemu /usr/bin/qemu-arm-static /usr/bin
# Install libpq for postgres
#RUN apk add libpq
RUN apt-get update && apt-get install -y libpq5
RUN addgroup --gid 1000 lemmy
# for alpine
#RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
# for debian
RUN adduser --disabled-password --shell /bin/sh --uid 1000 --ingroup lemmy lemmy
# Copy resources
COPY server/config /config
COPY --from=rust /app/server/ready /app/lemmy
COPY --from=node /app/ui/dist /app/dist
RUN chown lemmy:lemmy /app/lemmy
USER lemmy
EXPOSE 8536
CMD ["/app/lemmy"]

75
docker/dev/Dockerfile.libc vendored Normal file
View File

@ -0,0 +1,75 @@
# can be build on x64, arm32, arm64 platforms
# to build on target platform run
# docker build -f Dockerfile.libc -t dessalines/lemmy:version ../..
#
# to use docker buildx run
# docker buildx build --platform linux/amd64,linux/arm64 -f Dockerfile.libc -t YOURNAME/lemmy --push ../..
FROM node:12-buster as node
# use this if use docker buildx
#FROM --platform=$BUILDPLATFORM node:12-buster as node
WORKDIR /app/ui
# Cache deps
COPY ui/package.json ui/yarn.lock ./
RUN yarn install --pure-lockfile --network-timeout 100000
# Build
COPY ui /app/ui
RUN yarn build
FROM rust:1.37 as rust
# Cache deps
WORKDIR /app
RUN USER=root cargo new server
WORKDIR /app/server
COPY server/Cargo.toml server/Cargo.lock ./
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
#RUN cargo build --release
RUN cargo build && \
rm -f ./target/release/deps/lemmy_server* ; rm -f ./target/debug/deps/lemmy_server*
COPY server/src ./src/
COPY server/migrations ./migrations/
# build for release
# workaround for https://github.com/rust-lang/rust/issues/62896
#RUN RUSTFLAGS='-Ccodegen-units=1' cargo build --release
#RUN cargo build --release --frozen
RUN cargo build --frozen
# Get diesel-cli on there just in case
# RUN cargo install diesel_cli --no-default-features --features postgres
# make result place always the same for lemmy container
#RUN cp /app/server/target/release/lemmy_server /app/server/ready
RUN cp /app/server/target/debug/lemmy_server /app/server/ready
#FROM alpine:3.10
# debian because build with dynamic linking with debian:buster
FROM debian:buster as lemmy
# Install libpq for postgres
#RUN apk add libpq
RUN apt-get update && apt-get install -y libpq5
RUN addgroup --gid 1000 lemmy
# for alpine
#RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
# for debian
RUN adduser --disabled-password --shell /bin/sh --uid 1000 --ingroup lemmy lemmy
# Copy resources
COPY server/config /app/config
COPY --from=rust /app/server/ready /app/lemmy
COPY --from=node /app/ui/dist /app/dist
RUN chown lemmy:lemmy /app/lemmy
USER lemmy
EXPOSE 8536
CMD ["/app/lemmy"]

2
docker/dev/config/config.hjson vendored Normal file
View File

@ -0,0 +1,2 @@
{
}

58
docker/dev/deploy.sh vendored Executable file
View File

@ -0,0 +1,58 @@
#!/bin/sh
git checkout master
# Creating the new tag
new_tag="$1"
git tag $new_tag
# Setting the version on the front end
cd ../../
echo "export let version: string = '$(git describe --tags)';" > "ui/src/version.ts"
git add "ui/src/version.ts"
# Setting the version on the backend
echo "pub const VERSION: &'static str = \"$(git describe --tags)\";" > "server/src/version.rs"
git add "server/src/version.rs"
cd docker/dev
# Changing the docker-compose prod
sed -i "s/dessalines\/lemmy:.*/dessalines\/lemmy:$new_tag/" ../prod/docker-compose.yml
git add ../prod/docker-compose.yml
# The commit
git commit -m"Version $new_tag"
# Registering qemu binaries
docker run --rm --privileged multiarch/qemu-user-static:register --reset
# Rebuilding docker
docker-compose build
docker tag dev_lemmy:latest dessalines/lemmy:x64-$new_tag
docker push dessalines/lemmy:x64-$new_tag
# Build for Raspberry Pi / other archs
# Arm currently not working
# docker build -t lemmy:armv7hf -f Dockerfile.armv7hf ../../
# docker tag lemmy:armv7hf dessalines/lemmy:armv7hf-$new_tag
# docker push dessalines/lemmy:armv7hf-$new_tag
# aarch64
docker build -t lemmy:aarch64 -f Dockerfile.aarch64 ../../
docker tag lemmy:aarch64 dessalines/lemmy:arm64-$new_tag
docker push dessalines/lemmy:arm64-$new_tag
# Creating the manifest for the multi-arch build
docker manifest create dessalines/lemmy:$new_tag \
dessalines/lemmy:x64-$new_tag \
dessalines/lemmy:arm64-$new_tag
docker manifest push dessalines/lemmy:$new_tag
# Push
git push origin $new_tag
git push
# Pushing to any ansible deploys
cd ../../ansible
ansible-playbook lemmy.yml --become

9
docker/dev/dev_deploy.sh vendored Executable file
View File

@ -0,0 +1,9 @@
#!/bin/sh
# Building from the dev branch for dev servers
git checkout dev
# Rebuilding dev docker
docker-compose build
docker tag dev_lemmy:latest dessalines/lemmy:dev
docker push dessalines/lemmy:dev

53
docker/dev/docker-compose.yml vendored
View File

@ -1,48 +1,35 @@
version: '3.3'
services:
lemmy_db:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=${LEMMY_DATABASE_PASSWORD}
- POSTGRES_DB=lemmy
volumes:
- lemmy_db:/var/lib/postgresql/data
restart: always
lemmy:
build:
context: ../../
dockerfile: docker/dev/Dockerfile
ports:
- "127.0.0.1:8536:8536"
env_file:
- .env
restart: always
environment:
- RUST_LOG=debug
volumes:
- ../lemmy.hjson:/config/config.hjson
- ./config/config.hjson:/config/config.hjson:ro
depends_on:
- pictrs
- postgres
- iframely
postgres:
image: postgres:12-alpine
- lemmy_db
lemmy_pictshare:
image: shtripok/pictshare:latest
ports:
- "127.0.0.1:5432:5432"
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
- "127.0.0.1:8537:80"
volumes:
- ./volumes/postgres:/var/lib/postgresql/data
restart: always
pictrs:
image: asonix/pictrs:v0.1.13-r0
ports:
- "127.0.0.1:8537:8080"
user: 991:991
volumes:
- ./volumes/pictrs:/mnt
restart: always
iframely:
image: dogbin/iframely:latest
ports:
- "127.0.0.1:8061:80"
volumes:
- ../iframely.config.local.js:/iframely/config.local.js:ro
- lemmy_pictshare:/usr/share/nginx/html/data
restart: always
volumes:
lemmy_db:
lemmy_pictshare:

4
docker/dev/docker_update.sh vendored
View File

@ -1,6 +1,2 @@
#!/bin/sh
set -e
export COMPOSE_DOCKER_CLI_BUILD=1
export DOCKER_BUILDKIT=1
docker-compose up -d --no-deps --build

18
docker/dev/test_deploy.sh vendored
View File

@ -1,18 +0,0 @@
#!/bin/bash
set -e
BRANCH=$1
git checkout $BRANCH
export COMPOSE_DOCKER_CLI_BUILD=1
export DOCKER_BUILDKIT=1
# Rebuilding dev docker
sudo docker build ../../ -f . -t "dessalines/lemmy:$BRANCH"
sudo docker push "dessalines/lemmy:$BRANCH"
# Run the playbook
pushd ../../../lemmy-ansible
ansible-playbook -i test playbooks/site.yml
popd

36
docker/federation-test/run-tests.sh vendored
View File

@ -1,36 +0,0 @@
#!/bin/bash
set -e
# make sure there are no old containers or old data around
sudo docker-compose --file ../federation/docker-compose.yml --project-directory . down
sudo rm -rf volumes
pushd ../../server/
cargo build
popd
pushd ../../ui
yarn
popd
mkdir -p volumes/pictrs_{alpha,beta,gamma}
sudo chown -R 991:991 volumes/pictrs_{alpha,beta,gamma}
sudo docker build ../../ --file ../federation/Dockerfile --tag lemmy-federation:latest
sudo mkdir -p volumes/pictrs_alpha
sudo chown -R 991:991 volumes/pictrs_alpha
sudo docker-compose --file ../federation/docker-compose.yml --project-directory . up -d
pushd ../../ui
echo "Waiting for Lemmy to start..."
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8540/api/v1/site')" != "200" ]]; do sleep 1; done
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8550/api/v1/site')" != "200" ]]; do sleep 1; done
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8560/api/v1/site')" != "200" ]]; do sleep 1; done
yarn api-test || true
popd
sudo docker-compose --file ../federation/docker-compose.yml --project-directory . down
sudo rm -r volumes/

19
docker/federation-test/servers.sh vendored
View File

@ -1,19 +0,0 @@
#!/bin/bash
set -e
sudo rm -rf volumes
pushd ../../server/
cargo build
popd
pushd ../../ui
yarn
popd
mkdir -p volumes/pictrs_{alpha,beta,gamma}
sudo chown -R 991:991 volumes/pictrs_{alpha,beta,gamma}
sudo docker build ../../ --file ../federation/Dockerfile --tag lemmy-federation:latest
sudo docker-compose --file ../federation/docker-compose.yml --project-directory . up

10
docker/federation-test/tests.sh vendored
View File

@ -1,10 +0,0 @@
#!/bin/bash
set -xe
pushd ../../ui
echo "Waiting for Lemmy to start..."
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8540/api/v1/site')" != "200" ]]; do sleep 1; done
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8550/api/v1/site')" != "200" ]]; do sleep 1; done
while [[ "$(curl -s -o /dev/null -w '%{http_code}' 'localhost:8560/api/v1/site')" != "200" ]]; do sleep 1; done
yarn api-test || true
popd

17
docker/federation/Dockerfile vendored
View File

@ -1,17 +0,0 @@
FROM ekidd/rust-musl-builder:1.42.0-openssl11
USER root
RUN mkdir /app/dist/documentation/ -p \
&& addgroup --gid 1001 lemmy \
&& adduser --gecos "" --disabled-password --shell /bin/sh -u 1001 --ingroup lemmy lemmy
# Copy resources
COPY server/config/defaults.hjson /app/config/defaults.hjson
COPY ui/dist /app/dist
COPY server/target/debug/lemmy_server /app/lemmy
RUN chown lemmy:lemmy /app/ -R
USER lemmy
EXPOSE 8536
WORKDIR /app
CMD ["/app/lemmy"]

113
docker/federation/docker-compose.yml vendored
View File

@ -1,113 +0,0 @@
version: '3.3'
services:
nginx:
image: nginx:1.17-alpine
ports:
- "8540:8540"
- "8550:8550"
- "8560:8560"
volumes:
# Hack to make this work from both docker/federation/ and docker/federation-test/
- ../federation/nginx.conf:/etc/nginx/nginx.conf
restart: on-failure
depends_on:
- lemmy-alpha
- pictrs
- lemmy-beta
- lemmy-gamma
- iframely
pictrs:
restart: always
image: asonix/pictrs:v0.1.13-r0
user: 991:991
volumes:
- ./volumes/pictrs_alpha:/mnt
lemmy-alpha:
image: lemmy-federation:latest
environment:
- LEMMY_HOSTNAME=lemmy-alpha:8540
- LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_alpha:5432/lemmy
- LEMMY_JWT_SECRET=changeme
- LEMMY_FRONT_END_DIR=/app/dist
- LEMMY_FEDERATION__ENABLED=true
- LEMMY_FEDERATION__TLS_ENABLED=false
- LEMMY_FEDERATION__ALLOWED_INSTANCES=lemmy-beta,lemmy-gamma
- LEMMY_PORT=8540
- LEMMY_SETUP__ADMIN_USERNAME=lemmy_alpha
- LEMMY_SETUP__ADMIN_PASSWORD=lemmy
- LEMMY_SETUP__SITE_NAME=lemmy-alpha
- RUST_BACKTRACE=1
- RUST_LOG=debug
depends_on:
- postgres_alpha
postgres_alpha:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres_alpha:/var/lib/postgresql/data
lemmy-beta:
image: lemmy-federation:latest
environment:
- LEMMY_HOSTNAME=lemmy-beta:8550
- LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_beta:5432/lemmy
- LEMMY_JWT_SECRET=changeme
- LEMMY_FRONT_END_DIR=/app/dist
- LEMMY_FEDERATION__ENABLED=true
- LEMMY_FEDERATION__TLS_ENABLED=false
- LEMMY_FEDERATION__ALLOWED_INSTANCES=lemmy-alpha,lemmy-gamma
- LEMMY_PORT=8550
- LEMMY_SETUP__ADMIN_USERNAME=lemmy_beta
- LEMMY_SETUP__ADMIN_PASSWORD=lemmy
- LEMMY_SETUP__SITE_NAME=lemmy-beta
- RUST_BACKTRACE=1
- RUST_LOG=debug
depends_on:
- postgres_beta
postgres_beta:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres_beta:/var/lib/postgresql/data
lemmy-gamma:
image: lemmy-federation:latest
environment:
- LEMMY_HOSTNAME=lemmy-gamma:8560
- LEMMY_DATABASE_URL=postgres://lemmy:password@postgres_gamma:5432/lemmy
- LEMMY_JWT_SECRET=changeme
- LEMMY_FRONT_END_DIR=/app/dist
- LEMMY_FEDERATION__ENABLED=true
- LEMMY_FEDERATION__TLS_ENABLED=false
- LEMMY_FEDERATION__ALLOWED_INSTANCES=lemmy-alpha,lemmy-beta
- LEMMY_PORT=8560
- LEMMY_SETUP__ADMIN_USERNAME=lemmy_gamma
- LEMMY_SETUP__ADMIN_PASSWORD=lemmy
- LEMMY_SETUP__SITE_NAME=lemmy-gamma
- RUST_BACKTRACE=1
- RUST_LOG=debug
depends_on:
- postgres_gamma
postgres_gamma:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres_gamma:/var/lib/postgresql/data
iframely:
image: dogbin/iframely:latest
volumes:
- ../iframely.config.local.js:/iframely/config.local.js:ro
restart: always

125
docker/federation/nginx.conf vendored
View File

@ -1,125 +0,0 @@
events {
worker_connections 1024;
}
http {
server {
listen 8540;
server_name 127.0.0.1;
access_log off;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://lemmy-alpha:8540;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# pict-rs images
location /pictrs {
location /pictrs/image {
proxy_pass http://pictrs:8080/image;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Block the import
return 403;
}
location /iframely/ {
proxy_pass http://iframely:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 8550;
server_name 127.0.0.1;
access_log off;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://lemmy-beta:8550;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# pict-rs images
location /pictrs {
location /pictrs/image {
proxy_pass http://pictrs:8080/image;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Block the import
return 403;
}
location /iframely/ {
proxy_pass http://iframely:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 8560;
server_name 127.0.0.1;
access_log off;
# Upload limit for pictshare
client_max_body_size 50M;
location / {
proxy_pass http://lemmy-gamma:8560;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# WebSocket support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# pict-rs images
location /pictrs {
location /pictrs/image {
proxy_pass http://pictrs:8080/image;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Block the import
return 403;
}
location /iframely/ {
proxy_pass http://iframely:80/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}

28
docker/federation/run-federation-test.bash vendored
View File

@ -1,28 +0,0 @@
#!/bin/bash
set -e
# already start rust build in the background
pushd ../../server/ || exit
cargo build &
popd || exit
if [ "$1" = "-yarn" ]; then
pushd ../../ui/ || exit
yarn
yarn build
popd || exit
fi
# wait for rust build to finish
pushd ../../server/ || exit
cargo build
popd || exit
sudo docker build ../../ --file Dockerfile -t lemmy-federation:latest
for Item in alpha beta gamma ; do
sudo mkdir -p volumes/pictrs_$Item
sudo chown -R 991:991 volumes/pictrs_$Item
done
sudo docker-compose up

283
docker/iframely.config.local.js vendored
View File

@ -1,283 +0,0 @@
(function() {
var config = {
// Specify a path for custom plugins. Custom plugins will override core plugins.
// CUSTOM_PLUGINS_PATH: __dirname + '/yourcustom-plugin-folder',
DEBUG: false,
RICH_LOG_ENABLED: false,
// For embeds that require render, baseAppUrl will be used as the host.
baseAppUrl: "http://yourdomain.com",
relativeStaticUrl: "/r",
// Or just skip built-in renders altogether
SKIP_IFRAMELY_RENDERS: true,
// For legacy reasons the response format of Iframely open-source is
// different by default as it does not group the links array by rel.
// In order to get the same grouped response as in Cloud API,
// add `&group=true` to your request to change response per request
// or set `GROUP_LINKS` in your config to `true` for a global change.
GROUP_LINKS: true,
// Number of maximum redirects to follow before aborting the page
// request with `redirect loop` error.
MAX_REDIRECTS: 4,
SKIP_OEMBED_RE_LIST: [
// /^https?:\/\/yourdomain\.com\//,
],
/*
// Used to pass parameters to the generate functions when creating HTML elements
// disableSizeWrapper: Don't wrap element (iframe, video, etc) in a positioned div
GENERATE_LINK_PARAMS: {
disableSizeWrapper: true
},
*/
port: 80, //can be overridden by PORT env var
host: '0.0.0.0', // Dockers beware. See https://github.com/itteco/iframely/issues/132#issuecomment-242991246
//can be overridden by HOST env var
// Optional SSL cert, if you serve under HTTPS.
/*
ssl: {
key: require('fs').readFileSync(__dirname + '/key.pem'),
cert: require('fs').readFileSync(__dirname + '/cert.pem'),
port: 443
},
*/
/*
Supported cache engines:
- no-cache - no caching will be used.
- node-cache - good for debug, node memory will be used (https://github.com/tcs-de/nodecache).
- redis - https://github.com/mranney/node_redis.
- memcached - https://github.com/3rd-Eden/node-memcached
*/
CACHE_ENGINE: 'node-cache',
CACHE_TTL: 0, // In seconds.
// 0 = 'never expire' for memcached & node-cache to let cache engine decide itself when to evict the record
// 0 = 'no cache' for redis. Use high enough (e.g. 365*24*60*60*1000) ttl for similar 'never expire' approach instead
/*
// Redis cache options.
REDIS_OPTIONS: {
host: '127.0.0.1',
port: 6379
},
*/
/*
// Memcached options. See https://github.com/3rd-Eden/node-memcached#server-locations
MEMCACHED_OPTIONS: {
locations: "127.0.0.1:11211"
}
*/
/*
// Access-Control-Allow-Origin list.
allowedOrigins: [
"*",
"http://another_domain.com"
],
*/
/*
// Uncomment to enable plugin testing framework.
tests: {
mongodb: 'mongodb://localhost:27017/iframely-tests',
single_test_timeout: 10 * 1000,
plugin_test_period: 2 * 60 * 60 * 1000,
relaunch_script_period: 5 * 60 * 1000
},
*/
// If there's no response from remote server, the timeout will occur after
RESPONSE_TIMEOUT: 5 * 1000, //ms
/* From v1.4.0, Iframely supports HTTP/2 by default. Disable it, if you'd rather not.
Alternatively, you can also disable per origin. See `proxy` option below.
*/
// DISABLE_HTTP2: true,
// Customize API calls to oembed endpoints.
ADD_OEMBED_PARAMS: [{
// Endpoint url regexp array.
re: [/^http:\/\/api\.instagram\.com\/oembed/],
// Custom get params object.
params: {
hidecaption: true
}
}, {
re: [/^https:\/\/www\.facebook\.com\/plugins\/page\/oembed\.json/i],
params: {
show_posts: 0,
show_facepile: 0,
maxwidth: 600
}
}, {
// match i=user or i=moment or i=timeline to configure these types invidually
// see params spec at https://dev.twitter.com/web/embedded-timelines/oembed
re: [/^https?:\/\/publish\.twitter\.com\/oembed\?i=user/i],
params: {
limit: 1,
maxwidth: 600
}
/*
}, {
// Facebook https://developers.facebook.com/docs/plugins/oembed-endpoints
re: [/^https:\/\/www\.facebook\.com\/plugins\/\w+\/oembed\.json/i],
params: {
// Skip script tag and fb-root div.
omitscript: true
}
*/
}],
/*
// Configure use of HTTP proxies as needed.
// You don't have to specify all options per regex - just what you need to override
PROXY: [{
re: [/^https?:\/\/www\.domain\.com/],
proxy_server: 'http://1.2.3.4:8080',
user_agent: 'CHANGE YOUR AGENT',
headers: {
// HTTP headers
// Overrides previous params if overlapped.
},
request_options: {
// Refer to: https://github.com/request/request
// Overrides previous params if overlapped.
},
disable_http2: true
}],
*/
// Customize API calls to 3rd parties. At the very least - configure required keys.
providerOptions: {
locale: "en_US", // ISO 639-1 two-letter language code, e.g. en_CA or fr_CH.
// Will be added as highest priotity in accept-language header with each request.
// Plus is used in FB, YouTube and perhaps other plugins
"twitter": {
"max-width": 550,
"min-width": 250,
hide_media: false,
hide_thread: false,
omit_script: false,
center: false,
// dnt: true,
cache_ttl: 100 * 365 * 24 * 3600 // 100 Years.
},
readability: {
enabled: false
// allowPTagDescription: true // to enable description fallback to first paragraph
},
images: {
loadSize: false, // if true, will try an load first bytes of all images to get/confirm the sizes
checkFavicon: false // if true, will verify all favicons
},
tumblr: {
consumer_key: "INSERT YOUR VALUE"
// media_only: true // disables status embeds for images and videos - will return plain media
},
google: {
// https://developers.google.com/maps/documentation/embed/guide#api_key
maps_key: "INSERT YOUR VALUE"
},
/*
// Optional Camo Proxy to wrap all images: https://github.com/atmos/camo
camoProxy: {
camo_proxy_key: "INSERT YOUR VALUE",
camo_proxy_host: "INSERT YOUR VALUE"
// ssl_only: true // will only proxy non-ssl images
},
*/
// List of query parameters to add to YouTube and Vimeo frames
// Start it with leading "?". Or omit alltogether for default values
// API key is optional, youtube will work without it too.
// It is probably the same API key you use for Google Maps.
youtube: {
// api_key: "INSERT YOUR VALUE",
get_params: "?rel=0&showinfo=1" // https://developers.google.com/youtube/player_parameters
},
vimeo: {
get_params: "?byline=0&badge=0" // https://developer.vimeo.com/player/embedding
},
/*
soundcloud: {
old_player: true // enables classic player
},
giphy: {
media_only: true // disables branded player for gifs and returns just the image
}
*/
/*
bandcamp: {
get_params: '/size=large/bgcol=333333/linkcol=ffffff/artwork=small/transparent=true/',
media: {
album: {
height: 472,
'max-width': 700
},
track: {
height: 120,
'max-width': 700
}
}
}
*/
},
// WHITELIST_WILDCARD, if present, will be added to whitelist as record for top level domain: "*"
// with it, you can define what parsers do when they run accross unknown publisher.
// If absent or empty, all generic media parsers will be disabled except for known domains
// More about format: https://iframely.com/docs/qa-format
/*
WHITELIST_WILDCARD: {
"twitter": {
"player": "allow",
"photo": "deny"
},
"oembed": {
"video": "allow",
"photo": "allow",
"rich": "deny",
"link": "deny"
},
"og": {
"video": ["allow", "ssl", "responsive"]
},
"iframely": {
"survey": "allow",
"reader": "allow",
"player": "allow",
"image": "allow"
},
"html-meta": {
"video": ["allow", "responsive"],
"promo": "allow"
}
}
*/
// Black-list any of the inappropriate domains. Iframely will return 417
// At minimum, keep your localhosts blacklisted to avoid SSRF
BLACKLIST_DOMAINS_RE: [
/^https?:\/\/127\.0\.0\.1/i,
/^https?:\/\/localhost/i,
// And this is AWS metadata service
// https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
/^https?:\/\/169\.254\.169\.254/
]
};
module.exports = config;
})();

38
docker/lemmy.hjson vendored
View File

@ -1,38 +0,0 @@
{
# for more info about the config, check out the documentation
# https://dev.lemmy.ml/docs/administration_configuration.html
# the domain name of your instance (eg "dev.lemmy.ml")
hostname: "my_domain"
# address where lemmy should listen for incoming requests
bind: "0.0.0.0"
# port where lemmy should listen for incoming requests
port: 8536
# json web token for authorization between server and client
jwt_secret: "changeme"
# settings related to the postgresql database
database: {
# name of the postgres database for lemmy
database: "lemmy"
# username to connect to postgres
user: "lemmy"
# password to connect to postgres
password: "password"
# host where postgres is running
host: "postgres"
}
# The location of the frontend
front_end_dir: "/app/dist"
# # optional: email sending configuration
# email: {
# # hostname of the smtp server
# smtp_server: ""
# # login name for smtp server
# smtp_login: ""
# # password to login to the smtp server
# smtp_password: ""
# # address to send emails from, eg "noreply@your-instance.com"
# smtp_from_address: ""
# }
}

5
docker/prod/.env vendored Normal file
View File

@ -0,0 +1,5 @@
LEMMY_DOMAIN=my_domain
LEMMY_DATABASE_PASSWORD=password
LEMMY_DATABASE_URL=postgres://lemmy:password@lemmy_db:5432/lemmy
LEMMY_JWT_SECRET=changeme
LEMMY_FRONT_END_DIR=/app/dist

65
docker/prod/Dockerfile vendored
View File

@ -1,65 +0,0 @@
ARG RUST_BUILDER_IMAGE=shtripok/rust-musl-builder:arm
FROM $RUST_BUILDER_IMAGE as rust
#ARG RUSTRELEASEDIR="debug"
ARG RUSTRELEASEDIR="release"
# Cache deps
WORKDIR /app
RUN sudo chown -R rust:rust .
RUN USER=root cargo new server
WORKDIR /app/server
COPY server/Cargo.toml server/Cargo.lock ./
COPY server/lemmy_db ./lemmy_db
COPY server/lemmy_utils ./lemmy_utils
RUN mkdir -p ./src/bin \
&& echo 'fn main() { println!("Dummy") }' > ./src/bin/main.rs
RUN cargo build --release
RUN find target/$CARGO_BUILD_TARGET/$RUSTRELEASEDIR -type f -name "$(echo "lemmy_server" | tr '-' '_')*" -exec touch -t 200001010000 {} +
COPY server/src ./src/
COPY server/migrations ./migrations/
# build for release
# workaround for https://github.com/rust-lang/rust/issues/62896
RUN cargo build --frozen --release
# reduce binary size
RUN strip ./target/$CARGO_BUILD_TARGET/$RUSTRELEASEDIR/lemmy_server
RUN cp ./target/$CARGO_BUILD_TARGET/$RUSTRELEASEDIR/lemmy_server /app/server/
FROM $RUST_BUILDER_IMAGE as docs
WORKDIR /app
COPY --chown=rust:rust docs ./docs
RUN mdbook build docs/
FROM node:12-buster as node
WORKDIR /app/ui
# Cache deps
COPY ui/package.json ui/yarn.lock ./
RUN yarn install --pure-lockfile --network-timeout 600000
# Build
COPY ui /app/ui
RUN yarn build
FROM alpine:3.12 as lemmy
# Install libpq for postgres
RUN apk add libpq
RUN addgroup -g 1000 lemmy
RUN adduser -D -s /bin/sh -u 1000 -G lemmy lemmy
# Copy resources
COPY --chown=lemmy:lemmy server/config/defaults.hjson /config/defaults.hjson
COPY --chown=lemmy:lemmy --from=rust /app/server/lemmy_server /app/lemmy
COPY --chown=lemmy:lemmy --from=docs /app/docs/book/ /app/dist/documentation/
COPY --chown=lemmy:lemmy --from=node /app/ui/dist /app/dist
RUN chown lemmy:lemmy /app/lemmy
USER lemmy
EXPOSE 8536
CMD ["/app/lemmy"]

60
docker/prod/deploy.sh vendored
View File

@ -1,60 +0,0 @@
#!/bin/sh
set -e
git checkout master
# Import translations
git fetch weblate
git merge weblate/master
# Creating the new tag
new_tag="$1"
third_semver=$(echo $new_tag | cut -d "." -f 3)
# Setting the version on the front end
cd ../../
echo "export const version: string = '$new_tag';" > "ui/src/version.ts"
git add "ui/src/version.ts"
# Setting the version on the backend
echo "pub const VERSION: &str = \"$new_tag\";" > "server/src/version.rs"
git add "server/src/version.rs"
# Setting the version for Ansible
echo $new_tag > "ansible/VERSION"
git add "ansible/VERSION"
cd docker/prod || exit
# Changing the docker-compose prod
sed -i "s/dessalines\/lemmy:.*/dessalines\/lemmy:$new_tag/" ../prod/docker-compose.yml
sed -i "s/dessalines\/lemmy:.*/dessalines\/lemmy:$new_tag/" ../../ansible/templates/docker-compose.yml
git add ../prod/docker-compose.yml
git add ../../ansible/templates/docker-compose.yml
# The commit
git commit -m"Version $new_tag"
git tag $new_tag
export COMPOSE_DOCKER_CLI_BUILD=1
export DOCKER_BUILDKIT=1
# Rebuilding docker
if [ $third_semver -eq 0 ]; then
# TODO get linux/arm/v7 build working
# Build for Raspberry Pi / other archs too
docker buildx build --platform linux/amd64,linux/arm64 ../../ \
--file Dockerfile \
--tag dessalines/lemmy:$new_tag \
--push
else
docker buildx build --platform linux/amd64 ../../ \
--file Dockerfile \
--tag dessalines/lemmy:$new_tag \
--push
fi
# Push
git push origin $new_tag
git push
# Pushing to any ansible deploys
cd ../../../lemmy-ansible || exit
ansible-playbook -i prod playbooks/site.yml --vault-password-file vault_pass

44
docker/prod/docker-compose.yml vendored
View File

@ -1,44 +1,34 @@
version: '2.2'
version: '3.3'
services:
postgres:
lemmy_db:
image: postgres:12-alpine
environment:
- POSTGRES_USER=lemmy
- POSTGRES_PASSWORD=password
- POSTGRES_PASSWORD=${DATABASE_PASSWORD}
- POSTGRES_DB=lemmy
volumes:
- ./volumes/postgres:/var/lib/postgresql/data
- lemmy_db:/var/lib/postgresql/data
restart: always
lemmy:
image: dessalines/lemmy:v0.7.21
image: dessalines/lemmy:v0.5.0.2
ports:
- "127.0.0.1:8536:8536"
env_file:
- .env
restart: always
environment:
- RUST_LOG=error
volumes:
- ./lemmy.hjson:/config/config.hjson
- ./config.hjson:/config/config.hjson:ro
depends_on:
- postgres
- pictrs
- iframely
pictrs:
image: asonix/pictrs:v0.1.13-r0
ports:
- "127.0.0.1:8537:8080"
user: 991:991
volumes:
- ./volumes/pictrs:/mnt
restart: always
iframely:
image: dogbin/iframely:latest
- lemmy_db
lemmy_pictshare:
image: shtripok/pictshare:latest
ports:
- "127.0.0.1:8061:80"
- "127.0.0.1:8537:80"
volumes:
- ./iframely.config.local.js:/iframely/config.local.js:ro
- lemmy_pictshare:/usr/share/nginx/html/data
restart: always
mem_limit: 100m
volumes:
lemmy_db:
lemmy_pictshare:

92
docker/prod/migrate-pictshare-to-pictrs.bash vendored
View File

@ -1,92 +0,0 @@
#!/bin/bash
set -e
if [[ $(id -u) != 0 ]]; then
echo "This migration needs to be run as root"
exit
fi
if [[ ! -f docker-compose.yml ]]; then
echo "No docker-compose.yml found in current directory. Is this the right folder?"
exit
fi
if ! which jq > /dev/null; then
echo "jq must be installed to run this migration. On ubuntu systems, try 'sudo apt-get install jq'"
exit
fi
# Fixing pictrs permissions
mkdir -p volumes/pictrs
sudo chown -R 991:991 volumes/pictrs
echo "Restarting docker-compose, making sure that pictrs is started and pictshare is removed"
docker-compose up -d --remove-orphans
if [[ -z $(docker-compose ps | grep pictrs) ]]; then
echo "Pict-rs is not running, make sure you update Lemmy first"
exit
fi
# echo "Stopping Lemmy so that users dont upload new images during the migration"
# docker-compose stop lemmy
CRASHED_ON=()
pushd volumes/pictshare/
echo "Importing pictshare images to pict-rs..."
IMAGE_NAMES=*
for image in $IMAGE_NAMES; do
IMAGE_PATH="$(pwd)/$image/$image"
if [[ ! -f $IMAGE_PATH ]]; then
continue
fi
res=$(curl -s -F "images[]=@$IMAGE_PATH" http://127.0.0.1:8537/import | jq .msg)
if [ "${res}" == "" ]; then
echo -n "C" >&2
echo ""
CRASHED_ON+=("${IMAGE_PATH}")
echo "Failed to import $IMAGE_PATH with no error message"
echo " assuming crash, sleeping"
sleep 10
continue
fi
if [ "${res}" != "\"ok\"" ]; then
echo -n "F" >&2
echo ""
echo "Failed to import $IMAGE_PATH"
echo " Reason: ${res}"
else
echo -n "." >&2
fi
done
for image in ${CRASHED_ON[@]}; do
echo "Retrying ${image}"
res=$(curl -s -F "images[]=@$IMAGE_PATH" http://127.0.0.1:8537/import | jq .msg)
if [ "${res}" != "\"ok\"" ]; then
echo -n "F" >&2
echo ""
echo "Failed to upload ${image} on 2nd attempt"
echo " Reason: ${res}"
else
echo -n "." >&2
fi
done
echo "Fixing permissions on pictshare folder"
find . -type d -exec chmod 755 {} \;
find . -type f -exec chmod 644 {} \;
popd
echo "Rewrite image links in Lemmy database"
docker-compose exec -u postgres postgres psql -U lemmy -c "UPDATE user_ SET avatar = REPLACE(avatar, 'pictshare', 'pictrs/image') WHERE avatar is not null;"
docker-compose exec -u postgres postgres psql -U lemmy -c "UPDATE post SET url = REPLACE(url, 'pictshare', 'pictrs/image') WHERE url is not null;"
echo "Moving pictshare data folder to pictshare_backup"
mv volumes/pictshare volumes/pictshare_backup
echo "Migration done, starting Lemmy again"
echo "If everything went well, you can delete ./volumes/pictshare_backup/"
docker-compose start lemmy

1
docs/.gitignore vendored
View File

@ -1 +0,0 @@
book

664
docs/src/contributing_websocket_http_api.md → docs/api.md vendored
View File

File diff suppressed because it is too large Load Diff

0
docs/src/contributing_apub_api_outline.md → docs/apub_api_outline.md vendored
View File

6
docs/book.toml vendored
View File

@ -1,6 +0,0 @@
[book]
authors = ["Felix Ableitner"]
language = "en"
multilingual = false
src = "src"
title = "Lemmy Documentation"

28
docs/src/about_goals.md → docs/goals.md vendored
View File

@ -1,5 +1,4 @@
# Goals
- Come up with a name / codename.
- Must have communities.
- Must have threaded comments.
@ -8,7 +7,6 @@
- Use websockets for post / gets to your own instance.
# Questions
- How does voting work? Should we go back to the old way of showing up and downvote counts? Or just a score?
- Decide on tech to be used
- Backend: Actix, Diesel.
@ -19,7 +17,10 @@
- On mobile, allow you to switch between them. Default?
# Resources / Potential Libraries
- Use the [activitypub crate.](https://docs.rs/activitypub/0.1.4/activitypub/)
- https://docs.rs/activitypub/0.1.4/activitypub/
- [Activitypub vocab.](https://www.w3.org/TR/activitystreams-vocabulary/)
- [Activitypub main](https://www.w3.org/TR/activitypub/)
- [Diesel to Postgres data types](https://kotiri.com/2018/01/31/postgresql-diesel-rust-types.html)
- [helpful diesel examples](http://siciarz.net/24-days-rust-diesel/)
- [Recursive query for adjacency list for nested comments](https://stackoverflow.com/questions/192220/what-is-the-most-efficient-elegant-way-to-parse-a-flat-table-into-a-tree/192462#192462)
@ -35,20 +36,9 @@
- [Temp Icon](https://www.flaticon.com/free-icon/mouse_194242)
- [Rust docker build](https://shaneutt.com/blog/rust-fast-small-docker-image-builds/)
- [Zurb mentions](https://github.com/zurb/tribute)
- [TippyJS](https://github.com/atomiks/tippyjs)
- Activitypub guides
- https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/
- https://raw.githubusercontent.com/w3c/activitypub/gh-pages/activitypub-tutorial.txt
- https://github.com/tOkeshu/activitypub-example
- https://blog.joinmastodon.org/2018/07/how-to-make-friends-and-verify-requests/
## Activitypub guides
- https://blog.joinmastodon.org/2018/06/how-to-implement-a-basic-activitypub-server/
- https://raw.githubusercontent.com/w3c/activitypub/gh-pages/activitypub-tutorial.txt
- https://github.com/tOkeshu/activitypub-example
- https://blog.joinmastodon.org/2018/07/how-to-make-friends-and-verify-requests/
- Use the [activitypub crate.](https://docs.rs/activitypub/0.1.4/activitypub/)
- https://docs.rs/activitypub/0.1.4/activitypub/
- [Activitypub vocab.](https://www.w3.org/TR/activitystreams-vocabulary/)
- [Activitypub main](https://www.w3.org/TR/activitypub/)
- [Federation.md](https://github.com/dariusk/gathio/blob/7fc93dbe9d4d99457a0e85c6c532112f415b7af2/FEDERATION.md)
- [Activitypub implementers guide](https://socialhub.activitypub.rocks/t/draft-guide-for-new-activitypub-implementers/479)
- [Data storage questions](https://socialhub.activitypub.rocks/t/data-storage-questions/579/3)
- [Activitypub as it has been understood](https://flak.tedunangst.com/post/ActivityPub-as-it-has-been-understood)
- [Asonix http signatures in rust](https://git.asonix.dog/Aardwolf/http-signature-normalization)

BIN
docs/img/chat_screen.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 78 KiB

BIN
docs/img/main_screen.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 92 KiB

BIN
docs/img/rank_algorithm.png vendored
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 54 KiB

4
docs/src/about_ranking.md → docs/ranking.md vendored
View File

@ -18,7 +18,7 @@ Score = Upvotes - Downvotes
Time = time since submission (in hours)
Gravity = Decay gravity, 1.8 is default
```
- For posts, in order to bring up active posts, it uses the latest comment time (limited to a max creation age of a month ago)
- Use Max(1, score) to make sure all comments are affected by time decay.
- Add 3 to the score, so that everything that has less than 3 downvotes will seem new. Otherwise all new comments would stay at zero, near the bottom.
- The sign and abs of the score are necessary for dealing with the log of negative scores.
@ -26,4 +26,4 @@ Gravity = Decay gravity, 1.8 is default
A plot of rank over 24 hours, of scores of 1, 5, 10, 100, 1000, with a scale factor of 10k.
![](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/rank_algorithm.png)
![](https://i.imgur.com/w8oBLlL.png)

22
docs/src/SUMMARY.md vendored
View File

@ -1,22 +0,0 @@
# Summary
- [About](about.md)
- [Features](about_features.md)
- [Goals](about_goals.md)
- [Post and Comment Ranking](about_ranking.md)
- [Guide](about_guide.md)
- [Administration](administration.md)
- [Install with Docker](administration_install_docker.md)
- [Install with Ansible](administration_install_ansible.md)
- [Install with Kubernetes](administration_install_kubernetes.md)
- [Configuration](administration_configuration.md)
- [Backup and Restore](administration_backup_and_restore.md)
- [Contributing](contributing.md)
- [Docker Development](contributing_docker_development.md)
- [Local Development](contributing_local_development.md)
- [Tests](contributing_tests.md)
- [Federation Development](contributing_federation_development.md)
- [Websocket/HTTP API](contributing_websocket_http_api.md)
- [ActivityPub API Outline](contributing_apub_api_outline.md)
- [Theming Guide](contributing_theming.md)
- [Lemmy Council](lemmy_council.md)

30
docs/src/about.md vendored
View File

@ -1,30 +0,0 @@
## About The Project
Front Page|Post
---|---
![main screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/main_screen.png)|![chat screen](https://raw.githubusercontent.com/LemmyNet/lemmy/master/docs/img/chat_screen.png)
[Lemmy](https://github.com/LemmyNet/lemmy) is similar to sites like [Reddit](https://reddit.com), [Lobste.rs](https://lobste.rs), [Raddle](https://raddle.me), or [Hacker News](https://news.ycombinator.com/): you subscribe to forums you're interested in, post links and discussions, then vote, and comment on them. Behind the scenes, it is very different; anyone can easily run a server, and all these servers are federated (think email), and connected to the same universe, called the [Fediverse](https://en.wikipedia.org/wiki/Fediverse).
For a link aggregator, this means a user registered on one server can subscribe to forums on any other server, and can have discussions with users registered elsewhere.
The overall goal is to create an easily self-hostable, decentralized alternative to reddit and other link aggregators, outside of their corporate control and meddling.
Each lemmy server can set its own moderation policy; appointing site-wide admins, and community moderators to keep out the trolls, and foster a healthy, non-toxic environment where all can feel comfortable contributing.
*Note: Federation is still in active development*
### Why's it called Lemmy?
- Lead singer from [Motörhead](https://invidio.us/watch?v=pWB5JZRGl0U).
- The old school [video game](<https://en.wikipedia.org/wiki/Lemmings_(video_game)>).
- The [Koopa from Super Mario](https://www.mariowiki.com/Lemmy_Koopa).
- The [furry rodents](http://sunchild.fpwc.org/lemming-the-little-giant-of-the-north/).
### Built With
- [Rust](https://www.rust-lang.org)
- [Actix](https://actix.rs/)
- [Diesel](http://diesel.rs/)
- [Inferno](https://infernojs.org)
- [Typescript](https://www.typescriptlang.org/)

34
docs/src/about_features.md vendored
View File

@ -1,34 +0,0 @@
# Features
- Open source, [AGPL License](/LICENSE).
- Self hostable, easy to deploy.
- Comes with [Docker](#docker), [Ansible](#ansible), [Kubernetes](#kubernetes).
- Clean, mobile-friendly interface.
- Only a minimum of a username and password is required to sign up!
- User avatar support.
- Live-updating Comment threads.
- Full vote scores `(+/-)` like old reddit.
- Themes, including light, dark, and solarized.
- Emojis with autocomplete support. Start typing `:`
- User tagging using `@`, Community tagging using `#`.
- Integrated image uploading in both posts and comments.
- A post can consist of a title and any combination of self text, a URL, or nothing else.
- Notifications, on comment replies and when you're tagged.
- Notifications can be sent via email.
- i18n / internationalization support.
- RSS / Atom feeds for `All`, `Subscribed`, `Inbox`, `User`, and `Community`.
- Cross-posting support.
- A *similar post search* when creating new posts. Great for question / answer communities.
- Moderation abilities.
- Public Moderation Logs.
- Can sticky posts to the top of communities.
- Both site admins, and community moderators, who can appoint other moderators.
- Can lock, remove, and restore posts and comments.
- Can ban and unban users from communities and the site.
- Can transfer site and communities to others.
- Can fully erase your data, replacing all posts and comments.
- NSFW post / community support.
- High performance.
- Server is written in rust.
- Front end is `~80kB` gzipped.
- Supports arm64 / Raspberry Pi.

40
docs/src/about_guide.md vendored
View File

@ -1,40 +0,0 @@
# Lemmy Guide
Start typing...
- `@a_user_name` to get a list of usernames.
- `!a_community` to get a list of communities.
- `:emoji` to get a list of emojis.
## Sorting
*Applies to both posts and comments*
Type | Description
--- | ---
Hot | Shows *trending* posts, based on the score, and the most recent comment time.
New | Newest posts.
Top | Shows the highest scoring posts in the given time frame.
For more detail, check the [Post and Comment Ranking details](about_ranking.md).
## Markdown Guide
Type | Or | … to Get
--- | --- | ---
\*Italic\* | \_Italic\_ | _Italic_
\*\*Bold\*\* | \_\_Bold\_\_ | **Bold**
\# Heading 1 | Heading 1 <br> ========= | <h4>Heading 1</h4>
\## Heading 2 | Heading 2 <br>--------- | <h5>Heading 2</h5>
\[Link\](http://a.com) | \[Link\]\[1\]<br>⋮ <br>\[1\]: http://b.org | [Link](https://commonmark.org/)
!\[Image\](http://url/a.png) | !\[Image\]\[1\]<br>⋮ <br>\[1\]: http://url/b.jpg | ![Markdown](https://commonmark.org/help/images/favicon.png)
\> Blockquote | | <blockquote>Blockquote</blockquote>
\* List <br>\* List <br>\* List | \- List <br>\- List <br>\- List <br> | * List <br>* List <br>* List <br>
1\. One <br>2\. Two <br>3\. Three | 1) One<br>2) Two<br>3) Three | 1. One<br>2. Two<br>3. Three
Horizontal Rule <br>\--- | Horizontal Rule<br>\*\*\* | Horizontal Rule <br><hr>
\`Inline code\` with backticks | |`Inline code` with backticks
\`\`\`<br>\# code block <br>print '3 backticks or'<br>print 'indent 4 spaces' <br>\`\`\` | ····\# code block<br>····print '3 backticks or'<br>····print 'indent 4 spaces' | \# code block <br>print '3 backticks or'<br>print 'indent 4 spaces'
::: spoiler hidden or nsfw stuff<br>*a bunch of spoilers here*<br>::: | | <details><summary> hidden or nsfw stuff </summary><p><em>a bunch of spoilers here</em></p></details>
[CommonMark Tutorial](https://commonmark.org/help/tutorial/)

3
docs/src/administration.md vendored
View File

@ -1,3 +0,0 @@
# Admin info
Information for Lemmy instance admins, and those who want to start an instance.

44
docs/src/administration_backup_and_restore.md vendored
View File

@ -1,44 +0,0 @@
# Backup and Restore Guide
## Docker and Ansible
When using docker or ansible, there should be a `volumes` folder, which contains both the database, and all the pictures. Copy this folder to the new instance to restore your data.
### Incremental Database backup
To incrementally backup the DB to an `.sql` file, you can run:
```bash
docker exec -t FOLDERNAME_postgres_1 pg_dumpall -c -U lemmy > lemmy_dump_`date +%Y-%m-%d"_"%H_%M_%S`.sql
```
### A Sample backup script
```bash
#!/bin/sh
# DB Backup
ssh MY_USER@MY_IP "docker exec -t FOLDERNAME_postgres_1 pg_dumpall -c -U lemmy" > ~/BACKUP_LOCATION/INSTANCE_NAME_dump_`date +%Y-%m-%d"_"%H_%M_%S`.sql
# Volumes folder Backup
rsync -avP -zz --rsync-path="sudo rsync" MY_USER@MY_IP:/LEMMY_LOCATION/volumes ~/BACKUP_LOCATION/FOLDERNAME
```
### Restoring the DB
If you need to restore from a `pg_dumpall` file, you need to first clear out your existing database
```bash
# Drop the existing DB
docker exec -i FOLDERNAME_postgres_1 psql -U lemmy -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public;"
# Restore from the .sql backup
cat db_dump.sql | docker exec -i FOLDERNAME_postgres_1 psql -U lemmy # restores the db
# This also might be necessary when doing a db import with a different password.
docker exec -i FOLDERNAME_postgres_1 psql -U lemmy -c "alter user lemmy with password 'bleh'"
```
## More resources
- https://stackoverflow.com/questions/24718706/backup-restore-a-dockerized-postgresql-database

24
docs/src/administration_configuration.md vendored
View File

@ -1,24 +0,0 @@
# Configuration
The configuration is based on the file
[defaults.hjson](https://yerbamate.dev/LemmyNet/lemmy/src/branch/master/server/config/defaults.hjson).
This file also contains documentation for all the available options. To override the defaults, you
can copy the options you want to change into your local `config.hjson` file.
To use a different `config.hjson` location than the current directory, set the environment variable `LEMMY_CONFIG_LOCATION`.
Additionally, you can override any config files with environment variables. These have the same
name as the config options, and are prefixed with `LEMMY_`. For example, you can override the
`database.password` with `LEMMY_DATABASE__POOL_SIZE=10`.
An additional option `LEMMY_DATABASE_URL` is available, which can be used with a PostgreSQL
connection string like `postgres://lemmy:password@lemmy_db:5432/lemmy`, passing all connection
details at once.
If the Docker container is not used, manually create the database specified above by running the
following commands:
```bash
cd server
./db-init.sh
```

25
docs/src/administration_install_ansible.md vendored
View File

@ -1,25 +0,0 @@
# Ansible Installation
This is the same as the [Docker installation](administration_install_docker.md), except that Ansible handles all of it automatically. It also does some extra things like setting up TLS and email for your Lemmy instance.
First, you need to [install Ansible on your local computer](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) (e.g. using `sudo apt install ansible`) or the equivalent for you platform.
Then run the following commands on your local computer:
```bash
git clone https://github.com/LemmyNet/lemmy.git
cd lemmy/ansible/
cp inventory.example inventory
nano inventory # enter your server, domain, contact email
# If the command below fails, you may need to comment out this line
# In the ansible.cfg file:
# interpreter_python=/usr/bin/python3
ansible-playbook lemmy.yml --become
```
To update to a new version, just run the following in your local Lemmy repo:
```bash
git pull origin master
cd ansible
ansible-playbook lemmy.yml --become
```

41
docs/src/administration_install_docker.md vendored
View File

@ -1,41 +0,0 @@
# Docker Installation
Make sure you have both docker and docker-compose(>=`1.24.0`) installed. On Ubuntu, just run `apt install docker-compose docker.io`. Next,
```bash
# create a folder for the lemmy files. the location doesnt matter, you can put this anywhere you want
mkdir /lemmy
cd /lemmy
# download default config files
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/docker-compose.yml
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/lemmy.hjson
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/iframely.config.local.js
# Set correct permissions for pictrs folder
mkdir -p volumes/pictrs
sudo chown -R 991:991 volumes/pictrs
```
After this, have a look at the [config file](administration_configuration.md) named `lemmy.hjson`, and adjust it, in particular the hostname, and possibly the db password. Then run:
`docker-compose up -d`
To make Lemmy available outside the server, you need to setup a reverse proxy, like Nginx. [A sample nginx config](https://raw.githubusercontent.com/LemmyNet/lemmy/master/ansible/templates/nginx.conf), could be setup with:
```bash
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/ansible/templates/nginx.conf
# Replace the {{ vars }}
sudo mv nginx.conf /etc/nginx/sites-enabled/lemmy.conf
```
You will also need to setup TLS, for example with [Let's Encrypt](https://letsencrypt.org/). After this you need to restart Nginx to reload the config.
## Updating
To update to the newest version, you can manually change the version in `docker-compose.yml`. Alternatively, fetch the latest version from our git repo:
```bash
wget https://raw.githubusercontent.com/LemmyNet/lemmy/master/docker/prod/docker-compose.yml
docker-compose up -d
```

24
docs/src/administration_install_kubernetes.md vendored
View File

@ -1,24 +0,0 @@
# Kubernetes Installation
You'll need to have an existing Kubernetes cluster and [storage class](https://kubernetes.io/docs/concepts/storage/storage-classes/).
Setting this up will vary depending on your provider.
To try it locally, you can use [MicroK8s](https://microk8s.io/) or [Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/).
Once you have a working cluster, edit the environment variables and volume sizes in `docker/k8s/*.yml`.
You may also want to change the service types to use `LoadBalancer`s depending on where you're running your cluster (add `type: LoadBalancer` to `ports)`, or `NodePort`s.
By default they will use `ClusterIP`s, which will allow access only within the cluster. See the [docs](https://kubernetes.io/docs/concepts/services-networking/service/) for more on networking in Kubernetes.
**Important** Running a database in Kubernetes will work, but is generally not recommended.
If you're deploying on any of the common cloud providers, you should consider using their managed database service instead (RDS, Cloud SQL, Azure Databse, etc.).
Now you can deploy:
```bash
# Add `-n foo` if you want to deploy into a specific namespace `foo`;
# otherwise your resources will be created in the `default` namespace.
kubectl apply -f docker/k8s/db.yml
kubectl apply -f docker/k8s/pictshare.yml
kubectl apply -f docker/k8s/lemmy.yml
```
If you used a `LoadBalancer`, you should see it in your cloud provider's console.

39
docs/src/contributing.md vendored
View File

@ -1,39 +0,0 @@
# Contributing
Information about contributing to Lemmy, whether it is translating, testing, designing or programming.
## Issue tracking / Repositories
- [GitHub (for issues and pull requests)](https://github.com/LemmyNet/lemmy)
- [Gitea (only for pull requests)](https://yerbamate.dev/LemmyNet/lemmy)
- [GitLab (only code-mirror)](https://gitlab.com/dessalines/lemmy)
## Translating
Check out [Lemmy's Weblate](https://weblate.yerbamate.dev/projects/lemmy/) for translations.
## Architecture
### Front end
- The front end is written in `typescript`, using a react-like framework called [inferno](https://infernojs.org/). All UI elements are reusable `.tsx` components.
- The main page and routing are in `ui/src/index.tsx`.
- The components are located in `ui/src/components`.
### Back end
- The back end is written in `rust`, using `diesel`, and `actix`.
- The server source code is split into main sections in `server/src`. These include:
- `db` - The low level database actions.
- Database additions are done using diesel migrations. Run `diesel migration generate xxxxx` to add new things.
- `api` - The high level user interactions (things like `CreateComment`)
- `routes` - The server endpoints .
- `apub` - The activitypub conversions.
- `websocket` - Creates the websocket server.
## Linting / Formatting
- Every front and back end commit is automatically formatted then linted using `husky`, and `lint-staged`.
- Rust with `cargo fmt` and `cargo clippy`.
- Typescript with `prettier` and `eslint`.

24
docs/src/contributing_docker_development.md vendored
View File

@ -1,24 +0,0 @@
# Docker Development
## Running
```bash
sudo apt install git docker-compose
git clone https://github.com/LemmyNet/lemmy
cd lemmy/docker/dev
sudo docker-compose up --no-deps --build
```
and go to http://localhost:8536.
To speed up the Docker compile, add the following to `/etc/docker/daemon.json` and restart Docker.
```
{
"features": {
"buildkit": true
}
}
```
If the build is still too slow, you will have to use a
[local build](contributing_local_development.md) instead.

70
docs/src/contributing_federation_development.md vendored
View File

@ -1,70 +0,0 @@
# Federation Development
## Setup
If you don't have a local clone of the Lemmy repo yet, just run the following command:
```bash
git clone https://github.com/LemmyNet/lemmy
```
## Running locally
You need to have the following packages installed, the Docker service needs to be running.
- docker
- docker-compose
- cargo
- yarn
Then run the following
```bash
cd docker/federation
./run-federation-test.bash -yarn
```
The federation test sets up 3 instances:
Instance / Username | Location
--- | ---
lemmy_alpha | [127.0.0.1:8540](http://127.0.0.1:8540)
lemmy_beta | [127.0.0.1:8550](http://127.0.0.1:8550)
lemmy_gamma | [127.0.0.1:8560](http://127.0.0.1:8560)
You can log into each using the instance name, and `lemmy` as the password, IE (`lemmy_alpha`, `lemmy`).
Firefox containers are a good way to test them interacting.
## Integration tests
To run a suite of suite of federation integration tests:
```bash
cd docker/federation-test
./run-tests.sh
```
## Running on a server
Note that federation is currently in alpha. **Only use it for testing**, not on any production server, and be aware that turning on federation may break your instance.
Follow the normal installation instructions, either with [Ansible](administration_install_ansible.md) or
[manually](administration_install_docker.md). Then replace the line `image: dessalines/lemmy:v0.x.x` in
`/lemmy/docker-compose.yml` with `image: dessalines/lemmy:federation`. Also add the following in
`/lemmy/lemmy.hjson`:
```
federation: {
enabled: true
tls_enabled: true,
allowed_instances: example.com,
}
```
Afterwards, and whenever you want to update to the latest version, run these commands on the server:
```
cd /lemmy/
sudo docker-compose pull
sudo docker-compose up -d
```

67
docs/src/contributing_local_development.md vendored
View File

@ -1,67 +0,0 @@
### Ubuntu
#### Build requirements:
```
sudo apt install git cargo libssl-dev pkg-config libpq-dev yarn curl gnupg2 git
# install yarn
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt update && sudo apt install yarn
```
#### Get the source code
```
git clone https://github.com/LemmyNet/lemmy.git
# or alternatively from gitea
# git clone https://yerbamate.dev/LemmyNet/lemmy.git
```
All the following commands need to be run either in `lemmy/server` or `lemmy/ui`, as indicated
by the `cd` command.
#### Build the backend (Rust)
```
cd server
cargo build
# for development, use `cargo check` instead)
```
#### Build the frontend (Typescript)
```
cd ui
yarn
yarn build
```
#### Setup postgresql
```
sudo apt install postgresql
sudo systemctl start postgresql
# initialize postgres database
sudo -u postgres psql -c "create user lemmy with password 'password' superuser;" -U postgres
sudo -u postgres psql -c 'create database lemmy with owner lemmy;' -U postgres
export LEMMY_DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
# or execute server/db-init.sh
```
#### Run a local development instance
```
# run each of these in a seperate terminal
cd server && cargo run
ui & yarn start
```
Then open [localhost:4444](http://localhost:4444) in your browser. It will auto-refresh if you edit
any frontend files. For backend coding, you will have to rerun `cargo run`. You can use
`cargo check` as a faster way to find compilation errors.
To speed up incremental builds, you can add the following to `~/.cargo/config`:
```
[target.x86_64-unknown-linux-gnu]
rustflags = ["-Clink-arg=-fuse-ld=lld"]
```
Note that this setup doesn't include image uploads or link previews (provided by pict-rs and
iframely respectively). If you want to test those, you should use the
[Docker development](contributing_docker_development.md).

16
docs/src/contributing_tests.md vendored
View File

@ -1,16 +0,0 @@
### Tests
#### Rust
After installing [local development dependencies](contributing_local_development.md), run the
following commands in the `server` subfolder:
```bash
psql -U lemmy -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public;"
./test.sh
```
### Federation
Install the [Docker development dependencies](contributing_docker_development.md), and execute
`docker/federation-test/run-tests.sh`

18
docs/src/contributing_theming.md vendored
View File

@ -1,18 +0,0 @@
# Theming Guide
Lemmy uses [Bootstrap v4](https://getbootstrap.com/), and very few custom css classes, so any bootstrap v4 compatible theme should work fine.
## Creating
- Use a tool like [bootstrap.build](https://bootstrap.build/) to create a bootstrap v4 theme. Export the `bootstrap.min.css` once you're done, and save the `_variables.scss` too.
## Testing
- To test out a theme, you can either use your browser's web tools, or a plugin like stylus to copy-paste a theme, when viewing Lemmy.
## Adding
1. Copy `{my-theme-name}.min.css` to `ui/assets/css/themes`. (You can also copy the `_variables.scss` here if you want).
1. Go to `ui/src/utils.ts` and add `{my-theme-name}` to the themes list.
1. Test locally
1. Do a pull request with those changes.

79
docs/src/lemmy_council.md vendored
View File

@ -1,79 +0,0 @@
# Lemmy Council
- A group of lemmy developers and users that use a well-defined democratic process to steer the project in a positive direction, keep it aligned to community goals, and resolve conflicts.
- Council members are also added as administrators to any official Lemmy instances.
## 1. What gets voted on
This section describes all the aspects of Lemmy where the council has decision making power, namely:
- Coding direction
- Priorities / Emphasis
- Controversial features (For example, an unpopular feature should be removed)
- Moderation and conflict resolution on:
- [dev.lemmy.ml](https://dev.lemmy.ml/)
- [github.com/LemmyNet/lemmy](https://github.com/LemmyNet/lemmy)
- [yerbamate.dev/LemmyNet/lemmy](https://yerbamate.dev/LemmyNet/lemmy)
- [weblate.yerbamate.dev/projects/lemmy/](https://weblate.yerbamate.dev/projects/lemmy/)
- Technical administration of dev.lemmy.ml
- Official Lemmy accounts
- [Mastodon](https://mastodon.social/@LemmyDev)
- [Liberapay](https://liberapay.com/Lemmy/)
- [Patreon](https://www.patreon.com/dessalines)
- Council membership changes
- Changes to these rules
## 2. Feedback and Activity Reports
Every week, the council should make a thread on Lemmy that details its activity during the past week, be it development, moderation, or anything else mentioned in 1.
At the same time, users can give feedback and suggestions in this thread. This should be taken into account by the council. Council members can call for a vote on any controversial issues, if they can't be resolved by discussion.
## 2. Voting Process
Most of the time, we keep each other up to date through the Matrix chat, and take informal decisions on uncontroversial issues. For example, a user clearly violating the site rules could be banned by a single person, or ideally after discussing it with at least one other member.
If an issue can not be resolved in this way, then any council member can call for a vote, which works in the following way:
- Any council member can call for a vote, on any topic mentioned in 1.
- This should be used if there is any controversy in the community, or between council members.
- Before taking any decision, there needs to be a discussion where every council member can
explain their position.
- Discussion should be taken with the goal of reaching a compromise that is acceptable for
everyone.
- After the discussion, voting is done through Matrix emojis (👍: yes, 👎: no, X: abstain) and must
stay open for at least two days.
- All members of the Lemmy council have equal voting power.
- Decisions should be reached unanimously, or nearly so. If this is not possible, at least
2/3 of votes must be in favour for the motion to pass.
- Once a decision is reached in this way, every member needs to abide by it.
## 4. Joining
- We use the following process: anyone who is active around Lemmy can recommend any other active person to join the council. This has to be approved by a majority of the council.
- Active users are defined as those who contribute to Lemmy in some way for at least an hour per week on average, doing things like reporting bugs, discussing rules and features, translating, promoting, developing, or doing other things that aim to improve Lemmy as a whole.
-> people should have joined at least a month ago.
- The member list is public.
- Note: we would like to have a process where community members can elect candidates for the council, but this is not realistic because a single user could easily create multiple accounts and cheat the vote.
- Limit growth to one new member per month at most.
## 5. Removing members
- Inactive members should be removed from the council after a few months of inactivity, and after receiving a notification about this.
- Members that dont follow binding council decisions should be removed.
- Any member can be removed in a vote.
## 6. Goals
- We encourage the membership of groups such as LGBT, religious or ethnic minorities, abuse victims, etc etc, and strive to create a safe space for them to express their opinions. We also support measures to increase participation by the previously mentioned groups.
- The following are banned, and will always be harshly punished: fascism, abuse, racism, sexism, etc etc,
## 7. Communication
- A private Matrix chat for all council members.
- (Once private communities are done) A private community on dev.lemmy.ml for issues.
## 8. Member List / Contact Info
General Contact [@LemmyDev Mastodon](https://mastodon.social/@LemmyDev)
- [Dessalines](https://dev.lemmy.ml/u/dessalines)
- [Nutomic](https://dev.lemmy.ml/u/nutomic)
- [AgreeableLandscape](https://dev.lemmy.ml/u/AgreeableLandscape)
- [fruechtchen](https://dev.lemmy.ml/u/fruechtchen)
- [kixiQu](https://dev.lemmy.ml/u/kixiQu)

69
install.sh vendored
View File

@ -1,73 +1,16 @@
#!/bin/bash
#!/bin/sh
set -e
# Set the database variable to the default first.
# Don't forget to change this string to your actual database parameters
# if you don't plan to initialize the database in this script.
export LEMMY_DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
# Set other environment variables
export DATABASE_URL=postgres://lemmy:password@localhost:5432/lemmy
export JWT_SECRET=changeme
export HOSTNAME=rrr
yes_no_prompt_invalid() {
echo "Invalid input. Please enter either \"y\" or \"n\"." 1>&2
}
ask_to_init_db() {
init_db_valid=0
init_db_final=0
while [ "$init_db_valid" == 0 ]
do
read -p "Initialize database (y/n)? " init_db
case "$init_db" in
[yY]* ) init_db_valid=1; init_db_final=1;;
[nN]* ) init_db_valid=1; init_db_final=0;;
* ) yes_no_prompt_invalid;;
esac
echo
done
if [ "$init_db_final" = 1 ]
then
source ./server/db-init.sh
read -n 1 -s -r -p "Press ANY KEY to continue execution of this script, press CTRL+C to quit..."
echo
fi
}
ask_to_auto_reload() {
auto_reload_valid=0
auto_reload_final=0
while [ "$auto_reload_valid" == 0 ]
do
echo "Automagically reload the project when source files are changed?"
echo "ONLY ENABLE THIS FOR DEVELOPMENT!"
read -p "(y/n) " auto_reload
case "$auto_reload" in
[yY]* ) auto_reload_valid=1; auto_reload_final=1;;
[nN]* ) auto_reload_valid=1; auto_reload_final=0;;
* ) yes_no_prompt_invalid;;
esac
echo
done
if [ "$auto_reload_final" = 1 ]
then
cd ui && yarn start
cd server && cargo watch -x run
fi
}
# Optionally initialize the database
ask_to_init_db
# Build the web client
cd ui
yarn
yarn build
# Build and run the backend
cd ../server
RUST_LOG=debug cargo run
cargo run
# For live coding, where both the front and back end, automagically reload on any save
ask_to_auto_reload
# For live coding, where both the front and back end, automagically reload on any save, do:
# cd ui && yarn start
# cd server && cargo watch -x run

2
server/.gitignore vendored
View File

@ -2,5 +2,3 @@
.env
.idea
env_setup.sh
query_testing/*.json
query_testing/*.json.old

4
server/.rustfmt.toml vendored
View File

@ -1,5 +1 @@
tab_spaces = 2
edition="2018"
imports_layout="HorizontalVertical"
merge_imports=true
reorder_imports=true

3659
server/Cargo.lock generated vendored
View File

File diff suppressed because it is too large Load Diff

70
server/Cargo.toml vendored
View File

@ -1,54 +1,34 @@
[package]
name = "lemmy_server"
version = "0.0.1"
authors = ["Dessalines <happydooby@gmail.com>"]
edition = "2018"
[profile.release]
lto = true
[workspace]
members = [
"lemmy_utils",
"lemmy_db"
]
[dependencies]
lemmy_utils = { path = "./lemmy_utils" }
lemmy_db = { path = "./lemmy_db" }
diesel = "1.4.4"
diesel = { version = "1.4.2", features = ["postgres","chrono", "r2d2"] }
diesel_migrations = "1.4.0"
dotenv = "0.15.0"
activitystreams = "0.6.2"
activitystreams-new = { git = "https://git.asonix.dog/asonix/activitystreams-sketch" }
activitystreams-ext = { git = "https://git.asonix.dog/asonix/activitystreams-ext" }
bcrypt = "0.8.0"
dotenv = "0.14.1"
bcrypt = "0.5.0"
activitypub = "0.1.5"
chrono = { version = "0.4.7", features = ["serde"] }
serde_json = { version = "1.0.52", features = ["preserve_order"]}
failure = "0.1.8"
serde = { version = "1.0.105", features = ["derive"] }
actix = "0.10.0-alpha.2"
actix-web = { version = "3.0.0-alpha.3", features = ["rustls"] }
actix-files = "0.3.0-alpha.1"
actix-web-actors = "3.0.0-alpha.1"
actix-rt = "1.1.1"
awc = "2.0.0-alpha.2"
log = "0.4.0"
env_logger = "0.7.1"
rand = "0.7.3"
strum = "0.18.0"
strum_macros = "0.18.0"
jsonwebtoken = "7.0.1"
failure = "0.1.5"
serde_json = { version = "1.0.40", features = ["preserve_order"]}
serde = { version = "1.0.94", features = ["derive"] }
actix = "0.8.3"
actix-web = "1.0"
actix-files = "0.1.3"
actix-web-actors = "1.0"
env_logger = "0.6.2"
rand = "0.7.0"
strum = "0.15.0"
strum_macros = "0.15.0"
jsonwebtoken = "6.0.1"
regex = "1.1.9"
lazy_static = "1.3.0"
rss = "1.9.0"
url = { version = "2.1.1", features = ["serde"] }
percent-encoding = "2.1.0"
openssl = "0.10"
http = "0.2.1"
http-signature-normalization-actix = { version = "0.4.0-alpha.2", default-features = false, features = ["sha-2"] }
base64 = "0.12.1"
tokio = "0.2.21"
futures = "0.3.5"
itertools = "0.9.0"
uuid = { version = "0.8", features = ["serde", "v4"] }
sha2 = "0.9"
async-trait = "0.1.36"
lettre = "0.9.2"
lettre_email = "0.9.2"
rust-crypto = "^0.2"
rss = "1.8.0"
htmlescape = "0.3.1"
config = "0.9.3"
hjson = "0.8.2"

7
server/clean.sh vendored
View File

@ -1,7 +0,0 @@
#!/bin/sh
cargo update
cargo fmt
cargo check
cargo clippy
cargo outdated -R

4
server/config/config.hjson vendored
View File

@ -1,4 +0,0 @@
{
hostname: "localhost:8536"
federation_enabled: true
}

68
server/config/defaults.hjson vendored
View File

@ -1,75 +1,51 @@
{
# # optional: parameters for automatic configuration of new instance (only used at first start)
# setup: {
# # username for the admin user
# admin_username: ""
# # password for the admin user
# admin_password: ""
# # optional: email for the admin user (can be omitted and set later through the website)
# admin_email: ""
# # name of the site (can be changed later)
# site_name: ""
# }
# settings related to the postgresql database
database: {
# username to connect to postgres
user: "lemmy"
# password to connect to postgres
password: "password"
# host where postgres is running
host: "localhost"
# port where postgres can be accessed
port: 5432
# name of the postgres database for lemmy
database: "lemmy"
# maximum number of active sql connections
pool_size: 5
}
# username to connect to postgres
user: "lemmy"
# password to connect to postgres
password: "password"
# host where postgres is running
host: "localhost"
# port where postgres can be accessed
port: 5432
# name of the postgres database for lemmy
database: "lemmy"
# maximum number of active sql connections
pool_size: 5
}
# the domain name of your instance (eg "dev.lemmy.ml")
hostname: null
hostname: "rrr"
# address where lemmy should listen for incoming requests
bind: "0.0.0.0"
# port where lemmy should listen for incoming requests
port: 8536
# json web token for authorization between server and client
jwt_secret: "changeme"
# The location of the frontend
front_end_dir: "../ui/dist"
# rate limits for various user actions, by user ip
rate_limit: {
# maximum number of messages created in interval
message: 180
message: 30,
# interval length for message limit
message_per_second: 60
message_per_second: 60,
# maximum number of posts created in interval
post: 6
post: 6,
# interval length for post limit
post_per_second: 600
post_per_second: 600,
# maximum number of registrations in interval
register: 3
register: 3,
# interval length for registration limit
register_per_second: 3600
}
# settings related to activitypub federation
federation: {
# whether to enable activitypub federation. this feature is in alpha, do not enable in production.
enabled: false
# whether tls is required for activitypub. only disable this for debugging, never for producion.
tls_enabled: true
# comma seperated list of instances with which federation is allowed
allowed_instances: ""
register_per_second: 3600,
}
# # email sending configuration
# email: {
# # hostname and port of the smtp server
# # hostname of the smtp server
# smtp_server: ""
# # login name for smtp server
# smtp_login: ""
# # password to login to the smtp server
# smtp_password: ""
# # address to send emails from, eg "noreply@your-instance.com"
# # address to send emails from, eg "info@your-instance.com"
# smtp_from_address: ""
# # whether or not smtp connections should use tls
# use_tls: true
# }
}

107
server/db-init.sh vendored
View File

@ -1,107 +0,0 @@
#!/bin/bash
set -e
# Default configurations
username=lemmy
dbname=lemmy
port=5432
yes_no_prompt_invalid() {
echo "Invalid input. Please enter either \"y\" or \"n\"." 1>&2
}
print_config() {
echo " database name: $dbname"
echo " username: $username"
echo " port: $port"
}
ask_for_db_config() {
echo "The default database configuration is:"
print_config
echo
default_config_final=0
default_config_valid=0
while [ "$default_config_valid" == 0 ]
do
read -p "Use this configuration (y/n)? " default_config
case "$default_config" in
[yY]* ) default_config_valid=1; default_config_final=1;;
[nN]* ) default_config_valid=1; default_config_final=0;;
* ) yes_no_prompt_invalid;;
esac
echo
done
if [ "$default_config_final" == 0 ]
then
config_ok_final=0
while [ "$config_ok_final" == 0 ]
do
read -p "Database name: " dbname
read -p "Username: " username
read -p "Port: " port
#echo
#echo "The database configuration is:"
#print_config
#echo
config_ok_valid=0
while [ "$config_ok_valid" == 0 ]
do
read -p "Use this configuration (y/n)? " config_ok
case "$config_ok" in
[yY]* ) config_ok_valid=1; config_ok_final=1;;
[nN]* ) config_ok_valid=1; config_ok_final=0;;
* ) yes_no_prompt_invalid;;
esac
echo
done
done
fi
}
ask_for_password() {
password=""
password_confirm=""
password_valid=0
while [ "$password_valid" == 0 ]
do
read -p "Enter database password: " -s password
echo
read -p "Verify database password: " -s password_confirm
echo
echo
# Start the loop from the top if either check fails
if [ -z "$password" ]
then
echo "Error: Password cannot be empty." 1>&2
echo
continue
fi
if [ "$password" != "$password_confirm" ]
then
echo "Error: Passwords don't match." 1>&2
echo
continue
fi
# Set the password_valid variable to break out of the loop
password_valid=1
done
}
ask_for_db_config
ask_for_password
psql -c "CREATE USER $username WITH PASSWORD '$password' SUPERUSER;" -U postgres
psql -c "CREATE DATABASE $dbname WITH OWNER $username;" -U postgres
export LEMMY_DATABASE_URL=postgres://$username:$password@localhost:$port/$dbname
echo "The database URL is $LEMMY_DATABASE_URL"

2
server/diesel.toml vendored
View File

@ -2,4 +2,4 @@
# see diesel.rs/guides/configuring-diesel-cli
[print_schema]
file = "lemmy_db/src/schema.rs"
file = "src/schema.rs"

15
server/lemmy_db/Cargo.toml vendored
View File

@ -1,15 +0,0 @@
[package]
name = "lemmy_db"
version = "0.1.0"
edition = "2018"
[dependencies]
diesel = { version = "1.4.4", features = ["postgres","chrono","r2d2","64-column-tables","serde_json"] }
chrono = { version = "0.4.7", features = ["serde"] }
serde = { version = "1.0.105", features = ["derive"] }
serde_json = { version = "1.0.52", features = ["preserve_order"]}
strum = "0.18.0"
strum_macros = "0.18.0"
log = "0.4.0"
sha2 = "0.9"
bcrypt = "0.8.0"

165
server/lemmy_db/src/activity.rs
View File

@ -1,165 +0,0 @@
use crate::{schema::activity, Crud};
use diesel::{dsl::*, result::Error, *};
use log::debug;
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::{
fmt::Debug,
io::{Error as IoError, ErrorKind},
};
#[derive(Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
#[table_name = "activity"]
pub struct Activity {
pub id: i32,
pub user_id: i32,
pub data: Value,
pub local: bool,
pub published: chrono::NaiveDateTime,
pub updated: Option<chrono::NaiveDateTime>,
}
#[derive(Insertable, AsChangeset, Clone, Serialize, Deserialize)]
#[table_name = "activity"]
pub struct ActivityForm {
pub user_id: i32,
pub data: Value,
pub local: bool,
pub updated: Option<chrono::NaiveDateTime>,
}
impl Crud<ActivityForm> for Activity {
fn read(conn: &PgConnection, activity_id: i32) -> Result<Self, Error> {
use crate::schema::activity::dsl::*;
activity.find(activity_id).first::<Self>(conn)
}
fn delete(conn: &PgConnection, activity_id: i32) -> Result<usize, Error> {
use crate::schema::activity::dsl::*;
diesel::delete(activity.find(activity_id)).execute(conn)
}
fn create(conn: &PgConnection, new_activity: &ActivityForm) -> Result<Self, Error> {
use crate::schema::activity::dsl::*;
insert_into(activity)
.values(new_activity)
.get_result::<Self>(conn)
}
fn update(
conn: &PgConnection,
activity_id: i32,
new_activity: &ActivityForm,
) -> Result<Self, Error> {
use crate::schema::activity::dsl::*;
diesel::update(activity.find(activity_id))
.set(new_activity)
.get_result::<Self>(conn)
}
}
pub fn do_insert_activity<T>(
conn: &PgConnection,
user_id: i32,
data: &T,
local: bool,
) -> Result<Activity, IoError>
where
T: Serialize + Debug,
{
debug!("inserting activity for user {}, data {:?}", user_id, &data);
let activity_form = ActivityForm {
user_id,
data: serde_json::to_value(&data)?,
local,
updated: None,
};
let result = Activity::create(&conn, &activity_form);
match result {
Ok(s) => Ok(s),
Err(e) => Err(IoError::new(
ErrorKind::Other,
format!("Failed to insert activity into database: {}", e),
)),
}
}
#[cfg(test)]
mod tests {
use crate::{
activity::{Activity, ActivityForm},
tests::establish_unpooled_connection,
user::{UserForm, User_},
Crud,
ListingType,
SortType,
};
use serde_json::Value;
#[test]
fn test_crud() {
let conn = establish_unpooled_connection();
let creator_form = UserForm {
name: "activity_creator_pm".into(),
preferred_username: None,
password_encrypted: "nope".into(),
email: None,
matrix_user_id: None,
avatar: None,
admin: false,
banned: false,
updated: None,
show_nsfw: false,
theme: "darkly".into(),
default_sort_type: SortType::Hot as i16,
default_listing_type: ListingType::Subscribed as i16,
lang: "browser".into(),
show_avatars: true,
send_notifications_to_email: false,
actor_id: "http://fake.com".into(),
bio: None,
local: true,
private_key: None,
public_key: None,
last_refreshed_at: None,
};
let inserted_creator = User_::create(&conn, &creator_form).unwrap();
let test_json: Value = serde_json::from_str(
r#"{
"street": "Article Circle Expressway 1",
"city": "North Pole",
"postcode": "99705",
"state": "Alaska"
}"#,
)
.unwrap();
let activity_form = ActivityForm {
user_id: inserted_creator.id,
data: test_json.to_owned(),
local: true,
updated: None,
};
let inserted_activity = Activity::create(&conn, &activity_form).unwrap();
let expected_activity = Activity {
id: inserted_activity.id,
user_id: inserted_creator.id,
data: test_json,
local: true,
published: inserted_activity.published,
updated: None,
};
let read_activity = Activity::read(&conn, inserted_activity.id).unwrap();
let num_deleted = Activity::delete(&conn, inserted_activity.id).unwrap();
User_::delete(&conn, inserted_creator.id).unwrap();
assert_eq!(expected_activity, read_activity);
assert_eq!(expected_activity, inserted_activity);
assert_eq!(1, num_deleted);
}
}

192
server/lemmy_db/src/private_message.rs
View File

@ -1,192 +0,0 @@
use crate::{schema::private_message, Crud};
use diesel::{dsl::*, result::Error, *};
use serde::{Deserialize, Serialize};
#[derive(Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize)]
#[table_name = "private_message"]
pub struct PrivateMessage {
pub id: i32,
pub creator_id: i32,
pub recipient_id: i32,
pub content: String,
pub deleted: bool,
pub read: bool,
pub published: chrono::NaiveDateTime,
pub updated: Option<chrono::NaiveDateTime>,
pub ap_id: String,
pub local: bool,
}
#[derive(Insertable, AsChangeset, Clone)]
#[table_name = "private_message"]
pub struct PrivateMessageForm {
pub creator_id: i32,
pub recipient_id: i32,
pub content: String,
pub deleted: Option<bool>,
pub read: Option<bool>,
pub published: Option<chrono::NaiveDateTime>,
pub updated: Option<chrono::NaiveDateTime>,
pub ap_id: String,
pub local: bool,
}
impl Crud<PrivateMessageForm> for PrivateMessage {
fn read(conn: &PgConnection, private_message_id: i32) -> Result<Self, Error> {
use crate::schema::private_message::dsl::*;
private_message.find(private_message_id).first::<Self>(conn)
}
fn delete(conn: &PgConnection, private_message_id: i32) -> Result<usize, Error> {
use crate::schema::private_message::dsl::*;
diesel::delete(private_message.find(private_message_id)).execute(conn)
}
fn create(conn: &PgConnection, private_message_form: &PrivateMessageForm) -> Result<Self, Error> {
use crate::schema::private_message::dsl::*;
insert_into(private_message)
.values(private_message_form)
.get_result::<Self>(conn)
}
fn update(
conn: &PgConnection,
private_message_id: i32,
private_message_form: &PrivateMessageForm,
) -> Result<Self, Error> {
use crate::schema::private_message::dsl::*;
diesel::update(private_message.find(private_message_id))
.set(private_message_form)
.get_result::<Self>(conn)
}
}
impl PrivateMessage {
pub fn update_ap_id(
conn: &PgConnection,
private_message_id: i32,
apub_id: String,
) -> Result<Self, Error> {
use crate::schema::private_message::dsl::*;
diesel::update(private_message.find(private_message_id))
.set(ap_id.eq(apub_id))
.get_result::<Self>(conn)
}
pub fn read_from_apub_id(conn: &PgConnection, object_id: &str) -> Result<Self, Error> {
use crate::schema::private_message::dsl::*;
private_message
.filter(ap_id.eq(object_id))
.first::<Self>(conn)
}
}
#[cfg(test)]
mod tests {
use crate::{
private_message::*,
tests::establish_unpooled_connection,
user::*,
ListingType,
SortType,
};
#[test]
fn test_crud() {
let conn = establish_unpooled_connection();
let creator_form = UserForm {
name: "creator_pm".into(),
preferred_username: None,
password_encrypted: "nope".into(),
email: None,
matrix_user_id: None,
avatar: None,
admin: false,
banned: false,
updated: None,
show_nsfw: false,
theme: "darkly".into(),
default_sort_type: SortType::Hot as i16,
default_listing_type: ListingType::Subscribed as i16,
lang: "browser".into(),
show_avatars: true,
send_notifications_to_email: false,
actor_id: "http://fake.com".into(),
bio: None,
local: true,
private_key: None,
public_key: None,
last_refreshed_at: None,
};
let inserted_creator = User_::create(&conn, &creator_form).unwrap();
let recipient_form = UserForm {
name: "recipient_pm".into(),
preferred_username: None,
password_encrypted: "nope".into(),
email: None,
matrix_user_id: None,
avatar: None,
admin: false,
banned: false,
updated: None,
show_nsfw: false,
theme: "darkly".into(),
default_sort_type: SortType::Hot as i16,
default_listing_type: ListingType::Subscribed as i16,
lang: "browser".into(),
show_avatars: true,
send_notifications_to_email: false,
actor_id: "http://fake.com".into(),
bio: None,
local: true,
private_key: None,
public_key: None,
last_refreshed_at: None,
};
let inserted_recipient = User_::create(&conn, &recipient_form).unwrap();
let private_message_form = PrivateMessageForm {
content: "A test private message".into(),
creator_id: inserted_creator.id,
recipient_id: inserted_recipient.id,
deleted: None,
read: None,
published: None,
updated: None,
ap_id: "http://fake.com".into(),
local: true,
};
let inserted_private_message = PrivateMessage::create(&conn, &private_message_form).unwrap();
let expected_private_message = PrivateMessage {
id: inserted_private_message.id,
content: "A test private message".into(),
creator_id: inserted_creator.id,
recipient_id: inserted_recipient.id,
deleted: false,
read: false,
updated: None,
published: inserted_private_message.published,
ap_id: "http://fake.com".into(),
local: true,
};
let read_private_message = PrivateMessage::read(&conn, inserted_private_message.id).unwrap();
let updated_private_message =
PrivateMessage::update(&conn, inserted_private_message.id, &private_message_form).unwrap();
let num_deleted = PrivateMessage::delete(&conn, inserted_private_message.id).unwrap();
User_::delete(&conn, inserted_creator.id).unwrap();
User_::delete(&conn, inserted_recipient.id).unwrap();
assert_eq!(expected_private_message, read_private_message);
assert_eq!(expected_private_message, updated_private_message);
assert_eq!(expected_private_message, inserted_private_message);
assert_eq!(1, num_deleted);
}
}

136
server/lemmy_db/src/private_message_view.rs
View File

@ -1,136 +0,0 @@
use crate::{limit_and_offset, MaybeOptional};
use diesel::{pg::Pg, result::Error, *};
use serde::{Deserialize, Serialize};
// The faked schema since diesel doesn't do views
table! {
private_message_view (id) {
id -> Int4,
creator_id -> Int4,
recipient_id -> Int4,
content -> Text,
deleted -> Bool,
read -> Bool,
published -> Timestamp,
updated -> Nullable<Timestamp>,
ap_id -> Text,
local -> Bool,
creator_name -> Varchar,
creator_avatar -> Nullable<Text>,
creator_actor_id -> Text,
creator_local -> Bool,
recipient_name -> Varchar,
recipient_avatar -> Nullable<Text>,
recipient_actor_id -> Text,
recipient_local -> Bool,
}
}
#[derive(
Queryable, Identifiable, PartialEq, Debug, Serialize, Deserialize, QueryableByName, Clone,
)]
#[table_name = "private_message_view"]
pub struct PrivateMessageView {
pub id: i32,
pub creator_id: i32,
pub recipient_id: i32,
pub content: String,
pub deleted: bool,
pub read: bool,
pub published: chrono::NaiveDateTime,
pub updated: Option<chrono::NaiveDateTime>,
pub ap_id: String,
pub local: bool,
pub creator_name: String,
pub creator_avatar: Option<String>,
pub creator_actor_id: String,
pub creator_local: bool,
pub recipient_name: String,
pub recipient_avatar: Option<String>,
pub recipient_actor_id: String,
pub recipient_local: bool,
}
pub struct PrivateMessageQueryBuilder<'a> {
conn: &'a PgConnection,
query: super::private_message_view::private_message_view::BoxedQuery<'a, Pg>,
for_recipient_id: i32,
unread_only: bool,
page: Option<i64>,
limit: Option<i64>,
}
impl<'a> PrivateMessageQueryBuilder<'a> {
pub fn create(conn: &'a PgConnection, for_recipient_id: i32) -> Self {
use super::private_message_view::private_message_view::dsl::*;
let query = private_message_view.into_boxed();
PrivateMessageQueryBuilder {
conn,
query,
for_recipient_id,
unread_only: false,
page: None,
limit: None,
}
}
pub fn unread_only(mut self, unread_only: bool) -> Self {
self.unread_only = unread_only;
self
}
pub fn page<T: MaybeOptional<i64>>(mut self, page: T) -> Self {
self.page = page.get_optional();
self
}
pub fn limit<T: MaybeOptional<i64>>(mut self, limit: T) -> Self {
self.limit = limit.get_optional();
self
}
pub fn list(self) -> Result<Vec<PrivateMessageView>, Error> {
use super::private_message_view::private_message_view::dsl::*;
let mut query = self.query.filter(deleted.eq(false));
// If its unread, I only want the ones to me
if self.unread_only {
query = query
.filter(read.eq(false))
.filter(recipient_id.eq(self.for_recipient_id));
}
// Otherwise, I want the ALL view to show both sent and received
else {
query = query.filter(
recipient_id
.eq(self.for_recipient_id)
.or(creator_id.eq(self.for_recipient_id)),
)
}
let (limit, offset) = limit_and_offset(self.page, self.limit);
query
.limit(limit)
.offset(offset)
.order_by(published.desc())
.load::<PrivateMessageView>(self.conn)
}
}
impl PrivateMessageView {
pub fn read(conn: &PgConnection, from_private_message_id: i32) -> Result<Self, Error> {
use super::private_message_view::private_message_view::dsl::*;
let mut query = private_message_view.into_boxed();
query = query
.filter(id.eq(from_private_message_id))
.order_by(published.desc());
query.first::<Self>(conn)
}
}

22
server/lemmy_utils/Cargo.toml vendored
View File

@ -1,22 +0,0 @@
[package]
name = "lemmy_utils"
version = "0.1.0"
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
regex = "1.3.5"
config = { version = "0.10.1", default-features = false, features = ["hjson"] }
chrono = { version = "0.4.7", features = ["serde"] }
lettre = "0.9.3"
lettre_email = "0.9.4"
log = "0.4.0"
itertools = "0.9.0"
rand = "0.7.3"
serde = { version = "1.0.105", features = ["derive"] }
serde_json = { version = "1.0.52", features = ["preserve_order"]}
comrak = "0.7"
lazy_static = "1.3.0"
openssl = "0.10"
url = { version = "2.1.1", features = ["serde"] }

337
server/lemmy_utils/src/lib.rs
View File

@ -1,337 +0,0 @@
#[macro_use]
pub extern crate lazy_static;
pub extern crate comrak;
pub extern crate lettre;
pub extern crate lettre_email;
pub extern crate openssl;
pub extern crate rand;
pub extern crate regex;
pub extern crate serde_json;
pub extern crate url;
pub mod settings;
use crate::settings::Settings;
use chrono::{DateTime, FixedOffset, Local, NaiveDateTime, Utc};
use itertools::Itertools;
use lettre::{
smtp::{
authentication::{Credentials, Mechanism},
extension::ClientId,
ConnectionReuseParameters,
},
ClientSecurity,
SmtpClient,
Transport,
};
use lettre_email::Email;
use openssl::{pkey::PKey, rsa::Rsa};
use rand::{distributions::Alphanumeric, thread_rng, Rng};
use regex::{Regex, RegexBuilder};
use std::io::{Error, ErrorKind};
use url::Url;
pub fn to_datetime_utc(ndt: NaiveDateTime) -> DateTime<Utc> {
DateTime::<Utc>::from_utc(ndt, Utc)
}
pub fn naive_from_unix(time: i64) -> NaiveDateTime {
NaiveDateTime::from_timestamp(time, 0)
}
pub fn convert_datetime(datetime: NaiveDateTime) -> DateTime<FixedOffset> {
let now = Local::now();
DateTime::<FixedOffset>::from_utc(datetime, *now.offset())
}
pub fn is_email_regex(test: &str) -> bool {
EMAIL_REGEX.is_match(test)
}
pub fn remove_slurs(test: &str) -> String {
SLUR_REGEX.replace_all(test, "*removed*").to_string()
}
pub fn slur_check(test: &str) -> Result<(), Vec<&str>> {
let mut matches: Vec<&str> = SLUR_REGEX.find_iter(test).map(|mat| mat.as_str()).collect();
// Unique
matches.sort_unstable();
matches.dedup();
if matches.is_empty() {
Ok(())
} else {
Err(matches)
}
}
pub fn slurs_vec_to_str(slurs: Vec<&str>) -> String {
let start = "No slurs - ";
let combined = &slurs.join(", ");
[start, combined].concat()
}
pub fn generate_random_string() -> String {
thread_rng().sample_iter(&Alphanumeric).take(30).collect()
}
pub fn send_email(
subject: &str,
to_email: &str,
to_username: &str,
html: &str,
) -> Result<(), String> {
let email_config = Settings::get().email.ok_or("no_email_setup")?;
let email = Email::builder()
.to((to_email, to_username))
.from(email_config.smtp_from_address.to_owned())
.subject(subject)
.html(html)
.build()
.unwrap();
let mailer = if email_config.use_tls {
SmtpClient::new_simple(&email_config.smtp_server).unwrap()
} else {
SmtpClient::new(&email_config.smtp_server, ClientSecurity::None).unwrap()
}
.hello_name(ClientId::Domain(Settings::get().hostname))
.smtp_utf8(true)
.authentication_mechanism(Mechanism::Plain)
.connection_reuse(ConnectionReuseParameters::ReuseUnlimited);
let mailer = if let (Some(login), Some(password)) =
(&email_config.smtp_login, &email_config.smtp_password)
{
mailer.credentials(Credentials::new(login.to_owned(), password.to_owned()))
} else {
mailer
};
let mut transport = mailer.transport();
let result = transport.send(email.into());
transport.close();
match result {
Ok(_) => Ok(()),
Err(e) => Err(e.to_string()),
}
}
pub fn markdown_to_html(text: &str) -> String {
comrak::markdown_to_html(text, &comrak::ComrakOptions::default())
}
// TODO nothing is done with community / group webfingers yet, so just ignore those for now
#[derive(Clone, PartialEq, Eq, Hash)]
pub struct MentionData {
pub name: String,
pub domain: String,
}
impl MentionData {
pub fn is_local(&self) -> bool {
Settings::get().hostname.eq(&self.domain)
}
pub fn full_name(&self) -> String {
format!("@{}@{}", &self.name, &self.domain)
}
}
pub fn scrape_text_for_mentions(text: &str) -> Vec<MentionData> {
let mut out: Vec<MentionData> = Vec::new();
for caps in MENTIONS_REGEX.captures_iter(text) {
out.push(MentionData {
name: caps["name"].to_string(),
domain: caps["domain"].to_string(),
});
}
out.into_iter().unique().collect()
}
pub fn is_valid_username(name: &str) -> bool {
VALID_USERNAME_REGEX.is_match(name)
}
pub fn is_valid_community_name(name: &str) -> bool {
VALID_COMMUNITY_NAME_REGEX.is_match(name)
}
pub fn is_valid_post_title(title: &str) -> bool {
VALID_POST_TITLE_REGEX.is_match(title)
}
#[cfg(test)]
mod tests {
use crate::{
is_email_regex,
is_valid_community_name,
is_valid_post_title,
is_valid_username,
remove_slurs,
scrape_text_for_mentions,
slur_check,
slurs_vec_to_str,
};
#[test]
fn test_mentions_regex() {
let text = "Just read a great blog post by [@tedu@honk.teduangst.com](/u/test). And another by !test_community@fish.teduangst.com . Another [@lemmy@lemmy-alpha:8540](/u/fish)";
let mentions = scrape_text_for_mentions(text);
assert_eq!(mentions[0].name, "tedu".to_string());
assert_eq!(mentions[0].domain, "honk.teduangst.com".to_string());
assert_eq!(mentions[1].domain, "lemmy-alpha:8540".to_string());
}
#[test]
fn test_email() {
assert!(is_email_regex("gush@gmail.com"));
assert!(!is_email_regex("nada_neutho"));
}
#[test]
fn test_valid_register_username() {
assert!(is_valid_username("Hello_98"));
assert!(is_valid_username("ten"));
assert!(!is_valid_username("Hello-98"));
assert!(!is_valid_username("a"));
assert!(!is_valid_username(""));
}
#[test]
fn test_valid_community_name() {
assert!(is_valid_community_name("example"));
assert!(is_valid_community_name("example_community"));
assert!(!is_valid_community_name("Example"));
assert!(!is_valid_community_name("Ex"));
assert!(!is_valid_community_name(""));
}
#[test]
fn test_valid_post_title() {
assert!(is_valid_post_title("Post Title"));
assert!(is_valid_post_title(" POST TITLE 😃😃😃😃😃"));
assert!(!is_valid_post_title("\n \n \n \n ")); // tabs/spaces/newlines
}
#[test]
fn test_slur_filter() {
let test =
"coons test dindu ladyboy tranny retardeds. Capitalized Niggerz. This is a bunch of other safe text.";
let slur_free = "No slurs here";
assert_eq!(
remove_slurs(&test),
"*removed* test *removed* *removed* *removed* *removed*. Capitalized *removed*. This is a bunch of other safe text."
.to_string()
);
let has_slurs_vec = vec![
"Niggerz",
"coons",
"dindu",
"ladyboy",
"retardeds",
"tranny",
];
let has_slurs_err_str = "No slurs - Niggerz, coons, dindu, ladyboy, retardeds, tranny";
assert_eq!(slur_check(test), Err(has_slurs_vec));
assert_eq!(slur_check(slur_free), Ok(()));
if let Err(slur_vec) = slur_check(test) {
assert_eq!(&slurs_vec_to_str(slur_vec), has_slurs_err_str);
}
}
// These helped with testing
// #[test]
// fn test_send_email() {
// let result = send_email("not a subject", "test_email@gmail.com", "ur user", "<h1>HI there</h1>");
// assert!(result.is_ok());
// }
}
lazy_static! {
static ref EMAIL_REGEX: Regex = Regex::new(r"^[a-zA-Z0-9.!#$%&’*+/=?^_`{|}~-]+@[a-zA-Z0-9-]+(?:\.[a-zA-Z0-9-]+)*$").unwrap();
static ref SLUR_REGEX: Regex = RegexBuilder::new(r"(fag(g|got|tard)?|maricos?|cock\s?sucker(s|ing)?|\bn(i|1)g(\b|g?(a|er)?(s|z)?)\b|dindu(s?)|mudslime?s?|kikes?|mongoloids?|towel\s*heads?|\bspi(c|k)s?\b|\bchinks?|niglets?|beaners?|\bnips?\b|\bcoons?\b|jungle\s*bunn(y|ies?)|jigg?aboo?s?|\bpakis?\b|rag\s*heads?|gooks?|cunts?|bitch(es|ing|y)?|puss(y|ies?)|twats?|feminazis?|whor(es?|ing)|\bslut(s|t?y)?|\btr(a|@)nn?(y|ies?)|ladyboy(s?)|\b(b|re|r)tard(ed)?s?)").case_insensitive(true).build().unwrap();
static ref USERNAME_MATCHES_REGEX: Regex = Regex::new(r"/u/[a-zA-Z][0-9a-zA-Z_]*").unwrap();
// TODO keep this old one, it didn't work with port well tho
// static ref MENTIONS_REGEX: Regex = Regex::new(r"@(?P<name>[\w.]+)@(?P<domain>[a-zA-Z0-9._-]+\.[a-zA-Z0-9_-]+)").unwrap();
static ref MENTIONS_REGEX: Regex = Regex::new(r"@(?P<name>[\w.]+)@(?P<domain>[a-zA-Z0-9._:-]+)").unwrap();
static ref VALID_USERNAME_REGEX: Regex = Regex::new(r"^[a-zA-Z0-9_]{3,20}$").unwrap();
static ref VALID_COMMUNITY_NAME_REGEX: Regex = Regex::new(r"^[a-z0-9_]{3,20}$").unwrap();
static ref VALID_POST_TITLE_REGEX: Regex = Regex::new(r".*\S.*").unwrap();
pub static ref WEBFINGER_COMMUNITY_REGEX: Regex = Regex::new(&format!(
"^group:([a-z0-9_]{{3, 20}})@{}$",
Settings::get().hostname
))
.unwrap();
pub static ref WEBFINGER_USER_REGEX: Regex = Regex::new(&format!(
"^acct:([a-z0-9_]{{3, 20}})@{}$",
Settings::get().hostname
))
.unwrap();
pub static ref CACHE_CONTROL_REGEX: Regex =
Regex::new("^((text|image)/.+|application/javascript)$").unwrap();
}
pub struct Keypair {
pub private_key: String,
pub public_key: String,
}
/// Generate the asymmetric keypair for ActivityPub HTTP signatures.
pub fn generate_actor_keypair() -> Result<Keypair, Error> {
let rsa = Rsa::generate(2048)?;
let pkey = PKey::from_rsa(rsa)?;
let public_key = pkey.public_key_to_pem()?;
let private_key = pkey.private_key_to_pem_pkcs8()?;
let key_to_string = |key| match String::from_utf8(key) {
Ok(s) => Ok(s),
Err(e) => Err(Error::new(
ErrorKind::Other,
format!("Failed converting key to string: {}", e),
)),
};
Ok(Keypair {
private_key: key_to_string(private_key)?,
public_key: key_to_string(public_key)?,
})
}
pub enum EndpointType {
Community,
User,
Post,
Comment,
PrivateMessage,
}
pub fn get_apub_protocol_string() -> &'static str {
if Settings::get().federation.tls_enabled {
"https"
} else {
"http"
}
}
/// Generates the ActivityPub ID for a given object type and ID.
pub fn make_apub_endpoint(endpoint_type: EndpointType, name: &str) -> Url {
let point = match endpoint_type {
EndpointType::Community => "c",
EndpointType::User => "u",
EndpointType::Post => "post",
EndpointType::Comment => "comment",
EndpointType::PrivateMessage => "private_message",
};
Url::parse(&format!(
"{}://{}/{}/{}",
get_apub_protocol_string(),
Settings::get().hostname,
point,
name
))
.unwrap()
}

139
server/lemmy_utils/src/settings.rs
View File

@ -1,139 +0,0 @@
use config::{Config, ConfigError, Environment, File};
use serde::Deserialize;
use std::{env, fs, io::Error, net::IpAddr, sync::RwLock};
static CONFIG_FILE_DEFAULTS: &str = "config/defaults.hjson";
static CONFIG_FILE: &str = "config/config.hjson";
#[derive(Debug, Deserialize, Clone)]
pub struct Settings {
pub setup: Option<Setup>,
pub database: Database,
pub hostname: String,
pub bind: IpAddr,
pub port: u16,
pub jwt_secret: String,
pub front_end_dir: String,
pub rate_limit: RateLimitConfig,
pub email: Option<EmailConfig>,
pub federation: Federation,
}
#[derive(Debug, Deserialize, Clone)]
pub struct Setup {
pub admin_username: String,
pub admin_password: String,
pub admin_email: Option<String>,
pub site_name: String,
}
#[derive(Debug, Deserialize, Clone)]
pub struct RateLimitConfig {
pub message: i32,
pub message_per_second: i32,
pub post: i32,
pub post_per_second: i32,
pub register: i32,
pub register_per_second: i32,
}
#[derive(Debug, Deserialize, Clone)]
pub struct EmailConfig {
pub smtp_server: String,
pub smtp_login: Option<String>,
pub smtp_password: Option<String>,
pub smtp_from_address: String,
pub use_tls: bool,
}
#[derive(Debug, Deserialize, Clone)]
pub struct Database {
pub user: String,
pub password: String,
pub host: String,
pub port: i32,
pub database: String,
pub pool_size: u32,
}
#[derive(Debug, Deserialize, Clone)]
pub struct Federation {
pub enabled: bool,
pub tls_enabled: bool,
pub allowed_instances: String,
}
lazy_static! {
static ref SETTINGS: RwLock<Settings> = RwLock::new(match Settings::init() {
Ok(c) => c,
Err(e) => panic!("{}", e),
});
}
impl Settings {
/// Reads config from the files and environment.
/// First, defaults are loaded from CONFIG_FILE_DEFAULTS, then these values can be overwritten
/// from CONFIG_FILE (optional). Finally, values from the environment (with prefix LEMMY) are
/// added to the config.
///
/// Note: The env var `LEMMY_DATABASE_URL` is parsed in
/// `server/lemmy_db/src/lib.rs::get_database_url_from_env()`
fn init() -> Result<Self, ConfigError> {
let mut s = Config::new();
s.merge(File::with_name(CONFIG_FILE_DEFAULTS))?;
s.merge(File::with_name(&Self::get_config_location()).required(false))?;
// Add in settings from the environment (with a prefix of LEMMY)
// Eg.. `LEMMY_DEBUG=1 ./target/app` would set the `debug` key
// Note: we need to use double underscore here, because otherwise variables containing
// underscore cant be set from environmnet.
// https://github.com/mehcode/config-rs/issues/73
s.merge(Environment::with_prefix("LEMMY").separator("__"))?;
s.try_into()
}
/// Returns the config as a struct.
pub fn get() -> Self {
SETTINGS.read().unwrap().to_owned()
}
pub fn get_database_url(&self) -> String {
format!(
"postgres://{}:{}@{}:{}/{}",
self.database.user,
self.database.password,
self.database.host,
self.database.port,
self.database.database
)
}
pub fn api_endpoint(&self) -> String {
format!("{}/api/v1", self.hostname)
}
pub fn get_config_location() -> String {
env::var("LEMMY_CONFIG_LOCATION").unwrap_or_else(|_| CONFIG_FILE.to_string())
}
pub fn read_config_file() -> Result<String, Error> {
fs::read_to_string(Self::get_config_location())
}
pub fn save_config_file(data: &str) -> Result<String, Error> {
fs::write(Self::get_config_location(), data)?;
// Reload the new settings
// From https://stackoverflow.com/questions/29654927/how-do-i-assign-a-string-to-a-mutable-static-variable/47181804#47181804
let mut new_settings = SETTINGS.write().unwrap();
*new_settings = match Settings::init() {
Ok(c) => c,
Err(e) => panic!("{}", e),
};
Self::read_config_file()
}
}

224
server/migrations/2019-12-29-164820_add_avatar/down.sql vendored
View File

@ -1,224 +0,0 @@
-- the views
drop view user_mention_view;
drop view reply_view;
drop view comment_view;
drop view user_view;
-- user
create view user_view as
select id,
name,
fedi_name,
admin,
banned,
published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;
-- post
-- Recreate the view
drop view post_view;
create view post_view as
with all_post as
(
select
p.*,
(select u.banned from user_ u where p.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb where p.creator_id = cb.user_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where p.creator_id = user_.id) as creator_name,
(select name from community where p.community_id = community.id) as community_name,
(select removed from community c where p.community_id = c.id) as community_removed,
(select deleted from community c where p.community_id = c.id) as community_deleted,
(select nsfw from community c where p.community_id = c.id) as community_nsfw,
(select count(*) from comment where comment.post_id = p.id) as number_of_comments,
coalesce(sum(pl.score), 0) as score,
count (case when pl.score = 1 then 1 else null end) as upvotes,
count (case when pl.score = -1 then 1 else null end) as downvotes,
hot_rank(coalesce(sum(pl.score) , 0), p.published) as hot_rank
from post p
left join post_like pl on p.id = pl.post_id
group by p.id
)
select
ap.*,
u.id as user_id,
coalesce(pl.score, 0) as my_vote,
(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
from user_ u
cross join all_post ap
left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
union all
select
ap.*,
null as user_id,
null as my_vote,
null as subscribed,
null as read,
null as saved
from all_post ap
;
-- community
drop view community_view;
create view community_view as
with all_community as
(
select *,
(select name from user_ u where c.creator_id = u.id) as creator_name,
(select name from category ct where c.category_id = ct.id) as category_name,
(select count(*) from community_follower cf where cf.community_id = c.id) as number_of_subscribers,
(select count(*) from post p where p.community_id = c.id) as number_of_posts,
(select count(*) from comment co, post p where c.id = p.community_id and p.id = co.post_id) as number_of_comments,
hot_rank((select count(*) from community_follower cf where cf.community_id = c.id), c.published) as hot_rank
from community c
)
select
ac.*,
u.id as user_id,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
from user_ u
cross join all_community ac
union all
select
ac.*,
null as user_id,
null as subscribed
from all_community ac
;
-- Reply and comment view
create view comment_view as
with all_comment as
(
select
c.*,
(select community_id from post p where p.id = c.post_id),
(select u.banned from user_ u where c.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb, post p where c.creator_id = cb.user_id and p.id = c.post_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where c.creator_id = user_.id) as creator_name,
coalesce(sum(cl.score), 0) as score,
count (case when cl.score = 1 then 1 else null end) as upvotes,
count (case when cl.score = -1 then 1 else null end) as downvotes
from comment c
left join comment_like cl on c.id = cl.comment_id
group by c.id
)
select
ac.*,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
union all
select
ac.*,
null as user_id,
null as my_vote,
null as saved
from all_comment ac
;
create view reply_view as
with closereply as (
select
c2.id,
c2.creator_id as sender_id,
c.creator_id as recipient_id
from comment c
inner join comment c2 on c.id = c2.parent_id
where c2.creator_id != c.creator_id
-- Do union where post is null
union
select
c.id,
c.creator_id as sender_id,
p.creator_id as recipient_id
from comment c, post p
where c.post_id = p.id and c.parent_id is null and c.creator_id != p.creator_id
)
select cv.*,
closereply.recipient_id
from comment_view cv, closereply
where closereply.id = cv.id
;
-- user mention
create view user_mention_view as
select
c.id,
um.id as user_mention_id,
c.creator_id,
c.post_id,
c.parent_id,
c.content,
c.removed,
um.read,
c.published,
c.updated,
c.deleted,
c.community_id,
c.banned,
c.banned_from_community,
c.creator_name,
c.score,
c.upvotes,
c.downvotes,
c.user_id,
c.my_vote,
c.saved,
um.recipient_id
from user_mention um, comment_view c
where um.comment_id = c.id;
-- community tables
drop view community_moderator_view;
drop view community_follower_view;
drop view community_user_ban_view;
drop view site_view;
create view community_moderator_view as
select *,
(select name from user_ u where cm.user_id = u.id) as user_name,
(select name from community c where cm.community_id = c.id) as community_name
from community_moderator cm;
create view community_follower_view as
select *,
(select name from user_ u where cf.user_id = u.id) as user_name,
(select name from community c where cf.community_id = c.id) as community_name
from community_follower cf;
create view community_user_ban_view as
select *,
(select name from user_ u where cm.user_id = u.id) as user_name,
(select name from community c where cm.community_id = c.id) as community_name
from community_user_ban cm;
create view site_view as
select *,
(select name from user_ u where s.creator_id = u.id) as creator_name,
(select count(*) from user_) as number_of_users,
(select count(*) from post) as number_of_posts,
(select count(*) from comment) as number_of_comments,
(select count(*) from community) as number_of_communities
from site s;
alter table user_ rename column avatar to icon;
alter table user_ alter column icon type bytea using icon::bytea;

234
server/migrations/2019-12-29-164820_add_avatar/up.sql vendored
View File

@ -1,234 +0,0 @@
-- Rename to avatar
alter table user_ rename column icon to avatar;
alter table user_ alter column avatar type text;
-- Rebuild nearly all the views, to include the creator avatars
-- user
drop view user_view;
create view user_view as
select id,
name,
avatar,
fedi_name,
admin,
banned,
published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;
-- post
-- Recreate the view
drop view post_view;
create view post_view as
with all_post as
(
select
p.*,
(select u.banned from user_ u where p.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb where p.creator_id = cb.user_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where p.creator_id = user_.id) as creator_name,
(select avatar from user_ where p.creator_id = user_.id) as creator_avatar,
(select name from community where p.community_id = community.id) as community_name,
(select removed from community c where p.community_id = c.id) as community_removed,
(select deleted from community c where p.community_id = c.id) as community_deleted,
(select nsfw from community c where p.community_id = c.id) as community_nsfw,
(select count(*) from comment where comment.post_id = p.id) as number_of_comments,
coalesce(sum(pl.score), 0) as score,
count (case when pl.score = 1 then 1 else null end) as upvotes,
count (case when pl.score = -1 then 1 else null end) as downvotes,
hot_rank(coalesce(sum(pl.score) , 0), p.published) as hot_rank
from post p
left join post_like pl on p.id = pl.post_id
group by p.id
)
select
ap.*,
u.id as user_id,
coalesce(pl.score, 0) as my_vote,
(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
from user_ u
cross join all_post ap
left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
union all
select
ap.*,
null as user_id,
null as my_vote,
null as subscribed,
null as read,
null as saved
from all_post ap
;
-- community
drop view community_view;
create view community_view as
with all_community as
(
select *,
(select name from user_ u where c.creator_id = u.id) as creator_name,
(select avatar from user_ u where c.creator_id = u.id) as creator_avatar,
(select name from category ct where c.category_id = ct.id) as category_name,
(select count(*) from community_follower cf where cf.community_id = c.id) as number_of_subscribers,
(select count(*) from post p where p.community_id = c.id) as number_of_posts,
(select count(*) from comment co, post p where c.id = p.community_id and p.id = co.post_id) as number_of_comments,
hot_rank((select count(*) from community_follower cf where cf.community_id = c.id), c.published) as hot_rank
from community c
)
select
ac.*,
u.id as user_id,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
from user_ u
cross join all_community ac
union all
select
ac.*,
null as user_id,
null as subscribed
from all_community ac
;
-- reply and comment view
drop view reply_view;
drop view user_mention_view;
drop view comment_view;
create view comment_view as
with all_comment as
(
select
c.*,
(select community_id from post p where p.id = c.post_id),
(select u.banned from user_ u where c.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb, post p where c.creator_id = cb.user_id and p.id = c.post_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where c.creator_id = user_.id) as creator_name,
(select avatar from user_ where c.creator_id = user_.id) as creator_avatar,
coalesce(sum(cl.score), 0) as score,
count (case when cl.score = 1 then 1 else null end) as upvotes,
count (case when cl.score = -1 then 1 else null end) as downvotes
from comment c
left join comment_like cl on c.id = cl.comment_id
group by c.id
)
select
ac.*,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
union all
select
ac.*,
null as user_id,
null as my_vote,
null as saved
from all_comment ac
;
create view reply_view as
with closereply as (
select
c2.id,
c2.creator_id as sender_id,
c.creator_id as recipient_id
from comment c
inner join comment c2 on c.id = c2.parent_id
where c2.creator_id != c.creator_id
-- Do union where post is null
union
select
c.id,
c.creator_id as sender_id,
p.creator_id as recipient_id
from comment c, post p
where c.post_id = p.id and c.parent_id is null and c.creator_id != p.creator_id
)
select cv.*,
closereply.recipient_id
from comment_view cv, closereply
where closereply.id = cv.id
;
-- user mention
create view user_mention_view as
select
c.id,
um.id as user_mention_id,
c.creator_id,
c.post_id,
c.parent_id,
c.content,
c.removed,
um.read,
c.published,
c.updated,
c.deleted,
c.community_id,
c.banned,
c.banned_from_community,
c.creator_name,
c.creator_avatar,
c.score,
c.upvotes,
c.downvotes,
c.user_id,
c.my_vote,
c.saved,
um.recipient_id
from user_mention um, comment_view c
where um.comment_id = c.id;
-- community views
drop view community_moderator_view;
drop view community_follower_view;
drop view community_user_ban_view;
drop view site_view;
create view community_moderator_view as
select *,
(select name from user_ u where cm.user_id = u.id) as user_name,
(select avatar from user_ u where cm.user_id = u.id),
(select name from community c where cm.community_id = c.id) as community_name
from community_moderator cm;
create view community_follower_view as
select *,
(select name from user_ u where cf.user_id = u.id) as user_name,
(select avatar from user_ u where cf.user_id = u.id),
(select name from community c where cf.community_id = c.id) as community_name
from community_follower cf;
create view community_user_ban_view as
select *,
(select name from user_ u where cm.user_id = u.id) as user_name,
(select avatar from user_ u where cm.user_id = u.id),
(select name from community c where cm.community_id = c.id) as community_name
from community_user_ban cm;
create view site_view as
select *,
(select name from user_ u where s.creator_id = u.id) as creator_name,
(select avatar from user_ u where s.creator_id = u.id) as creator_avatar,
(select count(*) from user_) as number_of_users,
(select count(*) from post) as number_of_posts,
(select count(*) from comment) as number_of_comments,
(select count(*) from community) as number_of_communities
from site s;

15
server/migrations/2020-01-01-200418_add_email_to_user_view/down.sql vendored
View File

@ -1,15 +0,0 @@
-- user
drop view user_view;
create view user_view as
select id,
name,
avatar,
fedi_name,
admin,
banned,
published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;

16
server/migrations/2020-01-01-200418_add_email_to_user_view/up.sql vendored
View File

@ -1,16 +0,0 @@
-- user
drop view user_view;
create view user_view as
select id,
name,
avatar,
email,
fedi_name,
admin,
banned,
published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;

20
server/migrations/2020-01-02-172755_add_show_avatar_and_email_notifications_to_user/down.sql vendored
View File

@ -1,20 +0,0 @@
-- Drop the columns
drop view user_view;
alter table user_ drop column show_avatars;
alter table user_ drop column send_notifications_to_email;
-- Rebuild the view
create view user_view as
select id,
name,
avatar,
email,
fedi_name,
admin,
banned,
published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;

22
server/migrations/2020-01-02-172755_add_show_avatar_and_email_notifications_to_user/up.sql vendored
View File

@ -1,22 +0,0 @@
-- Add columns
alter table user_ add column show_avatars boolean default true not null;
alter table user_ add column send_notifications_to_email boolean default false not null;
-- Rebuild the user_view
drop view user_view;
create view user_view as
select id,
name,
avatar,
email,
fedi_name,
admin,
banned,
show_avatars,
send_notifications_to_email,
published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;

16
server/migrations/2020-01-11-012452_add_indexes/down.sql vendored
View File

@ -1,16 +0,0 @@
drop index idx_post_creator;
drop index idx_post_community;
drop index idx_post_like_post;
drop index idx_post_like_user;
drop index idx_comment_creator;
drop index idx_comment_parent;
drop index idx_comment_post;
drop index idx_comment_like_comment;
drop index idx_comment_like_user;
drop index idx_comment_like_post;
drop index idx_community_creator;
drop index idx_community_category;

17
server/migrations/2020-01-11-012452_add_indexes/up.sql vendored
View File

@ -1,17 +0,0 @@
-- Go through all the tables joins, optimize every view, CTE, etc.
create index idx_post_creator on post (creator_id);
create index idx_post_community on post (community_id);
create index idx_post_like_post on post_like (post_id);
create index idx_post_like_user on post_like (user_id);
create index idx_comment_creator on comment (creator_id);
create index idx_comment_parent on comment (parent_id);
create index idx_comment_post on comment (post_id);
create index idx_comment_like_comment on comment_like (comment_id);
create index idx_comment_like_user on comment_like (user_id);
create index idx_comment_like_post on comment_like (post_id);
create index idx_community_creator on community (creator_id);
create index idx_community_category on community (category_id);

223
server/migrations/2020-01-13-025151_create_materialized_views/down.sql vendored
View File

@ -1,223 +0,0 @@
-- functions and triggers
drop trigger refresh_user on user_;
drop function refresh_user();
drop trigger refresh_post on post;
drop function refresh_post();
drop trigger refresh_post_like on post_like;
drop function refresh_post_like();
drop trigger refresh_community on community;
drop function refresh_community();
drop trigger refresh_community_follower on community_follower;
drop function refresh_community_follower();
drop trigger refresh_community_user_ban on community_user_ban;
drop function refresh_community_user_ban();
drop trigger refresh_comment on comment;
drop function refresh_comment();
drop trigger refresh_comment_like on comment_like;
drop function refresh_comment_like();
-- post
-- Recreate the view
drop view post_view;
create view post_view as
with all_post as
(
select
p.*,
(select u.banned from user_ u where p.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb where p.creator_id = cb.user_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where p.creator_id = user_.id) as creator_name,
(select avatar from user_ where p.creator_id = user_.id) as creator_avatar,
(select name from community where p.community_id = community.id) as community_name,
(select removed from community c where p.community_id = c.id) as community_removed,
(select deleted from community c where p.community_id = c.id) as community_deleted,
(select nsfw from community c where p.community_id = c.id) as community_nsfw,
(select count(*) from comment where comment.post_id = p.id) as number_of_comments,
coalesce(sum(pl.score), 0) as score,
count (case when pl.score = 1 then 1 else null end) as upvotes,
count (case when pl.score = -1 then 1 else null end) as downvotes,
hot_rank(coalesce(sum(pl.score) , 0), p.published) as hot_rank
from post p
left join post_like pl on p.id = pl.post_id
group by p.id
)
select
ap.*,
u.id as user_id,
coalesce(pl.score, 0) as my_vote,
(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
from user_ u
cross join all_post ap
left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
union all
select
ap.*,
null as user_id,
null as my_vote,
null as subscribed,
null as read,
null as saved
from all_post ap
;
drop view post_mview;
drop materialized view post_aggregates_mview;
drop view post_aggregates_view;
-- user
drop materialized view user_mview;
drop view user_view;
create view user_view as
select id,
name,
avatar,
email,
fedi_name,
admin,
banned,
show_avatars,
send_notifications_to_email,
published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;
-- community
drop view community_mview;
drop materialized view community_aggregates_mview;
drop view community_view;
drop view community_aggregates_view;
create view community_view as
with all_community as
(
select *,
(select name from user_ u where c.creator_id = u.id) as creator_name,
(select avatar from user_ u where c.creator_id = u.id) as creator_avatar,
(select name from category ct where c.category_id = ct.id) as category_name,
(select count(*) from community_follower cf where cf.community_id = c.id) as number_of_subscribers,
(select count(*) from post p where p.community_id = c.id) as number_of_posts,
(select count(*) from comment co, post p where c.id = p.community_id and p.id = co.post_id) as number_of_comments,
hot_rank((select count(*) from community_follower cf where cf.community_id = c.id), c.published) as hot_rank
from community c
)
select
ac.*,
u.id as user_id,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
from user_ u
cross join all_community ac
union all
select
ac.*,
null as user_id,
null as subscribed
from all_community ac
;
-- reply and comment view
drop view reply_view;
drop view user_mention_view;
drop view comment_view;
drop view comment_mview;
drop materialized view comment_aggregates_mview;
drop view comment_aggregates_view;
create view comment_view as
with all_comment as
(
select
c.*,
(select community_id from post p where p.id = c.post_id),
(select u.banned from user_ u where c.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb, post p where c.creator_id = cb.user_id and p.id = c.post_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where c.creator_id = user_.id) as creator_name,
(select avatar from user_ where c.creator_id = user_.id) as creator_avatar,
coalesce(sum(cl.score), 0) as score,
count (case when cl.score = 1 then 1 else null end) as upvotes,
count (case when cl.score = -1 then 1 else null end) as downvotes
from comment c
left join comment_like cl on c.id = cl.comment_id
group by c.id
)
select
ac.*,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
union all
select
ac.*,
null as user_id,
null as my_vote,
null as saved
from all_comment ac
;
create view reply_view as
with closereply as (
select
c2.id,
c2.creator_id as sender_id,
c.creator_id as recipient_id
from comment c
inner join comment c2 on c.id = c2.parent_id
where c2.creator_id != c.creator_id
-- Do union where post is null
union
select
c.id,
c.creator_id as sender_id,
p.creator_id as recipient_id
from comment c, post p
where c.post_id = p.id and c.parent_id is null and c.creator_id != p.creator_id
)
select cv.*,
closereply.recipient_id
from comment_view cv, closereply
where closereply.id = cv.id
;
-- user mention
create view user_mention_view as
select
c.id,
um.id as user_mention_id,
c.creator_id,
c.post_id,
c.parent_id,
c.content,
c.removed,
um.read,
c.published,
c.updated,
c.deleted,
c.community_id,
c.banned,
c.banned_from_community,
c.creator_name,
c.creator_avatar,
c.score,
c.upvotes,
c.downvotes,
c.user_id,
c.my_vote,
c.saved,
um.recipient_id
from user_mention um, comment_view c
where um.comment_id = c.id;

437
server/migrations/2020-01-13-025151_create_materialized_views/up.sql vendored
View File

@ -1,437 +0,0 @@
-- post
create view post_aggregates_view as
select
p.*,
(select u.banned from user_ u where p.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb where p.creator_id = cb.user_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where p.creator_id = user_.id) as creator_name,
(select avatar from user_ where p.creator_id = user_.id) as creator_avatar,
(select name from community where p.community_id = community.id) as community_name,
(select removed from community c where p.community_id = c.id) as community_removed,
(select deleted from community c where p.community_id = c.id) as community_deleted,
(select nsfw from community c where p.community_id = c.id) as community_nsfw,
(select count(*) from comment where comment.post_id = p.id) as number_of_comments,
coalesce(sum(pl.score), 0) as score,
count (case when pl.score = 1 then 1 else null end) as upvotes,
count (case when pl.score = -1 then 1 else null end) as downvotes,
hot_rank(coalesce(sum(pl.score) , 0), p.published) as hot_rank
from post p
left join post_like pl on p.id = pl.post_id
group by p.id;
create materialized view post_aggregates_mview as select * from post_aggregates_view;
create unique index idx_post_aggregates_mview_id on post_aggregates_mview (id);
drop view post_view;
create view post_view as
with all_post as (
select
pa.*
from post_aggregates_view pa
)
select
ap.*,
u.id as user_id,
coalesce(pl.score, 0) as my_vote,
(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
from user_ u
cross join all_post ap
left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
union all
select
ap.*,
null as user_id,
null as my_vote,
null as subscribed,
null as read,
null as saved
from all_post ap
;
create view post_mview as
with all_post as (
select
pa.*
from post_aggregates_mview pa
)
select
ap.*,
u.id as user_id,
coalesce(pl.score, 0) as my_vote,
(select cf.id::bool from community_follower cf where u.id = cf.user_id and cf.community_id = ap.community_id) as subscribed,
(select pr.id::bool from post_read pr where u.id = pr.user_id and pr.post_id = ap.id) as read,
(select ps.id::bool from post_saved ps where u.id = ps.user_id and ps.post_id = ap.id) as saved
from user_ u
cross join all_post ap
left join post_like pl on u.id = pl.user_id and ap.id = pl.post_id
union all
select
ap.*,
null as user_id,
null as my_vote,
null as subscribed,
null as read,
null as saved
from all_post ap
;
-- user_view
drop view user_view;
create view user_view as
select
u.id,
u.name,
u.avatar,
u.email,
u.fedi_name,
u.admin,
u.banned,
u.show_avatars,
u.send_notifications_to_email,
u.published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;
create materialized view user_mview as select * from user_view;
create unique index idx_user_mview_id on user_mview (id);
-- community
create view community_aggregates_view as
select c.*,
(select name from user_ u where c.creator_id = u.id) as creator_name,
(select avatar from user_ u where c.creator_id = u.id) as creator_avatar,
(select name from category ct where c.category_id = ct.id) as category_name,
(select count(*) from community_follower cf where cf.community_id = c.id) as number_of_subscribers,
(select count(*) from post p where p.community_id = c.id) as number_of_posts,
(select count(*) from comment co, post p where c.id = p.community_id and p.id = co.post_id) as number_of_comments,
hot_rank((select count(*) from community_follower cf where cf.community_id = c.id), c.published) as hot_rank
from community c;
create materialized view community_aggregates_mview as select * from community_aggregates_view;
create unique index idx_community_aggregates_mview_id on community_aggregates_mview (id);
drop view community_view;
create view community_view as
with all_community as
(
select
ca.*
from community_aggregates_view ca
)
select
ac.*,
u.id as user_id,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
from user_ u
cross join all_community ac
union all
select
ac.*,
null as user_id,
null as subscribed
from all_community ac
;
create view community_mview as
with all_community as
(
select
ca.*
from community_aggregates_mview ca
)
select
ac.*,
u.id as user_id,
(select cf.id::boolean from community_follower cf where u.id = cf.user_id and ac.id = cf.community_id) as subscribed
from user_ u
cross join all_community ac
union all
select
ac.*,
null as user_id,
null as subscribed
from all_community ac
;
-- reply and comment view
create view comment_aggregates_view as
select
c.*,
(select community_id from post p where p.id = c.post_id),
(select u.banned from user_ u where c.creator_id = u.id) as banned,
(select cb.id::bool from community_user_ban cb, post p where c.creator_id = cb.user_id and p.id = c.post_id and p.community_id = cb.community_id) as banned_from_community,
(select name from user_ where c.creator_id = user_.id) as creator_name,
(select avatar from user_ where c.creator_id = user_.id) as creator_avatar,
coalesce(sum(cl.score), 0) as score,
count (case when cl.score = 1 then 1 else null end) as upvotes,
count (case when cl.score = -1 then 1 else null end) as downvotes
from comment c
left join comment_like cl on c.id = cl.comment_id
group by c.id;
create materialized view comment_aggregates_mview as select * from comment_aggregates_view;
create unique index idx_comment_aggregates_mview_id on comment_aggregates_mview (id);
drop view reply_view;
drop view user_mention_view;
drop view comment_view;
create view comment_view as
with all_comment as
(
select
ca.*
from comment_aggregates_view ca
)
select
ac.*,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
union all
select
ac.*,
null as user_id,
null as my_vote,
null as saved
from all_comment ac
;
create view comment_mview as
with all_comment as
(
select
ca.*
from comment_aggregates_mview ca
)
select
ac.*,
u.id as user_id,
coalesce(cl.score, 0) as my_vote,
(select cs.id::bool from comment_saved cs where u.id = cs.user_id and cs.comment_id = ac.id) as saved
from user_ u
cross join all_comment ac
left join comment_like cl on u.id = cl.user_id and ac.id = cl.comment_id
union all
select
ac.*,
null as user_id,
null as my_vote,
null as saved
from all_comment ac
;
create view reply_view as
with closereply as (
select
c2.id,
c2.creator_id as sender_id,
c.creator_id as recipient_id
from comment c
inner join comment c2 on c.id = c2.parent_id
where c2.creator_id != c.creator_id
-- Do union where post is null
union
select
c.id,
c.creator_id as sender_id,
p.creator_id as recipient_id
from comment c, post p
where c.post_id = p.id and c.parent_id is null and c.creator_id != p.creator_id
)
select cv.*,
closereply.recipient_id
from comment_view cv, closereply
where closereply.id = cv.id
;
-- user mention
create view user_mention_view as
select
c.id,
um.id as user_mention_id,
c.creator_id,
c.post_id,
c.parent_id,
c.content,
c.removed,
um.read,
c.published,
c.updated,
c.deleted,
c.community_id,
c.banned,
c.banned_from_community,
c.creator_name,
c.creator_avatar,
c.score,
c.upvotes,
c.downvotes,
c.user_id,
c.my_vote,
c.saved,
um.recipient_id
from user_mention um, comment_view c
where um.comment_id = c.id;
-- user
create or replace function refresh_user()
returns trigger language plpgsql
as $$
begin
refresh materialized view concurrently user_mview;
refresh materialized view concurrently comment_aggregates_mview; -- cause of bans
refresh materialized view concurrently post_aggregates_mview;
return null;
end $$;
create trigger refresh_user
after insert or update or delete or truncate
on user_
for each statement
execute procedure refresh_user();
-- post
create or replace function refresh_post()
returns trigger language plpgsql
as $$
begin
refresh materialized view concurrently post_aggregates_mview;
refresh materialized view concurrently user_mview;
return null;
end $$;
create trigger refresh_post
after insert or update or delete or truncate
on post
for each statement
execute procedure refresh_post();
-- post_like
create or replace function refresh_post_like()
returns trigger language plpgsql
as $$
begin
refresh materialized view concurrently post_aggregates_mview;
refresh materialized view concurrently user_mview;
return null;
end $$;
create trigger refresh_post_like
after insert or update or delete or truncate
on post_like
for each statement
execute procedure refresh_post_like();
-- community
create or replace function refresh_community()
returns trigger language plpgsql
as $$
begin
refresh materialized view concurrently post_aggregates_mview;
refresh materialized view concurrently community_aggregates_mview;
refresh materialized view concurrently user_mview;
return null;
end $$;
create trigger refresh_community
after insert or update or delete or truncate
on community
for each statement
execute procedure refresh_community();
-- community_follower
create or replace function refresh_community_follower()
returns trigger language plpgsql
as $$
begin
refresh materialized view concurrently community_aggregates_mview;
refresh materialized view concurrently post_aggregates_mview;
return null;
end $$;
create trigger refresh_community_follower
after insert or update or delete or truncate
on community_follower
for each statement
execute procedure refresh_community_follower();
-- community_user_ban
create or replace function refresh_community_user_ban()
returns trigger language plpgsql
as $$
begin
refresh materialized view concurrently comment_aggregates_mview;
refresh materialized view concurrently post_aggregates_mview;
return null;
end $$;
create trigger refresh_community_user_ban
after insert or update or delete or truncate
on community_user_ban
for each statement
execute procedure refresh_community_user_ban();
-- comment
create or replace function refresh_comment()
returns trigger language plpgsql
as $$
begin
refresh materialized view concurrently post_aggregates_mview;
refresh materialized view concurrently comment_aggregates_mview;
refresh materialized view concurrently community_aggregates_mview;
refresh materialized view concurrently user_mview;
return null;
end $$;
create trigger refresh_comment
after insert or update or delete or truncate
on comment
for each statement
execute procedure refresh_comment();
-- comment_like
create or replace function refresh_comment_like()
returns trigger language plpgsql
as $$
begin
refresh materialized view concurrently comment_aggregates_mview;
refresh materialized view concurrently user_mview;
return null;
end $$;
create trigger refresh_comment_like
after insert or update or delete or truncate
on comment_like
for each statement
execute procedure refresh_comment_like();

34
server/migrations/2020-01-21-001001_create_private_message/down.sql vendored
View File

@ -1,34 +0,0 @@
-- Drop the triggers
drop trigger refresh_private_message on private_message;
drop function refresh_private_message();
-- Drop the view and table
drop view private_message_view cascade;
drop table private_message;
-- Rebuild the old views
drop view user_view cascade;
create view user_view as
select
u.id,
u.name,
u.avatar,
u.email,
u.fedi_name,
u.admin,
u.banned,
u.show_avatars,
u.send_notifications_to_email,
u.published,
(select count(*) from post p where p.creator_id = u.id) as number_of_posts,
(select coalesce(sum(score), 0) from post p, post_like pl where u.id = p.creator_id and p.id = pl.post_id) as post_score,
(select count(*) from comment c where c.creator_id = u.id) as number_of_comments,
(select coalesce(sum(score), 0) from comment c, comment_like cl where u.id = c.creator_id and c.id = cl.comment_id) as comment_score
from user_ u;
create materialized view user_mview as select * from user_view;
create unique index idx_user_mview_id on user_mview (id);
-- Drop the columns
alter table user_ drop column matrix_user_id;

Some files were not shown because too many files have changed in this diff Show More