Rewrite instructions for manually installation

This commit is contained in:
Felix Ableitner 2021-09-13 17:10:43 +02:00
parent 5028afed50
commit 14bff73e41

View file

@ -2,324 +2,235 @@
> ⚠️ **Disclaimer:** this installation method is not recommended by the Lemmy developers. If you have any problems, you need to solve them yourself or ask the respective authors. If you notice any Lemmy bugs on an instance installed like this, please mention it in the bug report. > ⚠️ **Disclaimer:** this installation method is not recommended by the Lemmy developers. If you have any problems, you need to solve them yourself or ask the respective authors. If you notice any Lemmy bugs on an instance installed like this, please mention it in the bug report.
## Installing Lemmy from source code These instructions are written for Ubuntu 20.04.
Instructions for installing Lemmy natively, without relyng on Docker. Originally posted on [Some resources on setting Lemmy up from source - Lemmy dot C.A.](https://lemmy.ca/post/1066) **The current transcription has been adapted to improve readability**. ## Installation
### Important ### Lemmy Backend
> Software package references below are all Gentoo-based - there's information in the docker files about what's required on debian-like systems, and for anything else you'll probably be able to easily adjust as necessary. It is built from source, so this may take a while, especially on slow devices. For example, Lemmy v0.12.2 takes 17 minutes to build on a dual core VPS. If you prefer prebuilt binaries, use Docker.
Note that building Lemmy requires a lot of hardware resources. If you want to run Lemmy on a small VPS with very limited RAM (which seems like a perfectly acceptable way to run a production instance), maybe stick with the Docker image, or do your code building on another system that has more RAM. RAM is huge with rust builds.
Tag/release versions included in this note come from lemmy/docker/prod/docker-compose.yml and were current at the time this document was created. **Definitely adjust to appropriate versions as necessary**.
I had to switch from using `sudo` to `su`'ing in some places as the user was doing something odd/incomplete with the sudo env for pictrs
## Setup
| Dependencies | |
|------------------------------|-------------------------------------------|
| app-admin | sudo |
| dev-vcs | [git](https://git-scm.com/) |
| dev-lang | [rust](https://www.rust-lang.org/) |
| dev-db | [postgresql](https://www.postgresql.org/) |
| www-servers | [nginx](https://nginx.org/en/) |
| sys-apps | [yarn](https://yarnpkg.com/) |
| app-shells | bash-completion |
### Get postgresql service up & running
Compile and install Lemmy, setup database:
```bash ```bash
emerge --config dev-db/postgresql apt install pkg-config libssl-dev libpq-dev cargo postgresql
rc-service postgresql-12 start # installs latest release, you can also specify one with --version
useradd -m -s /bin/bash lemmy cargo install lemmy_server --target-dir /usr/bin/
cd ~lemmy # replace db-passwd with a randomly generated password
sudo -Hu lemmy git clone https://github.com/LemmyNet/lemmy.git sudo -iu postgres psql -c "CREATE USER lemmy WITH PASSWORD 'db-passwd';"
cd lemmy sudo -iu postgres psql -c "CREATE DATABASE lemmy WITH OWNER lemmy;"
adduser lemmy --system --disabled-login --no-create-home --group
``` ```
### List the tags/releases available Minimal Lemmy config, put this in `/etc/lemmy/lemmy.hjson` (see [here](https://github.com/LemmyNet/lemmy/blob/main/config/config.hjson) for more config options). Run `chown lemmy:lemmy /etc/lemmy/ -R` to set the correct owner.
```hjson
```bash
sudo -Hu lemmy git tag -l
sudo -Hu lemmy git checkout tags/v0.11.0
```
### Build prod? (Remove --release for dev)
```bash
sudo -Hu lemmy cargo build --release
cd ..
sudo -Hu lemmy git clone https://github.com/LemmyNet/lemmy-ui.git
cd lemmy-ui
```
### List the tags/releases available
```bash
sudo -Hu lemmy git tag -l
sudo -Hu lemmy git checkout tags/v0.8.10
sudo -Hu lemmy git submodule init
sudo -Hu lemmy git submodule update --remote
```
### Build the frontend
#### This is for dev
```bash
sudo -Hu lemmy yarn
```
#### This is for prod
```bash
sudo -Hu lemmy yarn install --pure-lockfile
sudo -Hu lemmy yarn build:prod
```
**And this is just to run a dev env, but I think we'll prefer prod - use the init script**
```bash
sudo -Hu lemmy yarn dev
```
For prod, we'll use the [init script](#initlemmy), but the prod commandline is:
```bash
sudo -Hu node dist/js/server.js
```
### Setup the DB
#### Adjust 'password' to an actual password here and execute:
```bash
sudo -u postgres psql -c "create user lemmy with password 'password' superuser;" -U postgres
sudo -u postgres psql -c 'create database lemmy with owner lemmy;' -U postgres
```
### pict-rs install
```bash
useradd -m -s /bin/bash pictrs
cp target/release/pict-rs .
```
Added hdri as magick_rust fails to compile if it's not there. Mentioned in [error[E0425]: cannot find value QuantumRange in module bindings](https://github.com/nlfiedler/magick-rust/issues/40)
```bash
echo "media-gfx/imagemagick hdri jpeg lzma png webp" >> /etc/portage/package.use
echo "*/* -llvm_targets_NVPTX -llvm_targets_AMDGPU" >> /etc/portage/package.use
```
Install extra packages required for pict-rs:
| packages | |
|-------------|--------------------------------------------------|
| media-libs | [gexiv2](https://gitlab.gnome.org/GNOME/gexiv2) |
| media-gfx | [imagemagick](https://imagemagick.org/index.php) |
| media-video | [ffmpeg](https://ffmpeg.org/) |
| sys-devel | [clang](https://clang.llvm.org/) |
Packages required for pict-rs (in case on separate system):
| packages | |
|----------|------------------------------------|
| dev-lang | [rust](https://www.rust-lang.org/) |
Script this or it has to be run manually as user...?
```bash
su - pictrs
git clone https://git.asonix.dog/asonix/pict-rs.git
cd pict-rs
git tag -l
git checkout tags/v0.2.5-r0
cargo build --release
cp target/release/pict-rs .
cd ~pictrs
mkdir pictrs-data
```
**Something missing in the pict-rs README - it created & uses a pict-rs folder in /tmp**
If you do anything funky like I have (changing the user pict-rs runs under) and you wind up with permission problems (which the logs don't tell you *what* it's having a permission issue with), this could be your issue. Also, time will tell whether this folder gets cleaned up appropriately or not.
Running pictrs is along the lines of:
```bash
pict-rs/pict-rs -a 127.0.0.1:9000 -p ~pictrs/pictrs-data/
```
But we'll just use the init script.
At this point, fire everything up via the [init scripts](#init-scripts), should all go. Setup the init scripts to run at boot time. Presumably you've setup nginx and you can reach your instance.
---
## Updating
```bash
su - lemmy
cd lemmy
```
BACKUP [config/config.hjson](#configuration) somewhere
```bash
git fetch
git tag -l
git checkout tags/WHATEVER
cargo build --release
cd ~/lemmy-ui
```
#### List the tags/releases available
```bash
git fetch
git tag -l
git checkout tags/WHATEVER
git submodule update --remote
```
### Build the frontend
#### This is for prod
```bash
yarn install --pure-lockfile # Is this step really needed?
#yarn upgrade --pure-lockfile # ?? Did I get it?
#yarn # Is this step really needed? One of these steps is for sure. (Should be unnecessary)
yarn build:prod
```
restart lemmy-ui
### pict-rs update
```bash
su - pictrs
cd pict-rs
git fetch
git tag -l
git checkout tags/v0.2.5-r0 # (or whatever is currently mentioned in the lemmy docker file)
cargo build --release
cp target/release/pict-rs .
```
restart pictrs
---
## Index
### configuration
config/config.hjson example
```json
{ {
database: { database: {
user: "lemmy" # put your db-passwd from above
password: "whatever" password: "db-passwd"
host: "localhost"
port: 5432
database: "lemmy"
pool_size: 5
} }
hostname: "lemmy.ca" # replace with your domain
hostname: example.com
bind: "127.0.0.1" bind: "127.0.0.1"
port: 8536 # put a random string here (required for login token encryption)
docs_dir: "/home/lemmy/lemmy/docs/book" jwt_secret: "changeme"
pictrs_url: "http://localhost:9000"
federation: {
enabled: true
allowed_instances: ""
blocked_instances: ""
}
email: {
smtp_server: "localhost:25"
smtp_from_address: "lemmy@lemmy.ca"
use_tls: false
}
} }
``` ```
### init scripts Systemd unit file, so that Lemmy automatically starts and stops, logs are handled via journalctl etc. Put this file into /etc/systemd/system/lemmy.service, then run `systemctl enable lemmy` and `systemctl start lemmy`.
```
[Unit]
Description=Lemmy - A link aggregator for the fediverse
After=network.target
##### init/lemmy [Service]
User=lemmy
ExecStart=/usr/bin/lemmy_server
Environment=LEMMY_CONFIG_LOCATION=/etc/lemmy/lemmy.hjson
Restart=on-failure
# Hardening
ProtectSystem=full
PrivateTmp=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true
[Install]
WantedBy=multi-user.target
```
If you did everything right, the Lemmy logs from `journalctl -u lemmy` should show "Starting http server at 127.0.0.1:8536". You can also run `curl localhost:8536/api/v3/site` which should give a successful response, looking like `{"site_view":null,"admins":[],"banned":[],"online":0,"version":"unknown version","my_user":null,"federated_instances":null}`.
### Install lemmy-ui (web frontend)
Install dependencies (nodejs and yarn in Ubuntu 20.04 repos are too old)
```bash
# https://classic.yarnpkg.com/en/docs/install/#debian-stable
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
# https://github.com/nodesource/distributions/blob/master/README.md#installation-instructions
curl -fsSL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt install nodejs yarn
```
Clone the git repo, checkout the version you want (0.12.2 in this case), and compile it.
```bash
mkdir /var/lib/lemmy-ui
cd /var/lib/lemmy-ui
chown lemmy:lemmy .
# dont compile as admin
sudo -u lemmy bash
git clone https://github.com/LemmyNet/lemmy-ui.git --recursive .
git checkout 0.12.2 # replace with the version you want to install
yarn install --pure-lockfile
yarn build:prod
exit
```
Add another systemd unit file, this time for lemmy-ui. You need to replace example.com with your actual domain. Put the file in `/etc/systemd/system/lemmy-ui.service`, then run `systemctl enable lemmy-ui` and `systemctl start lemmy-ui`.
```
[Unit]
Description=Lemmy UI - Web frontend for Lemmy
After=lemmy.service
Before=nginx.service
[Service]
User=lemmy
WorkingDirectory=/var/lib/lemmy-ui
ExecStart=/usr/bin/node dist/js/server.js
Environment=LEMMY_INTERNAL_HOST=localhost:8536
Environment=LEMMY_EXTERNAL_HOST=example.com
Environment=LEMMY_HTTPS=true
Restart=on-failure
# Hardening
ProtectSystem=full
PrivateTmp=true
NoNewPrivileges=true
[Install]
WantedBy=multi-user.target
```
If everything went right, the command `curl -I localhost:1234` should show `200 OK` at the top.
### Configure reverse proxy and TLS
Install dependencies
```bash
apt install nginx certbot python3-certbot-nginx
```
Request Let's Encrypt TLS certificate (just follow the instructions)
```bash
certbot certonly --nginx
```
Let's Encrypt certificates should be renewed automatically, so add the line below to your crontab, by running `sudo crontab -e`. Replace example.com with your actual domain.
```
@daily certbot certonly --nginx --cert-name example.com -d example.com --deploy-hook 'nginx -s reload'
```
Finally, add the nginx config. After downloading, you need to replace some variables in the file.
```bash
curl https://raw.githubusercontent.com/LemmyNet/lemmy/main/ansible/templates/nginx.conf \
--output /etc/nginx/sites-enabled/lemmy.conf
# put your actual domain instead of example.com
sed -i -e 's/{{domain}}/example.com/g' /etc/nginx/sites-enabled/lemmy.conf
# need to run this again because the variable is used both with and without spaces
# TODO: only use one variant in the original file
sed -i -e 's/{{ domain }}/example.com/g' /etc/nginx/sites-enabled/lemmy.conf
sed -i -e 's/{{ lemmy_port }}/8536/g' /etc/nginx/sites-enabled/lemmy.conf
sed -i -e 's/{{ lemmy_ui_port }}/1234/g' /etc/nginx/sites-enabled/lemmy.conf
nginx -s reload
```
Now open your Lemmy domain in the browser, and it should show you a configuration screen. Use it to create the first admin user and the default community.
### Pict-rs (for image hosting, optional)
Pict-rs requires a newer Rust version than the one available in Ubuntu 20.04 repos. So you need to install [Rustup](https://rustup.rs/) which installs the toolchain for you.
```bash ```bash
#!/sbin/openrc-run apt install ffmpeg imagemagick exiftool --no-install-recommends
adduser pictrs --system --disabled-login --no-create-home --group
name="Lemmy Backend" mkdir /var/lib/pictrs-source
cd /var/lib/pictrs
depend() { git clone https://git.asonix.dog/asonix/pict-rs.git .
need localmount # check docker-compose.yml for pict-rs version used by lemmy
need net # https://github.com/LemmyNet/lemmy/blob/main/ansible/templates/docker-compose.yml#L40
} git checkout v0.2.6-r2
# or simply add the bin folder to your $PATH
start() { $HOME/.cargo/bin/cargo build --release
ebegin "Starting Lemmy" cp target/release/pict-rs /usr/bin/
start-stop-daemon --start --background --make-pidfile --user lemmy --group lemmy --pidfile /home/lemmy/lemmy.pid --chdir /home/lemmy/lemmy -3 /usr/bin/logger -4 /usr/bin/logger --exec ~lemmy/lemmy/target/release/lemmy_server # create folder to store image data
eend $? mkdir /var/lib/pictrs
} chown pictrs:pictrs /var/lib/pictrs
stop() {
start-stop-daemon --stop --signal TERM --pidfile /home/lemmy/lemmy.pid
eend $?
}
``` ```
##### init/lemmy-ui Just like before, place the config below in `/etc/systemd/system/pictrs.service`, then run `systemctl enable pictrs` and `systemctl start pictrs`.
```
[Unit]
Description=pict-rs - A simple image host
After=network.target
[Service]
User=pictrs
ExecStart=/usr/bin/pict-rs
Environment=PICTRS_PATH=/var/lib/pictrs
Environment=PICTRS_ADDR=127.0.0.1:8080
Restart=on-failure
# Hardening
ProtectSystem=full
PrivateTmp=true
MemoryDenyWriteExecute=true
NoNewPrivileges=true
[Install]
WantedBy=multi-user.target
```
If it is working correctly, `curl 127.0.0.1:8080` should output nothing (particularly no errors).
Now add the line `pictrs_url: "http://127.0.0.1:8080"` to `/etc/lemmy/lemmy.hjson` so that Lemmy knows how to reach Pict-rs. Then restart Lemmy with `systemctl restart lemmy`, and image uploads should be working.
## Upgrading
### Lemmy
```bash ```bash
#!/sbin/openrc-run # installs latest release, you can also specify one with --version
cargo install lemmy_server --target-dir /usr/bin/
name="Lemmy UI" systemctl restart lemmy
depend() {
need localmount
need net
}
start() {
ebegin "Starting Lemmy UI"
start-stop-daemon --start --background --make-pidfile --user lemmy --group lemmy --pidfile /home/lemmy/lemmy-ui.pid --chdir /home/lemmy/lemmy-ui -3 /usr/bin/logger -4 /usr/bin/logger --exec node dist/js/server.js --env LEMMY_INTERNAL_HOST=127.0.0.1:8536 --env LEMMY_EXTERNAL_HOST=lemmy.ca --env LEMMY_HTTPS=true
eend $?
}
``` ```
##### init/pict-rs ### Lemmy UI
```bash ```bash
#!/sbin/openrc-run cd /var/lib/lemmy-ui
sudo -u lemmy
name="pict-rs Daemon" git checkout main
git pull --tags
depend() { git checkout 0.12.2 # replace with the version you want to install
need localmount git submodule update
need net yarn install --pure-lockfile
} yarn build:prod
exit
start() { systemctl restart lemmy-ui
ebegin "Starting pictrs"
start-stop-daemon --start --background --make-pidfile --user pictrs --group pictrs --pidfile /home/pictrs/pictrs.pid --chdir /home/pictrs/pict-rs -3 /usr/bin/logger -4 /usr/bin/logger --exec /home/pictrs/pict-rs/pict-rs -- -a 127.0.0.1:9000 -p ~pictrs/pictrs-data
eend $?
}
stop() {
start-stop-daemon --stop --signal TERM --pidfile /home/pictrs/pictrs.pid
eend $?
}
``` ```
### Pict-rs
```bash
rustup update
cd /var/lib/pictrs-source
git checkout main
git pull --tags
# check docker-compose.yml for pict-rs version used by lemmy
# https://github.com/LemmyNet/lemmy/blob/main/ansible/templates/docker-compose.yml#L40
git checkout v0.2.6-r2
# or simply add the bin folder to your $PATH
$HOME/.cargo/bin/cargo build --release
cp target/release/pict-rs /usr/bin/
systemctl restart pictrs
```