LemmyNet/lemmy-docs
2
0
Fork 0
mirror of https://github.com/LemmyNet/lemmy-docs.git synced 2025-01-15 14:15:48 +00:00
Code Issues Projects Releases Wiki Activity

Add nginx caching as possible solution for slow lemmy-ui (#334)

Browse source 
* Add nginx caching as possible solution for slow lemmy-ui

* Update troubleshooting.md

* Update troubleshooting.md
This commit is contained in:
Nutomic 2025-01-10 10:27:50 +00:00 committed by GitHub
parent e2d4089545
commit 6230839e45
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 3 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
src/administration/troubleshooting.md
View file

@ -223,11 +223,11 @@ Each of these folders contains a `down.sql` file. We need to run that against ou
If you notice that your lemmy-ui sometimes becomes sluggish or unresponsive over a period of minutes/hours and then it passes, you might be getting targeted by scraping bots.
There's a lot of scraping bots online and they can easily overwhelm your site when they're behaving too "greedily".
There's a lot of scraping bots online and they can easily overwhelm your site when they're behaving too "greedily". Unfortunately the existing lemmy-ui has a habit of falling over when polled too eagerly, while the backend still continues to work.
Unfortunately the existing lemmy-ui has a habit of falling over when polled too eagerly, while the backend still continues to work.
A solution is to cache responses in nginx as seen in lemmy-ansible [here](https://github.com/LemmyNet/lemmy-ansible/blob/1.5.3/templates/nginx.conf#L2-L3) and [here](https://github.com/LemmyNet/lemmy-ansible/blob/1.5.3/templates/nginx.conf#L66-L71). This way lemmy-ui doesn't have to generate all responses from scratch, which reduces CPU load. However it won't help if a single crawler goes through thousands of unique urls in a short time.
One solution to deal with the scraping bots is to block their user agents. To do so, you can modify your `nginx_internal.conf` to block some of the usual suspects, with this line under `server`
Another option is to block the scraper's user agents. To do so, you can modify your `nginx_internal.conf` to block some of the usual suspects, with this line under `server`
```bash
if ($http_user_agent ~* " Bytedance|Bytespider|Amazonbot|ClaudeBot") { return 444; }