|
|
418db7831f
|
Log crawl errors
|
2022-10-28 14:05:14 +02:00 |
|
|
|
d11febc7e8
|
Use exact version for Lemmy dependency
|
2022-05-16 22:59:06 +02:00 |
|
|
|
7ede38f584
|
Handle comma separated params correctly
|
2022-05-16 14:23:32 +02:00 |
|
|
|
21cf61f847
|
Add user agent
|
2022-05-13 16:57:53 +02:00 |
|
nutomic
|
575672cbe3
|
Merge pull request 'Recursive, parallel crawl' (#11) from recursive-crawl into main
Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy-stats-crawler/pulls/11
|
2022-05-13 11:37:52 +00:00 |
|
nutomic
|
0c3ba08d6c
|
Merge pull request 'Use lemmy structs' (#10) from lemmy-structs into main
Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy-stats-crawler/pulls/10
|
2022-05-13 11:37:45 +00:00 |
|
|
|
0079e72759
|
Add stderrlog, structopt for better logging and params
|
2022-05-11 00:56:42 +02:00 |
|
|
|
c254e50211
|
Recursive, parallel crawl
|
2022-05-10 11:49:41 +02:00 |
|
|
|
2e2a4888d0
|
Use lemmy structs
|
2022-05-10 01:29:42 +02:00 |
|
nutomic
|
8509c19f50
|
Merge pull request 'Add min version check to avoid including old instances' (#9) from min-version into main
Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy-stats-crawler/pulls/9
|
2022-05-03 12:19:39 +00:00 |
|
|
|
574536b151
|
Add min version check to avoid including old instances
|
2022-04-21 14:10:54 +02:00 |
|
|
|
9546936ddf
|
Convert instance name to lowercase
|
2022-04-04 13:14:34 +02:00 |
|
|
|
4e95487842
|
Change default crawl depth to 20
|
2022-01-31 23:45:21 +01:00 |
|
|
|
a1115d474b
|
Include require_application in crawl output
|
2022-01-29 17:18:29 +01:00 |
|
|
|
f110554993
|
Add command line param to exclude instances from crawl
|
2021-06-17 16:48:45 +02:00 |
|
nutomic
|
fefd708777
|
Merge pull request 'Sorting by active monthly desc.' (#6) from sort_active_monthly into main
Reviewed-on: https://yerbamate.ml/LemmyNet/lemmy-stats-crawler/pulls/6
|
2021-04-27 13:10:47 +00:00 |
|
|
|
60563fc7d9
|
Exclude test instances from crawl
|
2021-04-27 15:09:02 +02:00 |
|
|
|
d92de61d9c
|
Include description field (if it is under 150 chars)
|
2021-04-27 15:06:28 +02:00 |
|
|
|
1751035334
|
Add support for Lemmy api v2 and v3
|
2021-04-12 17:08:14 +02:00 |
|
|
|
cd04d54b9a
|
Update for API v3
|
2021-04-05 20:33:06 +02:00 |
|
|
|
f56a7dde4f
|
Sorting by active monthly desc.
|
2021-03-24 19:43:59 -04:00 |
|
|
|
0fcf8fe523
|
Remove duplicate code, clippy fixes
|
2021-03-17 03:09:21 +01:00 |
|
|
|
95107b6a39
|
Include number of failed instances
|
2021-03-17 03:04:57 +01:00 |
|
|
|
5d5a9b863d
|
Pretty print
|
2021-03-17 02:56:55 +01:00 |
|
|
|
9621b91f7c
|
Add support for command line parameters
|
2021-03-15 22:39:09 +01:00 |
|
|
|
f01e077020
|
Add max depth parameter for crawl
|
2021-03-15 22:19:48 +01:00 |
|
|
|
5a09fa46c4
|
Dont redo crawls for unreachable instances
|
2021-03-11 15:47:55 +01:00 |
|
|
|
48a164de5a
|
Move crawl code into separate file
|
2021-03-11 14:34:44 +01:00 |
|
|
|
ce909d5e72
|
Simplify crawl by using queue
|
2021-03-11 14:30:30 +01:00 |
|
|
|
0690824e9e
|
Minor reduction in dependencies
|
2021-03-11 12:44:41 +01:00 |
|
|
|
f3876b4745
|
Remove instance filtering/sorting
|
2021-03-11 12:22:28 +01:00 |
|
|
|
71cdf40bae
|
Simplified crawl loop, improved output
|
2021-03-11 03:25:51 +01:00 |
|
|
|
33580c52ea
|
Add total_instances count, use consts
|
2021-03-11 03:01:15 +01:00 |
|
|
|
4426b0edf9
|
Improved crawling code, allow null icon
|
2021-03-11 02:41:24 +01:00 |
|
|
|
342fe6ded8
|
Make requests to same instance in parallel
|
2021-03-11 02:30:56 +01:00 |
|
|
|
27c23bd5d8
|
Improve output, refactor code, add timeout
|
2021-03-11 02:25:10 +01:00 |
|
|
|
c45771d9e7
|
Add instance name and icon
|
2021-03-10 18:17:17 +01:00 |
|
|
|
9cbed8f2a9
|
Output instance info as json
|
2021-03-10 18:06:21 +01:00 |
|
|
|
cb4e6e221a
|
Add license
|
2021-03-10 17:39:43 +01:00 |
|
|
|
45ec994271
|
Crawl for number of online and total users
|
2021-03-09 19:51:35 +01:00 |
|
|
|
21be5c58ee
|
Fetch basic data from lemmy.ml
|
2021-03-09 19:11:13 +01:00 |
|
|
|
5f871dcc96
|
Initial commit
|
2021-03-09 18:19:23 +01:00 |
|
