False positives occuring with Nextcloud and eBook reader Kavita

I have CrowdSec running in front of a Nextcloud instance, and if one of my users attempts to sync files using the Nextcloud desktop client, they will usually get themselves banned. Similar things happen when a user is turning pages in a book on Kavita.

I’m pretty new to CrowdSec, so I’m not sure what information would be helpful here, but here’s the relevant alerts for Nextcloud:

| 455 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 15:12:16.780727354  |
| 454 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 15:12:02.230303367  |
| 453 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 15:05:46.160224937  |
| 452 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 15:04:16.460203323  |
| 451 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 15:00:55.198849852  |
| 450 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 14:58:45.905619787  |
| 449 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 14:53:49.400745564  |
| 448 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 14:53:14.750015725  |
| 447 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 14:51:22.467476162  |
| 446 | Ip:X.X.X.X           | crowdsecurity/http-probing           | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 14:38:03.256405305  |
| 445 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 14:38:58.110206824  |
| 444 | Ip:X.X.X.X           | crowdsecurity/http-probing           | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 14:21:51.721533815  |
| 443 | Ip:X.X.X.X           | crowdsecurity/http-crawl-non_statics | US      |  Time Warner Cable Internet    | ban:1     | 2021-10-31 14:21:08.864634398  |

Please let me know if I can provide any other information that would help.

Hi there,

Got more or less the same issue.
Installed crowdsec on my Nextcloud server and got banned at my first login attempt.

# cscli decisions list
+-------+----------+-----------------+--------------------------------------+--------+---------+---------+--------+--------------------+----------+
|  ID   |  SOURCE  |   SCOPE:VALUE   |                REASON                | ACTION | COUNTRY |   AS    | EVENTS |     EXPIRATION     | ALERT ID |
+-------+----------+-----------------+--------------------------------------+--------+---------+---------+--------+--------------------+----------+
| 43984 | crowdsec | Ip:x.x.x.x| crowdsecurity/http-crawl-non_statics | ban    | FR      |  Orange |     47 | 1h57m54.664254699s |       62 |
+-------+----------+-----------------+--------------------------------------+--------+---------+---------+--------+--------------------+----------+

Then took a look at the http-crawl-non_statics scenario :

https://hub.crowdsec.net/author/crowdsecurity/configurations/http-crawl-non_statics

Thus, it is just about high rate requests on non static ressources.
Unfortunately, the Nextcloud browser-side code is known to generate a lot of requests (which tends to offer poor user experience).

However, quicly parsed my Apache logs :

cat drive.access.log | cut -d' ' -f8 | cut -d'/' -f-4 | sort | uniq -c
...
    1 /index.php/js/core
      3 /index.php/login
      1 /index.php/svg/activity
      1 /index.php/svg/circles
     43 /index.php/svg/contacts
    187 /index.php/svg/core
      2 /index.php/svg/deck
      8 /index.php/svg/files
...

Hummm, looks that multiple requests to /index.php/svg is the root cause.

Then, took a look at :

https://discourse.crowdsec.net/t/help-with-whitelist-rules-expression-with-portion-of-url/41

but due to some syntax errors with usage of ‘expression’ in the yaml config file, had to adjust based on :

https://github.com/antonmedv/expr/blob/master/docs/Language-Definition.md

So, finally, I ended with creating the following whitelist file :

vi /etc/crowdsec/parsers/s02-enrich/mynextcloudwhitelist.yaml
name: crowdsecurity/mynextcloudwhitelist
description: "Whitelist events from nextcloud legitimate access"
whitelist:
  reason: "ignore index.php/svg"
  expression:
    - evt.Parsed.request contains "'/index.php/svg/'"

Then :

systemctl restart crowdsec

Removed the false-positive decision :

cscli decisions delete --id 43984

Job done (it looks).

Please, consider this as a candidate workaround ; not a perfect fix.

Regards.

1 Like

Hello,

Thanks for looking into it !
If either of you would be able to provide more logs of what is going on, we might be able to come up with an improved whitelist (hopefully).

On the side @caramb, can you tell me more on the issue you faced with :

1 Like

Hello Thibault,

Regarding the syntax error issue, the post suggests to use the following :

name: my-custom-whitelist
description: my custom whitelists
whitelist:
  reason: do not ban jellyfin users
  expression:
     - "'/jellyfin' in evt.Parsed.request"

So tried :

name: crowdsecurity/mynextcloudwhitelist
description: "Whitelist events from nextcloud legitimate access"
whitelist:
  reason: "ignore index.php/svg"
  expression:
    - "'/index.php/svg/' in evt.Parsed.request"

But this causes a syntax error :

-- L'unité (unit) crowdsec.service a commencé à démarrer.
nov. 08 13:08:30 nextcloud crowdsec[17901]: time="08-11-2021 13:08:30" level=fatal msg="Unable to compile whitelist expression ''/index.php/svg/' in evt.Parsed.request' : invalid operation: in (mismatched types string and string) (1:19)\
nov. 08 13:08:30 nextcloud systemd[1]: crowdsec.service: Control process exited, code=exited, status=1/FAILURE

This is the reason why I replaced

"'/index.php/svg/' in evt.Parsed.request"

with

evt.Parsed.request contains "'/index.php/svg/'"

Regards.

What logs are you looking for? I’m not sure where to find them in the Docker container.

It would be the logs of the webservice itself, you might be able to get them directly from docker logs -n XXXX of the container running the nextcloud service.