Frequent IP Bans by CrowdSec Due to Synapse Matrix Requests: Seeking Help and Solutions

1

I’ve been facing a recurring issue with CrowdSec and wanted to share my experience and seek advice. My IP address has been getting banned quite often, and I suspect that it’s related to the requests made to my Synapse matrix. However, there doesn’t seem to be an existing collection for the Synapse matrix in CrowdSec. I’m not even sure that a collection would help me out here.

The Problem:

  • My IP address is frequently banned by CrowdSec.
  • I believe this is linked to legitimate requests to my Synapse matrix.
  • I’m exploring solutions to address this issue while avoiding the use of whitelists, especially since the problem persists even when I’m not on my home IP address, such as when I’m at work

What I’ve Tried:

  • I’ve reviewed my server logs but can’t pinpoint the exact cause.
  • I’m considering creating a custom collection for the Synapse matrix.

Seeking Your Expertise:

  • Has anyone encountered a similar issue with CrowdSec?
  • Do you have any suggestions or insights on how to handle this situation?
  • Can you share tips on creating custom collections or optimizing CrowdSec settings?

I appreciate any help and advice you can provide.

Thank you in advance for your assistance!

2

So a custom collection will only help if you want to make some form of a whitelist. You need to inform the engine in some manner that these requests are infact legitimate, and dont count them towards a generic scenario like http crawling.

So it would help us to debug futher if you can provide these

  • Which scenario is triggered?
  • How is CrowdSec configured? EG: monitors a reverse proxy which synapse is a subdomain?
  • cscli alerts inspect can provide you the alert information which can help us debug further. (Make sure to redact any PII)
3

1. Triggered Scenarios:

  • LePresidente/http-generic-403-bf (Occurs most of the time)
  • crowdsecurity/http-probing (Occurs from time to time)
  • crowdsecurity/http-crawl-non_statics (Occurred once)
    (Sometimes, my local subnet is banned, and other times, it’s my public IP.)
    (and apperently synapse doesn’t look to be as much of a problem but I know / think it causes problems sometimes but I have too much alerts because of homeassistant to be able to check them all)

2. Configuration:
I’m using Traefik as a reverse proxy, which utilizes a CrowdSec bouncer as middleware to analyze connections.

CrowdSec is also using some collections for nextcloud, and some more apps

3. Observations with cscli alerts inspect:

I’ve noticed that HomeAssistant is often flagged, as well as Nextcloud for my calendars (Nextcloud is added as a collection). Home Assistant not yet, but I just noticed there was a first version released.

Finally, I’m uncertain about the specific alert information you expect, as the information I observed didn’t appear valuable unless I include the --debug flag or an alert UUID.

here one of the home asistant ones:

{"capacity":5,"created_at":"2023-10-28T19:23:48Z","decisions":[{"duration":"-1h36m41.597050911s","id":22759203,"origin":"crowdsec","scenario":"LePresidente/http-generic-403-bf","scope":"Ip","simulated":false,"type":"ban","value":"[IP ADDRESS]"}],"events":[{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[ISP]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[COUNTRY]"},{"key":"SourceRange","value":"[IP ADDRESS RANGE]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/api/webhook/0526b8fe7d840649449e4c535066f8fdda701ae9c2230028f48ab5e8b17e6d24"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"POST"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[IP ADDRESS]"},{"key":"timestamp","value":"2023-10-28T19:23:44Z"},{"key":"traefik_router_name","value":"homeassistant@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 19:23:44 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[ISP]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[COUNTRY]"},{"key":"SourceRange","value":"[IP ADDRESS RANGE]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/api/webhook/0526b8fe7d840649449e4c535066f8fdda701ae9c2230028f48ab5e8b17e6d24"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"POST"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[IP ADDRESS]"},{"key":"timestamp","value":"2023-10-28T19:23:44Z"},{"key":"traefik_router_name","value":"homeassistant@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 19:23:44 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[ISP]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[COUNTRY]"},{"key":"SourceRange","value":"[IP ADDRESS RANGE]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/api/webhook/0526b8fe7d840649449e4c535066f8fdda701ae9c2230028f48ab5e8b17e6d24"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"POST"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[IP ADDRESS]"},{"key":"timestamp","value":"2023-10-28T19:23:45Z"},{"key":"traefik_router_name","value":"homeassistant@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 19:23:45 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[ISP]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[COUNTRY]"},{"key":"SourceRange","value":"[IP ADDRESS RANGE]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/api/webhook/0526b8fe7d840649449e4c535066f8fdda701ae9c2230028f48ab5e8b17e6d24"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"POST"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[IP ADDRESS]"},{"key":"timestamp","value":"2023-10-28T19:23:46Z"},{"key":"traefik_router_name","value":"homeassistant@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 19:23:46 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[ISP]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[COUNTRY]"},{"key":"SourceRange","value":"[IP ADDRESS RANGE]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/api/webhook/0526b8fe7d840649449e4c535066f8fdda701ae9c2230028f48ab5e8b17e6d24"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"POST"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[IP ADDRESS]"},{"key":"timestamp","value":"2023-10-28T19:23:46Z"},{"key":"traefik_router_name","value":"homeassistant@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 19:23:46 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[ISP]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[COUNTRY]"},{"key":"SourceRange","value":"[IP ADDRESS RANGE]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/api/webhook/0526b8fe7d840649449e4c535066f8fdda701ae9c2230028f48ab5e8b17e6d24"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"POST"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[IP ADDRESS]"},{"key":"timestamp","value":"2023-10-28T19:23:47Z"},{"key":"traefik_router_name","value":"homeassistant@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 19:23:47 +0000 UTC"}],"events_count":6,"id":2489,"labels":null,"leakspeed":"10s","machine_id":"localhost","message":"Ip [IP ADDRESS] performed 'LePresidente/http-generic-403-bf' (6 events over 2.904844487s) at 2023-10-28 19:23:47.463010436 +0000 UTC","scenario":"LePresidente/http-generic-403-bf","scenario_hash":"d03fa7fbb3179407f221bc4e11d177422d21e5adcdcf408edf5f8b0ef492741f","scenario_version":"0.5","simulated":false,"source":{"as_name":"[ISP]","as_number":"6848","cn":"[COUNTRY]","ip":"[IP ADDRESS]","latitude":[lat],"longitude":[long],"range":"[IP ADDRESS RANGE]","scope":"Ip","value":"[IP ADDRESS]"},"start_at":"2023-10-28 19:23:44.55816845 +0000 UTC","stop_at":"2023-10-28 19:23:47.463012937 +0000 UTC","uuid":"1d7c0753-e280-4bf5-a733-0d843277ee23"}
0

and a nextcloud one:

{"capacity":10,"created_at":"2023-10-28T06:00:53Z","decisions":[{"duration":"-11h30m36.682694321s","id":22585642,"origin":"crowdsec","scenario":"crowdsecurity/http-probing","scope":"Ip","simulated":false,"type":"ban","value":"[ip address]"}],"events":[{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/addressbooks/users/[username]/z-server-generated--system/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:45Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:45 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/calendars/[username]/contact_birthdays/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:46Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:46 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/calendars/[username]/rveille-2/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:47Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:47 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/calendars/[username]/transport-1/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:47Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:47 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/calendars/[username]/activit-1/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:48Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:48 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/calendars/[username]/magasins-cuisiner-manger/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:48Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:48 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/calendars/[username]/rappelles-1/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:50Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:50 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/calendars/[username]/cours/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:51Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:51 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/calendars/[username]/devoirs/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:51Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:51 +0000 UTC"},{"meta":[{"key":"ASNNumber","value":"6848"},{"key":"ASNOrg","value":"[isp]"},{"key":"IsInEU","value":"true"},{"key":"IsoCode","value":"[country]"},{"key":"SourceRange","value":"[ip address range]"},{"key":"datasource_path","value":"/var/log/crowdsec/traefik.log"},{"key":"datasource_type","value":"file"},{"key":"http_args_len","value":"0"},{"key":"http_path","value":"/remote.php/dav/calendars/[username]/tudes/"},{"key":"http_status","value":"403"},{"key":"http_user_agent","value":"-"},{"key":"http_verb","value":"PROPFIND"},{"key":"log_type","value":"http_access-log"},{"key":"service","value":"http"},{"key":"source_ip","value":"[ip address]"},{"key":"timestamp","value":"2023-10-28T06:00:52Z"},{"key":"traefik_router_name","value":"nextcloud@file"},{"key":"user","value":"-"}],"timestamp":"2023-10-28 06:00:52 +0000 UTC"}],"events_count":12,"id":2429,"labels":null,"leakspeed":"10s","machine_id":"localhost","message":"Ip [ip address] performed 'crowdsecurity/http-probing' (12 events over 27.121386629s) at 2023-10-28 06:00:52.375028393 +0000 UTC","scenario":"crowdsecurity/http-probing","scenario_hash":"983c356924b6e01f709b3c2d901ceb4e4ce1abe6e840048558f2824a4c4a6719","scenario_version":"0.3","simulated":false,"source":{"as_name":"[isp]","as_number":"6848","cn":"[country]","ip":"[ip address]","latitude":[lat],"longitude":[long],"range":"[ip address range]","scope":"Ip","value":"[ip address]"},"start_at":"2023-10-28 06:00:25.253644037 +0000 UTC","stop_at":"2023-10-28 06:00:52.375030666 +0000 UTC","uuid":"a5ba2eb3-9ace-41a5-97a7-3ff5b3b56162"}

sorry for the delay