Hi all,
I hope one can help me out here, I’m new to crowdsec and just have a basic webserver.
I want to protect ssh from failed logins, as well as blocking IPs having repetitive HTTP 400-599 errors.
On fail2ban i use simple regex on the access logs of nginx: ^."(GET|HEAD|POST|PUT|DELETE|CONNECT|OPTIONS|TRACE|PATCH)." (4[0-9][0-9]|5[0-9][0-9]) .*$
How can I achieve this with crowdsec?
You can create a scenario for example
#/etc/crowdsec/scenarios/http-above-400.yaml
type: leaky
name: me/http-above-400
description: "Detect crawl from single ip with status above 400"
filter: "evt.Meta.log_type in ['http_access-log', 'http_error-log'] && evt.Meta.http_status != '' && int(evt.Meta.http_status) >= 400"
leakspeed: 5s
capacity: 20
#debug: true
cache_size: 5
groupby: "evt.Meta.source_ip + '/' + evt.Parsed.target_fqdn"
blackhole: 1m
labels:
remediation: true
I put a basic 20 and leakspeed of 5. However, you can fine tune those to your preferences, this triggers for all request including static content.
Remember we already have official hub scenario that detect crawlers, so you use case might already be covered. https://hub.crowdsec.net/
Hi I’m sorry, I’m honestly absolutely new to this. Do I just create that yaml file, how would I activate it or assign it to actual nginx logs?
p.s. it’s not just for crawlers, but people actively trying to download stuff from my site using HTTP Headers that are not verfied by my backend and I’m sending them to 401. But there are thousands of these logs.
You just create the file within the folder and then reload crowdsec with
sudo systemctl restart crowdsec
The scenario already knows if for nginx with
# Match on http log source
evt.Meta.log_type in ['http_access-log', 'http_error-log']
Then rest of the filter is just your use case of match anything above 400 status, we dont need to match on verb since you match for all
EDIT: this is under the presumption you have already setup the nginx log source
it actually works like that ! That’s awesome, thank you so much.
Just one question tho, I don’t quite understand the leakspeed, capacity, cache size.
As well as how to define the ‘punishment’
Leakspeed in simple terms, is like… the timeframe that the amount of ‘logs’ (capacity) are needed for this scenario to be triggered and ban the source.
What does the cache_size and blackhole do? And how to define the ‘punishment’ ?
I’d like to put it like… max. 5 occurences in last 5 minutes, and ban for 10 hours.
Also on 1 time removing the ban ( using cscli decisions delete -i < IP > ) the scenario is not triggered anymore for this ip?
Got it 
leak_time is indeed the duration for the remediation to be triggered.
capactiy is amount of ‘hits’ within the leak_time
blockhole is just to ignore more triggers within the blackhole’s time, just if it got triggered LOTS of times within the leak_time.
changing the ban to 10h could be done by setting remediation: 10h
I only don’t understand the cache_size, but that’s fine ! Thanks for the quick responses @iiAmLoz
Remediation time is handled by profiles. Format | CrowdSec setting it to anything but true in the scenario will cause no bans to happen.
cache_size
Cache size limits the amount of “in memory” objects it means that at any point it will only ever hold the last 5 to reduce RAM allocation. This is useful for high capacity bucket EG: 1000 capacity but only 10 in memory at once
I’m new to this too.
Do we see right away the compilation fo the alert on the online dashboard at crowdsec website or does it refresh only every few hours?
I’ve combined suricata and crowdsec on one of my setup for the moment.
Also is it okey to let fail2ban to run also alongside crowdsec?
Yes you should, it may take up to 30 seconds to reflect.
Also is it okey to let fail2ban to run also alongside crowdsec?
Of course! just note when an IP that gets banned you must be sure to check both f2b and CrowdSec
thanks for those specs