Hi. I’m just prototyping Crowdsec, inluding appsec. I have a multi node setups
- Some (1 for now) dedicated log-processors, which receive logs from an HTTP datasource
- Some (1 for now) local-api
The logs part is working as expected. I’m now playçing at adding appsec to the mix.
First issue I found : the appsec datasource can only be enabled on local-api, not on log processors when running a multi node setup. I couldn’t find any info about this in the docs, maybe I missed it. If we enable it on a log-processor, every requests will get a 401.
So, I moved this data source to my local-api node. Now appsec is mostly working. I’m using crowdsecurity/appsec-default and crowdsecurity/crs (plus some custom conf to disable false positives). Now, everything seems to be working. For example, I can trigger alerts when targeting the /crowdsec-test-NtktlJHV4TfBSK3wvlhiOBnl test URL. My issue is that although alerts are triggered, I couldn’t get decision to be taken by crowdsec.
I installed the crowdsecurity/appsec-generic-rules collections, which includes the appsec-logs parser. As I understand it, this parser works on internal events, not log lines like most parsers. But no matter how many in-band or out-of-band rules is triggered : all I have is alerts. No decision. And metrics show no scenario, nor parser data
/ $ cscli metrics show scenarios
╭────────────────────────────────────────────────────────────────────────╮
│ Scenario Metrics │
├──────────┬───────────────┬───────────┬──────────────┬────────┬─────────┤
│ Scenario │ Current Count │ Overflows │ Instantiated │ Poured │ Expired │
├──────────┼───────────────┼───────────┼──────────────┼────────┼─────────┤
╰──────────┴───────────────┴───────────┴──────────────┴────────┴─────────╯
/ $ cscli metrics show parsers
╭────────────────────────────────────╮
│ Parser Metrics │
├─────────┬──────┬────────┬──────────┤
│ Parsers │ Hits │ Parsed │ Unparsed │
├─────────┼──────┼────────┼──────────┤
╰─────────┴──────┴────────┴──────────╯
/ $
How is this supposed to work ? I’m running CrowdSec 1.7.8, with this config
api:
client:
credentials_path: /secrets/local_api_creds.yaml
insecure_skip_verify: false
server:
console_path: /secrets/console.yaml
enable: true
listen_uri: 127.0.0.1:8855
online_client:
credentials_path: /secrets/online_api_creds.yaml
pull:
blocklists: true
community: true
sharing: true
profiles_path: /secrets/profiles.yaml
trusted_ips:
- 127.0.0.1
trusted_proxies:
- 127.0.0.1
use_forwarded_for_headers: true
common:
log_media: stdout
config_paths:
data_dir: /local/data
hub_dir: /local/hub
index_path: /local/hub/.index.json
notification_dir: /secrets/notifications
pattern_dir: /etc/crowdsec/patterns
plugin_dir: /usr/local/lib/crowdsec
simulation_path: /etc/crowdsec/simulation.yaml
crowdsec_service:
acquisition_dir: /secrets/acquis.d
enable: true
cscli:
output: human
db_config:
db_name: crowdsec
flush:
agents_autodelete:
api_key: 30d
cert: 30d
bouncers_autodelete:
api_key: 30d
cert: 30d
max_age: 365d
host: ${PGHOST}
password: ${PGPASSWORD}
port: ${PGPORT}
type: pgx
user: ${PGUSER}
plugin_config:
group: crowdsec
user: crowdsec
prometheus:
enabled: true
level: full
listen_addr: 127.0.0.1
listen_port: 6060
With my profiles.yaml being
decisions:
- duration: 30m
type: ban
duration_expr: Sprintf('%dm', (GetDecisionsSinceCount(Alert.GetValue(), "168h") + 1) * (GetDecisionsSinceCount(Alert.GetValue(), "336h") + 1) * 20)
filters:
- Alert.Remediation == true && Alert.GetScope() in ["Ip", "Range"]
name: default_ip_remediation
notifications:
- http_default
on_success: break
(same profile working nicely on my log-processor nodes)
And my acquis.d/appsec.yaml
appsec_configs:
- crowdsecurity/appsec-default
- crowdsecurity/crs
- dbd/appsec-exceptions # These are just a few RemoveOutBandRuleByID for some vhosts
labels:
type: appsec
listen_addr: 127.0.0.1:8858
path: /
source: appsec