Syslog is not getting parsed

1

Hello there,

a few days ago i set up CrowdSec, but the problem is that my syslogs are not getting parsed and I don’t know why.

My Traefik logs are getting parsed, so this is working fine, also Nginx logs from another server are getting parsed.

This are the parsing metrics:

2023-02-14 09_43_45-Ubuntu Server
2023-02-14 09_43_45-Ubuntu Server1117×943 101 KB

Bouncers:
host-firewall-bouncer-server1 x.x.x.x :heavy_check_mark: 2023-02-14T09:01:52Z crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780 api-key
cloudflare-bouncer-server1 x.x.x.x :heavy_check_mark: 2023-02-14T09:01:44Z crowdsec-cloudflare-bouncer v0.2.1-6b30687c25027607083926cb2112dd06e04dae59 api-key
traefik-bouncer-server1 x.x.x.x :heavy_check_mark: 2023-02-14T09:01:32Z Go-http-client 1.1 api-key
host-firewall-bouncer-server2 x.x.x.x :heavy_check_mark: 2023-02-14T09:01:53Z crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780 api-key
host-firewall-bouncer-server3 x.x.x.x :heavy_check_mark: 2023-02-11T00:29:52Z crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780 api-key

Collections:
crowdsecurity/base-http-scenarios :heavy_check_mark: enabled 0.6 /etc/crowdsec/collections/base-http-scenarios.yaml
crowdsecurity/http-cve :heavy_check_mark: enabled 1.9 /etc/crowdsec/collections/http-cve.yaml
crowdsecurity/iptables :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/iptables.yaml
crowdsecurity/linux :heavy_check_mark: enabled 0.2 /etc/crowdsec/collections/linux.yaml
crowdsecurity/nginx :heavy_check_mark: enabled 0.2 /etc/crowdsec/collections/nginx.yaml
crowdsecurity/sshd :heavy_check_mark: enabled 0.2 /etc/crowdsec/collections/sshd.yaml
crowdsecurity/traefik :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/traefik.yaml
crowdsecurity/whitelist-good-actors :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/whitelist-good-actors.yaml

Parsers:
crowdsecurity/dateparse-enrich :heavy_check_mark: enabled 0.2 /etc/crowdsec/parsers/s02-enrich/dateparse-enrich.yaml
crowdsecurity/docker-logs :heavy_check_mark: enabled 0.1 /etc/crowdsec/parsers/s00-raw/docker-logs.yaml
crowdsecurity/geoip-enrich :heavy_check_mark: enabled 0.2 /etc/crowdsec/parsers/s02-enrich/geoip-enrich.yaml
crowdsecurity/http-logs :heavy_check_mark: enabled 1.1 /etc/crowdsec/parsers/s02-enrich/http-logs.yaml
crowdsecurity/iptables-logs :heavy_check_mark: enabled 0.3 /etc/crowdsec/parsers/s01-parse/iptables-logs.yaml
crowdsecurity/nginx-logs :heavy_check_mark: enabled 1.3 /etc/crowdsec/parsers/s01-parse/nginx-logs.yaml
crowdsecurity/sshd-logs :heavy_check_mark: enabled 2.0 /etc/crowdsec/parsers/s01-parse/sshd-logs.yaml
crowdsecurity/syslog-logs :heavy_check_mark: enabled 0.8 /etc/crowdsec/parsers/s00-raw/syslog-logs.yaml
crowdsecurity/traefik-logs :heavy_check_mark: enabled 0.5 /etc/crowdsec/parsers/s01-parse/traefik-logs.yaml
crowdsecurity/whitelists :heavy_check_mark: enabled 0.2 /etc/crowdsec/parsers/s02-enrich/whitelists.yaml
custom-whitelists.yaml :house: enabled,local /etc/crowdsec/parsers/s02-enrich/custom-whitelists.yaml

Does anyone has an idea what’s the probleme here and how to solve it?

Greetings

2

Could you post some example lines of those which reside in syslog folder? it may be they are not actually syslog format, or a different format we are not expecting.

Also could you provide your acquis.yaml

3

@iiAmLoz
it should be normal syslog format (I’m using Ubuntu 20.04 or 22.04 on all of my servers)

For example the auth.log:

2023-02-15 21_52_22-Ubuntu Server (intern)
2023-02-15 21_52_22-Ubuntu Server (intern)1587×229 110 KB

syslog:
Feb 15 22:01:58 docker systemd[1]: run-docker-runtime\x2drunc-moby-03e432402e988c74b3e271459a719c581237e4b46f3ad0b88e490eeb3292596a-runc.iMII42.mount: Succeeded.
Feb 15 22:02:03 docker systemd[1]: run-docker-runtime\x2drunc-moby-19ebcf7ea5ff8b3f946845e8f372efb1290170d2eb244def5f9eb543949ed20f-runc.bnbWt9.mount: Succeeded.
Feb 15 22:02:03 docker systemd[1472192]: run-docker-runtime\x2drunc-moby-19ebcf7ea5ff8b3f946845e8f372efb1290170d2eb244def5f9eb543949ed20f-runc.bnbWt9.mount: Succeeded.

kern.log:
Feb 15 22:00:03 docker kernel: [481676.656879] docker0: port 1(veth6630647) entered forwarding state
Feb 15 22:00:05 docker kernel: [481677.779408] docker0: port 1(veth6630647) entered disabled state
Feb 15 22:00:05 docker kernel: [481677.779637] veth746842f: renamed from eth0
Feb 15 22:00:05 docker kernel: [481677.935169] docker0: port 1(veth6630647) entered disabled state
Feb 15 22:00:05 docker kernel: [481677.942317] device veth6630647 left promiscuous mode
Feb 15 22:00:05 docker kernel: [481677.942325] docker0: port 1(veth6630647) entered disabled state

This is my acquis.yaml:

filenames:

  • /logs/syslog/auth.log
  • /logs/syslog/syslog
  • /logs/syslog/kern.log
  • /logs/syslog/mail.log
    labels:
    type: syslog

filenames:

  • /logs/traefik/*.log
    labels:
    type: traefik

(Sorry for the format, looks like it’s getting converted to markdown, I’m not allowed to post more than one picture per post)

Hope this helps you.

Greetings

4

Yeah so it seems parsing is fine, most likely your syslog have lines we dont care about, the only one I can see from the image is sshd but depending on how you configured it (accept keys only) we may not see password brute forcing.