i there,
I do have a problem with a running crowdsec docker container. I got everything set up except the ability to parse the docker logs. I create a new variable in docker for the docker.sock of Unraid. This is then correct passed through to the crwodsec docker container (like you could see in the cscli metrics screenshot)
I manually modified the acquis.yaml and added my docker container to it.
Quote
filenames:
-
/var/log/nginx/*.log
-
./tests/nginx/nginx.log
#this is not a syslog log, indicate which kind of logs it is
labels:
type: nginx
filenames:
-
/var/log/auth.log
-
/var/log/syslog
labels:
type: syslog
filename: /var/log/apache2/*.log
labels:
type: apache2
source: docker
container_name:
-
Nextcloud
-
immich
-
paperless-ngx
-
vaultwarden
labels:
type: log_type
When I now run cscli metrics from within the crowdsec container it will show:
So it looks like that the crowdsec container can’t parse the other docker containers but how can I achive that crowdsec can parse the other docker container logs?
Regards Shorty
So if you define all your containers in the same docker source it cant distinguish the right labels.type as they are all using the same type.
If you want to have a more simplistic configuration file and still achieve this single acquisition, I would recommend to use discovery via docker labels options.
So firstly change the docker acquistion to:
source: docker
use_container_labels: true
Then on each container that you want you to be discovered you would add labels to their compose or run command:
webserver:
image: ghcr.io/paperless-ngx/paperless-ngx:latest
restart: unless-stopped
depends_on:
- db
- broker
ports:
- "8000:8000"
volumes:
- data:/usr/src/paperless/data
- media:/usr/src/paperless/media
- ./export:/usr/src/paperless/export
- ./consume:/usr/src/paperless/consume
env_file: docker-compose.env
environment:
PAPERLESS_REDIS: redis://broker:6379
PAPERLESS_DBENGINE: mariadb
PAPERLESS_DBHOST: db
PAPERLESS_DBUSER: paperless # only needed if non-default username
PAPERLESS_DBPASS: paperless # only needed if non-default password
PAPERLESS_DBPORT: 3306
labels:
- crowdsec.enable=true
- crowdsec.labels.type=paperless-ngx
Example above is for paperless-ngx, crowdsec.enable=true informs that this container should be monitored and crowdsec.labels.type is the type that should be used when parsing the lines, most of them time on the Hub you will see a file example which shows the type that must be defined.
However, if you dont mind having an extended configuration you must define each container into a separate block example:
source: docker
container_name:
- vaultwarden
labels:
type: vaultwarden
---
source: docker
container_name:
- paperless-ngx
labels:
type: paperless-ngx
and etc etc you get the idea because the type is what drives which parser to use and since they are all the same type basically it cant parse any of them correctly.
thanks for your reply.
Can I also do this with UUID of the container instead of the label? I’m currently running on unraid and therefore it is a bit difficult to use labels with docker.
Regards Shorty
if you cant add labels, then simply you have to go to verbose configuration route as shown at the bottom and simply can switch container_name for container_id but most likely you can already use the name as you can see from your metrics CrowdSec finds them but it simply doesnt know the correct label.
ok, I modified it to
filenames:
- /var/log/auth.log
- /var/log/syslog
labels:
type: syslog
---
filename: /var/log/apache2/*.log
labels:
type: apache2
---
source: docker
container_name:
- Nextcloud
container_id:
- f365f1d51f2d
labels:
type: log_type
---
source: docker
container_name:
- immich
container_id:
- 57096ad4af3c
labels:
type: log_type
---
source: docker
container_name:
- paperless-ngx
container_id:
- 86d2a14328fd
labels:
type: log_type
---
source: docker
container_name:
- vaultwarden
container_id:
- 4aeef260312a
labels:
type: log_type
but now I’m stuck with the error message coming out of it
time="2025-06-20T15:20:56+02:00" level=fatal msg="crowdsec init: while loading acquisition config: while configuring datasource of type docker from /etc/crowdsec/acquis.yaml (position 2): while parsing DockerAcquisition configuration: yaml: unmarshal errors:\n line 7: field type not found in type dockeracquisition.DockerConfiguration"
but you dont put log_type, you replace it with the correct log_type like in my examples I shown you. Also you dont need to specify both container_name and container_id you just need to use one and it should be container_name.
it should be formatted like so:
source: docker
container_name:
- vaultwarden
container_id:
- 4aeef260312a
labels:
type: log_type
Note the type is indented
1 Like
sorry to say but outcome is still the same.
Lines parsed it listed with “-”
Like you mentioned earlier I changed my configuration to
filenames:
- /var/log/auth.log
- /var/log/syslog
labels:
type: syslog
---
filename: /var/log/apache2/*.log
labels:
type: apache2
---
source: docker
container_name:
- vaultwarden
container_id:
- 4aeef260312a
labels:
type: vaultwarden
---
source: docker
container_name:
- Nextcloud
container_id:
- f365f1d51f2d
labels:
type: Nextcloud
---
source: docker
container_name:
- npm
container_id:
- 6054fc637372
labels:
type: npm
I’m a bit stuck here.
This is how my docker log looks like
time="2025-06-28T11:33:34+02:00" level=info msg="Crowdsec v1.6.9-40b8cfe6"
time="2025-06-28T11:33:34+02:00" level=info msg="Loading prometheus collectors"
time="2025-06-28T11:33:34+02:00" level=info msg="Loading CAPI manager"
time="2025-06-28T11:33:35+02:00" level=info msg="CAPI manager configured successfully"
time="2025-06-28T11:33:35+02:00" level=error msg="Machine is not enrolled in the console, can't synchronize with the console"
time="2025-06-28T11:33:35+02:00" level=info msg="Start push to CrowdSec Central API (interval: 15s once, then 10s)"
time="2025-06-28T11:33:35+02:00" level=info msg="Start sending metrics to CrowdSec Central API (interval: 21m8s once, then 30m0s)"
time="2025-06-28T11:33:35+02:00" level=info msg="last CAPI pull is newer than 1h30, skip."
time="2025-06-28T11:33:35+02:00" level=info msg="Start pull from CrowdSec Central API (interval: 2h3m14s once, then 2h0m0s)"
time="2025-06-28T11:33:35+02:00" level=info msg="capi metrics: sending"
time="2025-06-28T11:33:35+02:00" level=info msg="CrowdSec Local API listening on 0.0.0.0:8080"
time="2025-06-28T11:33:35+02:00" level=info msg="Loading grok library /etc/crowdsec/patterns"
Local agent already registered
Check if lapi needs to register an additional agent
/etc/crowdsec was found in a volume
Running hub update
Nothing to do, the hub index is up to date.
/var/lib/crowdsec/data was found in a volume
Running hub upgrade
Action plan:
🔄 check & update data files
Running: cscli parsers install "crowdsecurity/docker-logs"
Nothing to do.
Running: cscli parsers install "crowdsecurity/cri-logs"
Nothing to do.
Running: cscli collections install "crowdsecurity/traefik"
Nothing to do.
Running: cscli collections install "crowdsecurity/http-cve"
Nothing to do.
Running: cscli collections install "LePresidente/authelia"
Nothing to do.
time="2025-06-28T11:33:36+02:00" level=info msg="Loading enrich plugins"
time="2025-06-28T11:33:36+02:00" level=info msg="Successfully registered enricher 'GeoIpCity'"
time="2025-06-28T11:33:36+02:00" level=info msg="Successfully registered enricher 'GeoIpASN'"
time="2025-06-28T11:33:36+02:00" level=info msg="Successfully registered enricher 'IpToRange'"
time="2025-06-28T11:33:36+02:00" level=info msg="Successfully registered enricher 'reverse_dns'"
time="2025-06-28T11:33:36+02:00" level=info msg="Successfully registered enricher 'ParseDate'"
time="2025-06-28T11:33:36+02:00" level=info msg="Successfully registered enricher 'UnmarshalJSON'"
time="2025-06-28T11:33:36+02:00" level=info msg="Loading parsers from 10 files"
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s00-raw/cri-logs.yaml stage=s00-raw
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s00-raw/docker-logs.yaml stage=s00-raw
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 2 parser nodes" file=/etc/crowdsec/parsers/s00-raw/syslog-logs.yaml stage=s00-raw
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/authelia-logs.yaml stage=s01-parse
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/sshd-logs.yaml stage=s01-parse
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s01-parse/traefik-logs.yaml stage=s01-parse
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/dateparse-enrich.yaml stage=s02-enrich
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/geoip-enrich.yaml stage=s02-enrich
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/http-logs.yaml stage=s02-enrich
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 1 parser nodes" file=/etc/crowdsec/parsers/s02-enrich/whitelists.yaml stage=s02-enrich
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 11 nodes from 3 stages"
time="2025-06-28T11:33:36+02:00" level=info msg="No postoverflow parsers to load"
time="2025-06-28T11:33:36+02:00" level=info msg="Loading 52 scenario files"
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=green-breeze name=crowdsecurity/apache_log4j2_cve-2021-44228
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=billowing-sky name=crowdsecurity/spring4shell_cve-2022-22965
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=bold-mountain name=crowdsecurity/grafana-cve-2021-43798
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=autumn-star name=crowdsecurity/CVE-2022-44877
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=autumn-wind name=crowdsecurity/http-backdoors-attempts
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=withered-river name=crowdsecurity/pulse-secure-sslvpn-cve-2019-11510
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=weathered-sun name=crowdsecurity/vmware-vcenter-vmsa-2021-0027
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=wispy-sun name=crowdsecurity/http-sensitive-files
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=lingering-grass name=crowdsecurity/CVE-2024-0012
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=cool-violet name=crowdsecurity/CVE-2022-41082
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=bold-sunset name=crowdsecurity/ssh-bf
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=muddy-mountain name=crowdsecurity/ssh-bf_user-enum
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=misty-rain name=crowdsecurity/ssh-refused-conn
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=hidden-snow name=crowdsecurity/fortinet-cve-2018-13379
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=icy-cherry name=crowdsecurity/CVE-2022-35914
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=red-sea name=crowdsecurity/netgear_rce
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=still-fog name=crowdsecurity/f5-big-ip-cve-2020-5902
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=weathered-field name=crowdsecurity/vmware-cve-2022-22954
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=sparkling-wave name=crowdsecurity/CVE-2024-9474
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=wispy-voice name=crowdsecurity/http-bad-user-agent
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=young-bird name=crowdsecurity/CVE-2023-49103
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=rough-waterfall name=crowdsecurity/http-generic-bf
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=patient-morning name=LePresidente/http-generic-401-bf
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=autumn-sound name=LePresidente/http-generic-403-bf
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=bold-breeze name=crowdsecurity/CVE-2024-38475
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=muddy-dew name=crowdsecurity/http-generic-test
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=cool-bush name=crowdsecurity/CVE-2023-22518
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=divine-haze name=crowdsecurity/CVE-2022-42889
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=morning-field name=crowdsecurity/CVE-2022-41697
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=floral-forest name=crowdsecurity/http-sap-interface-probing
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=young-haze name=crowdsecurity/CVE-2022-46169-bf
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=still-morning name=crowdsecurity/CVE-2022-46169-cmd
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=spring-waterfall name=crowdsecurity/http-cve-probing
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=red-darkness name=crowdsecurity/CVE-2017-9841
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=still-lake name=crowdsecurity/ssh-slow-bf
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=broken-cherry name=crowdsecurity/ssh-slow-bf_user-enum
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=lively-pond name=crowdsecurity/jira_cve-2021-26086
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=silent-snowflake name=crowdsecurity/ssh-cve-2024-6387
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=lively-grass name=crowdsecurity/http-cve-2021-41773
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=summer-forest name=crowdsecurity/http-sqli-probbing-detection
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=bold-water name=crowdsecurity/ssh-generic-test
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=snowy-silence name=crowdsecurity/http-admin-interface-probing
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=delicate-feather name=crowdsecurity/CVE-2022-37042
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=red-grass name=crowdsecurity/CVE-2023-22515
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=bold-tree name=crowdsecurity/http-open-proxy
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=withered-smoke name=crowdsecurity/http-crawl-non_statics
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=white-sun name=crowdsecurity/http-wordpress-scan
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=morning-cloud name=ltsich/http-w00tw00t
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=proud-mountain name=crowdsecurity/http-path-traversal-probing
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=spring-lake name=crowdsecurity/http-probing
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=lively-fog name=crowdsecurity/CVE-2019-18935
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=broken-thunder name=crowdsecurity/http-cve-2021-42013
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=solitary-sound name=LePresidente/authelia-bf
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=cool-lake name=LePresidente/authelia-bf_user-enum
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=purple-wave name=crowdsecurity/thinkphp-cve-2018-20062
time="2025-06-28T11:33:36+02:00" level=info msg="Adding leaky bucket" cfg=purple-field name=crowdsecurity/http-xss-probbing
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=wild-haze name=crowdsecurity/CVE-2022-26134
time="2025-06-28T11:33:36+02:00" level=info msg="Adding trigger bucket" cfg=late-darkness name=crowdsecurity/fortinet-cve-2022-40684
time="2025-06-28T11:33:36+02:00" level=info msg="Loaded 58 scenarios"
time="2025-06-28T11:33:36+02:00" level=info msg="127.0.0.1 - [Sat, 28 Jun 2025 11:33:36 CEST] \"POST /v1/watchers/login HTTP/1.1 200 85.736925ms \"crowdsec/v1.6.9-40b8cfe6-docker\" \""
time="2025-06-28T11:33:36+02:00" level=info msg="loading acquisition file : /etc/crowdsec/acquis.yaml"
time="2025-06-28T11:33:36+02:00" level=info msg="Adding file /var/log/auth.log to datasources" type=file
time="2025-06-28T11:33:36+02:00" level=warning msg="No matching files for pattern /var/log/syslog" type=file
time="2025-06-28T11:33:36+02:00" level=warning msg="No matching files for pattern /var/log/apache2/*.log" type=file
time="2025-06-28T11:33:36+02:00" level=info msg="Starting processing data"
time="2025-06-28T11:33:36+02:00" level=info msg="Starting docker acquisition" type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="Starting docker acquisition" type=docker
time="2025-06-28T11:33:36+02:00" level=warning msg="/var/log/auth.log is a directory, ignoring it." file=/var/log/auth.log type=file
time="2025-06-28T11:33:36+02:00" level=info msg="Starting docker acquisition" type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="DockerSource Manager started" type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="DockerSource Manager started" type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="DockerSource Manager started" type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="Subscribing to Docker events" type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="Subscribing to Docker events" type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="Subscribing to Docker events" type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="start tail for container npm" container_name=npm type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="start tail for container Nextcloud" container_name=Nextcloud type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="start tail for container vaultwarden" container_name=vaultwarden type=docker
time="2025-06-28T11:33:36+02:00" level=info msg="127.0.0.1 - [Sat, 28 Jun 2025 11:33:36 CEST] \"POST /v1/usage-metrics HTTP/1.1 201 26.160655ms \"crowdsec/v1.6.9-40b8cfe6-docker\" \""
For me that is looking good so far. Any ideas?
From these logs are you installing the collections you want to use like vaultwarden and nginx-proxy-manager?
Also the type isnt npm like I said please read the hub collection page Collections, AppSec Rules & Configurations | CrowdSec Hub - CrowdSec Console as there is an example already there that has the type you need to put…
I see, thanks for this explaination. I downloaded now the parser for vaultwarden, nextcloud and nginx-proxy-manager and set the correct type in the config.
But there is still one problem - it won’t parse the lines in the log
