Solution for parsing logs of docker containers

crowdsec
1

Hi,

Is there any solution for parsing logs of docker containers?

2

Very good Idea, maybe add it as an improvement suggestion?
I suggested to parse logs of traefik as I use it as a reverse proxy on my docker setup but forgot to ask for docker logs parsing in general, have a look: Improvement/hub add support for traefik · Issue #343 · crowdsecurity/crowdsec · GitHub

3

Hi @gergely, @ovizii,

We are writing a blog post about how to integrate crowdsec in a docker-compose environment. In the meantime, here is the example-docker-compose repository containing the example configuration.

4

Hey there @gergely @ovizii. Any news on this?
I have a docker nginx-proxy ingress that sends the logs to stderr and stdout as nginx default docker images do.
So it seems that is default to send logs on docker machines to stdout/stderr… seems to be the documented way to do.
As so, I can´t read the logs using volumes, like this example of yours do. (GitHub - crowdsecurity/example-docker-compose: Example integration of crowdsec in docker-compose)
How we could achieve the docker log reading from the std docker logs?
Thanks

5

Hey,

Crowdsec does support reading from containers. I have an example docker-compose mounting the docker socket into the crowdsec container here: broken

Edit Jul 2023: I forked the repository they showed above and changed it to support a docker scenario. If you have any questions please ask me :slight_smile:

Edit Oct 2023: The link above is now broken sorry, here is an updated link https://github.com/LaurenceJJones/example-docker-compose/tree/main/container-socket

6

Having several Docker containers in compose mode, I simply added, for example,

source: docker
container_name:
  - mailserver
labels:
  type: syslog
---
source: docker
container_name:
  - nginx
labels:
  type: nginx

to acquis.yml.

Check with cscli explain -d "docker://nginx?since=1h" -t nginx

Metrics (also example):

INFO[30-07-2022 10:43:18 AM] Acquisition Metrics:
+------------------------+------------+--------------+----------------+------------------------+
|         SOURCE         | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+------------------------+------------+--------------+----------------+------------------------+
| docker:mailserver      | 195.34k    | 27.56k       | 167.78k        | 2.51k                  |
| docker:nextcloud       | 97.95k     | -            | 97.95k         | -                      |
| docker:nginx           | 109.76k    | 107.75k      | 2.01k          | 52.40k                 |