# Crowdsec container and journald matches issues

**URL:** <https://discourse.crowdsec.net/t/crowdsec-container-and-journald-matches-issues/953>\
**Category:** Uncategorized\
**Created:** [September 12, 2022, 2:33pm UTC](https://discourse.crowdsec.net/t/crowdsec-container-and-journald-matches-issues/953 "2022-09-12T14:33:47Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![stephdl](https://avatars.discourse-cdn.com/v4/letter/s/439d5e/32.png) [@stephdl](https://discourse.crowdsec.net/u/stephdl)\
**Post date:** [September 12, 2022, 2:33pm UTC](https://discourse.crowdsec.net/t/crowdsec-container-and-journald-matches-issues/953/1 "2022-09-12T14:33:47Z")

</div>

Hi all

Testing crowdsec for a new fail2ban replacement in centos9 stream, I cannot have the log parsed with journald acquisition, only with file://var/log/secure

- using podman in a root container
- lauching metrics with journald and file acquisition

```auto
podman run -d --replace -v crowdsec_config:/etc/crowdsec \
-v crowdsec_data:/var/lib/crowdsec/data -v /run/log/journal/:/run/log/journal:Z \
-v /var/log/secure:/var/log/secure:Z -e COLLECTIONS="crowdsecurity/sshd" \
-p 8080:8080 -p 6060:6060 \
--name crowdsec docker.io/crowdsecurity/crowdsec:latest-debian

```

in `/var/lib/containers/storage/volumes/crowdsec_config/_data/acquis.yaml`

```auto
---
source: journalctl
journalctl_filter:
 - "_SYSTEMD_UNIT=sshd.service"
labels:
  type: journald
---
filenames:
- /var/log/secure
labels:
  type: syslog
---

```

I can see decisions only after I set the `/var/log/secure` in acquisition, when I watch the metcrics I have

```auto
Every 2.0s: podman exec -ti crowdsec cscli metrics droplet1.nethserver.fr: Mon Sep 12 10:30:27 2022

^[36mINFO^[0m[12-09-2022 02:30:28 PM] Buckets Metrics: g
+-------------------------------------+---------------+-----------+--------------+--------+---------+-
| BUCKET | CURRENT COUNT | OVERFLOWS | INSTANTIATED | POURED | EXPIRED |E
+-------------------------------------+---------------+-----------+--------------+--------+---------+-
| crowdsecurity/ssh-bf | 1 | - | 3 | 10 | 2 |
| crowdsecurity/ssh-bf_user-enum | 1 | - | 3 | 5 | 2 |-
| crowdsecurity/ssh-slow-bf | 3 | - | 3 | 10 | - |
| crowdsecurity/ssh-slow-bf_user-enum | 3 | - | 3 | 5 | - |
+-------------------------------------+---------------+-----------+--------------+--------+---------+
^[36mINFO^[0m[12-09-2022 02:30:28 PM] Acquisition Metrics:
+--------------------------------------------------+------------+--------------+----------------+------------------------+
| SOURCE | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+--------------------------------------------------+------------+--------------+----------------+------------------------+
| file:/var/log/secure | 16 | 10 | 6 | 30 |
| journalctl:journalctl-_SYSTEMD_UNIT=sshd.service | 15 | - | 15 | - |
+--------------------------------------------------+------------+--------------+----------------+------------------------+
^[36mINFO^[0m[12-09-2022 02:30:28 PM] Parser Metrics:
+---------------------------------+------+--------+----------+
| PARSERS | HITS | PARSED | UNPARSED |
+---------------------------------+------+--------+----------+
| child-crowdsecurity/sshd-logs | 110 | 10 | 100 |
| child-crowdsecurity/syslog-logs | 16 | 16 | - |
| crowdsecurity/dateparse-enrich | 10 | 10 | - |
| crowdsecurity/geoip-enrich | 10 | 10 | - |
| crowdsecurity/non-syslog | 15 | 15 | - |
| crowdsecurity/sshd-logs | 15 | 10 | 5 |
| crowdsecurity/syslog-logs | 16 | 16 | - |
| crowdsecurity/whitelists | 10 | 10 | - |
+---------------------------------+------+--------+----------+
^[36mINFO^[0m[12-09-2022 02:30:28 PM] Local Api Metrics:
+--------------------+--------+------+
|	ROUTE | METHOD | HITS |
+--------------------+--------+------+
| /v1/heartbeat | GET | 5 |
| /v1/watchers/login | POST | 2 |
+--------------------+--------+------+L
^[36mINFO^[0m[12-09-2022 02:30:28 PM] Local Api Machines Metrics:
+-----------+---------------+--------+------+O
| MACHINE | ROUTE | METHOD | HITS |-
+-----------+---------------+--------+------+
| localhost | /v1/heartbeat | GET | 5 |-
+-----------+---------------+--------+------+p
^[36mINFO^[0m[12-09-2022 02:30:28 PM] Local Api Decisions:
+---------------------------+----------+--------+-------+
| REASON | ORIGIN | ACTION | COUNT |
+---------------------------+----------+--------+-------+
| crowdsecurity/ssh-slow-bf | crowdsec | ban | 9 |
+---------------------------+----------+--------+-------+
^[36mINFO^[0m[12-09-2022 02:30:28 PM] Local Api Alerts:
+---------------------------+-------+
| REASON | COUNT |
+---------------------------+-------+
| crowdsecurity/ssh-slow-bf | 9 |

```

however the debian container seems to see well the journal

```auto
root@f164a0edaf92:/# journalctl -f -u sshd.service
-- Journal begins at Mon 2022-09-12 13:04:28 UTC. --
Sep 12 14:26:05 droplet1.nethserver.fr sshd[98228]: Invalid user admin from 89.252.140.21 port 44938
Sep 12 14:26:05 droplet1.nethserver.fr sshd[98228]: pam_unix(sshd:auth): check pass; user unknown
Sep 12 14:26:05 droplet1.nethserver.fr sshd[98228]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=89.252.140.21
Sep 12 14:26:07 droplet1.nethserver.fr sshd[98228]: Failed password for invalid user admin from 89.252.140.21 port 44938 ssh2
Sep 12 14:26:08 droplet1.nethserver.fr sshd[98228]: Received disconnect from 89.252.140.21 port 44938:11: Bye Bye [preauth]
Sep 12 14:26:08 droplet1.nethserver.fr sshd[98228]: Disconnected from invalid user admin 89.252.140.21 port 44938 [preauth]
Sep 12 14:29:37 droplet1.nethserver.fr sshd[100707]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=92.255.85.70 user=root
Sep 12 14:29:39 droplet1.nethserver.fr sshd[100707]: Failed password for root from 92.255.85.70 port 61768 ssh2
Sep 12 14:29:39 droplet1.nethserver.fr sshd[100707]: Received disconnect from 92.255.85.70 port 61768:11: Bye Bye [preauth]
Sep 12 14:29:39 droplet1.nethserver.fr sshd[100707]: Disconnected from authenticating user root 92.255.85.70 port 61768 [preauth]

```

---

<div class="post-metadata">

**Author:** ![stephdl](https://avatars.discourse-cdn.com/v4/letter/s/439d5e/32.png) [@stephdl](https://discourse.crowdsec.net/u/stephdl)\
**Post date:** [September 12, 2022, 2:39pm UTC](https://discourse.crowdsec.net/t/crowdsec-container-and-journald-matches-issues/953/2 "2022-09-12T14:39:12Z")

</div>

If I remove the `/var/log/secure` acquisition and restart crowdsec I can see something changed in the metrics

```auto
Every 2.0s: podman exec -ti crowdsec cscli metrics droplet1.nethserver.fr: Mon Sep 12 10:37:32 2022

^[36mINFO^[0m[12-09-2022 02:37:33 PM] Acquisition Metrics: g
+--------------------------------------------------+------------+--------------+----------------+------------------------+
| SOURCE | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+--------------------------------------------------+------------+--------------+----------------+------------------------+
| journalctl:journalctl-_SYSTEMD_UNIT=sshd.service | 1 | - | 1 | - |
+--------------------------------------------------+------------+--------------+----------------+------------------------+
^[36mINFO^[0m[12-09-2022 02:37:33 PM] Parser Metrics:
+--------------------------+------+--------+----------+
| PARSERS | HITS | PARSED | UNPARSED |
+--------------------------+------+--------+----------+
| crowdsecurity/non-syslog | 1 | 1 | - |
+--------------------------+------+--------+----------+
^[36mINFO^[0m[12-09-2022 02:37:33 PM] Local Api Metrics:
+--------------------+--------+------+
|	ROUTE | METHOD | HITS |
+--------------------+--------+------+
| /v1/heartbeat | GET | 2 |
| /v1/watchers/login | POST | 2 |
+--------------------+--------+------+L
^[36mINFO^[0m[12-09-2022 02:37:33 PM] Local Api Machines Metrics:
+-----------+---------------+--------+------+O
| MACHINE | ROUTE | METHOD | HITS |-
+-----------+---------------+--------+------+
| localhost | /v1/heartbeat | GET | 2 |-
+-----------+---------------+--------+------+p
^[36mINFO^[0m[12-09-2022 02:37:33 PM] Local Api Decisions:
+---------------------------+----------+--------+-------+
| REASON | ORIGIN | ACTION | COUNT |
+---------------------------+----------+--------+-------+
| crowdsecurity/ssh-slow-bf | crowdsec | ban | 9 |
+---------------------------+----------+--------+-------+
^[36mINFO^[0m[12-09-2022 02:37:33 PM] Local Api Alerts:
+---------------------------+-------+
| REASON | COUNT |
+---------------------------+-------+
| crowdsecurity/ssh-slow-bf | 9 |
+---------------------------+-------+

```

I miss all the buckets, not sure it is wanted ???

---

<div class="post-metadata">

**Author:** ![stephdl](https://avatars.discourse-cdn.com/v4/letter/s/439d5e/32.png) [@stephdl](https://discourse.crowdsec.net/u/stephdl)\
**Post date:** [September 12, 2022, 2:42pm UTC](https://discourse.crowdsec.net/t/crowdsec-container-and-journald-matches-issues/953/3 "2022-09-12T14:42:16Z")

</div>

And all the parsers ???

---

<div class="post-metadata">

**Author:** ![stephdl](https://avatars.discourse-cdn.com/v4/letter/s/439d5e/32.png) [@stephdl](https://discourse.crowdsec.net/u/stephdl)\
**Post date:** [September 12, 2022, 5:19pm UTC](https://discourse.crowdsec.net/t/crowdsec-container-and-journald-matches-issues/953/4 "2022-09-12T17:19:09Z")

</div>

answering myself I suppose the documentation send to a wrong direction, [the doc state for version 1.41](https://docs.crowdsec.net/docs/next/data_sources/journald#configuration-example)

```auto
---
source: journalctl
journalctl_filter:
  - "_SYSTEMD_UNIT=sshd.service"
labels:
  type: journald
---

```

but it does not work

however [for the version 1.0 the documentation states](https://docs.crowdsec.net/docs/v1.0/acquisitions/intro)

```auto
---
source: journalctl
journalctl_filter:
  - "_SYSTEMD_UNIT=sshd.service"
labels:
  type: syslog
---

```

`yaml filenames: - /var/log/nginx/access*.log - /var/log/nginx/error.log labels: type: nginx --- filenames: - /var/log/auth.log labels: type: syslog --- journalctl_filter: - "_SYSTEMD_UNIT=ssh.service" labels: type: syslog `

and this syntax is good and workable now

```auto
[root@droplet1 ~]# podman exec -ti crowdsec cscli metrics
INFO[12-09-2022 05:15:28 PM] Buckets Metrics:                             
+-------------------------------------+---------------+-----------+--------------+--------+---------+
| BUCKET | CURRENT COUNT | OVERFLOWS | INSTANTIATED | POURED | EXPIRED |
+-------------------------------------+---------------+-----------+--------------+--------+---------+
| crowdsecurity/ssh-bf | - | - | 4 | 19 | 4 |
| crowdsecurity/ssh-bf_user-enum | - | - | 4 | 7 | 4 |
| crowdsecurity/ssh-slow-bf | 3 | 1 | 4 | 19 | - |
| crowdsecurity/ssh-slow-bf_user-enum | 3 | - | 3 | 5 | - |
+-------------------------------------+---------------+-----------+--------------+--------+---------+
INFO[12-09-2022 05:15:28 PM] Acquisition Metrics:                         
+--------------------------------------------------+------------+--------------+----------------+------------------------+
| SOURCE | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+--------------------------------------------------+------------+--------------+----------------+------------------------+
| journalctl:journalctl-_SYSTEMD_UNIT=sshd.service | 39 | 19 | 20 | 50 |
+--------------------------------------------------+------------+--------------+----------------+------------------------+
INFO[12-09-2022 05:15:28 PM] Parser Metrics:                              
+---------------------------------+------+--------+----------+
| PARSERS | HITS | PARSED | UNPARSED |
+---------------------------------+------+--------+----------+
| child-crowdsecurity/sshd-logs | 315 | 19 | 296 |
| child-crowdsecurity/syslog-logs | 40 | 38 | 2 |
| crowdsecurity/dateparse-enrich | 19 | 19 | - |
| crowdsecurity/geoip-enrich | 19 | 19 | - |
| crowdsecurity/sshd-logs | 37 | 19 | 18 |
| crowdsecurity/syslog-logs | 39 | 38 | 1 |
| crowdsecurity/whitelists | 19 | 19 | - |
+---------------------------------+------+--------+----------+
INFO[12-09-2022 05:15:28 PM] Local Api Metrics:                           
+--------------------+--------+------+
| ROUTE | METHOD | HITS |
+--------------------+--------+------+
| /manager/text/list | GET | 1 |
| /v1/alerts | GET | 5 |
| /v1/alerts | POST | 1 |
| /v1/heartbeat | GET | 9 |
| /v1/watchers/login | POST | 7 |
+--------------------+--------+------+
INFO[12-09-2022 05:15:28 PM] Local Api Machines Metrics:                  
+-----------+---------------+--------+------+
| MACHINE | ROUTE | METHOD | HITS |
+-----------+---------------+--------+------+
| localhost | /v1/alerts | GET | 5 |
| localhost | /v1/alerts | POST | 1 |
| localhost | /v1/heartbeat | GET | 9 |
+-----------+---------------+--------+------+
INFO[12-09-2022 05:15:28 PM] Local Api Decisions:                         
+--------------------------------------------+----------+--------+-------+
| REASON | ORIGIN | ACTION | COUNT |
+--------------------------------------------+----------+--------+-------+
| crowdsecurity/http-probing | CAPI | ban | 434 |
| crowdsecurity/jira_cve-2021-26086 | CAPI | ban | 25 |
| crowdsecurity/ssh-bf | CAPI | ban | 254 |
| crowdsecurity/fortinet-cve-2018-13379 | CAPI | ban | 41 |
| crowdsecurity/http-backdoors-attempts | CAPI | ban | 44 |
| crowdsecurity/http-crawl-non_statics | CAPI | ban | 1692 |
| crowdsecurity/http-cve-2021-41773 | CAPI | ban | 4 |
| crowdsecurity/http-open-proxy | CAPI | ban | 76 |
| crowdsecurity/apache_log4j2_cve-2021-44228 | CAPI | ban | 15 |
| crowdsecurity/f5-big-ip-cve-2020-5902 | CAPI | ban | 2 |
| crowdsecurity/http-bad-user-agent | CAPI | ban | 2416 |
| crowdsecurity/http-sensitive-files | CAPI | ban | 96 |
| crowdsecurity/thinkphp-cve-2018-20062 | CAPI | ban | 163 |
| crowdsecurity/http-generic-bf | CAPI | ban | 16 |
| crowdsecurity/http-path-traversal-probing | CAPI | ban | 51 |
| crowdsecurity/ssh-slow-bf | crowdsec | ban | 10 |
| crowdsecurity/ssh-slow-bf | CAPI | ban | 7500 |
| ltsich/http-w00tw00t | CAPI | ban | 6 |
+--------------------------------------------+----------+--------+-------+
INFO[12-09-2022 05:15:28 PM] Local Api Alerts:                            
+---------------------------+-------+
| REASON | COUNT |
+---------------------------+-------+
| crowdsecurity/ssh-slow-bf | 10 |
+---------------------------+-------+

```

cc @klausagnoletti @thibault do I am right ?

I run the latest crowdsec debian container : `docker.io/crowdsecurity/crowdsec:latest-debian`

---

<div class="post-metadata">

**Author:** ![stephdl](https://avatars.discourse-cdn.com/v4/letter/s/439d5e/32.png) [@stephdl](https://discourse.crowdsec.net/u/stephdl)\
**Post date:** [October 25, 2022, 2:14pm UTC](https://discourse.crowdsec.net/t/crowdsec-container-and-journald-matches-issues/953/5 "2022-10-25T14:14:25Z")

</div>

answer from the developers in the discord channel, the doc is wrong, use `type: syslog` and not `type: journald`