No journal files were found

crowdsec
1

Hi,

I’m running crowdsec in a container using the latest-debian tag and it fails to find the journal files:

feb 19 09:17:48 crowdsec[610401]: time="2025-02-19T08:17:48Z" level=info msg="Crowdsec v1.6.5-72b4354b"
[...]
feb 19 09:17:50 crowdsec[610401]: time="2025-02-19T08:17:50Z" level=info msg="Running journalctl command: /usr/bin/journalctl [journalctl --follow -n 0 _SYSTEMD_UNIT=authelia.service]" src="journalctl-%s_SYSTEMD_UNIT=authelia.service" type=journalctl
feb 19 09:17:50 crowdsec[610401]: time="2025-02-19T08:17:50Z" level=warning msg="Got stderr message : No journal files were found." src="journalctl-%s_SYSTEMD_UNIT=authelia.service" type=journalctl
feb 19 09:17:50 crowdsec[610401]: time="2025-02-19T08:17:50Z" level=info msg="journalctl datasource journalctl-%s_SYSTEMD_UNIT=authelia.service stopping" src="journalctl-%s_SYSTEMD_UNIT=authelia.service" type=journalctl
feb 19 09:17:50 crowdsec[610401]: time="2025-02-19T08:17:50Z" level=info msg="Acquisition is finished, shutting down"
feb 19 09:17:50 crowdsec[610401]: time="2025-02-19T08:17:50Z" level=fatal msg="unable to start crowdsec routines: starting acquisition error: journalctl error : No journal files were found."

The journal files are readable from within the container:

$ ls -laR /var/log/journal/
/var/log/journal/:
total 12
drwxr-sr-x+ 1 root  190    64 May  7  2024 .
drwxr-xr-x. 1 root root  1810 Feb 18 23:09 ..
drwxr-sr-x+ 1 root  190 15706 Feb 19 09:42 5273c11607c34248a1a04ce5600cbd04

/var/log/journal/5273c11607c34248a1a04ce5600cbd04:
total 809952
drwxr-sr-x+ 1 root 190    15706 Feb 19 09:42 .
drwxr-sr-x+ 1 root 190       64 May  7  2024 ..
-rw-r-----+ 1 root 190  8388608 Feb 19 09:47 system.journal
-rw-r-----+ 1 root 190 50331648 Feb 17 17:06 system@12b0d3d715fe45f79375e42773581da9-0000000005e34819-00062e552e52adc1.journal
[...]
-rw-r-----+ 1 root 190 25165824 Feb 19 09:42 system@12b0d3d715fe45f79375e42773581da9-0000000005ef62bd-00062e7a6d2e7e09.journal
-rw-r-----+ 1 root 190  8388608 Jan 14 11:15 user-1000.journal
-rw-r-----+ 1 root 190  8388608 Jan 12 16:15 user-2147650180.journal
-rw-r-----+ 1 root 190  8388608 Dec  8 08:41 user-2147698654.journal

$ cat /var/log/journal/5273c11607c34248a1a04ce5600cbd04/system@12b0d3d715fe45f79375e42773581da9-0000000005ef62bd-00062e7a6d2e7e09.journal >/dev/null && echo OK
OK

$ journalctl --follow -n 100 _SYSTEMD_UNIT=authelia.service
No journal files were found.

The host system is Fedora Server, if that helps.

Any idea why is it failing?
Thank you.

2

hmmm very odd, if you run journalctl with the -D flag and set the value to /var/log/journal do that seems to inform the binary where to find the files?

3

Yes, that works:

$ journalctl --follow -n 10 -D /var/log/journal _SYSTEMD_UNIT=authelia.service
Feb 19 10:07:06 authelia[661484]: time="2025-02-19T10:07:06Z" level=info msg="Authelia v4.38.19 is starting"
Feb 19 10:07:06 authelia[661484]: time="2025-02-19T10:07:06Z" level=info msg="Log severity set to info"
4

then in your acquisition within the filter you can add

- "-D /var/log/journal"
- "_SYSTEMD_UNIT=authelia.service"

as the filter are basically just the arguments to journalctl

5

Thanks for the tip, but it looks there’s more problems:

With this acquis.yaml file:

---
source: journalctl
journalctl_filter:
  - "-D /var/log/journal"
  - "_SYSTEMD_UNIT=authelia.service"
labels:
  type: syslog

Crowdsec crashes at startup:

feb 19 11:26:19 crowdsec[670570]: time="2025-02-19T10:26:19Z" level=info msg="Running journalctl command: /usr/bin/journalctl [journalctl --follow -n 0 -D /var/log/journal _SYSTEMD_UNIT=authelia.service]" src="journalctl-%s-D /var/log/journal._SYSTEMD_UNIT=authelia.service" type=journalctl
feb 19 11:26:19 crowdsec[670570]: time="2025-02-19T10:26:19Z" level=warning msg="Got stderr message : Failed to open  /var/log/journal: No such file or directory" src="journalctl-%s-D /var/log/journal._SYSTEMD_UNIT=authelia.service" type=journalctl
feb 19 11:26:19 crowdsec[670570]: time="2025-02-19T10:26:19Z" level=info msg="journalctl datasource journalctl-%s-D /var/log/journal._SYSTEMD_UNIT=authelia.service stopping" src="journalctl-%s-D /var/log/journal._SYSTEMD_UNIT=authelia.service" type=journalctl
feb 19 11:26:19 crowdsec[670570]: time="2025-02-19T10:26:19Z" level=fatal msg="unable to start crowdsec routines: starting acquisition error: journalctl error : Failed to open  /var/log/journal: No such file or directory"
6

Ok, I’ve got it, it has to be like this:

---
source: journalctl
journalctl_filter:
  - "-D"
  - "/var/log/journal"
  - "_SYSTEMD_UNIT=authelia.service"
labels:
  type: syslog

Thank you!