Acquisition error : dead reader

1

Hello,
happy 2024 new year to all.

I have some fatal errors on boot with crowdsec : level=fatal msg="starting acquisition error : dead reader for

My acquis.yaml

filenames:
  - /var/www/*/var/log/apache2/access.*
  - /var/www/*/var/log/apache2/error.*
labels:
  type: apache2
---

Complete log :

time=“02-01-2024 15:17:57” level=fatal msg=“starting acquisition error : dead reader for /var/www/website/var/log/apache2/access.2022.09.20”
time=“02-01-2024 15:19:00” level=fatal msg=“starting acquisition error : dead reader for /var/www/website/var/log/apache2/access.2022.09.17”
time=“02-01-2024 15:20:04” level=fatal msg=“starting acquisition error : dead reader for /var/www/website/var/log/apache2/access.2022.09.21”

All works for a time but for somes weeks I have this fatal error.

Reference in Apache2 rotatelogs in acquisition

Thanks for you’re help

2

Hello, does someone have on idea ? I have same results in another server.

Does my acquisition apache logs is wrong ?

---
filenames:
  - /var/www/*/var/log/apache2/access*
  - /var/www/*/var/log/apache2/error*
labels:
  type: apache2
---

There is a solution to bypass errors and continue to work ?

Regards

3

So dead reader happens in two cases and both are linked to symlinked files:

  • Symlink is to a /dev/ property EG /dev/stdout, this commonly happen in containers
  • Symlink file is static, however, the underlying file is rotated and then since the symlink doesnt get changed the reader returns EOF

On dead reader I believe we log the file? does it pinpoint a direct location?

4

@iiAmLoz yes all file are direct files not symlinks

For exemple :

time="10-01-2024 15:14:02" level=fatal msg="starting acquisition error : dead reader for /var/www/<private>/var/log/apache2/access.2020.12.29"

# ls -la /var/www/www.monsite.fr/var/log/apache2/access.2020.12.29
-rw-r--r-- 1 monsite monsite 3684784 29 déc.   2020 /var/www/<private>/var/log/apache2/access.2020.12.29
5

Its a pretty old file, that hasnt been updated in 4 years (Closer to 3 years really)?

However, I see an area of improvement since we dont log the actual error that comes from the Tail package, so i opened an issue

6

yes but same problem with more recent file

 crowdsec
time="10-01-2024 17:10:30" level=fatal msg="starting acquisition error : dead reader for /var/www/www.monsite.fr/var/log/apache2/access.2023.12.03"
# ls -la /var/www/www.monsite.fr/var/log/apache2/access.2023.12.03
-rw-r--r-- 1 root monsite 7624458  3 déc.  23:59 /var/www/www.monsite.fr/var/log/apache2/access.2023.12.03
7

@iiAmLoz no idea about that ?

8

Can you upgrade to v1.6.0 as we merged a patch to log the reason for the death

If the error is empty then we have very little to go on.