Overzealous logging to /var/log/messages

crowdsec
1

In a a standard install this is getting logged to /var/log/messages every 10 seconds, effectively flooding that log at an info level:

Mar 19 08:20:50 emp87 crowdsec[364656]: 127.0.0.1 - [Fri, 19 Mar 2021 08:20:50 AEST] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 17.00232ms "cs-firewall-bouncer/v0.0.10-.....

Is there a way to mute that notice, or alter its level? As it is the messages are drowning any other notices in the log.

2

Hello @Swallowtail !

The default setup should make those logs land in /var/log/crowdsec_api.log. What version are you using and/or did you change some logging properties ?

thanks,

3

1.09, straight install, no changes… and I wouldn’t know how to change logging properties for crowdsec :slight_smile:

4

from the tarball or from the packages ? what distribution are you using ? (asking for extra info because i didn’t manage to reproduce)

5

RHEL 8, from tarball. Let me know if you want me to post config (you’ll need to tell me what to show you for this one though).

6

Hello,

If you’re using the tarball and default config, then I guess it must have to do with some rhel/centos specificity.

While I’m going to try to get my hands on a centos (unlikely rhel) to try to reproduce, can you check the logs you have in /var/log/crowdsec.log and /var/log/crowdsec_api.log ? Those are the two files where crowdsec itself is writing. (crowdsec_api.log should have those spammy access logs from api, and crowdsec.log should have the real crowdsec events). But I don’t know yet why/how this ends up in /var/log/messages

edit: out of curiosity, can you check that your /etc/crowdsec/config.yaml has :

common:
  ...
  log_media: file
  log_dir: /var/log/
7 
common:
  daemonize: true
  pid_dir: /var/run/
  log_media: file
  log_level: info
  log_dir: /var/log/
  working_dir: .

/var/log/crowdsec.log is logging every 30 minutes, e.g.:

time="19-03-2021 23:00:05" level=info msg="capi metrics: metrics sent successfully"

crowdsec_api.log is logging every 10 seconds:

127.0.0.1 - [Fri, 19 Mar 2021 23:12:40 AEST] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 21.732159ms "cs-firewall-bouncer/v0.0.10-..." "

…the same as is in /var/log/messages:

Mar 19 23:12:40 emp87 crowdsec[364656]: 127.0.0.1 - [Fri, 19 Mar 2021 23:12:40 AEST] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 21.732159ms "cs-firewall-bouncer/v0.0.10-..." "

8

BTW - different server, also with 1.0.9… from /var/log/messages

Mar 19 23:22:40 emp07 crowdsec: 127.0.0.1 - [Fri, 19 Mar 2021 23:22:40 AEST] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 20.9738ms "cs-firewall-bouncer/v0.0.5-

It’s doing the same (CentOS 7)

9

Hello !

After checking on a machine with journald, I can confirm that this is not related to your setup.

I’ve opened an issue to track it : Bug/LAPI (gin) writting logs to stdout *and* logfile, fills `/var/log/messages` · Issue #702 · crowdsecurity/crowdsec · GitHub

TL;DR : gin logs are written as well to stdout, and thus journald treats this as syslog and fills your logs.

I’ll keep you posted :slight_smile:

1 Like
10

I have the same problem on Linux Debian 11. I use NFTables, Firewalld + Crowdsec, Crowdsec-firewall-bouncer-nftables. Packages of latest (current) versions installed by standard instruction - repository packagecloud.io.

Every 10 seconds I get the following entries in crowdsec_api.log:

time=“23-01-2022 21:38:33” level=info msg="127.0.0.1 - [Sun, 23 Jan 2022 21:38:33 +05] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 3.299679ms “crowdsec-firewall-bouncer/v0.0.21-debian-pragmatic-<…>” “”
time=“23-01-2022 21:38:43” level=info msg="127.0.0.1 - [Sun, 23 Jan 2022 21:38:43 +05] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 4.134547ms “crowdsec-firewall-bouncer/v0.0.21-debian-pragmatic-<…>” “”
time=“23-01-2022 21:38:53” level=info msg="127.0.0.1 - [Sun, 23 Jan 2022 21:38:53 +05] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 3.120844ms “crowdsec-firewall-bouncer/v0.0.21-debian-pragmatic-<…>” “”
time=“23-01-2022 21:39:03” level=info msg="127.0.0.1 - [Sun, 23 Jan 2022 21:39:03 +05] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 3.88449ms “crowdsec-firewall-bouncer/v0.0.21-debian-pragmatic-<…>” “”
time=“23-01-2022 21:39:13” level=info msg="127.0.0.1 - [Sun, 23 Jan 2022 21:39:13 +05] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 3.250693ms “crowdsec-firewall-bouncer/v0.0.21-debian-pragmatic-<…>” “”

What do I need to do to fix this?