Overzealous logging to /var/log/messages

In a a standard install this is getting logged to /var/log/messages every 10 seconds, effectively flooding that log at an info level:

Mar 19 08:20:50 emp87 crowdsec[364656]: 127.0.0.1 - [Fri, 19 Mar 2021 08:20:50 AEST] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 17.00232ms "cs-firewall-bouncer/v0.0.10-.....

Is there a way to mute that notice, or alter its level? As it is the messages are drowning any other notices in the log.

Hello @Swallowtail !

The default setup should make those logs land in /var/log/crowdsec_api.log. What version are you using and/or did you change some logging properties ?

thanks,

1.09, straight install, no changes… and I wouldn’t know how to change logging properties for crowdsec :slight_smile:

from the tarball or from the packages ? what distribution are you using ? (asking for extra info because i didn’t manage to reproduce)

RHEL 8, from tarball. Let me know if you want me to post config (you’ll need to tell me what to show you for this one though).

Hello,

If you’re using the tarball and default config, then I guess it must have to do with some rhel/centos specificity.

While I’m going to try to get my hands on a centos (unlikely rhel) to try to reproduce, can you check the logs you have in /var/log/crowdsec.log and /var/log/crowdsec_api.log ? Those are the two files where crowdsec itself is writing. (crowdsec_api.log should have those spammy access logs from api, and crowdsec.log should have the real crowdsec events). But I don’t know yet why/how this ends up in /var/log/messages

edit: out of curiosity, can you check that your /etc/crowdsec/config.yaml has :

common:
  ...
  log_media: file
  log_dir: /var/log/

common:
  daemonize: true
  pid_dir: /var/run/
  log_media: file
  log_level: info
  log_dir: /var/log/
  working_dir: .

/var/log/crowdsec.log is logging every 30 minutes, e.g.:

time="19-03-2021 23:00:05" level=info msg="capi metrics: metrics sent successfully"

crowdsec_api.log is logging every 10 seconds:

127.0.0.1 - [Fri, 19 Mar 2021 23:12:40 AEST] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 21.732159ms "cs-firewall-bouncer/v0.0.10-..." "

…the same as is in /var/log/messages:

Mar 19 23:12:40 emp87 crowdsec[364656]: 127.0.0.1 - [Fri, 19 Mar 2021 23:12:40 AEST] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 21.732159ms "cs-firewall-bouncer/v0.0.10-..." "

BTW - different server, also with 1.0.9… from /var/log/messages

Mar 19 23:22:40 emp07 crowdsec: 127.0.0.1 - [Fri, 19 Mar 2021 23:22:40 AEST] "GET /v1/decisions/stream?startup=false HTTP/1.1 200 20.9738ms "cs-firewall-bouncer/v0.0.5-

It’s doing the same (CentOS 7)

Hello !

After checking on a machine with journald, I can confirm that this is not related to your setup.

I’ve opened an issue to track it : Bug/LAPI (gin) writting logs to stdout *and* logfile, fills `/var/log/messages` · Issue #702 · crowdsecurity/crowdsec · GitHub

TL;DR : gin logs are written as well to stdout, and thus journald treats this as syslog and fills your logs.

I’ll keep you posted :slight_smile:

1 Like