I’m trying to secure a cloudron server that runs NGINX as a proof of concept in an attempt to get Cloudron Dev’s to incorporate crowdsec as part of the build process. It seems that Cloudron uses a custom log format for NGINX and as such all my NGINX logs are showing up as “unparsed” in the metrics.
The NGINX.conf shows the following format
the collectd config depends on this log format
log_format combined2 '$remote_addr - [$time_local] '
'"$request" $status $body_bytes_sent $request_time '
'"$http_referer" "$host" "$http_user_agent"';
What can I do to get crowdsec to parse this format?
Hello @Mastadon !
If you have a custom log format for nginx, the best would be to copy the existing nginx parser and modify it for your needs. Once you have it figured out, it would be to open a pull request so that we can integrate it into the hub and make it available for all !
In order to make it work, here a few useful pointers :
- Grab some sample logs : it will be very useful for testing.
- If you are using the latest (1.2.1) crowdsec version,
cscli explain will hopefully make your life easier (it’s a new feature, so feedback is appreciated !)
- Otherwise, the “how to create parsers” documentation is what you might want to follow !
And when it comes to debugging the grok pattern itself grok debugger and debuggex are your friends as usual !
Hope this helps !