NGiNX Logs in GELF JSON Format

crowdsec
2

Hi,

I believe the produced logs are JSON, aren’t they ?

So we have no built-in parser that can be fed with json logs for now, but we have already some features to do it. You’ll find an example in the unit tests https://github.com/crowdsecurity/crowdsec/tree/master/pkg/parser/tests/base-json-extract

I guess the resulting parser file should look like this:

filter: "evt.Parsed.program startsWith 'nginx'"
onsuccess: next_stage
#debug: true
name: crowdsecurity/nginx-logs
description: "Parse nginx access and error logs"
statics:
  - target: evt.StrTime
    expression: JsonExtract(evt.Line.Raw, "timestamp8601")
  - parsed: "logsource"
    value: "gelf-nginx"
  - meta: source_ip
    expression: JsonExtract(evt.Line.Raw, "remote_addr")
  - meta: http_status
    expression: JsonExtract(evt.Line.Raw, "response_status")
  - meta: http_path
    expression: JsonExtract(evt.Line.Raw, "request")
  - meta: log_type
    value: http_access-log

This file should be put in the /etc/crowdsec/config/parsers/s00-raw directory but I can’t test it because I don’t have any of your logs. If you want to, you can provide us a sample of your logs and we’ll have a better chance to provide you a ready to use parser file.