Caddy JSON float‑ts isn’t parsed on OPNsense leading to false http‑crawl‑non_statics bans

crowdsec
1

Versions and settings:

  • OPNsense using official crowdsec and caddy plugins
  • Caddy version 2.9.1
  • Crowdsec version 1.6.5
  • OPNsense 25.1.4_1
  • Caddy files in json format

When I run
cscli explain --file sample.log --type caddy -v

I see two timestamps set:

  • evt.Line.Time - the time CrowdSec read the file (UTC)
  • evt.Time - supposed to be the parsed request time

But for most lines, evt.Time remains the “read‐time” and never becomes the real event time from the "ts" float. A few lines that are parse correctly get the right time (e.g. 10:26:03 +0100 BST), but the rest stay at the moment cscli explain was run (e.g. 17:02:10 UTC).

I’m getting false positives with http-crawl-non_statics and I think it’s because http-crawl-non_statics sees a hundred distinct GETs all stamped within 0.8 s (the read‑time window), it overflows its leaky bucket (capacity 40, leakspeed 0.5 s), and false bans with a “Begin/End” window of < 1 s—even though no visitor hit the site at the time stated (confirmed by checking caddy and nginx timestamps, note caddy proxies to nginx in this instance so I was able to check that caddy matched nginx).

Possible cause:
On OPNsense, possible due to message size or 1.6.5’s helper UnmarshalJSON() sometimes fails so the whole node is skipped and evt.StrTime and evt.Meta.timestamp are never set. That means dateparse-enrich can’t set evt.Time from the JSON, so it leaves the fallback read‑time.

I can private message some sample logs with ip addresses changed if required.

How to reproduce:

cscli explain --file sample.log --type caddy -v > output.txt
grep -m3 '"ts":' output.log
grep -m3 'update evt.Line.Time' output.txt | sed -E 's/.*-> //'
grep -m3 'update evt.Time :' output.txt | sed -E 's/.*-> //'
grep -m3 'create evt.Parsed.message' output.txt | sed -E 's/.*"ts":([0-9]+(\.[0-9]+)?).*/\1/'

grep -m3 ‘update evt.Time :’ output.txt | sed -E ‘s/.*-> //’ will show a handful correctly parse but most not.

I tried making a fallback parser to run before the installed caddy-log parser to extract the raw float ts from the json but I couldn’t get it to work and it’s beyond my ability to work it out.

I’ve read as much documentation as I can take in but I don’t seem to be able to fix it. Any help is greatly appreciated.

2

I’m investigating modifying the time format of caddy logs to match.

Recognized values are: unix_seconds_float, unix_milli_float, unix_nano, iso8601, rfc3339, rfc3339_nano, wall, wall_milli, wall_nano, common_log. The value may also be custom format per the Go time package layout specification

I may need to post on the opnsense forums to work out how it can be done with the caddy plugin for it.

3

My latest thoughts is that Crowdsec on OPNSense should protect the OPNSense but not use it to protect web servers running under it (with caddy logs at least) and that installing Crowdsec on the webservers is the best way to do it (feeding back to OPNSense Crowdsec also so that the bans are enforced at the firewall level).