CrowdSec Parser for Stalwart Logs

2.687

Hello CrowdSec Community,

I’ve been using CrowdSec as an extension in the reverse proxy Caddy for quite some time, but I’m new to the community. Everything is running in a Docker Compose instance with its own MacVLAN. In my small, private home lab, I’ve installed Stalwart Mail in Docker, which generates a continuous log file called stalwart.log. I’ve built a CrowdSec parser using AI, but I’d like to know if it’s correct.

I’ve been using CrowdSec for a while now, but I’d like your feedback on whether it’s working correctly. In the stalwart.log file, I find the following log lines, for example, because Stalwart also has its own security measures, which I would like to address with CrowdSec:

2026-04-26T00:17:59Z INFO Banned due to scan (security.scan-ban) listenerId = "submissions", localPort = 465, remoteIp = 104.248.203.156, remotePort = 47964, remoteIp = 104.248.203.156, reason = "Invalid SMTP command"

To address this, I added the following entry to the acquise.yaml file:

[...]
---
filenames:
  - /remotelogs/stalwart/stalwart.log
labels:
  type: stalwart

I also created the file /parsers/s00-raw/stalwart-logs.yaml created:

name: stalwart/parse-logs
description: Raw parser for Stalwart logs
stage: s00-raw
onsuccess: next_stage

nodes:
  - grok:
      apply_on: Line.Raw
      pattern: '^%{TIMESTAMP_ISO8601:timestamp}\s+%{WORD:log_level}\s+%{DATA:message_text}\s*\(%{DATA:event_type}\)\s*(%{GREEDYDATA:kvpairs})?$'

Another file /parsers/s01-parse/stalwart-logs-extended.yaml looks like this:

Eine weitere Datei /parsers/s01-parse/stalwart-logs-extended.yaml sieht so aus:
name: stalwart/parse-extended
description: Parse Stalwart logs including key fields without kv parser
stage: s01-parse
onsuccess: next_stage

# This Parser extracts:
# - timestamp
# - log_level
# - message_text
# - event_type
# - listenerId
# - localPort
# - remoteIp
# - remotePort
# - reason

nodes:
  - grok:
      apply_on: Line.Raw
      pattern: '^%{TIMESTAMP_ISO8601:timestamp}\s+%{WORD:log_level}\s+%{DATA:message_text}\s+\(%{DATA:event_type}\)\s*(?:listenerId\s*=\s*"%{DATA:listenerId}",\s*)?(?:localPort\s*=\s*%{INT:localPort},\s*)?(?:remoteIp\s*=\s*%{IP:remoteIp},\s*)?(?:remotePort\s*=\s*%{INT:remotePort},\s*)?(?:reason\s*=\s*"%{DATA:reason}")?.*$'

Finally there is /scenarios/stalwart-smtp-bruteforce.yaml:

type: leaky
name: stalwart/smtp-smtp-bruteforce
description: Detect SMTP bruteforce and scanners on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.invalid-ehlo",
    "smtp.auth-not-allowed",
    "smtp.auth-mechanism-not-supported"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true

My question for the parser experts is whether this is correct and can work. At least when starting Stalwart, there are no error messages.

Thank you for your help.

Regards,
Mic.

Hello… is anybody out there? :thinking:

Seems not much people reading, but I just want to mention I had the same, only with Exim.
I did asked Claude (you also mentioned using AI) and that was a big help for me, and in the end had an improved version for it.

You can see with the command “cscli metrics” and than look under Parser Metrics.

Good luck :slight_smile:

Thank you. I am also supriesed that nobody answers. Are there no developers or experts in this fourm?

The metrcs show:

# docker exec crowdsec cscli metrics
+----------------------------------------------------------------------------------------------------------------------------------+
| Acquisition Metrics                                                                                                              |
+----------------------------------------+------------+--------------+----------------+------------------------+-------------------+
| Source                                 | Lines read | Lines parsed | Lines unparsed | Lines poured to bucket | Lines whitelisted |
+----------------------------------------+------------+--------------+----------------+------------------------+-------------------+
| file:/remotelogs/stalwart/stalwart.log | 155        | 155          | -              | -                      | -                 |
| file:/var/log/caddy/access.log         | 45         | 45           | -              | 20                     | -                 |
+----------------------------------------+------------+--------------+----------------+------------------------+-------------------+

[...]

+---------------------------------------------------------------+
| Parser Metrics                                                |
+------------------------------------+------+--------+----------+
| Parsers                            | Hits | Parsed | Unparsed |
+------------------------------------+------+--------+----------+
| child-crowdsecurity/http-logs      | 135  | 90     | 45       |
| child-stalwart/parse-extended      | 155  | 155    | -        |
| child-stalwart/parse-logs          | 200  | 155    | 45       |
| crowdsecurity/caddy-logs           | 45   | 45     | -        |
| crowdsecurity/dateparse-enrich     | 45   | 45     | -        |
| crowdsecurity/geoip-enrich         | 45   | 45     | -        |
| crowdsecurity/http-logs            | 45   | 45     | -        |
| crowdsecurity/nextcloud-whitelist  | 45   | 45     | -        |
| crowdsecurity/non-syslog           | 45   | 45     | -        |
| crowdsecurity/public-dns-allowlist | 200  | 200    | -        |
| crowdsecurity/whitelists           | 200  | 200    | -        |
| stalwart/parse-extended            | 155  | 155    | -        |
| stalwart/parse-logs                | 200  | 155    | 45       |
| whitelist_immich_server            | 200  | 200    | -        |
+------------------------------------+------+--------+----------+

[...]

I hope this is good news. Sometimes I am not shure how to interpret the activities of CrowdSec.

I encountered an issue with the Stalwart log parser where malicious IPs were not being banned at all. After debugging with cscli explain -v, I found two critical bugs in the parser configurations that prevent CrowdSec from processing logs properly:

- Missing Timestamp

- IP Extraction Failure

Second, I do not use Caddy, but with an Nginx proxy_pass, some Stalwart logs contain two remote IP addresses (the first is 127.0.0.1 and the second is the attacker’s IP). In this situation, your pattern hits 127.0.0.1, which gets whitelisted.

Below you can find the modified parsers.

name: stalwart/parse-logs
description: Raw parser for Stalwart logs
stage: s00-raw
onsuccess: next_stage

nodes:
  - grok:
      apply_on: Line.Raw
      pattern: '^%{TIMESTAMP_ISO8601:timestamp}\s+%{WORD:log_level}\s+%{DATA:message_text}\s*\(%{DATA:event_type}\)\s*(%{GREEDYDATA:kvpairs})?$'
    statics:
      - target: evt.StrTime
        expression: "evt.Parsed.timestamp"
name: stalwart/parse-extended
description: Parse Stalwart logs including key fields without kv parser
stage: s01-parse
onsuccess: next_stage

# This Parser extracts:
# - timestamp
# - log_level
# - message_text
# - event_type
# - listenerId
# - localPort
# - remoteIp
# - remotePort
# - reason

nodes:
  - grok:
      apply_on: Line.Raw
      pattern: '^%{TIMESTAMP_ISO8601:timestamp}\s+%{WORD:log_level}\s+%{DATA:message_text}\s+\(%{DATA:event_type}\)\s+.*remoteIp\s*=\s*%{IP:remoteIp},.*$'

    statics:
      - meta: log_type
        value: stalwart
      - meta: source_ip
        expression: "evt.Parsed.remoteIp"
      - target: evt.StrAlertSource
        expression: "evt.Parsed.remoteIp"

You can try testing it by

sudo cscli explain --type stalwart --log 'your log line'
echo 'your log line' | sudo tee -a /var/log/stalwart/stalwart.log > /dev/null

PS I am not an expert, but this has been tested and works perfectly.

Hello @Lukes ,

thank you for our feedback. Your parser works much better than my one. It extracts different security error messages from my stalwart.log. Thank you.

I checked several lines from the log file and defined scenario files to react on it.

stalwart-imap-bruteforce.yaml
stalwart-pop3-bruteforce.yaml
stalwart-smtp-auth-bruteforce.yaml
stalwart-smtp-bruteforce.yaml
stalwart-smtp-dkim-anomaly.yaml
stalwart-smtp-relay-abuse.yaml

All of them can be “explained” by CrowdSec.

Here is an example:

docker exec -it crowdsec cscli explain --type stalwart --log 'my log line'
line: my log line
        ├ s00-raw
        |       └ 🟢 stalwart/parse-logs (+6 ~9)
        ├ s01-parse
        |       └ 🟢 stalwart/parse-extended (+3 ~1)
        ├ s02-enrich
        ├-------- parser success 🟢
        ├ Scenarios
                └ 🟢 stalwart/imap-bruteforce

Thank you and have a nice weekend.

Regards
Mic.

Could you please share your dkim-anomaly.yaml, relay-abuse.yaml, and imap-bruteforce.yaml scenarios?

Here are my scenarios.

type: leaky
name: stalwart/imap-bruteforce
description: Detect IMAP bruteforce attempts on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "imap.auth-failed"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: imap
  remediation: true
type: leaky
name: stalwart/pop3-bruteforce
description: Detect POP3 bruteforce attempts on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "pop3.auth-failed"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: pop3
  remediation: true
type: leaky
name: stalwart/smtp-auth-bruteforce
description: Detect SMTP AUTH bruteforce attempts on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.auth-failed"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true
type: leaky
name: stalwart/smtp-smtp-bruteforce
description: Detect SMTP bruteforce and scanners on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.invalid-ehlo",
    "smtp.auth-not-allowed",
    "smtp.auth-mechanism-not-supported"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true
type: leaky
name: stalwart/smtp-dkim-anomaly
description: Detect repeated DKIM verification failures on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.dkim-fail"
  ]
groupby: evt.Parsed.source_ip
capacity: 10
leakspeed: 30m
blackhole: 1h
labels:
  type: anomaly
  service: smtp
  remediation: false
type: leaky
name: stalwart/smtp-relay-abuse
description: Detect unauthenticated SMTP relay attempts on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.relay-denied"
  ]
groupby: evt.Parsed.source_ip
capacity: 3
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true

Thanks. Personally, I recommend setting IPREV to ‘strict’ in the Stalwart UI. It filters out a lot of garbage.

You can also use scenarios with smtp.iprev-fail. Something like:

filter: |
  evt.Line.Labels.type == 'stalwart' &&
  evt.Line.Raw contains "smtp.iprev-fail" &&
  (
    evt.Line.Raw contains "Non-Existent Domain" ||
    evt.Line.Raw contains "No Error" ||
    evt.Line.Raw contains "DNS record not found"
  )

Hi Lukas,
thanks for your feedback. Clearly, my scenarios aren’t entirely off the mark; otherwise, you surely would have said so.

How did you configure the IPREV settings in Stalwart 0.16.x WebGUI? Can you please share me a screenshot?

I think you mean the settings in:

www.your-mail-domain.com/admin/Settings/x:SenderAuth/singleton

I made it like this now:

I’ve added another scenario:

type: leaky
name: stalwart/smtp-iprev-fail
description: Detect SMTP reverse DNS (IPREV) failures on Stalwart logs
filter: |
  evt.Parsed.event_type == "smtp.iprev-fail" &&
  (
    evt.Parsed.kvpairs contains "reason = \"Non-Existent Domain\"" ||
    evt.Parsed.kvpairs contains "reason = \"DNS record not found\"" ||
    evt.Parsed.kvpairs contains "reason = \"No Error\""
  )
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true

Thank you for the hint.

Regards
Mic.

For the IPREV scenario, you can make it more aggressive, for example

type: trigger
...
...
capacity: 0
leakspeed: 0s
...
...

Be extremely careful with “smtp.dkim-fail” scenario. Dropping emails based solely on a strict DKIM failure will lead to false positives.

Also, be careful, Stalwart automatically triggers the sender authentication pipeline even for clients authenticated on submission ports (465/587).

Since your mail client hasn’t signed the outbound mail, Stalwart treats it as an unsigned inbound message at first. If you configure a strict aggressive rule for the smtp-dkim-anomaly scenario, you will literally ban your own authenticated users the moment they hit “Send”.

Below is exmaple of log: 2026-06-30T14:49:02Z INFO DKIM verification failed (smtp.dkim-fail) listenerId = "submissions", localPort = 465, remoteIp = xxxx, remotePort = 19988, strict = false, result = , elapsed = 0ms

Okay, I’ve configured it that way in Stalwart now. Maybe I was being a bit too strict after all. I deleted my additional rule and set the rule for port 25 to ‘strict’.

However, I’m going to leave my scenario as it is rather than making it even tougher. In my opinion, banning a bot after five failed attempts is sufficient.

By the way, my parser can’t “explain” the example line you provided. It also looks different from the lines in my log. Are you already using the new Stalwart 0.16.x?

I’ve also made my scenario a bit more robust. What do you think?

type: leaky
name: stalwart/smtp-iprev-fail
description: Detect SMTP reverse DNS (IPREV) failures on Stalwart logs
filter: |
  evt.Parsed.event_type == "smtp.iprev-fail" &&
  (
    evt.Parsed.result contains "iprev.perm-error" ||
    evt.Parsed.result contains "DNS record not found" ||
    evt.Parsed.result contains "Non-Existent Domain"
  )
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true