CrowdSec Parser for Stalwart Logs

2.687

Hello CrowdSec Community,

I’ve been using CrowdSec as an extension in the reverse proxy Caddy for quite some time, but I’m new to the community. Everything is running in a Docker Compose instance with its own MacVLAN. In my small, private home lab, I’ve installed Stalwart Mail in Docker, which generates a continuous log file called stalwart.log. I’ve built a CrowdSec parser using AI, but I’d like to know if it’s correct.

I’ve been using CrowdSec for a while now, but I’d like your feedback on whether it’s working correctly. In the stalwart.log file, I find the following log lines, for example, because Stalwart also has its own security measures, which I would like to address with CrowdSec:

2026-04-26T00:17:59Z INFO Banned due to scan (security.scan-ban) listenerId = "submissions", localPort = 465, remoteIp = 104.248.203.156, remotePort = 47964, remoteIp = 104.248.203.156, reason = "Invalid SMTP command"

To address this, I added the following entry to the acquise.yaml file:

[...]
---
filenames:
  - /remotelogs/stalwart/stalwart.log
labels:
  type: stalwart

I also created the file /parsers/s00-raw/stalwart-logs.yaml created:

name: stalwart/parse-logs
description: Raw parser for Stalwart logs
stage: s00-raw
onsuccess: next_stage

nodes:
  - grok:
      apply_on: Line.Raw
      pattern: '^%{TIMESTAMP_ISO8601:timestamp}\s+%{WORD:log_level}\s+%{DATA:message_text}\s*\(%{DATA:event_type}\)\s*(%{GREEDYDATA:kvpairs})?$'

Another file /parsers/s01-parse/stalwart-logs-extended.yaml looks like this:

Eine weitere Datei /parsers/s01-parse/stalwart-logs-extended.yaml sieht so aus:
name: stalwart/parse-extended
description: Parse Stalwart logs including key fields without kv parser
stage: s01-parse
onsuccess: next_stage

# This Parser extracts:
# - timestamp
# - log_level
# - message_text
# - event_type
# - listenerId
# - localPort
# - remoteIp
# - remotePort
# - reason

nodes:
  - grok:
      apply_on: Line.Raw
      pattern: '^%{TIMESTAMP_ISO8601:timestamp}\s+%{WORD:log_level}\s+%{DATA:message_text}\s+\(%{DATA:event_type}\)\s*(?:listenerId\s*=\s*"%{DATA:listenerId}",\s*)?(?:localPort\s*=\s*%{INT:localPort},\s*)?(?:remoteIp\s*=\s*%{IP:remoteIp},\s*)?(?:remotePort\s*=\s*%{INT:remotePort},\s*)?(?:reason\s*=\s*"%{DATA:reason}")?.*$'

Finally there is /scenarios/stalwart-smtp-bruteforce.yaml:

type: leaky
name: stalwart/smtp-smtp-bruteforce
description: Detect SMTP bruteforce and scanners on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.invalid-ehlo",
    "smtp.auth-not-allowed",
    "smtp.auth-mechanism-not-supported"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true

My question for the parser experts is whether this is correct and can work. At least when starting Stalwart, there are no error messages.

Thank you for your help.

Regards,
Mic.

Hello… is anybody out there? :thinking:

Seems not much people reading, but I just want to mention I had the same, only with Exim.
I did asked Claude (you also mentioned using AI) and that was a big help for me, and in the end had an improved version for it.

You can see with the command “cscli metrics” and than look under Parser Metrics.

Good luck :slight_smile:

Thank you. I am also supriesed that nobody answers. Are there no developers or experts in this fourm?

The metrcs show:

# docker exec crowdsec cscli metrics
+----------------------------------------------------------------------------------------------------------------------------------+
| Acquisition Metrics                                                                                                              |
+----------------------------------------+------------+--------------+----------------+------------------------+-------------------+
| Source                                 | Lines read | Lines parsed | Lines unparsed | Lines poured to bucket | Lines whitelisted |
+----------------------------------------+------------+--------------+----------------+------------------------+-------------------+
| file:/remotelogs/stalwart/stalwart.log | 155        | 155          | -              | -                      | -                 |
| file:/var/log/caddy/access.log         | 45         | 45           | -              | 20                     | -                 |
+----------------------------------------+------------+--------------+----------------+------------------------+-------------------+

[...]

+---------------------------------------------------------------+
| Parser Metrics                                                |
+------------------------------------+------+--------+----------+
| Parsers                            | Hits | Parsed | Unparsed |
+------------------------------------+------+--------+----------+
| child-crowdsecurity/http-logs      | 135  | 90     | 45       |
| child-stalwart/parse-extended      | 155  | 155    | -        |
| child-stalwart/parse-logs          | 200  | 155    | 45       |
| crowdsecurity/caddy-logs           | 45   | 45     | -        |
| crowdsecurity/dateparse-enrich     | 45   | 45     | -        |
| crowdsecurity/geoip-enrich         | 45   | 45     | -        |
| crowdsecurity/http-logs            | 45   | 45     | -        |
| crowdsecurity/nextcloud-whitelist  | 45   | 45     | -        |
| crowdsecurity/non-syslog           | 45   | 45     | -        |
| crowdsecurity/public-dns-allowlist | 200  | 200    | -        |
| crowdsecurity/whitelists           | 200  | 200    | -        |
| stalwart/parse-extended            | 155  | 155    | -        |
| stalwart/parse-logs                | 200  | 155    | 45       |
| whitelist_immich_server            | 200  | 200    | -        |
+------------------------------------+------+--------+----------+

[...]

I hope this is good news. Sometimes I am not shure how to interpret the activities of CrowdSec.

I encountered an issue with the Stalwart log parser where malicious IPs were not being banned at all. After debugging with cscli explain -v, I found two critical bugs in the parser configurations that prevent CrowdSec from processing logs properly:

- Missing Timestamp

- IP Extraction Failure

Second, I do not use Caddy, but with an Nginx proxy_pass, some Stalwart logs contain two remote IP addresses (the first is 127.0.0.1 and the second is the attacker’s IP). In this situation, your pattern hits 127.0.0.1, which gets whitelisted.

Below you can find the modified parsers.

name: stalwart/parse-logs
description: Raw parser for Stalwart logs
stage: s00-raw
onsuccess: next_stage

nodes:
  - grok:
      apply_on: Line.Raw
      pattern: '^%{TIMESTAMP_ISO8601:timestamp}\s+%{WORD:log_level}\s+%{DATA:message_text}\s*\(%{DATA:event_type}\)\s*(%{GREEDYDATA:kvpairs})?$'
    statics:
      - target: evt.StrTime
        expression: "evt.Parsed.timestamp"
name: stalwart/parse-extended
description: Parse Stalwart logs including key fields without kv parser
stage: s01-parse
onsuccess: next_stage

# This Parser extracts:
# - timestamp
# - log_level
# - message_text
# - event_type
# - listenerId
# - localPort
# - remoteIp
# - remotePort
# - reason

nodes:
  - grok:
      apply_on: Line.Raw
      pattern: '^%{TIMESTAMP_ISO8601:timestamp}\s+%{WORD:log_level}\s+%{DATA:message_text}\s+\(%{DATA:event_type}\)\s+.*remoteIp\s*=\s*%{IP:remoteIp},.*$'

    statics:
      - meta: log_type
        value: stalwart
      - meta: source_ip
        expression: "evt.Parsed.remoteIp"
      - target: evt.StrAlertSource
        expression: "evt.Parsed.remoteIp"

You can try testing it by

sudo cscli explain --type stalwart --log 'your log line'
echo 'your log line' | sudo tee -a /var/log/stalwart/stalwart.log > /dev/null

PS I am not an expert, but this has been tested and works perfectly.

Hello @Lukes ,

thank you for our feedback. Your parser works much better than my one. It extracts different security error messages from my stalwart.log. Thank you.

I checked several lines from the log file and defined scenario files to react on it.

stalwart-imap-bruteforce.yaml
stalwart-pop3-bruteforce.yaml
stalwart-smtp-auth-bruteforce.yaml
stalwart-smtp-bruteforce.yaml
stalwart-smtp-dkim-anomaly.yaml
stalwart-smtp-relay-abuse.yaml

All of them can be “explained” by CrowdSec.

Here is an example:

docker exec -it crowdsec cscli explain --type stalwart --log 'my log line'
line: my log line
        ├ s00-raw
        |       └ 🟢 stalwart/parse-logs (+6 ~9)
        ├ s01-parse
        |       └ 🟢 stalwart/parse-extended (+3 ~1)
        ├ s02-enrich
        ├-------- parser success 🟢
        ├ Scenarios
                └ 🟢 stalwart/imap-bruteforce

Thank you and have a nice weekend.

Regards
Mic.

Could you please share your dkim-anomaly.yaml, relay-abuse.yaml, and imap-bruteforce.yaml scenarios?

Here are my scenarios.

type: leaky
name: stalwart/imap-bruteforce
description: Detect IMAP bruteforce attempts on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "imap.auth-failed"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: imap
  remediation: true
type: leaky
name: stalwart/pop3-bruteforce
description: Detect POP3 bruteforce attempts on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "pop3.auth-failed"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: pop3
  remediation: true
type: leaky
name: stalwart/smtp-auth-bruteforce
description: Detect SMTP AUTH bruteforce attempts on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.auth-failed"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true
type: leaky
name: stalwart/smtp-smtp-bruteforce
description: Detect SMTP bruteforce and scanners on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.invalid-ehlo",
    "smtp.auth-not-allowed",
    "smtp.auth-mechanism-not-supported"
  ]
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true
type: leaky
name: stalwart/smtp-dkim-anomaly
description: Detect repeated DKIM verification failures on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.dkim-fail"
  ]
groupby: evt.Parsed.source_ip
capacity: 10
leakspeed: 30m
blackhole: 1h
labels:
  type: anomaly
  service: smtp
  remediation: false
type: leaky
name: stalwart/smtp-relay-abuse
description: Detect unauthenticated SMTP relay attempts on Stalwart logs
filter: |
  evt.Parsed.event_type in [
    "smtp.relay-denied"
  ]
groupby: evt.Parsed.source_ip
capacity: 3
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true

Thanks. Personally, I recommend setting IPREV to ‘strict’ in the Stalwart UI. It filters out a lot of garbage.

You can also use scenarios with smtp.iprev-fail. Something like:

filter: |
  evt.Line.Labels.type == 'stalwart' &&
  evt.Line.Raw contains "smtp.iprev-fail" &&
  (
    evt.Line.Raw contains "Non-Existent Domain" ||
    evt.Line.Raw contains "No Error" ||
    evt.Line.Raw contains "DNS record not found"
  )

Hi Lukas,
thanks for your feedback. Clearly, my scenarios aren’t entirely off the mark; otherwise, you surely would have said so.

How did you configure the IPREV settings in Stalwart 0.16.x WebGUI? Can you please share me a screenshot?

I think you mean the settings in:

www.your-mail-domain.com/admin/Settings/x:SenderAuth/singleton

I made it like this now:

I’ve added another scenario:

type: leaky
name: stalwart/smtp-iprev-fail
description: Detect SMTP reverse DNS (IPREV) failures on Stalwart logs
filter: |
  evt.Parsed.event_type == "smtp.iprev-fail" &&
  (
    evt.Parsed.kvpairs contains "reason = \"Non-Existent Domain\"" ||
    evt.Parsed.kvpairs contains "reason = \"DNS record not found\"" ||
    evt.Parsed.kvpairs contains "reason = \"No Error\""
  )
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true

Thank you for the hint.

Regards
Mic.

For the IPREV scenario, you can make it more aggressive, for example

type: trigger
...
...
capacity: 0
leakspeed: 0s
...
...

Be extremely careful with “smtp.dkim-fail” scenario. Dropping emails based solely on a strict DKIM failure will lead to false positives.

Also, be careful, Stalwart automatically triggers the sender authentication pipeline even for clients authenticated on submission ports (465/587).

Since your mail client hasn’t signed the outbound mail, Stalwart treats it as an unsigned inbound message at first. If you configure a strict aggressive rule for the smtp-dkim-anomaly scenario, you will literally ban your own authenticated users the moment they hit “Send”.

Below is exmaple of log: 2026-06-30T14:49:02Z INFO DKIM verification failed (smtp.dkim-fail) listenerId = "submissions", localPort = 465, remoteIp = xxxx, remotePort = 19988, strict = false, result = , elapsed = 0ms

Okay, I’ve configured it that way in Stalwart now. Maybe I was being a bit too strict after all. I deleted my additional rule and set the rule for port 25 to ‘strict’.

However, I’m going to leave my scenario as it is rather than making it even tougher. In my opinion, banning a bot after five failed attempts is sufficient.

By the way, my parser can’t “explain” the example line you provided. It also looks different from the lines in my log. Are you already using the new Stalwart 0.16.x?

I’ve also made my scenario a bit more robust. What do you think?

type: leaky
name: stalwart/smtp-iprev-fail
description: Detect SMTP reverse DNS (IPREV) failures on Stalwart logs
filter: |
  evt.Parsed.event_type == "smtp.iprev-fail" &&
  (
    evt.Parsed.result contains "iprev.perm-error" ||
    evt.Parsed.result contains "DNS record not found" ||
    evt.Parsed.result contains "Non-Existent Domain"
  )
groupby: evt.Parsed.source_ip
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true

About IPREV scenario, The current setup uses capacity: 5 and a leakspeed: 10m. However, from a practical standpoint, this configuration doesn’t make much sense for IPREV failures on Stalwart.
1 - Stalwart drops the incoming mail immediately if the sender’s IP lacks valid DNS records.
2 - In real-world scenarios, misconfigured mail servers or single-burst spam bots do not immediately retry the connection multiple times within such a short window.

Your IPREV scenario doesn’t work:

sudo cscli explain --type stalwart --log '2026-07-02T05:54:51Z INFO IPREV check failed (smtp.iprev-fail) listenerId = "smtp", localPort = 25, remoteIp = 138.128.245.98, remotePort = 52120, domain = "sz-yuhong.com", result = IPREV permanent error (iprev.perm-error) { causedBy = DNS record not found (mail-auth.dns-record-not-found) { code = "Non-Existent Domain" } }, elapsed = 1ms'
line: 2026-07-02T05:54:51Z INFO IPREV check failed (smtp.iprev-fail) listenerId = "smtp", localPort = 25, remoteIp = 138.128.245.98, remotePort = 52120, domain = "sz-yuhong.com", result = IPREV permanent error (iprev.perm-error) { causedBy = DNS record not found (mail-auth.dns-record-not-found) { code = "Non-Existent Domain" } }, elapsed = 1ms
	├ s00-raw
	|	└ 🟢 stalwart/parse-logs (+6 ~9)
	├ s01-parse
	|	├ 🔴 crowdsecurity/appsec-logs
	|	├ 🔴 crowdsecurity/nginx-logs
	|	├ 🔴 crowdsecurity/sshd-logs
	|	├ 🔴 crowdsecurity/sshd-success-logs
	|	└ 🟢 stalwart/parse-extended (+3 ~1)
	├ s02-enrich
	|	├ 🟢 crowdsecurity/dateparse-enrich (+2 ~2)
	|	├ 🟢 crowdsecurity/geoip-enrich (+13)
	|	├ 🔴 crowdsecurity/http-logs
	|	├ 🟢 crowdsecurity/public-dns-allowlist (unchanged)
	|	└ 🟢 crowdsecurity/whitelists (unchanged)
	├-------- parser success 🟢
	├ Scenarios

Corrected version:

type: leaky
name: stalwart/smtp-iprev-fail
description: Detect SMTP reverse DNS (IPREV) failures on Stalwart logs
filter: |
  evt.Parsed.event_type == "smtp.iprev-fail" &&
  (
    evt.Parsed.kvpairs contains 'code = "Non-Existent Domain"' ||
    evt.Parsed.kvpairs contains 'code = "No Error" ||
    evt.Parsed.kvpairs contains 'code = "DNS record not found"
  )
groupby: evt.Parsed.remoteIp
capacity: 5
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true

Debug:

sudo cscli explain --type stalwart --log '2026-07-02T05:54:51Z INFO IPREV check failed (smtp.iprev-fail) listenerId = "smtp", localPort = 25, remoteIp = 138.128.245.98, remotePort = 52120, domain = "sz-yuhong.com", result = IPREV permanent error (iprev.perm-error) { causedBy = DNS record not found (mail-auth.dns-record-not-found) { code = "Non-Existent Domain" } }, elapsed = 1ms'  
line: 
	├ s00-enrich
	|	└ 🟢 crowdsecurity/rdns (+39 ~6)
	├ s01-whitelist
	|	├ 🟢 crowdsecurity/cdn-whitelist (unchanged)
	|	├ 🟢 crowdsecurity/google-special-crawlers-whitelist (unchanged)
	|	└ 🟢 crowdsecurity/seo-bots-whitelist (unchanged)
	└-------- parser failure 🔴

line: 2026-07-02T05:54:51Z INFO IPREV check failed (smtp.iprev-fail) listenerId = "smtp", localPort = 25, remoteIp = 138.128.245.98, remotePort = 52120, domain = "sz-yuhong.com", result = IPREV permanent error (iprev.perm-error) { causedBy = DNS record not found (mail-auth.dns-record-not-found) { code = "Non-Existent Domain" } }, elapsed = 1ms
	├ s00-raw
	|	└ 🟢 stalwart/parse-logs (+6 ~9)
	├ s01-parse
	|	├ 🔴 crowdsecurity/appsec-logs
	|	├ 🔴 crowdsecurity/nginx-logs
	|	├ 🔴 crowdsecurity/sshd-logs
	|	├ 🔴 crowdsecurity/sshd-success-logs
	|	└ 🟢 stalwart/parse-extended (+3 ~1)
	├ s02-enrich
	|	├ 🟢 crowdsecurity/dateparse-enrich (+2 ~2)
	|	├ 🟢 crowdsecurity/geoip-enrich (+13)
	|	├ 🔴 crowdsecurity/http-logs
	|	├ 🟢 crowdsecurity/public-dns-allowlist (unchanged)
	|	└ 🟢 crowdsecurity/whitelists (unchanged)
	├-------- parser success 🟢
	├ Scenarios
		└ 🟢 my/stalwart-iprev-fail

Test:

Also, take a look at Stalwart’s built-in Abuse Protection ( Auto-banning | Stalwart ). If you prefer using CrowdSec for all blocks, you can easily build a scenario around it.

Thanks. What does “code =” actually do in the scenario? And how does it differ from “reason =”? As I said, I’m not very familiar with CrowdSec parsers.

If Stalwart issues a ban after just a single failed attempt, is “type: leaky” really the right choice, or should one switch to “type: trigger”?

Does your test work better than using “explain” in CrowdSec?

I asked Gemini now and he told me your described s01 parser may not work with the scenarios. He recommended this adaptation:

name: stalwart/parse-extended
description: Parse Stalwart logs including key fields without kv parser
stage: s01-parse
onsuccess: next_stage

nodes:
  - grok:
      apply_on: Line.Raw
      # Optimized pattern:
      pattern: '^%{TIMESTAMP_ISO8601:timestamp}\s+%{WORD:log_level}\s+%{DATA:message_text}(?:\s+\(%{DATA:event_type}\))?.*remoteIp\s*=\s*%{IP:remoteIp}.*$'


    statics:
      - meta: log_type
        value: stalwart
      - meta: source_ip
        expression: "evt.Parsed.remoteIp"
      - target: evt.StrAlertSource
        expression: "evt.Parsed.remoteIp"
      # This line fixes the scenarios:
      - target: evt.Parsed.source_ip
        expression: "evt.Parsed.remoteIp"

And this is the adapted IPREV scenario:

type: leaky
name: stalwart/smtp-iprev-fail
description: Detect SMTP reverse DNS (IPREV) failures on Stalwart logs
filter: |
  evt.Parsed.event_type == "smtp.iprev-fail" &&
  (
    evt.Parsed.message_text contains "IPREV" ||
    evt.Parsed.message_text contains "failed" ||
    evt.Parsed.message_text contains "error"
  )
groupby: evt.Parsed.source_ip
capacity: 3
leakspeed: 10m
blackhole: 1h
labels:
  type: attack
  service: smtp
  remediation: true

I had no chance to try it out up to now.

The code parameter captures the precise DNS failure returned by Stalwart.

It works. Check the photo in my post above. I think type: trigger is much better.

There are many ways to write a parser and a scenario. Anyway, make sure to check all scenarios after modifying the parser. I believe it is bad practice to write a scenario without using the full error message, such as evt.Parsed.message_text contains "failed".

Also, You can test your CrowdSec secnario using echo command. Whether it results in a ban or not.

So, you think your parser and IPREV scenario work best? In that case, I’d be happy to just adopt them. I find the whole parsing process in CrowdSec rather complicated and annoying. Unfortunately, I don’t have much time to deal with it right now, either. If there were a ready-made solution - like the one for NextCloud - I’d use that instead.

....
filter: |
  evt.Parsed.event_type == "smtp.iprev-fail" &&
  (
    evt.Parsed.kvpairs contains 'code = "Non-Existent Domain"' ||
    evt.Parsed.kvpairs contains 'code = "No Error" ||
    evt.Parsed.kvpairs contains 'code = "DNS record not found"
  )
....

I teste it code = causes a syntax error in CrowdSec.

The parser is yours, I only tweaked it to get it working. I think my scenario is better. There isn’t a ready-made solution :(. Usually, people use the built-in F2B in Stalwart or write a custom parser.

I would say the same. :grinning_face_with_smiling_eyes: I have now used the combination of the optimised s01-parser and the scenario without code = or reason =. Two AI systems told me this is the best solution that all my scenarios will run and the IPREV scenario is more robust.

It is hard for me to prove if it is right. If you create your own log lines for testing they also can be wrong.

The reason I want to have it in CrowdSec additional to the security feaures of Stalwart is, that I have an OpenWRT router with a CrouwdSec Bouncer upstream my PC where Stalwart is running in a Docker. This router can also make a static blocking before an attack reaches the PC and CrowdSec in Caddy (in this Docker) cannot block traffic directly to the Stalwart mail ports.

Thank you for your support so far.