Whitelist if dovecot authentication is successful

crowdsec
1

Good morning,

I use Dovecot as an IMAP server and I have a few users (association/company) who share an IP… I have had cases where the “dovecot-spam” scenario https://app.crowdsec.net/hub/author/crowdsecurity/configurations/dovecot-spam is triggered rightly because an email client had a poorly configured mailbox (bad password) and/or a password is changed and the time to modify that on the messaging client it’s already too late, it’s blacklisted… And when it’s blacklisted all other users (who use the same IP) of the entity are blocked.

To avoid this I would like to create a dynamic whitelist: when Dovecot authentication is successful, we add this IP to the whitelist for ~24 hours.

I think I managed to do the parser for that:

onsuccess: next_stage
filter: "evt.Parsed.program == 'dovecot'"
name: crowdsecurity/dovecot-auth-parser
description: "Parse dovecot logs"
pattern_syntax:
  DOVECOT_SUCCESS_LOGIN: 'imap-login: Login: user=<%{EMAILADDRESS:email}>, method=%{WORD:method}, rip=%{IPV4:source_ip}'
nodes:
  - grok:
      name: "DOVECOT_SUCCESS_LOGIN" 
      apply_on: message
      statics:
        - meta: log_type
          value: dovecot_auth_succcess

It works (I followed the doc to create a test environment: Creating parsers | CrowdSec) and here is the result:

results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Parsed["logsource"] == "syslog"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Parsed["message"] == "imap-login: Login: user=<user@domaine.com>, method=PLAIN, rip=X.Y.124.127, lip=A.B.A.B, mpid=2191284, TLS, session=<XkY34nwc5MGfRXx/>"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Parsed["method"] == "PLAIN"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Parsed["program"] == "dovecot"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Parsed["source_ip"] == "X.Y.124.127"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Parsed["timestamp"] == "Jul  5 11:42:37"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Meta["datasource_path"] == "myservice-logs.log"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Meta["datasource_type"] == "file"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Meta["log_type"] == "dovecot_auth_succcess"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Meta["machine"] == "master"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Meta["timestamp"] == "2024-07-05T11:42:37Z"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Enriched["MarshaledTime"] == "2024-07-05T11:42:37Z"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][1].Evt.Whitelisted == false

But I can’t understand/know where to create my whitelist, if it’s in a parsers/s02-enrich:

name: crowdsecurity/whitelists-dovecot
description: "Whitelist dovect"
whitelist:
 reason: "Dovecot logins success"
 expression:
 - "evt.Meta.log_type == 'dovecot_auth_success'"

Or if it’s more of a scenario:

name: crowdsecurity/dovecot-whitelist
description: "Automatically whitelist IPs based on successful authentication"
filter: "evt.Meta.log_type == 'dovecot_auth_success'"
groupby: evt.Meta.source_ip
whitelist:
 reason: "Successful authentication"
 duration: 24h

None seem to have any effect (and/or I don’t know how to consult the whitelist…) I tell myself that if it’s a scenario it should be visible in “cscli alert” but if it’s in the parser, how to know the whitelisted IPs?

Thanks for your help,
David

2

So currently the whitelist are mainly based on static values EG IP or CIDR or an EXPRresssion.

So in theory you could achieve this by using the stash feature of a parser so example:

Please test these examples, I dont currently have a system to test but knowledge of stash and how the enricher should work, it should work in practice as there is no mechanism to whitelist for a duration so we need to use the stash feature

onsuccess: next_stage
filter: "evt.Parsed.program == 'dovecot'"
name: crowdsecurity/dovecot-auth-parser
description: "Parse dovecot logs"
pattern_syntax:
  DOVECOT_SUCCESS_LOGIN: 'imap-login: Login: user=<%{DATA:dovecot_user}>, method=%{WORD:method}, rip=%{IPV4:source_ip}'
nodes:
  - grok:
      name: "DOVECOT_SUCCESS_LOGIN" 
      apply_on: message
      statics:
        - meta: log_type
          value: dovecot_auth_succcess
    stash:
      - name: dovecot_valid_auth
        key: "evt.Parsed.dovecot_user+evt.Parsed.source_ip"
        value: string("valid_auth")
        ttl: 24h
        size: 100 #Make this as small as possible with usernames + ip beaware its FIFO so if a user get pushed out the cache then they loose the 24hrs
statics:
  - meta: source_ip
    expression: evt.Parsed.source_ip

The parser is simply to same as yours (I use dovecot_user instead of email as it needs to track same variables as the original dovecot parser), but we added the stash section which caches a valid for a TTL, simply I use the “dovecot_user” and append the IP of the valid authentication to use that as the key incase somebody bruteforces the username I dont want to cache just either as that will lead to false negatives.

Then the whitelist can be used to filter with the same dovecot scenario filters to ignore them if the stashed value is valid auth

name: me/dovecot-whitelist
description: "Automatically whitelist IPs based on successful authentication"
filter: "evt.Meta.log_type == 'dovecot_auth_succcess' || evt.Meta.dovecot_login_result == 'auth_failed'"
whitelist:
 reason: "Successful authentication"
 expression:
   - "let stashValue = GetFromStash('dovecot_valid_auth', evt.Parsed.dovecot_user+evt.Meta.source_ip);
      stashValue == 'valid_auth'"

This hasn’t been extensively tested so I would make sure you keep on eye on the performance and check for false negatives.

Also each time the user successfully logs in the TTL will be constantly updated to 24hrs, so meaning the last time they logged in they will have 24hrs of whitelisting.

3

Thanks @iiAmLoz

I think the parser stores correctly, given “cscli metrics”:

Whitelist Metrics:
╭──────────────────────────────────┬────────────────────────────────────┬────────┬─────────────╮
│             Whitelist            │               Reason               │  Hits  │ Whitelisted │
├──────────────────────────────────┼────────────────────────────────────┼────────┼─────────────┤
│ crowdsecurity/whitelists         │ Retzo ipv4/ipv6 ip/ranges          │ 113343 │ 13148       │
│ me/dovecot-whitelist             │ Successful authentication          │ 12110  │ 11822       │
╰──────────────────────────────────┴────────────────────────────────────┴────────┴─────────────╯

But if I test a case with “cscli explain” I don’t have the impression that the system considers the white list:

line: A.B.136.202 - - [08/Jul/2024:15:47:42 +0200] "GET / HTTP/1.0" 200 670 "-" "ZoominfoBot (zoominfobot at zoominfo dot com)"
	├ s00-raw
	|	├ 🔴 crowdsecurity/syslog-logs
	|	└ 🟢 crowdsecurity/non-syslog (+5 ~8)
	├ s01-parse
	|	└ 🟢 crowdsecurity/apache2-logs (+21 ~2)
	├ s02-enrich
	|	├ 🟢 crowdsecurity/dateparse-enrich (+2 ~2)
	|	├ 🔴 me/dovecot-whitelist
	|	├ 🟢 crowdsecurity/geoip-enrich (+13)
	|	├ 🟢 crowdsecurity/http-logs (+6)
	|	└ 🟢 crowdsecurity/whitelists (unchanged)
	├-------- parser success 🟢
	├ Scenarios
		├ 🟢 crowdsecurity/http-bad-user-agent
		└ 🟢 crowdsecurity/http-crawl-non_statics

However, the IP A.B.136.202 has been present in my dovecot log for less than 24 hours:

Jul 8 18:33:47 master dovecot: imap-login: Login: user=marion.michel@dom.fr, method=PLAIN, rip=A.B.136.202, lip=w.x.y.z, mpid=509625, TLS, session=<Zrov+r4cYsxaMYjK>
To be sure :

Do we agree that it was good to put in parser/s02-enrich? (because I get an error if I put it in scenario => line 4: field whitelist not found in type leakybucket.BucketFactory" )

Yes I missed something to allow me to test?
Can we list the IPs in the whitelist with “cscli”?

David

4

cscli explain doesnt work with the stash feature since to explain the log line it creates a new instance of crowdsec behind the scenes to process the logs so it doesnt share the same memory space.

Do we agree that it was good to put in parser/s02-enrich?

Yes its a whitelist enricher so it goes in that folder.

Can we list the IPs in the whitelist with “cscli”?

You can just add a general whitelist as per the docs I linked before but they are static and wont allow users to lets so log in from home IP and get the same treatment.

You can see if this is working by checking the CrowdSec logs it should state the log line was discarded due to whitelisting.

But the log line you shown was for http not for dovecot? remember the example I shown was only to whitelist dovecot logins / failed logins if user had logged in previously in 24 hours TTL

5

Ha ok, I thought / wanted the IP to be in the “general” whitelist (for all filters). I imagine that for this you simply have to remove “|| evt.Meta.dovecot_login_result == ‘auth_failed’” from the “filter”?

I have the impression that it’s a bit out of the box… Shouldn’t I go for a simpler solution with an external script that populates a “text” whitelist? In terms of saving machine resources, wouldn’t that be a good thing? (seen your comments on “stash”)

With a bash script of this type: PrivateBin

And a whitelist like this:

name: me/dovecot-whitelist
description: "Automatically whitelist IPs based on successful authentication (external design)"
whitelist:
 reason: "Dovecot success login IP"
 expression:
 - "any(File('dovecot-whitelist-ip.log'), { len(#) > 0 && IpInRange(evt.Overflow.Alert.Source.IP ,#)})"
6

I didn’t get that impression from the first post, I just thought you want to whitelist dovecot events if the user had previously logged in correct. You can completely just remove the filter then it will trigger for all lines. However, you need to update the stash requirements to drop the usernames prefix as if you want all events then you can only check the IP. (Which means you could get alot of false negatives as if a user machine is compromied and they attack you then all events are whitelisted for 24 hours)

I have the impression that it’s a bit out of the box… Shouldn’t I go for a simpler solution with an external script that populates a “text” whitelist? In terms of saving machine resources, wouldn’t that be a good thing? (seen your comments on “stash”)

You can only issue is CrowdSec only loads IP’s from the file at startup so everytime you modify the file you have to restart meaning you could loose tracked events and sceanrios may not trigger as the memory gets restarted.

Edit: im am thinking of a different way, but it may mean a new functionality in CrowdSec

7

Ha ok, actually it’s not great to systematically restart Crowdsec, I didn’t have this data.

Yes, sorry, I may be expressing myself poorly (I’m using a translator… that doesn’t help)

The intention is to “save” each IP in the whitelist for 24 hours for which Dovecot authentication has been successful. And that this white list applies to the entire system / all scenarios (for example the “crowdsecurity/http-probing” scenario (which is triggered in my case as a false positive on a virtualhost hosting Nextcloud) but suddenly it was not maybe not the filter that needed to be touched in your proposal? Or did it already do what I wanted but I didn’t realize it?

I had big stability problems with Crowdsec:

  • full I/O occupation if I want to restart crowdsec-firewall-boucer and very slow…
  • Crowdsec launched fine but after a few seconds kept restarting with the message in sysctl and nothing specific in the crowdsec logs
crowdsec Main process exited, code=exited, status=1/FAILURE in sysctl

So I’m starting from scratch on my config and I’m putting the bricks back little by little to try to detect where the problem comes from…)