New install, postfix / IMAP

I’ve installed crowdsec and netfilter-blocker. The server is a postfix server, also running IMAP.

The install did not detect postfix (!) so I did this in acquis.yaml, is this correct?

#Generated acquisition file - wizard.sh (service: sshd) / files : /var/log/secure
filenames:
  - /var/log/secure
labels:
  type: syslog
---
#Generated acquisition file - wizard.sh (service: linux) / files : /var/log/messages
filenames:
  - /var/log/messages
labels:
  type: syslog
---

filenames:
  - /var/log/maillog
labels:
  type: syslog
---

I have this now after running for 10 minutes:

time="2020-10-24T14:43:41+10:00" level=info msg="Buckets Metrics:"
+--------+---------------+-----------+--------------+--------+---------+
| BUCKET | CURRENT COUNT | OVERFLOWS | INSTANCIATED | POURED | EXPIRED |
+--------+---------------+-----------+--------------+--------+---------+
+--------+---------------+-----------+--------------+--------+---------+
time="2020-10-24T14:43:41+10:00" level=info msg="Acquisition Metrics:"
+-------------------+------------+--------------+----------------+------------------------+
|      SOURCE       | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+-------------------+------------+--------------+----------------+------------------------+
| /var/log/maillog  |        326 | -            |            326 | -                      |
| /var/log/messages |          7 | -            |              7 | -                      |
| /var/log/secure   |          5 | -            |              5 | -                      |
+-------------------+------------+--------------+----------------+------------------------+
time="2020-10-24T14:43:41+10:00" level=info msg="Parser Metrics:"
+----------------------------------+------+--------+----------+
|             PARSERS              | HITS | PARSED | UNPARSED |
+----------------------------------+------+--------+----------+
| child-crowdsecurity/postfix-logs |   68 | -      |       68 |
| crowdsecurity/postfix-logs       |   34 | -      |       34 |
| crowdsecurity/syslog-logs        |  338 |    338 | -        |
+----------------------------------+------+--------+----------+

I assume this is now reading Postfix logs ok and work as intended?

I have temporarily disabled fail2ban to see how this goes. In fail2ban I have had the following rules active on this server: postfix, postfix-rbl, postfix-postscreen, postfix-sasl.

I’m not sure yet that crowdsec is going to cover all of those scenarios - what is included in the out of the box postfix collection? All of those jails trigger regularly in fail2ban.

Thanks for your feedback.

I guess what we’ve now is something near postfix+postfix-sasl. In fact, we are kind of stuck with the log I managed to get. If you care to provide the logs you want to be taken care of, we will improve the scenario.

By the way postfix-rbl works by detecting when postfix rejects any mail because of sender being in rbl and then fail2ban is blocking them. We could improve our bouncers or add a new one to directly ban rbl blacklisted sender without further postfix configuration.

I hope I answered your questions. Feel free to raise any more issues, we’ll definetely look into it, as we want postfix senario to be well fitted.

OK, the current setup is as follows:
I use fail2ban’s default postfix filter (mode = more) with default 3 attempts, then IP block. Additionally I run postfix-sasl and postfix-rbl with maxretry of 1. My setup is well tuned with blacklists over a few years, and if a RBL is failed it’s an instant ban, same with a sasl auth failure on the submission port.
My postscreen filter is a custom one (also called with a maxretry of 1) I use to block a botnet which is a fairly constant source of attempts:

[INCLUDES]
before = common.conf

[Definition]
_daemon = postfix/postscreen
failregex = ^%(__prefix_line)sPREGREET \d+ after \d+\.\d+ from \[<HOST>\]:\d+: EHLO ylmf-pc\\r\\n
ignoreregex =

[Init]

journalmatch = _SYSTEMD_UNIT=postfix.service

I have also at times had other regex in there getting blocked at the postscreen stage, but the current setup seems to work OK.

I can send logs and a snapshot of fail2ban-regex printed hits - let me know where I can securely submit.

Oh, and a way to disable ipv6 configuration to prevent unnecessary fatal errors in the log would be good.

Hello @Swallowtail,

What do you mean about fatal errors and ipv6 ? Normally crowdsec (and the bouncers) should handle this pretty well !

ps: we’re going to provide you an email address and gpg pubkey soon so you can provide some logs to us

thanks,

Ok, to securely send us logs, you can use gpg encrypted mail or text via support@crowdsec.net with the gpg public fingerprint 0x8c332d31 (the key is stored on http://keys.gnupg.net/)

Or either use the the following public armored output:

-----BEGIN PGP PUBLIC KEY BLOCK-----

mQGNBF+VOSUBDADP6bxKDv88CdLBNhQMFNI37LE82vyfIAQmrGszON1m1EtL/LSQ
b/vC9mmlkUmJHM+bdxJ0BSl/xlWwrXjHVpaZNoluQDngVUe62cybN4tpFCvtVTMr
lo4Y0UhETgOmBFxaQLVd7Xc/jaSZGoHtSzh9hpGHg9pKrcYviG0MR173JYQfilw3
L8yJ+K/oUUpvh2MRRwXiCNUVLtTppb7oxlcdExb0Px2PcaC34e/M30xFwiu7VJFj
0D7IIdKs6gvZuqwkNSUBF8/jtuzzM/YGzJHIdvOj15z+81/o/e6p3xvY/IKmyXC/
1FMD8f4g5T/5fNDVq6QgJLel/g0bJ+kG75ccXfY45xKFo/YhdQ2Wg9JQX5Yjc5k7
5AI0iuJjatXlym2Ek1niPEqR5H0C/KXFG4mPyCu9wzJu11jtY34e5TNYl9DA31F6
81BbMmVFg4EbhYSN/2DuxpCvt2qQpk33bmdT7tFWcd2hYB/bSq2f8+K6ho50Sqwk
PK68LNZzi5ZXqGEAEQEAAbQnQ3Jvd2RTZWMgc3VwcG9ydCA8c3VwcG9ydEBjcm93
ZHNlYy5uZXQ+iQHUBBMBCgA+FiEEpRXNfWM+DON/Satp2MpQXYwzLTEFAl+VOSUC
GwMFCQPCZwAFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ2MpQXYwzLTEhuwwA
wWdsuSrTC4ryVOYnfHRcT2b/rfbJXIUYXqAy75qsdUGwvueYdYSBMCMXqRB65J+J
juofCF0kTQKuhjtyJezwUfr5C+Sd08JWlZwf9F7CO83/ztLOPIUUp69H3m9heW7C
+A/Lpq3epALytC/QSkDHYnKBBZbLhoR/7WXhdLFvh+A475/ggn4GAOnZMg8WULpR
Kisu1GbEBPcVr1Xl6VTYVX5ghA/1W2WTY/NxAcLhCiJO/ENeka7xy4EKdCE5pDxM
QO/fnpCHsWDIHTxpCx+JAhdkb2BIvzSiF2+o+9y+vwzcPxdGemx7y8MjSGXIp1xp
TJparq309nljh+wqI6w/K+NjzNn/qJL0tpGqiHQXtYDbi86KaAXT9IYCGAIP36w8
XUHYGgo0s6zMEP1NEFHWAgGy5elO403vm+NO5vpHv59FTjgoK2UcjeSjqtAYwzvc
bWQ6wZHwhoqD0WevFcAMmgdbebyOdPoA7+8eCPnkjER4eKxE23ffFU75HDuQNRYk
uQGNBF+VOSUBDADNHEm33IcwhO+uJQxjKtcF0DdAMqbjU5cXxeryo1i7A1WkTH5/
wHfyJAmtLrY4abkQ1LEJ4bMYKdJz2vmvWq0fKCAXC18yLnxU+l0Ld4tWME8hJ/Wh
p+aePsW5BdLpHQeqmQ5MCsw1cZllbURcee22hLJ/PIM2bRsZp7goSj4wXBFjhJyq
EepVmasI17dBbIBFWBSSIJW4UnSBk+Zqbj6C6PDmsket68qcEebsqduWXPxegAzh
IIFD2qhC5t+nn5i+hPwKZN5ZYLQJeAjI4Z7wi3FIBZCzZ214421BbohxPo+GKkFp
mUQ7ZrIa+goHXAcj6ZHMeNNP0lsJRl91lK6NVu3p+Ygl0+wbMOAqDRguMfFdbnV8
gcoYpAyk4YFCfgVQLuKGaYcGjcMP8+nZnPsbaTwbUKkjDAUo+JGmrB4XyAQPugZq
TiUN+lYgTs0cJALEQkKTh2w10TPyV6/YsYDSSnwJeVDIpNCQVg5EB0eRvhaCs9fd
dVni1C5RMcb+Q4MAEQEAAYkBvAQYAQoAJhYhBKUVzX1jPgzjf0mradjKUF2MMy0x
BQJflTklAhsMBQkDwmcAAAoJENjKUF2MMy0xkIcL/johqZbyHskQIaTfQUgASbbu
bdLXSrIkB8Ort9WULxdqs8hveFy6RjXFJWFitFHk46Bj6FJ1ZykfozL+k9uOGrL9
lBk1e3bhqMVhW1o00DufgawNU2FU9NuH/rCuGpum9DE0cc1fFmQ3pjeiHV55GYxr
BGuyyals1ORwK06h+1VFMHrGB12SR7Imgo7FWuexhgLyOK4t1MXg3E4h72qaowpj
5B45qG9jUXgIFKR1D8G8tPeDYLbd37pskNDFozzfAe/H2fqmEjQxMLHrk7J8I3wQ
FPvKIvUF8M3NqZjyaFSiisOn32AS3RAsI8RuD4T2XgpE2L6e29u3RpJkvhPbcAN6
w0W8yw3z1/2uHSvYbwoH1cn4akAikYR9aVVHv86AvNlr0BguqWdzEfiGT6mcJ/hH
2sGQJ1nJRgGpAlx/2HpsLJxhJwLVbXSDSk6Bu2T9G/VIda95niVgq6MfE9GSS+MS
ucVcwqjIXn/9V6+pFZ11soXNKuTk4Wx+uO2r/i5bVA==
=Edl+
-----END PGP PUBLIC KEY BLOCK-----

Re the errors it’s not a huge issue, but the netfilter blocker tries to run ipv6 rules, even though my machine does not have the protocol active:

time="24-10-2020 14:17:56" level=fatal msg="iptables init failed: Error while insert set in iptables (/usr/sbin/ip6tables -I INPUT -m set --match-set crowdsec6-blacklists src -j DROP): exit status 1 --> Could not open socket to kernel: Address family not supported by protocol\n"
time="24-10-2020 14:18:34" level=error msg="error while removing set entry in iptables : exit status 1 --> Could not open socket to kernel: Address family not supported by protocol\n"

Ideally there should be a config to disable the attempt, or detection if IPv6 is active before attempting to insert the rules.

Hey @Swallowtail

Nice spot indeed, I opened an issue here : https://github.com/crowdsecurity/cs-netfilter-blocker/issues/12

Great.

Logs are emailed with pgp.

Hello @Swallowtail

Thanks for the logs, we have created the missing scenarios and we should come back to you tomorrow so you/we can test it!

Hello there,

I just finished this postfix/postscreen scenarios.

You can test this scenario. For now, we are only testing it, so it’s kind of hidden in the hub. You can install it using cscli:

cscli update --branch postfix
cscli install collection crowdsecurity/postfix --branch postfix --force

collection with 2 L :slight_smile:
I will check that :wink:

Did you by any chance get some time to check on this ? We would be very thrilled to have some feedback :slight_smile:

I just checked, the service crashed last week:

time="02-11-2020 05:00:48" level=error msg="API init failed, won't push/pull : api signin: HTTP request creation failed: Post https://tmsov6x2n9.execute-api.eu-west-1.amazonaws.com/v1/signin: dial tcp: lookup tmsov6x2n9.execute-api.eu-west-1.amazonaws.com on 192.168.1.145:53: read udp 192.168.1.235:54434->192.168.1.145:53: i/o timeout"
time="02-11-2020 05:00:48" level=fatal msg="failed to initialize outputs : failed to load api : api signin: HTTP request creation failed: Post https://tmsov6x2n9.execute-api.eu-west-1.amazonaws.com/v1/signin: dial tcp: lookup tmsov6x2n9.execute-api.eu-west-1.amazonaws.com on 192.168.1.145:53: read udp 192.168.1.235:54434->192.168.1.145:53: i/o timeout"

It looks like my DNS server was non-responsive - but the service crashing needs some work…

In terms of the bans, I have fail2ban set more aggressively than crowdsec is running, so I am not sure I am going to see much additional benefit. But I’ll keep running for a while and post some logs back here.

Hello @Swallowtail !

Thanks for the feedback !

Would you mind sharing what kind of changes you made, so that we can offer similar behaviour with crowdsec ?

Thanks !