Checking bans really work and how to extend bans?

crowdsec
1

Here is my output from cscli metrics

> cscli metrics
INFO[10-12-2021 11:19:57 PM] Buckets Metrics:
+-------------------------------------------+---------------+-----------+--------------+--------+---------+
|                  BUCKET                   | CURRENT COUNT | OVERFLOWS | INSTANCIATED | POURED | EXPIRED |
+-------------------------------------------+---------------+-----------+--------------+--------+---------+
| crowdsecurity/http-bad-user-agent         | -             |         1 |            1 |      2 | -       |
| crowdsecurity/http-crawl-non_statics      | -             | -         |           20 |     21 |      20 |
| crowdsecurity/http-path-traversal-probing | -             | -         |            1 |      1 |       1 |
| crowdsecurity/http-probing                | -             | -         |           12 |     13 |      12 |
+-------------------------------------------+---------------+-----------+--------------+--------+---------+
INFO[10-12-2021 11:19:57 PM] Acquisition Metrics:
+----------------------------------------+------------+--------------+----------------+------------------------+
|                 SOURCE                 | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+----------------------------------------+------------+--------------+----------------+------------------------+
| file:/var/log/auth.log                 |         20 | -            |             20 | -                      |
| file:/var/log/kern.log                 |       1712 | -            |           1712 | -                      |
| file:/var/log/nginx/access.log         |         30 |           30 | -              |                     27 |
| file:/var/log/nginx/XXXX_access.log    |        678 |          678 | -              |                      9 |
| file:/var/log/nginx/XXXX_access.log    |         12 |           12 | -              | -                      |
| file:/var/log/nginx/XXXXXXX_access.log |        162 |          162 | -              | -                      |
| file:/var/log/nginx/XXXXX_access.log   |          8 |            8 | -              |                      1 |
| file:/var/log/nginx/XXXXX_errors.log   |          4 |            4 | -              | -                      |
| file:/var/log/syslog                   |       1826 | -            |           1826 | -                      |
+----------------------------------------+------------+--------------+----------------+------------------------+
INFO[10-12-2021 11:19:57 PM] Parser Metrics:
+--------------------------------+------+--------+----------+
|            PARSERS             | HITS | PARSED | UNPARSED |
+--------------------------------+------+--------+----------+
| child-crowdsecurity/http-logs  | 2682 |   1968 |      714 |
| child-crowdsecurity/nginx-logs |  898 |    894 |        4 |
| crowdsecurity/dateparse-enrich |  894 |    894 | -        |
| crowdsecurity/geoip-enrich     |  894 |    894 | -        |
| crowdsecurity/http-logs        |  894 |    850 |       44 |
| crowdsecurity/nginx-logs       |  894 |    894 | -        |
| crowdsecurity/non-syslog       |  894 |    894 | -        |
| crowdsecurity/syslog-logs      | 3558 |   3558 | -        |
| crowdsecurity/whitelists       | 1788 |   1788 | -        |
+--------------------------------+------+--------+----------+
INFO[10-12-2021 11:19:57 PM] Local Api Metrics:
+----------------------+--------+------+
|        ROUTE         | METHOD | HITS |
+----------------------+--------+------+
| /v1/alerts           | GET    |    2 |
| /v1/alerts           | POST   |    1 |
| /v1/decisions/stream | GET    | 1670 |
| /v1/watchers/login   | POST   |    9 |
+----------------------+--------+------+
INFO[10-12-2021 11:19:57 PM] Local Api Machines Metrics:
+--------------------------------------------------+------------+--------+------+
|                     MACHINE                      |   ROUTE    | METHOD | HITS |
+--------------------------------------------------+------------+--------+------+
| xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | /v1/alerts | GET    |    2 |
| xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx | /v1/alerts | POST   |    1 |
+--------------------------------------------------+------------+--------+------+
INFO[10-12-2021 11:19:57 PM] Local Api Bouncers Metrics:
+----------------------------+----------------------+--------+------+
|          BOUNCER           |        ROUTE         | METHOD | HITS |
+----------------------------+----------------------+--------+------+
| FirewallBouncer-1639155106 | /v1/decisions/stream | GET    | 1670 |
+----------------------------+----------------------+--------+------+

I have two questions about this output:

  • How do I know bans are working?
  • Why are some lines marked as UNPARSED?

Iā€™d also like to extend bans because they seem to last about 3 hours by default which isnā€™t nearly enough for me (Iā€™d like a few months at least), how can I do this?

Iā€™ve just started using this tool but I already like it a lot, thank you for developing it!

Thanks in advance for your answers!

2

I found how to extend ban duration in this post Default Ban time duration , however Iā€™m wondering where I can find a list of units for this setting (for example is ā€œmā€ minutes or months?).

3

Hi
I canā€™t help on all your questions but Iā€™ll answer those I can.

First of all: Why do you want to ban ips for so long? By design we donā€™t ban ips globally for more than 72 hrs because the internet and the world of hackers stealing accesses to servers is so volatile that a lot can happen in those 72 hrs that makes it impractical to ban for longer. What if hackers took over an ip, started using it to abuse, it got banned, the original owner took back the server and ip? In that case we donā€™t think it makes sense to punish anyone with an extended ban. If, however, the ip is still bad after 72 hrs it will be refreshed in our database and still be banned.

In terms of units, Iā€™ll direct you to our docs. This is the only place itā€™s mentioned so I would assume the units are the same globally in CrowdSec. For reasons mentioned above we donā€™t need units longer than a day.

In terms of how you know if your ban is working, it will depend on your bouncer. Some will log in a separate log file (e.g. the firewall bouncer) - some will log via an associated application like nginx that will log blockings from the nginx bouncer or wordpress or generic php if your website runs on that.

I hope that answered all the questions I tried to answer. If not, feel free to write again.

4

My server has a few open directories that donā€™t allow indexes and Iā€™m only using it to distribute files to friends so a large majority of the hits it gets are either indexing scans or malicious scans, I do not care if an IP is banned forever as the chances a friendā€™s IP is banned is extremely low.

I donā€™t really need to use Crowdsec but Iā€™m happy to contribute to the database nonetheless and I understand having longer ban durations will likely reduce the amount I contribute (because a banned IP wonā€™t be able to hit me again during their ban time).

Thank you for the link to the docs about units, the units are good enough for me.

I checked my bouncerā€™s logs and found this kind of lines in it so I guess itā€™s working:

time=ā€œ10-12-2021 18:44:05ā€ level=info msg=ā€œProcessing new and deleted decisions . . .ā€
time=ā€œ10-12-2021 18:44:05ā€ level=info msg=ā€œdeleting ā€˜1ā€™ decisionsā€
time=ā€œ10-12-2021 18:44:05ā€ level=info msg=ā€œadding ā€˜1982ā€™ decisionsā€
time=ā€œ10-12-2021 19:42:45ā€ level=info msg=ā€œdeleting ā€˜1ā€™ decisionsā€
time=ā€œ10-12-2021 20:42:45ā€ level=info msg=ā€œdeleting ā€˜3ā€™ decisionsā€
time=ā€œ10-12-2021 20:52:25ā€ level=info msg=ā€œadding ā€˜1985ā€™ decisionsā€
time=ā€œ10-12-2021 21:42:25ā€ level=info msg=ā€œdeleting ā€˜1ā€™ decisionsā€
time=ā€œ10-12-2021 21:42:45ā€ level=info msg=ā€œdeleting ā€˜4ā€™ decisionsā€
time=ā€œ10-12-2021 22:10:35ā€ level=info msg=ā€œadding ā€˜1ā€™ decisionsā€
time=ā€œ10-12-2021 22:52:25ā€ level=info msg=ā€œdeleting ā€˜6ā€™ decisionsā€
time=ā€œ10-12-2021 22:52:25ā€ level=info msg=ā€œadding ā€˜1982ā€™ decisionsā€

Whatā€™s strange is that even though my bouncer is ā€œFirewallBouncerā€ I canā€™t find any iptables rules and looking at the logs it seems I have an issue with iptables.
Iā€™m using iptables-save |less to make sure I list all the rules but I still canā€™t find any rule matching the active bans.

I also found a few errors in the FirewallBouncer logs:

time="10-12-2021 18:41:28" level=error msg="Get \"http://localhost:8085/v1/decisions/stream?startup=false\": dial tcp [::1]:8085: connect: connection refused"
time="10-12-2021 18:41:38" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp [::1]:8085: connect: connection refused"

and

time="10-12-2021 18:44:04" level=info msg="Checking existing set"
time="10-12-2021 18:44:04" level=info msg="ipset set-up : /sbin/ipset -exist create crowdsec6-blacklists nethash timeout 300 family inet6"
time="10-12-2021 18:44:05" level=warning msg="iptables check command (/usr/sbin/ip6tables -C INPUT -m set --match-set crowdsec6-blacklists src -j DROP) failed : exit status 1"
time="10-12-2021 18:44:05" level=info msg="iptables set-up : /usr/sbin/ip6tables -I INPUT -m set --match-set crowdsec6-blacklists src -j DROP"
time="10-12-2021 18:44:05" level=info msg="Processing new and deleted decisions . . ."

For the first error I just changed the value of api_url: from http://localhost:8085/ to http://127.0.0.1:8085/ to force it to use ip v4 as crowdsec is only listening on v4 on my setup so this should be solved now.
Could this have been the reason I canā€™t find any iptables rules matching the bans?

PS: Forcing the IP v4 address fixed this first issue above.
Iā€™ve also disabled IP v6 in crowdsec-firewall-bouncer.yaml because my server only has a public v4 one.

For the second one Iā€™m not sure what the issue is but Iā€™ll try to debug this properly.

PS: As I disabled IP v6 in the config this exact error disappeared but Iā€™m still getting some kind of warning:

time="11-12-2021 21:00:06" level=info msg="iptables clean-up : /usr/sbin/iptables -D INPUT -m set --match-set crowdsec-blacklists src -j DROP"
time="11-12-2021 21:00:06" level=info msg="ipset clean-up : /sbin/ipset -exist destroy crowdsec-blacklists"
time="11-12-2021 21:00:06" level=info msg="Shutting down firewall-bouncer service"
time="11-12-2021 21:00:06" level=info msg="config is valid"
time="11-12-2021 21:00:06" level=info msg="backend type : iptables"
time="11-12-2021 21:00:06" level=info msg="IPV6 is disabled"
time="11-12-2021 21:00:06" level=info msg="iptables for ipv4 initiated"
time="11-12-2021 21:00:06" level=info msg="iptables clean-up : /usr/sbin/iptables -D INPUT -m set --match-set crowdsec-blacklists src -j DROP"
time="11-12-2021 21:00:06" level=info msg="ipset 'crowdsec-blacklists' doesn't exist, skip"
time="11-12-2021 21:00:06" level=info msg="ipset clean-up : /sbin/ipset -exist destroy crowdsec-blacklists"
time="11-12-2021 21:00:06" level=info msg="ipset 'crowdsec-blacklists' doesn't exist, skip"
time="11-12-2021 21:00:06" level=info msg="Checking existing set"
time="11-12-2021 21:00:06" level=info msg="ipset set-up : /sbin/ipset -exist create crowdsec-blacklists nethash timeout 300"
time="11-12-2021 21:00:07" level=info msg="Rule doesn't exist (/usr/sbin/iptables -C INPUT -m set --match-set crowdsec-blacklists src -j DROP)"
time="11-12-2021 21:00:07" level=info msg="iptables set-up : /usr/sbin/iptables -I INPUT -m set --match-set crowdsec-blacklists src -j DROP"
time="11-12-2021 21:00:07" level=info msg="Processing new and deleted decisions . . ."
time="11-12-2021 21:00:07" level=info msg="deleting '71' decisions"
5

@thibault or @blotus could one of you help out here? :slight_smile:

6

If you are talking about:
Rule doesnā€™t exist (/usr/sbin/iptables -C INPUT -m set --match-set crowdsec-blacklists src -j DROP)
then you can see that the rule is configured on the next step.
You can check if the rule is there by list iptable rules.
Just execute: iptables-save | grep crowdsec-blacklists
and you should see:
-A INPUT -m set --match-set crowdsec-blacklists src -j LOG --log-prefix "crowdsec drop: "
-A INPUT -m set --match-set crowdsec-blacklists src -j DROP

The LOG rule will be added if you set deny_log: true
/etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

P.S.: Donā€™t forget to restart the bouncer service after the modification

Cheers,
-Tiho

7

Alright, itā€™s working then but I thought I would have been able to see the rules for each banned IP in iptables-save (Iā€™m rusty with iptables).
Iā€™ve also enabled the deny log so it should be more clear from now on.

The last remaining thing I was wondering about is the UNPARSED lines, if they are due to parsing errors is there a way to get more details about what lines failed exactly so that I can try to fix the issue?

Thank you for your answers!

8

Hello @discword ,

Having unparsed lines is ok since crowdsec only care about important line. For example for SSH, crowdsec will only parse logs for failed authentication attempt.

1 Like
9

I find it rather strange to call them unparsed because I thought there was an issue somewhere but if itā€™s just ignoring these lines because they are not useful itā€™s fine.

And just a note about my previous comment above, enabling the deny log makes it way more clear whatā€™s happening with iptable as I can now see the drops ini syslog.

1 Like