Cs-firewall-bouncer and nftables

absolem · May 26, 2021, 5:03pm

Hey,

I’m new to crowdsec and just ran into an issue I am stuck right now. Hope you can help me out.

When using cs-firewall-bouncer in nftables mode it cannot ban any IPs.
When I have a look at the log I see a lot of these lines either for the capi provided IPs and for IPs that should be blocked according to my scenarios as well:

time="26-05-2021 18:39:29" level=error msg="unable to insert decision for '20.43.10.201/32': Receive: netlink receive: no such file or directory"
time="26-05-2021 18:39:29" level=error msg="unable to insert decision for '176.218.42.235/32': Receive: netlink receive: no such file or directory"
time="26-05-2021 18:39:29" level=error msg="unable to insert decision for '62.85.37.244/32': Receive: netlink receive: no such file or directory"
time="26-05-2021 18:39:29" level=error msg="unable to insert decision for '40.84.157.160/32': Receive: netlink receive: no such file or directory"
time="26-05-2021 18:39:29" level=error msg="unable to insert decision for '54.78.26.52/32': Receive: netlink receive: no such file or directory"

Here’s my cs-firewall-bouncer.yaml:

mode: nftables
pid_dir: /run/
update_frequency: 10s
daemonize: true
log_mode: file
log_dir: /var/log/
log_level: info
api_url: http://localhost:8080/
api_key: <api_key>
disable_ipv6: true
deny_action: DROP
deny_log: true
#to change log prefix
deny_log_prefix: "[nftables] crowdsec initiated block: "
#if present, insert rule in those chains
#iptables_chains:
#  - INPUT
#  - FORWARD
#  - DOCKER-USER

Does nftables mode interfere with my own ruleset? It seems that it cannot create any of the crowdsec tables…

Does anybody have an idea or hint for me? Would be great.

he2ss · May 28, 2021, 8:39am

Hi @absolem,

Welcome to the crowdsec community

First question, is the bouncer installed in the same machine where crowdsec is already installed ?
If yes, could you check if you have a registered bouncer with this command : cscli bouncers list.

Does nftables mode interfere with my own ruleset? It seems that it cannot create any of the crowdsec tables…

the bouncer create it own table when it start correctly.

absolem · May 28, 2021, 9:10am

Hey he2ss,

the connection to LAPI works just fine. In my post I just replaced the API token

Yesterday I did some more research and it seems that cs-firewall-bouncer isn’t creating the “crowdsec” table and corresponding chain within nftables. (I only use IPv4)
When I set the cs-firewall-bouncer loglevel to debug it tells me:

time="27-05-2021 17:22:51" level=debug msg="nftables: ipv4 table created"

But when I run nft list ruleset no “crowdsec” table shows up (this only shows me my own table).

I found this thread on stackexchange, where someone was facing the exact same error message when the table&chain he tries to insert some rules were missing: https://unix.stackexchange.com/a/537935

Next I tried to create the following structure within my nftables.conf manually, to make sure the table and chain exist:

table ip crowdsec {
    chain crowdsec_chain {
        type filter hook input priority -1; policy accept;
    }
}

I chose priority -1 for it to be evaluated before my own table which has priority 0.

After creating the table manually and verified it’s there via nft list ruleset I restarted cs-firewall-bouncer and all it does, is removing the table according to the shutdown sequence but it never creates a new table…

I really don’t know what I am missing…

edit: Here’s the output I get from cs-firewall-bouncer.log when I restart the service.

time="27-05-2021 18:55:46" level=info msg="removing 'crowdsec' table"
time="27-05-2021 18:55:46" level=fatal msg="shutdown fail: Receive: netlink receive: no such file or directory"
time="27-05-2021 18:55:46" level=info msg="backend type : nftables"
time="27-05-2021 18:55:46" level=info msg="IPV6 is disabled"
time="27-05-2021 18:55:46" level=info msg="nftables initiated"
time="27-05-2021 18:55:46" level=info msg="Processing new and deleted decisions . . ."
time="27-05-2021 18:55:46" level=info msg="deleting '3012' decisions"
time="27-05-2021 18:56:33" level=info msg="adding '820' decisions"
time="27-05-2021 18:56:33" level=error msg="unable to insert decision for '49.88.112.75/32': Receive: netlink receive: no such file or directory"
time="27-05-2021 18:56:33" level=error msg="unable to insert decision for '222.186.30.112/32': Receive: netlink receive: no such file or directory"
time="27-05-2021 18:56:33" level=error msg="unable to insert decision for '222.186.175.23/32': Receive: netlink receive: no such file or directory"
....
time="28-05-2021 02:36:56" level=info msg="adding '1' decisions"
time="28-05-2021 02:36:57" level=error msg="unable to insert decision for '45.146.164.125': Receive: netlink receive: no such file or directory"

In this example you can see the “fatal” error message on shutdown. This is due to the table it expected doesn’t exist…and it cannot delete what doesn’t exist. It also contains one entry where it is trying to ban an IP as part of a decision, so LAPI connection is working.

he2ss · May 28, 2021, 10:03am

Could you please tell us on which OS are you running the bouncer and the bouncer version.
We’ll try to reproduce.

absolem · May 28, 2021, 10:12am

It’s a Debian 10 machine and issuing the command cs-firewall-bouncer --help returns the following version information:

INFO[0000] cs-firewall-bouncer v0.0.12-1c94ce7a62a02843d14d4ddc9e7f144d0244a355

alteredCoder · May 28, 2021, 3:37pm

Hello @absolem,

Can you please uninstall the bouncer, undo the change you did in the /etc/nftables.conf and reinstall the bouncer to have a fresh install?

Then if you can paste the output of the following commands please:

nft list tables
nft list table crowdsec
nft list chains
nft list ruleset

Thanks

absolem · May 28, 2021, 4:28pm

Sure. And thank you for your investigations

Steps I did

Uninstalled it via ./uninstall.sh
removed it in crowdsec via cscli bouncers delete cs-firewall-bouncer-uFlKYrzC
systemctl restart crowdsec
cleared /etc/nftables.conf to only include my rules
systemctl restart nftables
Downloaded cs-firewall-bouncer again from Link and installed it via ./install.sh
leave cs-firewall-bouncer.yaml as it is per default
Same issue as before I reinstalled it (also tried restarting all the services…)

Here’s the output (I replaced some information)

root@<host>:/etc/crowdsec/cs-firewall-bouncer# nft list tables
table inet filter
root@<host>:/etc/crowdsec/cs-firewall-bouncer# nft list chains
table inet filter {
        chain input {
                type filter hook input priority 0; policy drop;
        }
        chain output {
                type filter hook output priority 0; policy drop;
        }
        chain forward {
                type filter hook forward priority 0; policy drop;
        }
}
root@<host>:/etc/crowdsec/cs-firewall-bouncer# nft list ruleset
table inet filter {
        chain input {
                type filter hook input priority 0; policy drop;
                ip protocol icmp icmp type echo-request limit rate over 1/second drop
                ct state established,related accept
                ct state invalid drop
                iif "lo" accept
                ip frag-off & 8191 != 0 counter packets 0 bytes 0 drop
                tcp flags & (fin | syn | rst | ack) != syn ct state new counter packets 0 bytes 0 log prefix "[nftables] dropped NON-SYN packet in new connection: " flags all drop
                tcp flags & (fin | syn | rst | psh | ack | urg) == fin | syn | rst | psh | ack | urg counter packets 0 bytes 0 log prefix "[nftables] dropped XMAS packet: " flags all drop
                tcp flags & (fin | syn | rst | psh | ack | urg) == 0x0 counter packets 0 bytes 0 log prefix "[nftables] dropped NULL packet: " flags all drop
                ip protocol icmp icmp type { echo-reply, destination-unreachable, source-quench, echo-request, time-exceeded } accept
                tcp dport <port> ct state new log prefix "[nftables] inbound SSH connection: " flags all accept
                tcp dport { http, https } ct state new accept
                tcp dport <port> ct state new accept
                ip saddr { <ip>, <ip> } udp sport domain ct state new accept
                log prefix "[nftables] inbound denied: " flags all counter packets 103 bytes 5895 drop
        }

        chain output {
                type filter hook output priority 0; policy drop;
                ct state established,related,new accept
                ct state invalid drop
        }

        chain forward {
                type filter hook forward priority 0; policy drop;
        }
}

I didn’t include the output for nft list table crowdsec since it throws an error, indicating there is no such table.

Here again ist the output I get, when I restart cs-firewall-bouncer service (loglevel debug):

time="28-05-2021 18:21:17" level=info msg="removing 'crowdsec' table"
time="28-05-2021 18:21:17" level=fatal msg="shutdown fail: Receive: netlink receive: no such file or directory"
time="28-05-2021 18:21:18" level=info msg="backend type : nftables"
time="28-05-2021 18:21:18" level=info msg="IPV6 is disabled"
time="28-05-2021 18:21:18" level=debug msg="nftables: ipv4 table created"
time="28-05-2021 18:21:18" level=info msg="nftables initiated"
time="28-05-2021 18:21:18" level=info msg="Processing new and deleted decisions . . ."
time="28-05-2021 18:21:18" level=debug msg="req-api: GET http://localhost:8080/v1/decisions/stream?startup=true"
time="28-05-2021 18:21:18" level=debug msg="resp-api: http 200"
time="28-05-2021 18:21:18" level=info msg="deleting '3834' decisions"
time="28-05-2021 18:21:28" level=debug msg="req-api: GET http://localhost:8080/v1/decisions/stream?startup=false"
time="28-05-2021 18:21:28" level=debug msg="resp-api: http 200"
time="28-05-2021 18:22:31" level=info msg="adding '819' decisions"
time="28-05-2021 18:22:31" level=error msg="unable to insert decision for '185.153.198.173/32': Receive: netlink receive: no such file or directory"
time="28-05-2021 18:22:31" level=error msg="unable to insert decision for '52.255.151.46/32': Receive: netlink receive: no such file or directory"
...

alteredCoder · June 1, 2021, 9:47am

Hello @absolem,

Sorry for the delay.
Thanks for those informations. Indeed, the crowdsec table is not even created.
Can you paste me the output of nft add table ip crowdsec please to see what happen when creating the table ?

absolem · June 1, 2021, 12:34pm

Adding a table manually works as expected.

root@host:/etc/crowdsec# nft add table ip crowdsec
root@host:/etc/crowdsec# nft list ruleset
table ip crowdsec {
}
table inet filter {
        chain input {
              ...
        }

But even when I add the table this way, cs-firewall-bouncer isn’t able to add the rules. I get the exact same error message.

When I then restart cs-firewall-bouncer service it sucessfully removes the table due to the shutdown sequence, but actually it doesn’t add a new one while initializing.

alteredCoder · June 1, 2021, 1:34pm

Hello @absolem,

This is really weird. Can you please come on our Gitter so we can debug this easier ?

Topic		Replies	Views
Ip ranges, cs-firewall-bouncer and nftables	1	579	September 28, 2021
CS-Firewall-Bouncer nftables crowdsec	6	825	March 16, 2022
Firewall bouncer problem	1	233	June 11, 2024
[docker setup] local bouncer on host doesn't ban IPs	3	1070	September 30, 2023
Same IP keeps getting banned, CTI says it's in community blocklist - firewall rule issue? crowdsec	1	16	November 1, 2024

Cs-firewall-bouncer and nftables

Related Topics