Quick how-to for firewall bouncer on Ubiquity EdgeRouter

Hi!

I would like to share my recipe for running a CrowdSec
firewall bouncer on an EdgeOS router – it was tested on a ER-6P and it worked
quite well for the last 4 months on my company’s router.

I’m assuming the following:

  • you have admin access to a Linux VM or PC with Ubuntu or Arch
  • you have basic Linux CLI skills (can open terminal and run commands)
  • you are familiar with SSH and EdgeOS configuration
  • you are running a separate machine with CrowdSec’s LocalAPI
    (running full CrowdSec installation on router should be possible,
    but I didn’t test it)
  • your router is available at yourroutersaddress. Every time you see this word
    in the instructions replace it with your real router’s address
    (for ex. 192.168.1.1)

Things you should run on your PC are prefixed with pc ,
the ones to run on your router router and the ones run on a LocalAPI Server - la.

Anyway, let’s get into business:

  1. You have to install Go toolchain and Git on your Linux PC or VM:

    Debian/Ubuntu:

    pc $ sudo apt install git golang-go

    Arch:

    pc $ sudo pacman -S go git

  2. Then, clone CrowdSec bouncer repository to folder on your PC:

    pc $ git clone https://github.com/crowdsecurity/cs-firewall-bouncer.git

  3. Now you have to establish your router’s architecture.
    For modern Cavium-based routers (ER-4, ER-6P, ER-12, …) it’s mips64.
    For Mediatek-based ones (ER-X, ER-10X, …) it’s mipsle.

  4. Build your CrowdSec bouncer’s binary

    pc $ export GOOS=linux
    pc $ # export GOARCH=<your router's architecture> # for example
    pc $ export GOARCH=mips64
    pc $ make
    
  5. After a short while, the build process should finish and you’ll have a
    bouncer binary in your current folder named crowdsec-firewall-bouncer.

  6. Connect to your LocalAPI server and create a bouncer token:

    la $ cscli bouncers add yourroutersname

    You’ll get a response similar to:

    
    Api key for 'yourroutersname':
    
    aaaaaaaaaabbbbbbbbbbccccccccddff
    
    Please keep this key since you will not be able to retrieve it!
    

    Save the API key, it will be needed on next step.

  7. It’s time to SSH to your router and prepare your bouncers configuration.
    We’ll install everything int the /config folder as it’s the only location
    I know that survives firmware upgrades.

    pc $ ssh yourroutersaddress
    router $ sudo -i
    
    router # mkdir -p /config/user-data/crowdsec/bouncers
    
    router # cat >/config/user-data/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml <<EOF
    mode: iptables
    pid_dir: /var/run/
    update_frequency: 10s
    daemonize: true
    log_mode: file
    log_dir: /var/log/
    log_level: info
    api_url: <your LocalApi's URL>
    api_key: <your LocalApi's bouncer key from step 6.>
    disable_ipv6: false
    deny_action: DROP
    deny_log: false
    iptables_chains:
      - "INPUT"
      - "FORWARD"
    EOF
    
    router # cat > /config/user-data/crowdsec-firewall-bouncer.service <<EOF
    [Unit]
    Description=The firewall bouncer for CrowdSec
    After=syslog.target network.target remote-fs.target nss-lookup.target crowdsec.service
    
    [Service]
    Type=notify   
    ExecStart=/config/user-data/crowdsec-firewall-bouncer -c /config/user-data/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
    ExecStartPost=/bin/sleep 0.1
    
    [Install]
    WantedBy=multi-user.target
    EOF
    
    router # ln -s /config/user-data/crowdsec-firewall-bouncer.service /etc/systemd/system
    
    
  8. Now we need to copy our bouncer’s binary from our PC to EdgeRouter.
    From the git checkout directory on your local pc from step 2. run:

    pc $ scp crowdsec-firewall-bouncer yourroutersaddress:/config/user-data/crowdsec-firewall-bouncer

  9. Now the last step: On your router run

    router # systemctl enable crowdsec-firewall-bouncer
    router # systemctl restart crowdsec-firewall-bouncer
    

    and your router should start blocking malicious traffic in few seconds.

    You can watch logs by running

    router # tail -f /var/log/crowdsec-firewall-bouncer.log

That’s it!

I tried to proof this tutorial, but I’m sure there are at least some errors and omissions. Feel free
to post your corrections into this thread, I’ll try to update this guide accordingly.

Enjoy!

Edit: I removed integration with EdgeOS firewall – it caused a lot of race conditions. Right now the bouncer just inserts two rules to INPUT and FORWARD chains.

2 Likes

That’s great, thank you very much :slight_smile:

Is there any issues you faced or suggested improvements based on your experience ?

thankyou so much! your guide (2nd half) was extremely helpful as i’m trying to do the same on OpenWrt and no instructions are available…

i did not manage to get it running yet (no log is created) so difficult to debug… but I would have been really lost without these simple step by step guide…

I wonder why it’s so hard to find simple guides like this? many seem to just address peices or are oversimplified and miss critical steps…

Wow, nice to hear it was useful to someone :slight_smile: Regarding your problems with CrowdSec bouncer on OpenWRT:

  • mine is writing logs to /var/log/crowdsec-firewall.bouncer.log, check if you have anything there,

  • if there is no log file, check if the bouncer is really runing, by running on your router:

    pgrep cs-firewall-bouncer
    

    if you don’t get a process ID, then it’s not running

  • in that case, run

    cs-firewall-bouncer -t -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
    

    It will check if you have any problems with your configuration.

Best of luck!

2 Likes

Thank you for valuable information Tried this on Edgerouter-x and i am seeing this error

systemctl status crowdsec-firewall-bouncer.service

  • crowdsec-firewall-bouncer.service - The firewall bouncer for CrowdSec
    Loaded: loaded (/config/user-data/crowdsec-firewall-bouncer.service; linked; vendor preset: enabled)
    Active: failed (Result: exit-code) since Wed 2022-01-19 15:22:52 EST; 10min ago
    Process: 5109 ExecStart=/config/user-data/crowdsec-firewall-bouncer -c /config/user-data/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml (code=exited, status=203/EXEC)
    Process: 5104 ExecStartPre=/sbin/ipset create crowdsec6-blacklists hash:net timeout 3600 (code=exited, status=0/SUCCESS)
    Process: 5101 ExecStartPre=/sbin/ipset create crowdsec-blacklists hash:net timeout 3600 (code=exited, status=0/SUCCESS)
    Process: 5098 ExecStartPre=/sbin/ipset destroy crowdsec6-blacklists (code=exited, status=0/SUCCESS)
    Process: 5095 ExecStartPre=/sbin/ipset destroy crowdsec-blacklists (code=exited, status=0/SUCCESS)
    Main PID: 5109 (code=exited, status=203/EXEC)

i have compiled the binary using MIPS …any idea how i can i get this corrected ?

Hi! Please run

/config/user-data/crowdsec-firewall-bouncer -t -c /config/user-data/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

and post results.

root@EdgeRouter~# /config/user-data/crowdsec-firewall-bouncer -t -c /config/user-data/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
-vbash: /config/user-data/crowdsec-firewall-bouncer: cannot execute binary file

also tried using the configure prompt
root@EdgeRouter# /config/user-data/crowdsec-firewall-bouncer -t -c /config/user-data/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
vbash: /config/user-data/crowdsec-firewall-bouncer: cannot execute binary file
[edit]

These are commands i have ran on the linux box
root@homeassistant:/tmp#git clone GitHub - crowdsecurity/cs-firewall-bouncer: Crowdsec bouncer written in golang for firewalls
root@homeassistant:/tmp/cs-firewall-bouncer# export GOARCH=mips
root@homeassistant:/tmp/cs-firewall-bouncer# export GOOS=linux
root@homeassistant:/tmp/cs-firewall-bouncer# make
root@homeassistant:/tmp/cs-firewall-bouncer# scp crowdsec-firewall-bouncer root@10.0.0.1:/config/user-data/crowdsec-firewall-bouncer

Here is my router uname -a Information
root@EdgeRouter# uname -a
Linux EdgeRouter 4.14.54-UBNT #1 SMP Tue May 11 13:23:28 UTC 2021 mips GNU/Linux

It seems the crowdsec-firewall-bouncer binary is build for wrong architecture. Can you post results of running

file crowdsec-firewall-bouncer

on the PC you built the binary?

Mine look more or less like this:

crowdsec-firewall-bouncer: ELF 64-bit MSB executable, MIPS, MIPS-III version 1 (SYSV), statically linked, Go BuildID=C-r7kcaeDcuP6zU20pGF/PnlGdSSuAdo7RLErzG_G/GWGw0pfd9_ujG4Gz9K5t/OdRPzq1q8ScP3VebY19w, stripped

for a MIPS64 build.

file crowdsec-firewall-bouncer
crowdsec-firewall-bouncer: ELF 32-bit MSB executable, MIPS, MIPS32 version 1 (SYSV), statically linked, Go BuildID=AkG2AM_R7ZIpNEYzlbJp/TlmqWEUDxk_Pxyc7pAz1/sP5iD2AL256FGnUHGfb_/KZQYtT_JIvxOJ-ygnfF4, stripped

Hmm, it also seems fine. Can you double check you have the same file on your router?

Post your uname -ar to check the arch.
It must be same than the one with
file crowdsec-firewall-bouncer
You may also checked another binary of the system with file…like @mwegrzynek already suggest.
Also take a look to your binary with
ls -hal crowdsec-firewall-bouncer
to see the status of the eXecute permission. If not already sets, add it with
chmod a+x crowdsec-firewall-bouncer

Thankyou @Gandalf

here is the output of uname -ar

root@EdgeRouter:/config/user-data# ls -hal crowdsec-firewall-bouncer
-rwxr-xr-x 1 admin vyattacf 8.6M Jan 19 14:39 crowdsec-firewall-bouncer
root@EdgeRouter:/config/user-data# uname -ar
Linux EdgeRouter 4.14.54-UBNT #1 SMP Tue May 11 13:23:28 UTC 2021 mips GNU/Linux

Can you simply test crowdsec from command-line ?

$ crowdsec --version

OK, I found a solution. For EdgeRouter-X please use

export GOARCH=mipsle

instead of

export GOARCH=mips

and it should work.

1 Like

Now i am not getting any errors …I will update my results shortly

time="01-03-2022 14:28:20" level=info msg="backend type : ipset"
time="01-03-2022 14:28:20" level=info msg="iptables for ipv4 initiated"
time="01-03-2022 14:28:20" level=info msg="ipset clean-up : /sbin/ipset -exist flush crowdsec-blacklists"
time="01-03-2022 14:28:20" level=info msg="Checking existing set"
time="01-03-2022 14:28:21" level=info msg="iptables for ipv6 initiated"
time="01-03-2022 14:28:21" level=info msg="ipset clean-up : /sbin/ipset -exist flush crowdsec6-blacklists"
time="01-03-2022 14:28:21" level=info msg="Checking existing set"
time="01-03-2022 14:28:22" level=info msg="Processing new and deleted decisions . . ."
time="01-03-2022 14:28:34" level=info msg="Error while inserting in set (/sbin/ipset -exist add crowdsec6-blacklists 2620:96:e000:b0cc:e:2:7:4 timeout 592377): exit status 1 --> ipset v6.30: Syntax error: cannot parse 2620:96:e000:b0cc:e:2:7:4: resolving to IPv4 address failed\n"
time="01-03-2022 14:29:06" level=info msg="Error while inserting in set (/sbin/ipset -exist add crowdsec6-blacklists 2604:a880:4:1d0::2b8:0 timeout 379977): exit status 1 --> ipset v6.30: Syntax error: cannot parse 2604:a880:4:1d0::2b8:0: resolving to IPv4 address failed\n"
time="01-03-2022 14:29:09" level=info msg="Error while inserting in set (/sbin/ipset -exist add crowdsec6-blacklists 2a03:b0c0:2:f0::16c:e001 timeout 135177): exit status 1 --> ipset v6.30: Syntax error: cannot parse 2a03:b0c0:2:f0::16c:e001: resolving to IPv4 address failed\n"
time="01-03-2022 14:31:14" level=info msg="11138 decisions added"

It is finally working … i see IPV6 errors …i can ignore them

I cannot get it to work on an EdgeRouter 12

file crowdsec-firewall-bouncer
ELF 64-bit MSB executable, MIPS, MIPS-III version 1 (SYSV), statically linked, Go BuildID=fFO2yqjI0IwwQh_Bc-4y/o3wYN3TSY2F4foRfaKiw/Qf7TVLRfh3JbkWnwBmMB/PLA48R4IceCkaWPKIPCy, stripped

the service is running but with some errors:

sudo systemctl status crowdsec-firewall-bouncer
* crowdsec-firewall-bouncer.service
   Loaded: loaded (/config/user-data/crowdsec-firewall-bouncer.service; linked; vendor preset: enabled)
   Active: active (running) since Wed 2022-03-09 16:10:26 EET; 2min 29s ago
  Process: 7409 ExecStartPost=/bin/sleep 0.1 (code=exited, status=0/SUCCESS)
  Process: 7387 ExecStartPre=/sbin/ipset create crowdsec6-blacklists hash:net timeout 3600 family inet6 (code=exited, status=0/SUCCESS)
  Process: 7383 ExecStartPre=/sbin/ipset create crowdsec-blacklists hash:net timeout 3600 (code=exited, status=1/FAILURE)
  Process: 7381 ExecStartPre=/sbin/ipset destroy crowdsec6-blacklists (code=exited, status=0/SUCCESS)
  Process: 7378 ExecStartPre=/sbin/ipset destroy crowdsec-blacklists (code=exited, status=1/FAILURE)
 Main PID: 7389 (crowdsec-firewa)
   CGroup: /system.slice/crowdsec-firewall-bouncer.service
           `-7389 /config/user-data/crowdsec-firewall-bouncer -c /config/user-data/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
/config/user-data/crowdsec-firewall-bouncer -t -c /config/user-data/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
INFO[0000] crowdsec-firewall-bouncer v0.0.23-rc3-3-g701f619-701f6199a1ab4de24fe1fc3dbd3254f1514fff02
INFO[0000] config is valid

I get an error:

time="09-03-2022 16:10:29" level=info msg="Error while inserting in set (/sbin/ipset -exist add crowdsec-blacklists 222.255.148.167 timeout 596087): exit status 1 --> ipset v6.30: Timeout cannot be used: set was created without timeout support\n"
time="09-03-2022 16:10:29" level=info msg="Error while inserting in set (/sbin/ipset -exist add crowdsec-blacklists 137.184.77.9 timeout 596087): exit status 1 --> ipset v6.30: Timeout cannot be used: set was created without timeout support\n"
time="09-03-2022 16:10:29" level=info msg="Error while inserting in set (/sbin/ipset -exist add crowdsec-blacklists 106.12.176.108 timeout 596087): exit status 1 --> ipset v6.30: Timeout cannot be used: set was created without timeout support\n"
time="09-03-2022 16:10:29" level=info msg="Error while inserting in set (/sbin/ipset -exist add crowdsec-blacklists 165.232.154.178 timeout 596087): exit status 1 --> ipset v6.30: Timeout cannot be used: set was created without timeout support\n"
time="09-03-2022 16:10:29" level=info msg="Error while inserting in set (/sbin/ipset -exist add crowdsec-blacklists 119.159.234.131 timeout 596088): exit status 1 --> ipset v6.30: Timeout cannot be used: set was created without timeout support\n"
time="09-03-2022 16:10:29" level=info msg="Error while inserting in set (/sbin/ipset -exist add crowdsec-blacklists 46.101.149.216 timeout 596088): exit status 1 --> ipset v6.30: Timeout cannot be used: set was created without timeout support\n"
time="09-03-2022 16:10:29" level=info msg="300 decisions added"

I have noticed that if a firewall address group is created before the service stars, it throws errors an it is unable to add ip addresses to it but it is not created beforehand it loasa with no errors, it is able to add addresses to the list but it is imposible to add the grup to a firewall rule…

I have tried on an ER-X and on ER12, the same errors

Please try updated HOWTO, without integration with EdgeOS firewall configuration – should fix your errors.