Beginner issues, not everything getting blocked

[Fronti113]

Hey all,

first post, first two issues (guess a point in config is missing) and new to Crowdsec, but find the idea awesome.

So far it seems kind of working for me but not everything as expected.

So let me try to explain what is happening.

Im running a VM as Dockerhost (Debian 12).
The Security Agent is running on the host VM.
There are two Portainer/Docker stacks on the host that I also want to monitor/secure with crowdsec.
Immich and paperless-ngx,

As immich is exposed to the web, I want to make sure it is kind of safe.

So, for testing I added two ip adresses manually to the ban. One address is in my LAN, that block works. Not able to access the Dockerhost in any way - perfect
But, when I now try to access the immich webinterface from a different machine via the exposed address, the IP that is also set to ban can access the webserver - not perfect

second is also with the immich access. it seems that even when I try multiple times (fun fact: from the already blocked address), the ban does not getting automatically added to the ban list. would guess it is with the parser, but from what I see I don’t get why…

guess some log and conifg files are helpful that I can get some help with these issues, so here we go

lets start with the blacklist output

so as per this the 147.x address should get blocked, right? at least it works for the 192.x that is also in the list, only not shown on this screenshot
sudo ipset -L crowdsec-blacklists | grep 147.161
147.161.138.110 timeout 10275
164.90.147.161 timeout 427033

parser epxlain output
line: [Nest] 7 - 01/12/2024, 5:00:36 PM WARN [AuthService] Failed login attempt for user test@test.com from ip address 147.161.138.110
├ s00-raw
| ├ crowdsecurity/non-syslog (+5 ~8)
| └ crowdsecurity/syslog-logs
├ s01-parse
| └ gauth-fr/immich-logs (+7 ~3)
├ s02-enrich
| ├ crowdsecurity/dateparse-enrich (+2 ~2)
| ├ crowdsecurity/geoip-enrich (+13)
| └ crowdsecurity/whitelists (unchanged)
├-------- parser success
├ Scenarios
├ gauth-fr/immich-bf
└ gauth-fr/immich-bf_user-enum
the parser also seems to recognize and the log from the immich_server container seems accesable

here log directly from docker log shortend, tried it like 20 times more
[Nest] 7 - 01/12/2024, 5:00:34 PM WARN [AuthService] Failed login attempt for user -testATtest.com from ip address 147.161.138.110
[Nest] 7 - 01/12/2024, 5:00:34 PM WARN [AuthService] Failed login attempt for user -testATtest.com from ip address 147.161.138.110
[Nest] 7 - 01/12/2024, 5:00:34 PM WARN [AuthService] Failed login attempt for user -testATtest.com from ip address 147.161.138.110

for reference the hub list
INFO[12-01-2024 18:58:53] Loaded 111 collecs, 117 parsers, 205 scenarios, 7 post-overflow parsers

COLLECTIONS
────────────────────────────────────────────────────────────────────────────────────────────────
Name Status Version Local Path
────────────────────────────────────────────────────────────────────────────────────────────────
andreasbrett/paperless-ngx enabled 0.1 /etc/crowdsec/collections/paperless-ngx.yml
crowdsecurity/iptables enabled 0.1 /etc/crowdsec/collections/iptables.yaml
crowdsecurity/linux enabled 0.2 /etc/crowdsec/collections/linux.yaml
crowdsecurity/smb enabled 0.1 /etc/crowdsec/collections/smb.yaml
crowdsecurity/sshd enabled 0.2 /etc/crowdsec/collections/sshd.yaml
gauth-fr/immich enabled 0.1 /etc/crowdsec/collections/immich.yml
────────────────────────────────────────────────────────────────────────────────────────────────

PARSERS
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Name Status Version Local Path
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
andreasbrett/paperless-ngx-logs enabled 0.4 /etc/crowdsec/parsers/s01-parse/paperless-ngx-logs.yaml
crowdsecurity/dateparse-enrich enabled 0.2 /etc/crowdsec/parsers/s02-enrich/dateparse-enrich.yaml
crowdsecurity/geoip-enrich enabled 0.2 /etc/crowdsec/parsers/s02-enrich/geoip-enrich.yaml
crowdsecurity/iptables-logs enabled 0.5 /etc/crowdsec/parsers/s01-parse/iptables-logs.yaml
crowdsecurity/smb-logs enabled 0.2 /etc/crowdsec/parsers/s01-parse/smb-logs.yaml
crowdsecurity/sshd-logs enabled 2.2 /etc/crowdsec/parsers/s01-parse/sshd-logs.yaml
crowdsecurity/syslog-logs enabled 0.8 /etc/crowdsec/parsers/s00-raw/syslog-logs.yaml
crowdsecurity/whitelists enabled 0.2 /etc/crowdsec/parsers/s02-enrich/whitelists.yaml
gauth-fr/immich-logs enabled 0.2 /etc/crowdsec/parsers/s01-parse/immich-logs.yaml
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────

SCENARIOS
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Name Status Version Local Path
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
andreasbrett/paperless-ngx-bf enabled 0.3 /etc/crowdsec/scenarios/paperless-ngx-bf.yaml
crowdsecurity/iptables-scan-multi_ports enabled 0.2 /etc/crowdsec/scenarios/iptables-scan-multi_ports.yaml
crowdsecurity/smb-bf enabled 0.2 /etc/crowdsec/scenarios/smb-bf.yaml
crowdsecurity/ssh-bf enabled 0.3 /etc/crowdsec/scenarios/ssh-bf.yaml
crowdsecurity/ssh-slow-bf enabled 0.4 /etc/crowdsec/scenarios/ssh-slow-bf.yaml
gauth-fr/immich-bf enabled 0.2 /etc/crowdsec/scenarios/immich-bf.yaml
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

POSTOVERFLOWS
─────────────────────────────────────────
Name Status Version Local Path
─────────────────────────────────────────

the metrics

and the config.yaml
common:
daemonize: true
log_media: file
log_level: info
log_dir: /var/log/
log_max_size: 20
compress_logs: true
log_max_files: 10
working_dir: .
config_paths:
config_dir: /etc/crowdsec/
data_dir: /var/lib/crowdsec/data/
simulation_path: /etc/crowdsec/simulation.yaml
hub_dir: /etc/crowdsec/hub/
index_path: /etc/crowdsec/hub/.index.json
notification_dir: /etc/crowdsec/notifications/
plugin_dir: /usr/lib/crowdsec/plugins/
crowdsec_service:
#console_context_path: /etc/crowdsec/console/context.yaml
acquisition_path: /etc/crowdsec/acquis.yaml
acquisition_dir: /etc/crowdsec/acquis.d
parser_routines: 1
cscli:
output: human
color: auto
db_config:
log_level: info
type: sqlite
db_path: /var/lib/crowdsec/data/crowdsec.db
use_wal: true
#max_open_conns: 100
#user:
#password:
#db_name:
#host:
#port:
flush:
max_items: 5000
max_age: 7d
plugin_config:
user: nobody # plugin process would be ran on behalf of this user
group: nogroup # plugin process would be ran on behalf of this group
api:
client:
insecure_skip_verify: false
credentials_path: /etc/crowdsec/local_api_credentials.yaml
server:
log_level: info
listen_uri: 127.0.0.1:8080
profiles_path: /etc/crowdsec/profiles.yaml
console_path: /etc/crowdsec/console.yaml
online_client: # Central API credentials (to push signals and receive bad IPs)
credentials_path: /etc/crowdsec/online_api_credentials.yaml
trusted_ips: # IP ranges, or IPs which can have admin API access
- 127.0.0.1
- ::1
#- tls:
#- cert_file: /etc/crowdsec/ssl/cert.pem
#- key_file: /etc/crowdsec/ssl/key.pem
prometheus:
enabled: true
level: full
listen_addr: 127.0.0.1
listen_port: 6060

acquis.yaml
#Generated acquisition file - wizard.sh (service: ssh) / files :
journalctl_filter:

• _SYSTEMD_UNIT=ssh.service
  labels:
  type: syslog

---

#Generated acquisition file - wizard.sh (service: smb) / files :
journalctl_filter:

• _SYSTEMD_UNIT=smb.service
  labels:
  type: smb

---

#Generated acquisition file - wizard.sh (service: linux) / files : /var/log/installer/syslog
filenames:

• /var/log/installer/syslog
  labels:
  type: syslog

---

source: docker
container_name:

• immich_server
  #- container_id:
  #- 227cac8d08ce
  labels:
  type: immmich

---

source: docker
container_name:

• paperlessngx-webserver-1
  container_id:
• 0740dd0c079c
  labels:
  type: paperless-ngx

---

not sure what else could be helpful. LAPI and CAPI status are reporting fine. I can see the manual bans in the decisions on -app.crowdsec.net

So any help is appreciated

Cheers

[iiAmLoz]

Are you exposing the webinterface directly using mapped ports in docker or using a webserver to proxy requests?

What bouncers are you using? I can see firewall within the image

if firewall most likely you havent enabled DOCKER_USER chain within the configuration. Because docker mangles NAT rules to work if you don’t enable the previous stated chain no rules get enforced from the docker network.

EDIT: the immich as well from the parser stats means the type is invalid and looking at the bottom you have an extra m

source: docker
container_name:
  immich_server
labels:
  type: "immmich" ## <- should be immich

Once type is valid the detection part should start working, however, that wont resolve the issue with the firewall bouncer.

[Fronti113]

dumb me removing the third m did the trick, wrote immich so many times didn’t see that.
so it seems crowdsec is now doing what it should, the alerts are now hitting the console and are shown in the metrics, it get also added to the iptables.

ATM I’m running cloudflared on my Dockerhost, so a subdomain is pointing to immich in that case.
working on setting up traefik as reverse proxy.

and you are again right, docker-user was not enabled, unfortunately, as soon as I set docker-user the firewall bouncer fails to start.
from the bouncer log

time="13-01-2024 22:22:47" level=info msg="ipset set-up : /usr/sbin/ipset -exist create crowdsec-blacklists nethash timeout 300 maxelem 131072"
time="13-01-2024 22:22:48" level=info msg="Rule doesn't exist (/usr/sbin/iptables -C INPUT -m set --match-set crowdsec-blacklists src -j DROP)"
time="13-01-2024 22:22:48" level=info msg="Rule doesn't exist (/usr/sbin/iptables -C DOCKER-USER -m set --match-set crowdsec-blacklists src -j DROP)"
time="13-01-2024 22:22:48" level=info msg="iptables set-up : /usr/sbin/iptables -I INPUT -m set --match-set crowdsec-blacklists src -j DROP"
time="13-01-2024 22:22:48" level=info msg="iptables set-up : /usr/sbin/iptables -I DOCKER-USER -m set --match-set crowdsec-blacklists src -j DROP"
time="13-01-2024 22:22:48" level=info msg="iptables for ipv6 initiated"
time="13-01-2024 22:22:48" level=info msg="iptables clean-up : /usr/sbin/ip6tables -D INPUT -m set --match-set crowdsec6-blacklists src -j DROP"
time="13-01-2024 22:22:48" level=info msg="iptables clean-up : /usr/sbin/ip6tables -D DOCKER-USER -m set --match-set crowdsec6-blacklists src -j DROP"
time="13-01-2024 22:22:48" level=error msg="error while removing set entry in iptables : exit status 1 --> ip6tables: Bad rule (does a matching rule exist in that chain?).\n"
time="13-01-2024 22:22:48" level=info msg="ipset clean-up : /usr/sbin/ipset -exist destroy crowdsec6-blacklists"
time="13-01-2024 22:22:48" level=error msg="set destroy error : exit status 1 - ipset v7.17: Set cannot be destroyed: it is in use by a kernel component\n"
time="13-01-2024 22:22:48" level=info msg="Checking existing set"
time="13-01-2024 22:22:49" level=warning msg="iptables check command (/usr/sbin/ip6tables -C INPUT -m set --match-set crowdsec6-blacklists src -j DROP) failed : exit status 1"
time="13-01-2024 22:22:49" level=warning msg="iptables check command (/usr/sbin/ip6tables -C DOCKER-USER -m set --match-set crowdsec6-blacklists src -j DROP) failed : exit status 1"
time="13-01-2024 22:22:49" level=info msg="iptables set-up : /usr/sbin/ip6tables -I INPUT -m set --match-set crowdsec6-blacklists src -j DROP"
time="13-01-2024 22:22:49" level=info msg="iptables set-up : /usr/sbin/ip6tables -I DOCKER-USER -m set --match-set crowdsec6-blacklists src -j DROP"
time="13-01-2024 22:22:49" level=warning msg="Error inserting set in iptables (/usr/sbin/ip6tables -I DOCKER-USER -m set --match-set crowdsec6-blacklists src -j DROP): exit status 1 : ip6tables: No chain/targ>time="13-01-2024 22:22:49" level=fatal msg="iptables init failed: while inserting set in iptables: exit status 1"

seems something wrong with the iptables, right?

chain is present

hain INPUT (policy ACCEPT)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere             match-set crowdsec6-blacklists src

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

thanks so far!

seems crowdstrike was or is not the issue. Anything with the system. But if you have another hint, I’m happy

[iiAmLoz]

Most likely you dont have ipv6 enabled the docker network so if fails to create the chain

time="13-01-2024 22:22:49" level=warning msg="Error inserting set in iptables (/usr/sbin/ip6tables -I DOCKER-USER -m set --match-set crowdsec6-blacklists src -j DROP)

There is an option within the bouncer config to turn off ipv6 if that okay with you

[Fronti113]

Awesome that did the trick as well.

But still not blocking the external IP address even I can see the docker user chain using the blacklist.

Seems I need to play a bit with the chains.

Thanks so much !