Practically begging for help here! I have been dealing with something that has been killing me for over a month now. I simply cannot get Crowdsec to work within my Kubernetes cluster. Which is where I usually deploy all my network infrastructure like Traefik for reverse proxying. It seems everything works okay, but when I apply the Crowdsec-Traefik plugin middleware to an Ingress route. It simply does not work whatsoever. I 100% have set up the Crowdsec-Traefik plugin correctly, with the correct LAPI key and everything. The Traefik dashboard also reports the middleware is recognized within Traefik and the cluster.
I will drop some info on my cluster for context: I am running a six node cluster with metallb as a load balancer for services. Everything has worked great so far. I run an HA 3 pod Traefik deployment within it. I primarily deploy everything with Helm charts.
I will drop some configuration down below:
Crowdsec’s values.yaml:
container_runtime: containerd
# Here you can specify your own custom configuration to be loaded in crowdsec agent or lapi
# Each config needs to be a multi-line using '|' in YAML specs
# for the agent those configs will be loaded : parsers, scenarios, postoverflows, simulation.yaml
# for the lapi those configs will be loaded : profiles.yaml, notifications, console.yaml
tls:
enabled: true
bouncer:
reflector:
namespaces: ["traefik"]
agent:
# Specify each pod whose logs you want to process
persistentVolume:
config:
enabled: false
data:
enabled: true
storageClassName: "longhorn"
acquisition:
# The namespace where the pod is located
- namespace: traefik
# The pod name
podName: traefik-*
# as in crowdsec configuration, we need to specify the program name to find a matching parser
program: traefik
env:
- name: PARSERS
value: "crowdsecurity/cri-logs crowdsecurity/whitelists crowdsecurity/nextcloud-whitelist"
- name: COLLECTIONS
value: "crowdsecurity/linux crowdsecurity/k8s-audit crowdsecurity/apache2 crowdsecurity/traefik crowdsecurity/home-assistant Dominic-Wagner/vaultwarden timokoessler/uptime-kuma firix/authentik LePresidente/jellyseerr LePresidente/jellyfin LePresidente/adguardhome crowdsecurity/nextcloud gauth-fr/immich"
# When testing, allow bans on private networks
#- name: DISABLE_PARSERS
# value: "crowdsecurity/whitelists"
image:
pullPolicy: Always
lapi:
dashboard:
enabled: false
ingress:
host: dashboard.local
enabled: false
persistentVolume:
config:
enabled: false
data:
enabled: true
storageClassName: "longhorn"
resources:
limits:
memory: 200Mi
requests:
cpu: 250m
memory: 200Mi
env:
# For an internal test, disable the Online API by setting 'DISABLE_ONLINE_API' to "true"
- name: DISABLE_ONLINE_API
value: "false"
- name: ENROLL_KEY
value: "placeholder-correct-key"
- name: ENROLL_INSTANCE_NAME
value: "k3s"
- name: ENROLL_TAGS
value: "homelab"
image:
pullPolicy: Always
Here is my Traefik’s values.yaml:
globalArguments:
- "--global.sendanonymoususage=false"
- "--global.checknewversion=false"
additionalArguments:
- "--serversTransport.insecureSkipVerify=true"
- "--log.level=INFO"
- "--providers.kubernetesingress.namespaces="
- "--providers.kubernetescrd.namespaces="
deployment:
enabled: true
replicas: 3
annotations: {}
podAnnotations: {}
additionalContainers: []
initContainers: []
ports:
web:
redirectTo:
port: websecure
websecure:
tls:
enabled: true
ingressRoute:
dashboard:
enabled: false
providers:
kubernetesCRD:
enabled: true
ingressClass: traefik-external
allowExternalNameServices: true
kubernetesIngress:
enabled: true
allowExternalNameServices: true
allowCrossNamespace: true
publishedService:
enabled: false
rbac:
enabled: true
service:
enabled: true
type: LoadBalancer
annotations: {}
labels: {}
spec:
loadBalancerIP: 192.168.30.150 # this should be an IP in the MetalLB range
externalTrafficPolicy: Local # for crowdsec
loadBalancerSourceRanges: []
externalIPs: []
logs:
access:
enabled: true
experimental:
plugins:
crowdsec-bouncer-traefik-plugin:
moduleName: "github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
version: "v1.3.0-dev2"
volumes:
- name: my-crowdsec-bouncer-tls
mountPath: /etc/traefik/certs/
type: secret
image:
pullPolicy: Always
Now for the middleware:
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: my-crowdsec-bouncer-traefik-plugin
namespace: default
spec:
plugin:
crowdsec-bouncer-traefik-plugin:
CrowdsecLapiKey: place-holder-correct-key
Enabled: "true"
Now, here is an Ingress route format I’ve used with no avail:
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: nextcloud-external
namespace: default
annotations:
kubernetes.io/ingress.class: traefik-external
spec:
entryPoints:
- websecure
routes:
- match: Host(`www.nextcloud.xxx.xyz`)
kind: Rule
services:
- name: nextcloud-external
port: 80
middlewares:
- name: my-crowdsec-bouncer-traefik-plugin
- match: Host(`nextcloud.xxx.xyz`)
kind: Rule
services:
- name: nextcloud-external
port: 80
tls:
secretName: xxx-xyz-tls
I have also experimented with specifying the middleware on the "Host(nextcloud.xxx.xyz) portion such as this:
match: Host(`nextcloud.xxx.xyz`)
kind: Rule
services:
- name: nextcloud-external
port: 80
middlewares:
- name: my-crowdsec-bouncer-traefik-plugin
When I specify the middleware here, I simply get a blank webpage that does not work at all. When I specify the middleware at the Host(www.nextcloud.xxx.xyz) I can access the webpage, but it’s clear crowdsec will not work as I create a cscli decision to ban an IP I can VPN into, and the ban goes into place, yet when I VPN to the IP address, I can still access the webpage no problem.
I have troubleshooted this to no end and I have simply gotten nowhere.