Hi,
I have a k8s 1.28 + traefik 3.3.4 which run great so far. I started to add security engine w/o problem (seen in web console). Things got harder after adding the bouncer plugin and associated middleware: every single request get 403 response.
Traefik is deployed as Daemonset, here the relevant params for traefik:
- --log.level=INFO
- --log.maxsize=2048
- --log.maxage=7
- --core.defaultrulesyntax=v2
- --accesslog
- --entrypoints.web.address=:80
- --entrypoints.web.http.middlewares=default-bouncer@kubernetescrd
- --entrypoints.websecure.address=:443
- --entrypoints.websecure.http.middlewares=default-bouncer@kubernetescrd
- --api=true
- --api.dashboard=true
- --providers.kubernetescrd
- --providers.kubernetesIngress=true
- --providers.kubernetescrd.throttleDuration=10s
- --providers.kubernetescrd.allowCrossNamespace=true
- --experimental.plugins.bouncer.modulename=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
- --experimental.plugins.bouncer.version=v1.4.1
Traefik logs show the plugin got correctly loaded:
2025-03-24T09:25:10Z WRN v2 rules syntax is now deprecated, please use v3 instead...
2025-03-24T09:25:10Z INF Traefik version 3.3.4 built on 2025-02-25T10:11:01Z version=3.3.4
2025-03-24T09:25:10Z INF
Stats collection is disabled.
Help us improve Traefik by turning this feature on :)
More details on: https://doc.traefik.io/traefik/contributing/data-collection/
2025-03-24T09:25:10Z INF Enabling ProxyProtocol for trusted IPs [173.245.48.0/20 103.21.244.0/22 103.22.200.0/22 103.31.4.0/22 141.101.64.0/18 108.162.192.0/18 190.93.240.0/20 188.114.96.0/20 197.234.240.0/22 198.41.128.0/17 162.158.0.0/15 104.16.0.0/13 104.24.0.0/14 172.64.0.0/13 131.0.72.0/22 2400:cb00::/32 2606:4700::/32 2803:f800::/32 2405:b500::/32 2405:8100::/32 2a06:98c0::/29 2c0f:f248::/32] entryPointName=websecure
2025-03-24T09:25:10Z INF Loading plugins... plugins=["bouncer","cloudflarewarp"]
2025-03-24T09:25:20Z ERR Request failed error={"Err":{},"Op":"Get","URL":"https://plugins.traefik.io/public/download/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/v1.4.1"} method=GET url=https://plugins.traefik.io/public/download/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/v1.4.1
2025-03-24T09:25:21Z INF Plugins loaded. plugins=["bouncer","cloudflarewarp"]
2025-03-24T09:25:21Z INF Starting provider aggregator *aggregator.ProviderAggregator
2025-03-24T09:25:21Z INF Starting provider *file.Provider
2025-03-24T09:25:21Z INF Starting provider *traefik.Provider
2025-03-24T09:25:21Z INF Starting provider *acme.ChallengeTLSALPN
2025-03-24T09:25:21Z INF Starting provider *ingress.Provider
2025-03-24T09:25:21Z INF Starting provider *crd.Provider
2025-03-24T09:25:21Z INF Starting provider *acme.Provider
2025-03-24T09:25:21Z INF ingress label selector is: "" providerName=kubernetes
2025-03-24T09:25:21Z INF Creating in-cluster Provider client providerName=kubernetes
2025-03-24T09:25:21Z INF Testing certificate renew... acmeCA=https://acme-v02.api.letsencrypt.org/directory providerName=leweb.acme
2025-03-24T09:25:21Z INF label selector is: "" providerName=kubernetescrd
2025-03-24T09:25:21Z INF Creating in-cluster Provider client providerName=kubernetescrd
2025-03-24T09:25:21Z WRN Cross-namespace reference between IngressRoutes and resources is enabled, please ensure that this is expected (see AllowCrossNamespace option) providerName=kubernetescrd
When connecting to LAPI, CAPI and LAPI are ok:
crowdsec-lapi-77c57dcfdb-57zfp:/# cscli lapi status
Loaded credentials from /etc/crowdsec/local_api_credentials.yaml
Trying to authenticate with username crowdsec-lapi-77c57dcfdb-57zfp on http://localhost:8080/
You can successfully interact with Local API (LAPI)
crowdsec-lapi-77c57dcfdb-57zfp:/# cscli capi status
Loaded credentials from /etc/crowdsec//online_api_credentials.yaml
Trying to authenticate with username ca47d73a9c4e465ab1326d376cc79fddFkBI9ktT1kYWDgE9 on https://api.crowdsec.net/
You can successfully interact with Central API (CAPI)
Your instance is enrolled in the console
Sharing signals is enabled
Pulling community blocklist is enabled
Pulling blocklists from the console is enabled
But I see no acquisition:
crowdsec-lapi-77c57dcfdb-57zfp:/# cscli metrics show acquisition
╭──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Acquisition Metrics │
├────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────┤
│ Source │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
╰────────┴────────────┴──────────────┴────────────────┴────────────────────────┴───────────────────╯
When getting logs from agent, I see each agent try to get the traefik container log on the host node:
time="2025-03-24T13:11:21Z" level=info msg="loading acquisition file : /etc/crowdsec/acquis.yaml"
time="2025-03-24T13:11:21Z" level=info msg="Force add watch on /var/log/containers" type=file
time="2025-03-24T13:11:21Z" level=info msg="Adding file /var/log/containers/traefik-ingress-controller-qz5nt_default_traefik-ingress-lb-02c97c963e6fb93cd46b9fbe49322c3bc34279ecd2c3fcbe6fc4a9e24ef0db97.log to datasources" type=file
time="2025-03-24T13:11:21Z" level=warning msg="No matching files for pattern /var/log/traefik/access.log" type=file
time="2025-03-24T13:11:21Z" level=info msg="Starting processing data"
time="2025-03-24T13:11:21Z" level=error msg="unable to read /var/log/containers/traefik-ingress-controller-qz5nt_default_traefik-ingress-lb-02c97c963e6fb93cd46b9fbe49322c3bc34279ecd2c3fcbe6fc4a9e24ef0db97.log : open /var/log/containers/traefik-ingress-controller-qz5nt_default_traefik-ingress-lb-02c97c963e6fb93cd46b9fbe49322c3bc34279ecd2c3fcbe6fc4a9e24ef0db97.log: no such file or directory" type=file
On the host, I confirm the presence of the file.
How can I configure the agents so they can access le traefik logs?
Thank you