Problems with Keycloak collection

crowdsec
1

I have installed my first Crowdsec instance and have installed a collection for Keycloak.
But it does not ban any IPs.

Keycloak is running in a Docker container.

Collection:
inherent-io/keycloak

Acquiss:

#Generated acquisition file - wizard.sh (service: nginx) / files :
journalctl_filter:
 - _SYSTEMD_UNIT=nginx.service
labels:
  type: nginx
---
#Generated acquisition file - wizard.sh (service: ssh) / files :
journalctl_filter:
 - _SYSTEMD_UNIT=ssh.service
labels:
  type: syslog
---
source: docker
container_name:
 - keycloak
#container_id:
# - 6ae66a948965
labels:
  type: keycloak

Metrics:

╭─────────────────────────────────┬──────┬────────┬──────────╮
│             Parsers             │ Hits │ Parsed │ Unparsed │
├─────────────────────────────────┼──────┼────────┼──────────┤
│ child-inherent-io/keycloak-logs │ 18   │ -      │ 18       │
│ crowdsecurity/non-syslog        │ 18   │ 18     │ -        │
│ inherent-io/keycloak-logs       │ 18   │ -      │ 18       │
╰─────────────────────────────────┴──────┴────────┴───────

Docker logs:

2024-05-06 20:39:13,513 WARN  [org.keycloak.events] (executor-thread-1) type=LOGIN_ERROR, realmId=7fb34d1f-ea30-4f13-ab20-f6910f191f59, clientId=security-admin-console, userId=null, ipAddress=91.xx.xx.xx, error=user_not_found, auth_method=openid-connect, auth_type=code, redirect_uri=https://keycloak.xxxxx.xxxx/admin/master/console/, code_id=8a8cd4b2-5236-435e-80ef-14e672d6f895, username=dsfsd@sdfsdf.de
2024-05-06 20:40:15,094 WARN  [org.keycloak.events] (executor-thread-1) type=LOGIN_ERROR, realmId=7fb34d1f-ea30-4f13-ab20-f6910f191f59, clientId=security-admin-console, userId=null, ipAddress=xx.xx.xx.xx, error=user_not_found, auth_method=openid-connect, auth_type=code, redirect_uri=https://keycloak.xxxx.xxx/admin/master/console/, code_id=8a8cd4b2-5236-435e-80ef-14e672d6f895, username=sadas
2024-05-06 20:40:16,353 WARN  [org.keycloak.events] (executor-thread-1) type=LOGIN_ERROR, realmId=7fb34d1f-ea30-4f13-ab20-f6910f191f59, clientId=security-admin-console, userId=null, ipAddress=xx.xx.xx.xx, error=user_not_found, auth_method=openid-connect, auth_type=code, redirect_uri=https://keycloak.xxxx.xxxx/admin/master/console/, code_id=8a8cd4b2-5236-435e-80ef-14e672d6f895, username=sadas
2024-05-06 20:40:19,124 WARN  [org.keycloak.events] (executor-thread-1) type=LOGIN_ERROR, realmId=7fb34d1f-ea30-4f13-ab20-f6910f191f59, clientId=security-admin-console, userId=null, ipAddress=xx.xx.xx.xx, error=user_not_found, auth_method=openid-connect, auth_type=code, redirect_uri=https://keycloak.xxxxx.xxxxx/admin/master/console/, code_id=8a8cd4b2-5236-435e-80ef-14e672d6f895, username=sadas

i don’t have the experience with crowdsec to find the error. Can anyone give me a tip?

Greetings
Hank75

(Sorry for bad English)

2

Most likely the issue is at the end of the pattern we expect these data points

authSessionParentId=%{GREEDYDATA:authSessionParentId}, authSessionTabId=%{GREEDYDATA:authSessionTabId}

Making these optional makes the parser parse again

image
image2269×554 114 KB

(ignore the log level parsing I didnt have the datetime parsers loaded)

I can work on pushing these changes

3
2 Likes
4

Thank you very much for your work. What do I have to change now so that I can import the changes into my system?

5

You can just manually download the parser file to /etc/crowdsec/parsers/s01-parse

6

I have the same problem, but my log lines are including quotation marks, for instance:

2024-05-08 06:16:19,457 WARN  [org.keycloak.events] (executor-thread-1) type="LOGIN_ERROR", realmId="8735605f-6ae9-4f41-8436-db5d4cf50025", clientId="am-qa", userId="null", ipAddress="90.152.181.38", error="user_not_found", auth_method="openid-connect", auth_type="code", redirect_uri="https://am116bigserver.qa.fakedomain.net/oidc/callback/", code_id="e44d80b4-058d-4b45-b2ee-fac3d174e10c", username="admindeddeedede"
7

Okay I can make the quotes optional for both yourself and other user, I will add tests and update my PR

8

Thanks very much. My keycloak deployment was not using docker it was in a standalone server, so probably that’s the reason because it had a different output, I’m using the default keycloak log format:

log-format=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n

And keycloak version: 24.0.1

9

by any chance does anyone have a successful login line? you can redact any PII data I just need to see if my changes may break other lines than fail login attempts

10

At least in my keycloak instance there’s no line in keycloak log when using “info” log-level for successful login. When using next log-level (debug) I can see a lot of new lines related to my user (and ldap federation) when login, and probably this is the best one I found for my login:

2024-05-09 01:11:47,962 DEBUG [org.keycloak.events] (executor-thread-1) type="LOGIN", realmId="8735605f-6ae9-4f41-8436-db5d4cf50025", clientId="am-qa", userId="aa4ac2f0-088f-4eb9-b695-3d1198c6af62", ipAddress="90.162.181.38", auth_method="openid-connect", auth_type="code", response_type="code", redirect_uri="https://am116bigserver.fakedomain.net/oidc/callback/", consent="no_consent_required", code_id="13dc04db-567e-4e2c-bc33-e43371d6a055", username="mamedin", response_mode="query", authSessionParentId="13dc04db-567e-4e2c-bc33-e43371d6a055", authSessionTabId="D-DR4Cxy_nU"
1 Like
11

Okay my PR has been updated to support quoted strings and allows for debug logs also. Until it is merged you can manually download the file from github if you want to test it out :+1:

12

Excellent! I can see now that the login lines are parsed when using wrong credentials:

[root@test-keycloak crowdsec]# cscli metrics

Acquisition Metrics:
╭─────────────────────────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────╮
│               Source                │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├─────────────────────────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ file:/var/log/keycloak/keycloak.log │ 3          │ 3            │ -              │ 4                      │ -                 │
│ file:/var/log/messages              │ 1          │ -            │ 1              │ -                      │ -                 │
╰─────────────────────────────────────┴────────────┴──────────────┴────────────────┴────────────────────────┴───────────────────╯
1 Like
13

I have now also tested everything. The entries are shown as parsed and under cscli desisions list, the IP addresses are shown as banned.

Acquisition Metrics:
╭─────────────────┬────────────┬──────────────┬────────────────┬────────────────────────┬───────────────────╮
│     Source      │ Lines read │ Lines parsed │ Lines unparsed │ Lines poured to bucket │ Lines whitelisted │
├─────────────────┼────────────┼──────────────┼────────────────┼────────────────────────┼───────────────────┤
│ docker:keycloak │ 19         │ 19           │ -              │ 48                     │ -                 │
╰─────────────────┴────────────┴──────────────┴────────────────┴────────────────────────┴───────────────────╯

╭────────┬──────────┬──────────────────┬──────────────────────────────┬────────┬─────────┬───────────────────────────┬────────┬────────────────────┬──────────╮
│ ID │ Source │ Scope:Value │ Reason │ Action │ Country │ AS │ Events │ expiration │ Alert ID │
├────────┼──────────┼──────────────────┼──────────────────────────────┼────────┼─────────┼───────────────────────────┼────────┼────────────────────┼──────────┤
│ 249003 │ crowdsec │ Ip:10xx.xxx.xxx.xxx │ inherent-io/keycloak-bf │ ban │ DE │ 3209 Vodafone GmbH │ 6 │ 3h29m35.49629614s │ 38 │
│ 249002 │ crowdsec │ Ip:9xx.xxx.xxx.xxx │ inherent-io/keycloak-slow-bf │ ban │ DE │ 24940 Hetzner Online GmbH │ 11 │ 3h27m50.272299992s │ 37 │
╰────────┴──────────┴──────────────────┴──────────────────────────────┴────────┴─────────┴───────────────────────────┴────────┴────────────────────┴──────────╯
1 duplicated entries skipped

However, the fact that I can still access the page via the banned IP address is probably due to another problem.

Many thanks for your help.