Hi all,
I cannot get the email plugin to run properly. I have created the folder /etc/crowdsec/notifications and added email.yaml according to the documentation. I also enabled the email plugin in the profiles.yaml
When I try to restart the crowdsec service it throws the following error
level=info msg="Crowdsec v1.4.6-6~deb12u1-linux-debian"
level=info msg="Loading prometheus collectors"
level=info msg="Loading CAPI pusher"
level=info msg="initiating plugin broker"
level=fatal msg="api server init: unable to run local API: while loading plugin: plugin name /usr/lib/crowdsec/plugins/dummy is invalid. Name should be like {type-name}"
What is the issue on this?
Hmmm the default notifications directory is /usr/local/lib/crowdsec/plugins/ did you install outside of repositories?
What the contents of /usr/lib/crowdsec/plugins/?
Thanks for your respnse,
no, I have used the official repo for bookworm.
here the apt sources:
cat /etc/apt/sources.list.d/crowdsec_crowdsec.list
# this file was generated by packagecloud.io for
# the repository at https://packagecloud.io/crowdsec/crowdsec
deb [signed-by=/etc/apt/keyrings/crowdsec_crowdsec-archive-keyring.gpg] https://packagecloud.io/crowdsec/crowdsec/debian/ bookworm main
deb-src [signed-by=/etc/apt/keyrings/crowdsec_crowdsec-archive-keyring.gpg] https://packagecloud.io/crowdsec/crowdsec/debian/ bookworm main
here the contenct of the plugins directory:
ls -al /usr/lib/crowdsec/plugins/
total 44640
drwxr-xr-x 2 root root 4096 Aug 18 07:57 .
drwxr-xr-x 4 root root 4096 Aug 18 07:57 ..
-rwxr-xr-x 1 root root 8760464 Jul 15 11:29 dummy
-rwxr-xr-x 1 root root 8908848 Jul 15 11:29 email
-rwxr-xr-x 1 root root 9266416 Jul 15 11:29 http
-rwxr-xr-x 1 root root 9488400 Jul 15 11:29 slack
-rwxr-xr-x 1 root root 9266416 Jul 15 11:29 splunk
The directory /usr/local/lib/crowdsec/plugins/ is empty
I also needed to manually create /etc/crowdsec/notifications directory as well as email.yaml file. It was not created automatically
What version you running? as they are not package correctly what?!?
Did you previously install via the debian repository then installed ours?
You can fix it by running
for i in $(ls /usr/lib/crowdsec/plugins/); do mv "/usr/lib/crowdsec/plugins/$i" "/usr/lib/crowdsec/plugins/notification-$i"; done
But it seems it missing a whole bunch of stuff. Can you also run ls -la /etc/crowdsec/parsers/s01-parse/
Hi,
no, I only used your repository. I upgraded recently from bullseye to bookworm.
the version I am running is:
sudo cscli version
2023/08/18 11:19:44 version: v1.4.6-6~deb12u1-debian
2023/08/18 11:19:44 Codename: alphaga
2023/08/18 11:19:44 BuildDate: 2023-07-15_09:29:33
2023/08/18 11:19:44 GoVersion: 1.19.8
2023/08/18 11:19:44 Platform: linux
2023/08/18 11:19:44 Constraint_parser: >= 1.0, <= 2.0
2023/08/18 11:19:44 Constraint_scenario: >= 1.0, < 3.0
2023/08/18 11:19:44 Constraint_api: v1
2023/08/18 11:19:44 Constraint_acquis: >= 1.0, < 2.0
Yeah we dont use version naming like v1.4.6-6~deb12u1-debian ours has the github hash
2023/08/18 10:24:21 version: v1.5.3-rc4-12-ge8e2ade8-e8e2ade8f099fbd9b6bcf352d131985eba986f19
2023/08/18 10:24:21 Codename: alphaga
2023/08/18 10:24:21 BuildDate: 2023-08-18_10:24:01
2023/08/18 10:24:21 GoVersion: 1.21.0
2023/08/18 10:24:21 Platform: linux
2023/08/18 10:24:21 libre2: C++
2023/08/18 10:24:21 Constraint_parser: >= 1.0, <= 2.0
2023/08/18 10:24:21 Constraint_scenario: >= 1.0, < 3.0
2023/08/18 10:24:21 Constraint_api: v1
2023/08/18 10:24:21 Constraint_acquis: >= 1.0, < 2.0
So I think you may have downloaded via the debian repositories. And yeah /var/lib/crowdsec/hub/parsers/ is debian repositories that means if you have an updated config.yaml you will stop receiving updates to your parsers
ls -la /etc/crowdsec/parsers/s01-parse/
total 28
drwxr-xr-x 2 root root 4096 Aug 17 08:45 .
drwxr-xr-x 5 root root 4096 Aug 15 08:15 ..
lrwxrwxrwx 1 root root 71 Aug 17 08:34 jellyfin-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/LePresidente/jellyfin-logs.yaml
lrwxrwxrwx 1 root root 73 Aug 15 08:19 nextcloud-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/crowdsecurity/nextcloud-logs.yaml
lrwxrwxrwx 1 root root 69 Aug 15 08:15 nginx-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/crowdsecurity/nginx-logs.yaml
lrwxrwxrwx 1 root root 68 Aug 15 08:15 sshd-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/crowdsecurity/sshd-logs.yaml
lrwxrwxrwx 1 root root 76 Aug 17 08:12 vaultwarden-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/Dominic-Wagner/vaultwarden-logs.yaml
parsers look good to me. Bouncers also working. It’s just the notifications that were throwing errors.
I renamed all the plugins now with the notification prefix. The error is gone
Thanks for your support on this.
well, then the official crowdsec debian repo for bookworm seems to be outdated.
Maybe you should update the official repo then to the latest stable release.
We are not the maintainers for the Debian repository that is handled by a third party
Just to ensure you wont run into any issues can you paste your config.yaml here?
Ok I see. Then it would be better to manually install crowdsec without apt currently.
Here my config.yaml and config.yaml.local:
common:
daemonize: true
log_media: file
log_level: info
log_dir: /var/log/
log_max_size: 20
compress_logs: true
log_max_files: 10
working_dir: .
config_paths:
config_dir: /etc/crowdsec/
data_dir: /var/lib/crowdsec/data/
simulation_path: /etc/crowdsec/simulation.yaml
hub_dir: /var/lib/crowdsec/hub/
index_path: /var/lib/crowdsec/hub/.index.json
notification_dir: /etc/crowdsec/notifications/
plugin_dir: /usr/lib/crowdsec/plugins/
crowdsec_service:
acquisition_path: /etc/crowdsec/acquis.yaml
acquisition_dir: /etc/crowdsec/acquis.d
parser_routines: 1
cscli:
output: human
color: auto
db_config:
log_level: info
type: sqlite
db_path: /var/lib/crowdsec/data/crowdsec.db
#max_open_conns: 100
#user:
#password:
#db_name:
#host:
#port:
flush:
max_items: 5000
max_age: 7d
plugin_config:
user: nobody # plugin process would be ran on behalf of this user
group: nogroup # plugin process would be ran on behalf of this group
api:
client:
insecure_skip_verify: false
credentials_path: /etc/crowdsec/local_api_credentials.yaml
server:
log_level: info
listen_uri: 127.0.0.1:8080
profiles_path: /etc/crowdsec/profiles.yaml
console_path: /etc/crowdsec/console.yaml
online_client: # Central API credentials (to push signals and receive bad IPs)
credentials_path: /etc/crowdsec/online_api_credentials.yaml
trusted_ips: # IP ranges, or IPs which can have admin API access
- 127.0.0.1
- ::1
# tls:
# cert_file: /etc/crowdsec/ssl/cert.pem
# key_file: /etc/crowdsec/ssl/key.pem
prometheus:
enabled: true
level: full
listen_addr: 127.0.0.1
listen_port: 6060
and config.yaml.local
# Optimization for sqlite, see README.Debian:
db_config:
use_wal: true
Perfect, I see no issues in your config.yaml just for context sake if your
hub_dir: /var/lib/crowdsec/hub/
Was
hub_dir: /etc/crowdsec/hub/
Then you will have issues as cscli will update the hub but your symlinks are pointing to wrong location (Hence why I asked you to ls -la earlier)
I created this fix script that will automatically dump all local configuration parser,scenarios,collections and fix the symlinks but you dont need to run it as you dont have the issue
https://gist.githubusercontent.com/LaurenceJJones/6960107296145e8e365009973b9d7f6d/raw/8a304d60c6f340a9e9c6c4e308a31462b5de3c28/debian_update_symlinks.sh
Thanks for clarifying. Hope this will help others too.
Ok, I digged deeper into the issue.
It seems that the current apt priority of debian is still pointing to the old stable version
apt-cache madison crowdsec
crowdsec | 1.5.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.5.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.5.0 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.4.6-6~deb12u1 | https://ftp.debian.org/debian bookworm/main amd64 Packages
crowdsec | 1.4.6 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.4.5 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.4.4 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.4.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.4.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.4.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.3.4 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.3.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.3.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.3.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.3.0 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.2.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.2.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.2.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.2.0-1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
crowdsec | 1.4.6-4 | https://ftp.debian.org/debian bookworm/main Sources
crowdsec | 1.4.6-6~deb12u1 | https://ftp.debian.org/debian bookworm/main Sources
crowdsec | 1.2.0-1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
crowdsec | 1.2.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
crowdsec | 1.2.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
crowdsec | 1.2.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
crowdsec | 1.3.0 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
crowdsec | 1.3.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
crowdsec | 1.3.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
crowdsec | 1.3.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
crowdsec | 1.3.4 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
crowdsec | 1.4.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
so I simply added the packagecloud entry to the top of /etc/apt/sources.list and removed /etc/apt/sources.list.d/crowdsec_crowdsec.list
After that version 1.5.2 was taken as the latest and had been updated. Also all directories and plugins are correct.
but now, when I test the mail plugin the mail is sent correctly but the log gives me an error:
DEBU[0001] received EOF, stopping recv loop err="rpc error: code = Unavailable desc = error reading from server: EOF"
DEBU[0001] plugin process exited path=/usr/lib/crowdsec/plugins/notification-email pid=2280226
DEBU[0001] plugin exited