Error running Email Plugin on Debian 12

crowdsec
1

Hi all,
I cannot get the email plugin to run properly. I have created the folder /etc/crowdsec/notifications and added email.yaml according to the documentation. I also enabled the email plugin in the profiles.yaml

When I try to restart the crowdsec service it throws the following error

level=info msg="Crowdsec v1.4.6-6~deb12u1-linux-debian"
level=info msg="Loading prometheus collectors"
level=info msg="Loading CAPI pusher"
level=info msg="initiating plugin broker"
level=fatal msg="api server init: unable to run local API: while loading plugin: plugin name /usr/lib/crowdsec/plugins/dummy is invalid. Name should be like {type-name}"

What is the issue on this?

2

Hmmm the default notifications directory is /usr/local/lib/crowdsec/plugins/ did you install outside of repositories?

What the contents of /usr/lib/crowdsec/plugins/?

3

Thanks for your respnse,

no, I have used the official repo for bookworm.

here the apt sources:

cat /etc/apt/sources.list.d/crowdsec_crowdsec.list
# this file was generated by packagecloud.io for
# the repository at https://packagecloud.io/crowdsec/crowdsec

deb [signed-by=/etc/apt/keyrings/crowdsec_crowdsec-archive-keyring.gpg] https://packagecloud.io/crowdsec/crowdsec/debian/ bookworm main
deb-src [signed-by=/etc/apt/keyrings/crowdsec_crowdsec-archive-keyring.gpg] https://packagecloud.io/crowdsec/crowdsec/debian/ bookworm main

here the contenct of the plugins directory:

ls -al /usr/lib/crowdsec/plugins/
total 44640
drwxr-xr-x 2 root root    4096 Aug 18 07:57 .
drwxr-xr-x 4 root root    4096 Aug 18 07:57 ..
-rwxr-xr-x 1 root root 8760464 Jul 15 11:29 dummy
-rwxr-xr-x 1 root root 8908848 Jul 15 11:29 email
-rwxr-xr-x 1 root root 9266416 Jul 15 11:29 http
-rwxr-xr-x 1 root root 9488400 Jul 15 11:29 slack
-rwxr-xr-x 1 root root 9266416 Jul 15 11:29 splunk

The directory /usr/local/lib/crowdsec/plugins/ is empty

I also needed to manually create /etc/crowdsec/notifications directory as well as email.yaml file. It was not created automatically

4

What version you running? as they are not package correctly what?!?

Did you previously install via the debian repository then installed ours?

5

You can fix it by running

for i in $(ls /usr/lib/crowdsec/plugins/); do mv "/usr/lib/crowdsec/plugins/$i" "/usr/lib/crowdsec/plugins/notification-$i"; done

But it seems it missing a whole bunch of stuff. Can you also run ls -la /etc/crowdsec/parsers/s01-parse/

6

Hi,

no, I only used your repository. I upgraded recently from bullseye to bookworm.

the version I am running is:

sudo cscli version
2023/08/18 11:19:44 version: v1.4.6-6~deb12u1-debian
2023/08/18 11:19:44 Codename: alphaga
2023/08/18 11:19:44 BuildDate: 2023-07-15_09:29:33
2023/08/18 11:19:44 GoVersion: 1.19.8
2023/08/18 11:19:44 Platform: linux
2023/08/18 11:19:44 Constraint_parser: >= 1.0, <= 2.0
2023/08/18 11:19:44 Constraint_scenario: >= 1.0, < 3.0
2023/08/18 11:19:44 Constraint_api: v1
2023/08/18 11:19:44 Constraint_acquis: >= 1.0, < 2.0
7

Yeah we dont use version naming like v1.4.6-6~deb12u1-debian ours has the github hash

2023/08/18 10:24:21 version: v1.5.3-rc4-12-ge8e2ade8-e8e2ade8f099fbd9b6bcf352d131985eba986f19
2023/08/18 10:24:21 Codename: alphaga
2023/08/18 10:24:21 BuildDate: 2023-08-18_10:24:01
2023/08/18 10:24:21 GoVersion: 1.21.0
2023/08/18 10:24:21 Platform: linux
2023/08/18 10:24:21 libre2: C++
2023/08/18 10:24:21 Constraint_parser: >= 1.0, <= 2.0
2023/08/18 10:24:21 Constraint_scenario: >= 1.0, < 3.0
2023/08/18 10:24:21 Constraint_api: v1
2023/08/18 10:24:21 Constraint_acquis: >= 1.0, < 2.0

So I think you may have downloaded via the debian repositories. And yeah /var/lib/crowdsec/hub/parsers/ is debian repositories that means if you have an updated config.yaml you will stop receiving updates to your parsers

8 
ls -la /etc/crowdsec/parsers/s01-parse/
total 28
drwxr-xr-x 2 root root 4096 Aug 17 08:45 .
drwxr-xr-x 5 root root 4096 Aug 15 08:15 ..
lrwxrwxrwx 1 root root   71 Aug 17 08:34 jellyfin-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/LePresidente/jellyfin-logs.yaml
lrwxrwxrwx 1 root root   73 Aug 15 08:19 nextcloud-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/crowdsecurity/nextcloud-logs.yaml
lrwxrwxrwx 1 root root   69 Aug 15 08:15 nginx-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/crowdsecurity/nginx-logs.yaml
lrwxrwxrwx 1 root root   68 Aug 15 08:15 sshd-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/crowdsecurity/sshd-logs.yaml
lrwxrwxrwx 1 root root   76 Aug 17 08:12 vaultwarden-logs.yaml -> /var/lib/crowdsec/hub/parsers/s01-parse/Dominic-Wagner/vaultwarden-logs.yaml

parsers look good to me. Bouncers also working. It’s just the notifications that were throwing errors.
I renamed all the plugins now with the notification prefix. The error is gone

Thanks for your support on this.

9

well, then the official crowdsec debian repo for bookworm seems to be outdated.

Maybe you should update the official repo then to the latest stable release.

10

We are not the maintainers for the Debian repository that is handled by a third party

11

Just to ensure you wont run into any issues can you paste your config.yaml here?

12

Ok I see. Then it would be better to manually install crowdsec without apt currently.

Here my config.yaml and config.yaml.local:

common:
  daemonize: true
  log_media: file
  log_level: info
  log_dir: /var/log/
  log_max_size: 20
  compress_logs: true
  log_max_files: 10
  working_dir: .
config_paths:
  config_dir: /etc/crowdsec/
  data_dir: /var/lib/crowdsec/data/
  simulation_path: /etc/crowdsec/simulation.yaml
  hub_dir: /var/lib/crowdsec/hub/
  index_path: /var/lib/crowdsec/hub/.index.json
  notification_dir: /etc/crowdsec/notifications/
  plugin_dir: /usr/lib/crowdsec/plugins/
crowdsec_service:
  acquisition_path: /etc/crowdsec/acquis.yaml
  acquisition_dir: /etc/crowdsec/acquis.d
  parser_routines: 1
cscli:
  output: human
  color: auto
db_config:
  log_level: info
  type: sqlite
  db_path: /var/lib/crowdsec/data/crowdsec.db
  #max_open_conns: 100
  #user:
  #password:
  #db_name:
  #host:
  #port:
  flush:
    max_items: 5000
    max_age: 7d
plugin_config:
  user: nobody # plugin process would be ran on behalf of this user
  group: nogroup # plugin process would be ran on behalf of this group
api:
  client:
    insecure_skip_verify: false
    credentials_path: /etc/crowdsec/local_api_credentials.yaml
  server:
    log_level: info
    listen_uri: 127.0.0.1:8080
    profiles_path: /etc/crowdsec/profiles.yaml
    console_path: /etc/crowdsec/console.yaml
    online_client: # Central API credentials (to push signals and receive bad IPs)
      credentials_path: /etc/crowdsec/online_api_credentials.yaml
    trusted_ips: # IP ranges, or IPs which can have admin API access
      - 127.0.0.1
      - ::1
#    tls:
#      cert_file: /etc/crowdsec/ssl/cert.pem
#      key_file: /etc/crowdsec/ssl/key.pem
prometheus:
  enabled: true
  level: full
  listen_addr: 127.0.0.1
  listen_port: 6060

and config.yaml.local

# Optimization for sqlite, see README.Debian:
db_config:
  use_wal: true
13

Perfect, I see no issues in your config.yaml just for context sake if your

hub_dir: /var/lib/crowdsec/hub/

Was

hub_dir: /etc/crowdsec/hub/

Then you will have issues as cscli will update the hub but your symlinks are pointing to wrong location (Hence why I asked you to ls -la earlier)

I created this fix script that will automatically dump all local configuration parser,scenarios,collections and fix the symlinks but you dont need to run it as you dont have the issue

https://gist.githubusercontent.com/LaurenceJJones/6960107296145e8e365009973b9d7f6d/raw/8a304d60c6f340a9e9c6c4e308a31462b5de3c28/debian_update_symlinks.sh

14

Thanks for clarifying. Hope this will help others too.

15

Ok, I digged deeper into the issue.

It seems that the current apt priority of debian is still pointing to the old stable version

apt-cache madison crowdsec
  crowdsec |      1.5.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.5.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.5.0 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec | 1.4.6-6~deb12u1 | https://ftp.debian.org/debian bookworm/main amd64 Packages
  crowdsec |      1.4.6 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.4.5 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.4.4 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.4.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.4.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.4.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.3.4 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.3.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.3.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.3.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.3.0 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.2.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.2.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |      1.2.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |    1.2.0-1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main amd64 Packages
  crowdsec |    1.4.6-4 | https://ftp.debian.org/debian bookworm/main Sources
  crowdsec | 1.4.6-6~deb12u1 | https://ftp.debian.org/debian bookworm/main Sources
  crowdsec |    1.2.0-1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
  crowdsec |      1.2.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
  crowdsec |      1.2.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
  crowdsec |      1.2.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
  crowdsec |      1.3.0 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
  crowdsec |      1.3.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
  crowdsec |      1.3.2 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
  crowdsec |      1.3.3 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
  crowdsec |      1.3.4 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources
  crowdsec |      1.4.1 | https://packagecloud.io/crowdsec/crowdsec/debian bookworm/main Sources

so I simply added the packagecloud entry to the top of /etc/apt/sources.list and removed /etc/apt/sources.list.d/crowdsec_crowdsec.list

After that version 1.5.2 was taken as the latest and had been updated. Also all directories and plugins are correct.

but now, when I test the mail plugin the mail is sent correctly but the log gives me an error:

DEBU[0001] received EOF, stopping recv loop              err="rpc error: code = Unavailable desc = error reading from server: EOF"
DEBU[0001] plugin process exited                         path=/usr/lib/crowdsec/plugins/notification-email pid=2280226
DEBU[0001] plugin exited