Error on email notification plugin

crowdsec
1

Hi all,
the email notification plugin worked until yesterday but yesterday I noticed these errors in the crowdsec.log file and the email does not arrive:

time="25-05-2022 10:27:50" level=debug msg="making smtp connection" @module=email-plugin.email_default
time="25-05-2022 10:27:50" level=error msg="rpc error: code = Unknown desc = Mail Error on Hello: EOF error, retry num 1" plugin=email_default
time="25-05-2022 10:27:51" level=error msg="rpc error: code = Unknown desc = Mail Error on Hello: EOF" plugin:=email_default

Nothing changed in the email configuration in /etc/crowdsec/notifications/email.yaml file:

type: email           # Don't change
name: email_default   # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
#log_level: info
log_level: trace

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
timeout: 20s          # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the email message body
format: |
  {{range . -}}
    {{$alert := . -}}
    {{range .Decisions -}}
      <a href=https://www.whois.com/whois/{{.Value}}>{{.Value}}</a> will get <b>{{.Type}}</b> for next <b>{{.Duration}}</b> for triggering <b>{{.Scenario}}</b> on machine <b>{{$alert.MachineID}}</b>. <a href=https://www.shodan.io/host/{{.Value}}>Shodan</a>
      #<html><body><p><a href=https://www.whois.com/whois/{{.Value}}>{{.Value}}</a> will get <b>{{.Type}}</b> for next <b>{{.Duration}}</b> for triggering <b>{{.Scenario}}</b> on machine <b>{{$alert.MachineID}}</b>.</p> <p><a href=https://www.shodan.io/host/{{.Value}}>Shodan</a></p></body></html>
    {{end -}}
  {{end -}}

smtp_host: smtp-relay.gmail.com
smtp_username: username@mydomain.com
smtp_password: my_password
#smtp_port:            # Common values are any of [25, 465, 587, 2525]
smtp_port: 587
#auth_type:            # Valid choices are "none", "crammd5", "login", "plain"
auth_type: login
sender_name: "CrowdSec"
sender_email: sender@domain.com
email_subject: "CrowdSec Notification"
receiver_emails:
        - receiver@domain.com
# - email1@gmail.com
# - email2@gmail.com

# One of "ssltls", "none"
encryption_type: ssltls

I can send an email via telnet with the same config parameters from the file above.

Do you have any idea?

Thanks in advance.

2

Hello,

According to the error message, it’s during the Hello phase that the main plugin gets a closed connection from the server. Are you really using gmail smtp ? in this case we can try to reproduce on our side ! Please let me know :slight_smile:

3

Hi @thibault,
yes I’m using Google Workspace account with our configured domain (not @gmail.com) and with this configuration everything worked until some days ago.

Thanks.

4

did it broke when you upgraded crowdsec, or did it stop working out of the blue ? :slight_smile:

5

I checked old logs but I did not found anything about a correlation between an upgrade and the errors I’m having, so I think it stops working out of the blue…

I found these old strange logs:

time="14-03-2022 14:41:29" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default
time="14-03-2022 15:36:26" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default
time="14-03-2022 15:36:42" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default
time="14-03-2022 15:51:33" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default
time="14-03-2022 15:52:39" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default.email_default
time="14-03-2022 15:53:29" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default.email_default.email_default
time="14-03-2022 16:42:58" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default.email_default.email_default.email_default
time="14-03-2022 16:44:00" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default
time="14-03-2022 16:45:09" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default
time="14-03-2022 16:46:16" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default
time="14-03-2022 16:47:24" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default
time="14-03-2022 16:54:38" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default
time="14-03-2022 17:11:02" level=info msg="sent email to [myemail@domain.com]" @module=email-plugin.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default.email_default

time="14-03-2022 11:43:58" level=error msg="rpc error: code = Unknown desc = 550 5.7.1 Invalid credentials for relay [111.222.333.444]. The IP address you've\n5.7.1 registered in your G Suite SMTP Relay service doesn't match domain of\n5.7.1 the account this email is being sent from. If you are trying to relay\n5.7.1 mail from a domain that isn't registered under your G Suite account\n5.7.1 or has empty envelope-from, you must configure your mail server\n5.7.1 either to use SMTP AUTH to identify the sending domain or to present\n5.7.1 one of your domain names in the HELO or EHLO command. For more\n5.7.1 information, please visit\n5.7.1  https://support.google.com/a/answer/6140680#invalidcred k21-20020a05651c10b500b00247bc56d317sm444446ljn.49 - gsmtp error, retry num 1" plugin=email_default

time="14-03-2022 20:41:03" level=error msg="rpc error: code = Unknown desc = Mail Error on STARTTLS: 421 4.7.0 Try again later, closing connection. (EHLO) m5-20020a7bcf25000000b00389a5247b08sm23368wmg.18 - gsmtp error, retry num 1" plugin=email_default

But the last email notified from crowdsec was sent on 17/04/2022…

6

Hi, are there any news about this issue?

7

Hey! sometimes I had issues with these settings:

May since the new enforcement this has blocked the application and you must allow it via your g suite settings.

8

Mmmm,I think it’s not this the problem because we have a google Workspace account (not @gmail.com) and we set a “password for app” for the Google account that should send the email from Crowdsec.
And with the same smtp settings of Crowdsec I’m able to send an email from a linux console via smtp (like Crowdsec)…

9

Hi, no news about this problem?

I don’t understand if this is a configuration problem (but I don’t think for the reasons of previous posts) or a crowdsec’s issue…

10

Hello,

unfortunately I was not able to reproduce the bug :frowning:
I would tend to say it’s a configuration issue, but it’s a shot in the dark.

11

Hi, I did some tests and:

  • I remind you that we use Google Workspace accounts and not the basic gmail
  • if I use the same SMTP configuration posted above changing only the smtp_host from smtp-relay.gmail.com to smtp.gmail.com it works and I’m able to send email
  • I suppose the correct smtp server is smtp-relay.gmail.com for Google Workspace accounts
  • I enabled and whitelisted the crowdsec’s machine IP address as permit sender for our domain
  • And for the reason just above I’m able to send an email from a linux console on the same crowdsec machine (with whitelisted IP address) using the same SMTP configuration posted above and using smtp-relay.gmail.com

Maybe this documentation could be helpful?

I try with this consideration: the crowdsec description error was Mail Error on Hello and in the previous link there is something about like this:

We recommend that servers presents unique identifiers in the HELO or EHLO arguments during SMTP connections. For example, use your domain name or the server name, instead of generic identifiers such as localhost or smtp-relay.gmail.com

Thanks.

12

Looking at the bottom point you mentioned I ran into recently with a user using postfix and we didnt have configuration to override it, the email plugin sends localhost in the HELO connect statement. I have created a PR Add helo config by LaurenceJJones · Pull Request #1765 · crowdsecurity/crowdsec · GitHub