Telegram Notification Bug

1

Hey \0

I have problems with sending notifications in telegram

profiles.yaml

name: default_ip_remediation
debug: true
filters:
 - Alert.Remediation == true && Alert.GetScope() == "Ip"
decisions:
 - type: ban
   duration: 4h
#duration_expr: Sprintf('%dh', (GetDecisionsCount(Alert.GetValue()) + 1) * 4)
notifications:
#   - slack_default  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
#   - splunk_default # Set the splunk url and token in /etc/crowdsec/notifications/splunk.yaml before enabling this.
 - http_default   # Set the required http parameters in /etc/crowdsec/notifications/http.yaml before enabling this.
#   - email_default  # Set the required email parameters in /etc/crowdsec/notifications/email.yaml before enabling this.
on_success: break

http.yaml

type: http          # Don't change
name: http_default  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the http request body

# Replace XXXXXXXXX with your Telegram chat ID
format: |
  {
   "chat_id": "My ID",
   "text": "
     {{range . -}}
     {{$alert := . -}}
     {{range .Decisions -}}
     {{.Value}} will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}}.
     {{end -}}
     {{end -}}
   ",
   "reply_markup": {
      "inline_keyboard": [
          {{ $arrLength := len . -}}
          {{ range $i, $value := . -}}
          {{ $V := $value.Source.Value -}}
          [
              {
                  "text": "See {{ $V }} on shodan.io",
                  "url": "https://www.shodan.io/host/{{ $V -}}"
              },
              {
                  "text": "See {{ $V }} on crowdsec.net",
                  "url": "https://app.crowdsec.net/cti/{{ $V -}}"
              }
          ]{{if lt $i ( sub $arrLength 1) }},{{end }}
      {{end -}}
      ]
  }

url: https://api.telegram.org/bot<MY API>/sendMessage # Replace <TELEGRAM_APY_KEY> with your APi key

method: POST
headers:
  Content-Type: "application/json"

Log

time="27-04-2023 13:01:40" level=debug msg="req-jwt: POST http://0.0.0.0:8080/v1/alerts"
time="27-04-2023 13:01:40" level=debug msg="alert sent to Plugin channel"
time="27-04-2023 13:01:40" level=debug msg="(*models.Decision)(0xc0007a93b0)({\n Duration: (*string)(0xc00157ca10)((len=2) \"4h\"),\n ID: (int64) 0,\n Origin: (*string)(0xc00157ca40)((len=8) \"crowdsec\"),\n Scenario: (*string)(0xc00157ca50)((len=32) \"LePresidente/http-generic-401-bf\"),\n Scope: (*string)(0xc00157ca00)((len=2) \"Ip\"),\n Simulated: (*bool)(<nil>),\n Type: (*string)(0xc00157ca20)((len=3) \"ban\"),\n Until: (string) \"\",\n Value: (*string)(0xc00157ca30)((len=14) \"IP\")\n})\n"
time="27-04-2023 13:01:40" level=info msg="(localhost/crowdsec) LePresidente/http-generic-401-bf by ip IP (RU/57371) : 4h ban on Ip IP"
time="27-04-2023 13:01:40" level=debug msg="alert sent to CAPI channel"
time="27-04-2023 13:01:40" level=info msg="127.0.0.1 - [Thu, 27 Apr 2023 13:01:40 UTC] \"POST /v1/alerts HTTP/1.1 201 93.312212ms \"crowdsec/v1.4.6-linux-5f71037b40c498045e1b59923504469e2b8d0140\" \""
time="27-04-2023 13:01:40" level=debug msg="resp-jwt: 201"
time="27-04-2023 13:01:40" level=debug msg="[headers] Date : [Thu, 27 Apr 2023 13:01:40 GMT]"
time="27-04-2023 13:01:40" level=debug msg="[headers] Content-Length : [6]"
time="27-04-2023 13:01:40" level=debug msg="[headers] Content-Type : [application/json; charset=utf-8]"
time="27-04-2023 13:01:40" level=debug msg="Response: HTTP/1.1 201 Created\r\nContent-Length: 6\r\nContent-Type: application/json; charset=utf-8\r\nDate: Thu, 27 Apr 2023 13:01:40 GMT\r\n\r\n[\"25\"]"
time="27-04-2023 13:01:40" level=debug msg="pushing 1 alerts to plugin" plugin=http_default
time="27-04-2023 13:01:40" level=info msg="received signal for http_default config" @module=http-plugin
time="27-04-2023 13:01:40" level=error msg="Failed to make HTTP request : Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\"" @module=http-plugin
time="27-04-2023 13:01:40" level=error msg="rpc error: code = Unknown desc = Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\" error, retry num 1" plugin=http_default
time="27-04-2023 13:01:41" level=error msg="rpc error: code = Unknown desc = Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\"" plugin:=http_default
2

From what it seems are you trying to do

url: ${http_url}

within the http.yaml? cause it doesnt support environment variables at the moment.

Well decoding it goes to <HTTP_URL> that doesn’t exists in the yaml at all?

4

the code is completely from the official instructions CrowdSec

5

There must be an error the only thing I could think it

url: https://api.telegram.org/bot<MY API>/sendMessage

Make sure the url doesnt have <> in but im sure you don’t as its just to visualise the placeholder value.

6

So I just got it working with the current format

url: https://api.telegram.org/botXXX:YYY/sendMessage
## Replace XXX:YYY with your bot token

If you are sending to a private chat the chat-id might not need a - infront

If you need to get your chat-id, send a message to your bot then go to

https://api.telegram.org/botXXX:YYY/getUpdates
## Replace XXX:YYY with your bot token

Which will list all latest events, you will find a chat property and id within the object if it is a private chat the ID will match your user ID.

I can see how the documentation can be confusing so I will update this.

7

all tokens and chat id were specified by me. I just removed them from the example

8

Well I updated the docs here to simulate better what the token looks like. There doesn’t seem to be a bug within the http notification as I got it to work with no modifications.

9

Here is http.yaml.

token and chat_id fictitious

type: http          # Don't change
name: http_default  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

group_wait: 30s       # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
group_threshold: 5    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
max_retry: 4          # Number of attempts to relay messages to plugins in case of error
timeout: 4s           # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

format: |
  {
   "chat_id": "-6354634534567", 
   "text": "
     {{range . -}}  
     {{$alert := . -}}  
     {{range .Decisions -}}
     {{.Value}} will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}}.
     {{end -}}
     {{end -}}
   ",
   "reply_markup": {
      "inline_keyboard": [
          {{ $arrLength := len . -}}
          {{ range $i, $value := . -}}
          {{ $V := $value.Source.Value -}}
          [
              {
                  "text": "See {{ $V }} on shodan.io",
                  "url": "https://www.shodan.io/host/{{ $V -}}"
              },
              {
                  "text": "See {{ $V }} on crowdsec.net",
                  "url": "https://app.crowdsec.net/cti/{{ $V -}}"
              }
          ]{{if lt $i ( sub $arrLength 1) }},{{end }}
      {{end -}}
      ]
  }

url: https://api.telegram.org/bot54578563453:sdfgsdf_823454k3454345kdfgs/sendMessage

method: POST
headers:
  Content-Type: "application/json"

Here is log

time="09-05-2023 10:50:36" level=info msg="received signal for http_default config" @module=http-plugin

time="09-05-2023 10:50:36" level=error msg="Failed to make HTTP request : Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\"" @module=http-plugin

time="09-05-2023 10:50:36" level=error msg="rpc error: code = Unknown desc = Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\" error, retry num 1" plugin=http_default

time="09-05-2023 10:50:37" level=error msg="rpc error: code = Unknown desc = Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\"" plugin:=http_default
10

Can you comment out these

group_wait: 30s       # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
group_threshold: 5    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
max_retry: 4          # Number of attempts to relay messages to plugins in case of error
timeout: 4s           # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

As with json body these are not recommended to turn on hence our example doesnt have them on. It will help debug if some of these options are causing an issue.

11

It does not help

type: http          # Don't change
name: http_default  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait: 30s       # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold: 5    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry: 4          # Number of attempts to relay messages to plugins in case of error
# timeout: 4s           # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

format: |
  {
   "chat_id": "-6354634534567", 
   "text": "
     {{range . -}}  
     {{$alert := . -}}  
     {{range .Decisions -}}
     {{.Value}} will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}}.
     {{end -}}
     {{end -}}
   ",
   "reply_markup": {
      "inline_keyboard": [
          {{ $arrLength := len . -}}
          {{ range $i, $value := . -}}
          {{ $V := $value.Source.Value -}}
          [
              {
                  "text": "See {{ $V }} on shodan.io",
                  "url": "https://www.shodan.io/host/{{ $V -}}"
              },
              {
                  "text": "See {{ $V }} on crowdsec.net",
                  "url": "https://app.crowdsec.net/cti/{{ $V -}}"
              }
          ]{{if lt $i ( sub $arrLength 1) }},{{end }}
      {{end -}}
      ]
  }

url: https://api.telegram.org/bot54578563453:sdfgsdf_823454k3454345kdfgs/sendMessage

method: POST
headers:
  Content-Type: "application/json"
12

And when you updated the yaml you restarted the crowdsec service?

13

Yes, I restarted the docker container

14

And if you exec into the crowdsec container you can curl

curl -s https://api.telegram.org/botXXX:YYY/getUpdates

This with no errors? obviously replace the bot token as you did.

15

done, no errors. command runs fine

16

Okay, I need to test a docker setup. You running a compose stack or independent?

Could you also provide output of

cscli version
17

Compose Stack.
Traefik - CrowdSec - Bouncer-traefik

cscli version

2023/05/09 11:25:56 version: v1.4.6-5f71037b40c498045e1b59923504469e2b8d0140
2023/05/09 11:25:56 Codename: alphaga
2023/05/09 11:25:56 BuildDate: 2023-02-09_14:37:12
2023/05/09 11:25:56 GoVersion: 1.19.5
2023/05/09 11:25:56 Platform: linux
2023/05/09 11:25:56 Constraint_parser: >= 1.0, <= 2.0
2023/05/09 11:25:56 Constraint_scenario: >= 1.0, < 3.0
2023/05/09 11:25:56 Constraint_api: v1
2023/05/09 11:25:56 Constraint_acquis: >= 1.0, < 2.0

Telegram works fine with other containers. The problem is only with CrowdSec Notifications

18

So. I put curl in the crowdsec container and ran the command to send a message from the container

curl -s -o /dev/null -X POST -H "Content-Type: application/json" -d "{\"chat_id\": \"{-MY_ID}\", \"text\": \"test_message\", \"disable_notification\": false}" https://api.telegram.org/bot{MY_TOKEN}/sendMessage

And everything went perfectly, the message came to my telegram

So from the container itself, everything works

19

I just tried to replicate here my http_tele.yaml

type: http          # Don't change
name: http_tele  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the http request body

# Replace XXXXXXXXX with your Telegram chat ID
format: |
  {
   "chat_id": "MYID", 
   "text": "
     {{range . -}}  
     {{$alert := . -}}  
     {{range .Decisions -}}
     {{.Value}} will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}}.
     {{end -}}
     {{end -}}
   ",
   "reply_markup": {
      "inline_keyboard": [
          {{ $arrLength := len . -}}
          {{ range $i, $value := . -}}
          {{ $V := $value.Source.Value -}}
          [
              {
                  "text": "See {{ $V }} on shodan.io",
                  "url": "https://www.shodan.io/host/{{ $V -}}"
              },
              {
                  "text": "See {{ $V }} on crowdsec.net",
                  "url": "https://app.crowdsec.net/cti/{{ $V -}}"
              }
          ]{{if lt $i ( sub $arrLength 1) }},{{end }}
      {{end -}}
      ]
    }
  }

url: https://api.telegram.org/botXXX:YYY/sendMessage # Replace <TELEGRAM_API_KEY> with your API key

method: POST
headers:
  Content-Type: "application/json"

I mounted the file to /etc/crowdsec/notifications/http.yaml and updated my profiles to have:

notifications:
#   - slack_default  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
#   - splunk_default # Set the splunk url and token in /etc/crowdsec/notifications/splunk.yaml before enabling this.
- http_tele   # Set the required http parameters in /etc/crowdsec/notifications/http.yaml before enabling this.
#   - email_default  # Set the required email parameters in /etc/crowdsec/notifications/email.yaml before enabling this.

And restarted the container and it just worked.

20

it’s worked. But… o_O

21

Do you have 2 yaml files with the same http_default name?