Telegram Notification Bug

ScreN · April 27, 2023, 1:42pm

Hey \0

I have problems with sending notifications in telegram

profiles.yaml

name: default_ip_remediation
debug: true
filters:
 - Alert.Remediation == true && Alert.GetScope() == "Ip"
decisions:
 - type: ban
   duration: 4h
#duration_expr: Sprintf('%dh', (GetDecisionsCount(Alert.GetValue()) + 1) * 4)
notifications:
#   - slack_default  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
#   - splunk_default # Set the splunk url and token in /etc/crowdsec/notifications/splunk.yaml before enabling this.
 - http_default   # Set the required http parameters in /etc/crowdsec/notifications/http.yaml before enabling this.
#   - email_default  # Set the required email parameters in /etc/crowdsec/notifications/email.yaml before enabling this.
on_success: break

http.yaml

type: http          # Don't change
name: http_default  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the http request body

# Replace XXXXXXXXX with your Telegram chat ID
format: |
  {
   "chat_id": "My ID",
   "text": "
     {{range . -}}
     {{$alert := . -}}
     {{range .Decisions -}}
     {{.Value}} will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}}.
     {{end -}}
     {{end -}}
   ",
   "reply_markup": {
      "inline_keyboard": [
          {{ $arrLength := len . -}}
          {{ range $i, $value := . -}}
          {{ $V := $value.Source.Value -}}
          [
              {
                  "text": "See {{ $V }} on shodan.io",
                  "url": "https://www.shodan.io/host/{{ $V -}}"
              },
              {
                  "text": "See {{ $V }} on crowdsec.net",
                  "url": "https://app.crowdsec.net/cti/{{ $V -}}"
              }
          ]{{if lt $i ( sub $arrLength 1) }},{{end }}
      {{end -}}
      ]
  }

url: https://api.telegram.org/bot<MY API>/sendMessage # Replace <TELEGRAM_APY_KEY> with your APi key

method: POST
headers:
  Content-Type: "application/json"

Log

time="27-04-2023 13:01:40" level=debug msg="req-jwt: POST http://0.0.0.0:8080/v1/alerts"
time="27-04-2023 13:01:40" level=debug msg="alert sent to Plugin channel"
time="27-04-2023 13:01:40" level=debug msg="(*models.Decision)(0xc0007a93b0)({\n Duration: (*string)(0xc00157ca10)((len=2) \"4h\"),\n ID: (int64) 0,\n Origin: (*string)(0xc00157ca40)((len=8) \"crowdsec\"),\n Scenario: (*string)(0xc00157ca50)((len=32) \"LePresidente/http-generic-401-bf\"),\n Scope: (*string)(0xc00157ca00)((len=2) \"Ip\"),\n Simulated: (*bool)(<nil>),\n Type: (*string)(0xc00157ca20)((len=3) \"ban\"),\n Until: (string) \"\",\n Value: (*string)(0xc00157ca30)((len=14) \"IP\")\n})\n"
time="27-04-2023 13:01:40" level=info msg="(localhost/crowdsec) LePresidente/http-generic-401-bf by ip IP (RU/57371) : 4h ban on Ip IP"
time="27-04-2023 13:01:40" level=debug msg="alert sent to CAPI channel"
time="27-04-2023 13:01:40" level=info msg="127.0.0.1 - [Thu, 27 Apr 2023 13:01:40 UTC] \"POST /v1/alerts HTTP/1.1 201 93.312212ms \"crowdsec/v1.4.6-linux-5f71037b40c498045e1b59923504469e2b8d0140\" \""
time="27-04-2023 13:01:40" level=debug msg="resp-jwt: 201"
time="27-04-2023 13:01:40" level=debug msg="[headers] Date : [Thu, 27 Apr 2023 13:01:40 GMT]"
time="27-04-2023 13:01:40" level=debug msg="[headers] Content-Length : [6]"
time="27-04-2023 13:01:40" level=debug msg="[headers] Content-Type : [application/json; charset=utf-8]"
time="27-04-2023 13:01:40" level=debug msg="Response: HTTP/1.1 201 Created\r\nContent-Length: 6\r\nContent-Type: application/json; charset=utf-8\r\nDate: Thu, 27 Apr 2023 13:01:40 GMT\r\n\r\n[\"25\"]"
time="27-04-2023 13:01:40" level=debug msg="pushing 1 alerts to plugin" plugin=http_default
time="27-04-2023 13:01:40" level=info msg="received signal for http_default config" @module=http-plugin
time="27-04-2023 13:01:40" level=error msg="Failed to make HTTP request : Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\"" @module=http-plugin
time="27-04-2023 13:01:40" level=error msg="rpc error: code = Unknown desc = Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\" error, retry num 1" plugin=http_default
time="27-04-2023 13:01:41" level=error msg="rpc error: code = Unknown desc = Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\"" plugin:=http_default

iiAmLoz · May 4, 2023, 2:44pm

From what it seems are you trying to do

url: ${http_url}

within the http.yaml? cause it doesnt support environment variables at the moment.

Well decoding it goes to <HTTP_URL> that doesn’t exists in the yaml at all?

ScreN · May 5, 2023, 1:50am

the code is completely from the official instructions CrowdSec

iiAmLoz · May 5, 2023, 4:41pm

There must be an error the only thing I could think it

url: https://api.telegram.org/bot<MY API>/sendMessage

Make sure the url doesnt have <> in but im sure you don’t as its just to visualise the placeholder value.

iiAmLoz · May 5, 2023, 5:16pm

So I just got it working with the current format

url: https://api.telegram.org/botXXX:YYY/sendMessage
## Replace XXX:YYY with your bot token

If you are sending to a private chat the chat-id might not need a - infront

If you need to get your chat-id, send a message to your bot then go to

https://api.telegram.org/botXXX:YYY/getUpdates
## Replace XXX:YYY with your bot token

Which will list all latest events, you will find a chat property and id within the object if it is a private chat the ID will match your user ID.

I can see how the documentation can be confusing so I will update this.

ScreN · May 6, 2023, 1:02am

all tokens and chat id were specified by me. I just removed them from the example

iiAmLoz · May 9, 2023, 8:44am

Well I updated the docs here to simulate better what the token looks like. There doesn’t seem to be a bug within the http notification as I got it to work with no modifications.

ScreN · May 9, 2023, 10:53am

Here is http.yaml.

token and chat_id fictitious

type: http          # Don't change
name: http_default  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

group_wait: 30s       # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
group_threshold: 5    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
max_retry: 4          # Number of attempts to relay messages to plugins in case of error
timeout: 4s           # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

format: |
  {
   "chat_id": "-6354634534567", 
   "text": "
     {{range . -}}  
     {{$alert := . -}}  
     {{range .Decisions -}}
     {{.Value}} will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}}.
     {{end -}}
     {{end -}}
   ",
   "reply_markup": {
      "inline_keyboard": [
          {{ $arrLength := len . -}}
          {{ range $i, $value := . -}}
          {{ $V := $value.Source.Value -}}
          [
              {
                  "text": "See {{ $V }} on shodan.io",
                  "url": "https://www.shodan.io/host/{{ $V -}}"
              },
              {
                  "text": "See {{ $V }} on crowdsec.net",
                  "url": "https://app.crowdsec.net/cti/{{ $V -}}"
              }
          ]{{if lt $i ( sub $arrLength 1) }},{{end }}
      {{end -}}
      ]
  }

url: https://api.telegram.org/bot54578563453:sdfgsdf_823454k3454345kdfgs/sendMessage

method: POST
headers:
  Content-Type: "application/json"

Here is log

time="09-05-2023 10:50:36" level=info msg="received signal for http_default config" @module=http-plugin

time="09-05-2023 10:50:36" level=error msg="Failed to make HTTP request : Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\"" @module=http-plugin

time="09-05-2023 10:50:36" level=error msg="rpc error: code = Unknown desc = Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\" error, retry num 1" plugin=http_default

time="09-05-2023 10:50:37" level=error msg="rpc error: code = Unknown desc = Post \"%3CHTTP_url%3E\": unsupported protocol scheme \"\"" plugin:=http_default

iiAmLoz · May 9, 2023, 10:55am

Can you comment out these

group_wait: 30s       # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
group_threshold: 5    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
max_retry: 4          # Number of attempts to relay messages to plugins in case of error
timeout: 4s           # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

As with json body these are not recommended to turn on hence our example doesnt have them on. It will help debug if some of these options are causing an issue.

ScreN · May 9, 2023, 10:59am

It does not help

type: http          # Don't change
name: http_default  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait: 30s       # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold: 5    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry: 4          # Number of attempts to relay messages to plugins in case of error
# timeout: 4s           # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

format: |
  {
   "chat_id": "-6354634534567", 
   "text": "
     {{range . -}}  
     {{$alert := . -}}  
     {{range .Decisions -}}
     {{.Value}} will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}}.
     {{end -}}
     {{end -}}
   ",
   "reply_markup": {
      "inline_keyboard": [
          {{ $arrLength := len . -}}
          {{ range $i, $value := . -}}
          {{ $V := $value.Source.Value -}}
          [
              {
                  "text": "See {{ $V }} on shodan.io",
                  "url": "https://www.shodan.io/host/{{ $V -}}"
              },
              {
                  "text": "See {{ $V }} on crowdsec.net",
                  "url": "https://app.crowdsec.net/cti/{{ $V -}}"
              }
          ]{{if lt $i ( sub $arrLength 1) }},{{end }}
      {{end -}}
      ]
  }

url: https://api.telegram.org/bot54578563453:sdfgsdf_823454k3454345kdfgs/sendMessage

method: POST
headers:
  Content-Type: "application/json"

iiAmLoz · May 9, 2023, 11:01am

And when you updated the yaml you restarted the crowdsec service?

ScreN · May 9, 2023, 11:02am

Yes, I restarted the docker container

iiAmLoz · May 9, 2023, 11:21am

And if you exec into the crowdsec container you can curl

curl -s https://api.telegram.org/botXXX:YYY/getUpdates

This with no errors? obviously replace the bot token as you did.

ScreN · May 9, 2023, 11:22am

done, no errors. command runs fine

iiAmLoz · May 9, 2023, 11:25am

Okay, I need to test a docker setup. You running a compose stack or independent?

Could you also provide output of

cscli version

ScreN · May 9, 2023, 11:28am

Compose Stack.
Traefik - CrowdSec - Bouncer-traefik

cscli version

2023/05/09 11:25:56 version: v1.4.6-5f71037b40c498045e1b59923504469e2b8d0140
2023/05/09 11:25:56 Codename: alphaga
2023/05/09 11:25:56 BuildDate: 2023-02-09_14:37:12
2023/05/09 11:25:56 GoVersion: 1.19.5
2023/05/09 11:25:56 Platform: linux
2023/05/09 11:25:56 Constraint_parser: >= 1.0, <= 2.0
2023/05/09 11:25:56 Constraint_scenario: >= 1.0, < 3.0
2023/05/09 11:25:56 Constraint_api: v1
2023/05/09 11:25:56 Constraint_acquis: >= 1.0, < 2.0

Telegram works fine with other containers. The problem is only with CrowdSec Notifications

ScreN · May 22, 2023, 12:43pm

So. I put curl in the crowdsec container and ran the command to send a message from the container

curl -s -o /dev/null -X POST -H "Content-Type: application/json" -d "{\"chat_id\": \"{-MY_ID}\", \"text\": \"test_message\", \"disable_notification\": false}" https://api.telegram.org/bot{MY_TOKEN}/sendMessage

And everything went perfectly, the message came to my telegram

So from the container itself, everything works

iiAmLoz · May 22, 2023, 12:49pm

I just tried to replicate here my http_tele.yaml

type: http          # Don't change
name: http_tele  # Must match the registered plugin in the profile

# One of "trace", "debug", "info", "warn", "error", "off"
log_level: info

# group_wait:         # Time to wait collecting alerts before relaying a message to this plugin, eg "30s"
# group_threshold:    # Amount of alerts that triggers a message before <group_wait> has expired, eg "10"
# max_retry:          # Number of attempts to relay messages to plugins in case of error
# timeout:            # Time to wait for response from the plugin before considering the attempt a failure, eg "10s"

#-------------------------
# plugin-specific options

# The following template receives a list of models.Alert objects
# The output goes in the http request body

# Replace XXXXXXXXX with your Telegram chat ID
format: |
  {
   "chat_id": "MYID", 
   "text": "
     {{range . -}}  
     {{$alert := . -}}  
     {{range .Decisions -}}
     {{.Value}} will get {{.Type}} for next {{.Duration}} for triggering {{.Scenario}}.
     {{end -}}
     {{end -}}
   ",
   "reply_markup": {
      "inline_keyboard": [
          {{ $arrLength := len . -}}
          {{ range $i, $value := . -}}
          {{ $V := $value.Source.Value -}}
          [
              {
                  "text": "See {{ $V }} on shodan.io",
                  "url": "https://www.shodan.io/host/{{ $V -}}"
              },
              {
                  "text": "See {{ $V }} on crowdsec.net",
                  "url": "https://app.crowdsec.net/cti/{{ $V -}}"
              }
          ]{{if lt $i ( sub $arrLength 1) }},{{end }}
      {{end -}}
      ]
    }
  }

url: https://api.telegram.org/botXXX:YYY/sendMessage # Replace <TELEGRAM_API_KEY> with your API key

method: POST
headers:
  Content-Type: "application/json"

I mounted the file to /etc/crowdsec/notifications/http.yaml and updated my profiles to have:

notifications:
#   - slack_default  # Set the webhook in /etc/crowdsec/notifications/slack.yaml before enabling this.
#   - splunk_default # Set the splunk url and token in /etc/crowdsec/notifications/splunk.yaml before enabling this.
- http_tele   # Set the required http parameters in /etc/crowdsec/notifications/http.yaml before enabling this.
#   - email_default  # Set the required email parameters in /etc/crowdsec/notifications/email.yaml before enabling this.

And restarted the container and it just worked.

ScreN · May 22, 2023, 1:04pm

it’s worked. But… o_O

iiAmLoz · May 22, 2023, 1:05pm

Do you have 2 yaml files with the same http_default name?

Topic		Replies	Views
Telegram notification : plugin not enabled? crowdsec	4	479	August 10, 2023
Telegram Notification not working	6	1313	May 12, 2022
Enable notification service won't start crowdsec	5	430	April 20, 2023
Api key telegram not found crowdsec	2	511	June 22, 2022
Error on email notification plugin crowdsec	11	1315	October 5, 2022

Telegram Notification Bug

Related Topics