Crowdsec stopped working : while pushing to api : failed sending alert to LAPI: API error: unable to create alerts: creating alert events: ent: validator failed for field \"serialized\": value is greater than the required length: unable to insert bulk"

crowdsec
1

Hi guys !
2 instances (out of 4) of Crowdsec stopped working in production (see logs below)

time="07-09-2021 13:10:16" level=info msg="(3c1f443a494fb47370e37e34680787eeEuNLMEfCnStMWJnu/crowdsec) crowdsecurity/http-probing by ip 173.212.221.154 (DE) : 4h ban on Ip 173.212.221.154"
time="07-09-2021 13:10:16" level=error msg="while pushing to api : failed sending alert to LAPI: API error: unable to create alerts: creating alert events: ent: validator failed for field \"serialized\": value is greater than the required
 length: unable to insert bulk"
time="07-09-2021 13:10:17" level=info msg="(3c1f443a494fb47370e37e34680787eeEuNLMEfCnStMWJnu/crowdsec) crowdsecurity/http-probing by ip 173.212.221.154 (DE) : 4h ban on Ip 173.212.221.154"
time="07-09-2021 13:10:17" level=error msg="while pushing to api : failed sending alert to LAPI: API error: unable to create alerts: creating alert events: ent: validator failed for field \"serialized\": value is greater than the required
 length: unable to insert bulk"

Those are continous messages in the crowdsec.log

Crowdsec_api.log extract

time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=77.130.108.242 HTTP/1.1 200 2.69644ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=52.12.231.31 HTTP/1.1 200 2.758018ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=46.101.150.34 HTTP/1.1 200 2.694697ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"POST /v1/alerts HTTP/1.1 500 152.480213ms \"crowdsec/v1.1.1-debian-pragmatic-linux-73e0bbaf93070f4a640eb5a22212b5dcf26699de\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=90.31.229.88 HTTP/1.1 200 2.958256ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=35.231.49.189 HTTP/1.1 200 3.926729ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=91.134.184.236 HTTP/1.1 200 3.947829ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=173.249.48.46 HTTP/1.1 200 4.172169ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=41.92.3.33 HTTP/1.1 200 3.101118ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=156.67.218.117 HTTP/1.1 200 2.33337ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:22" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:22 UTC] \"GET /v1/decisions?ip=114.119.137.103 HTTP/1.1 200 2.58566ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:23" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:23 UTC] \"GET /v1/decisions?ip=91.134.184.236 HTTP/1.1 200 2.464307ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:23" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:23 UTC] \"GET /v1/decisions?ip=170.210.156.52 HTTP/1.1 200 2.616362ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:23" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:23 UTC] \"GET /v1/decisions?ip=90.90.190.147 HTTP/1.1 200 33.011672ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:23" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:23 UTC] \"GET /v1/decisions?ip=213.222.112.184 HTTP/1.1 200 3.010775ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:23" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:23 UTC] \"GET /v1/decisions?ip=146.148.61.110 HTTP/1.1 200 2.873714ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:23" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:23 UTC] \"GET /v1/decisions?ip=92.184.108.117 HTTP/1.1 200 2.5572ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:23" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:23 UTC] \"GET /v1/decisions?ip=52.162.252.179 HTTP/1.1 200 2.755405ms \"crowdsec-nginx-bouncer/v0.0.6\" \""
time="07-09-2021 13:22:23" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:23 UTC] \"POST /v1/alerts HTTP/1.1 500 256.275565ms \"crowdsec/v1.1.1-debian-pragmatic-linux-73e0bbaf93070f4a640eb5a22212b5dcf26699de\" \""
time="07-09-2021 13:22:23" level=info msg="127.0.0.1 - [Tue, 07 Sep 2021 13:22:23 UTC] \"GET /v1/decisions?ip=143.198.229.94 HTTP/1.1 200 2.686136ms \"crowdsec-nginx-bouncer/v0.0.6\" \""

Had to restart instances to make crowdsec work again :confused:

Crowdsec 1.1.1 on Ubuntu 20.04.3 LTS

2

Hello,

This issue has already been fixed on our end (https://github.com/crowdsecurity/crowdsec/pull/877), and will be included with the 1.2 release.

3

Thanks !
same problem again, prod instances were crashed. Can you tell me when 1.1.2 will be released ?

4

Hello,

We have just released the v1.2.0, it will be available very soon in our repositories.

5

1.2.0 is now available in repositories :slight_smile:

1 Like