Issues in Upgrading CorwdSec to Latest version

Have tried upgrading CrowdSec to the latest version and getting some errors … Please find below errors faced

[11/26/2020:01:40:46 PM][INF] crowdsec_wizard: Backing up existing configuration
WARN[0000] Starting configuration backup                
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/apache2-logs type=parsers
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/http-logs type=parsers
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/sshd-logs type=parsers
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/syslog-logs type=parsers
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/dateparse-enrich type=parsers
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/whitelists type=parsers
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/geoip-enrich type=parsers
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/iptables-logs type=parsers
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/modsecurity type=parsers
INFO[0000] Wrote 9 entries for parsers to /tmp/tmp.8s0zqmPPUl/parsers//upstream-parsers.json  file=crowdsecurity/smb-logs type=parsers
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/cdn-whitelist type=postoverflows
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/rdns type=postoverflows
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/seo-bots-whitelist type=postoverflows
INFO[0000] Wrote 3 entries for postoverflows to /tmp/tmp.8s0zqmPPUl/postoverflows//upstream-postoverflows.json  file=crowdsecurity/seo-bots-whitelist type=postoverflows
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/http-crawl-non_statics type=scenarios
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/http-probing type=scenarios
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/http-sensitive-files type=scenarios
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/http-sqli-probing type=scenarios
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/http-xss-probing type=scenarios
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/http-backdoors-attempts type=scenarios
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/iptables-scan-multi_ports type=scenarios
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/modsecurity type=scenarios
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/ssh-bf type=scenarios
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/http-bad-user-agent type=scenarios
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/http-bf-wordpress_bf type=scenarios
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/http-path-traversal-probing type=scenarios
INFO[0000] Wrote 12 entries for scenarios to /tmp/tmp.8s0zqmPPUl/scenarios//upstream-scenarios.json  file=crowdsecurity/http-path-traversal-probing type=scenarios
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/base-http-scenarios type=collections
INFO[0000] saving, version:0.2, up-to-date:true          file=crowdsecurity/linux type=collections
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/modsecurity type=collections
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/whitelist-good-actors type=collections
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/apache2 type=collections
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/sshd type=collections
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/wordpress type=collections
INFO[0000] saving, version:0.1, up-to-date:true          file=crowdsecurity/iptables type=collections
INFO[0000] Wrote 8 entries for collections to /tmp/tmp.8s0zqmPPUl/collections//upstream-collections.json  file=crowdsecurity/iptables type=collections
INFO[0000] Saved acquis to /tmp/tmp.8s0zqmPPUl/acquis.yaml 
INFO[0000] Saved default yaml to /tmp/tmp.8s0zqmPPUl/default.yaml 
INFO[0000] api load configuration: configuration loaded successfully (base:https://tmsov6x2n9.execute-api.eu-west-1.amazonaws.com/v1/) 
INFO[0000] Saved configuration to /tmp/tmp.8s0zqmPPUl   
[11/26/2020:01:40:46 PM][INF] crowdsec_wizard: Saving default database content
[11/26/2020:01:40:46 PM][INF] crowdsec_wizard: Cleanup existing crowdsec configuration
FATA[0000] failed to read config file: open /etc/crowdsec/config.yaml: no such file or directory 
[11/26/2020:01:40:46 PM][INF] crowdsec_wizard: Removing crowdsec binaries
[11/26/2020:01:40:47 PM][INF] crowdsec_wizard: crowdsec successfully uninstalled
[11/26/2020:01:40:47 PM][INF] crowdsec_wizard: Installing crowdsec
‘./config/local_api_credentials.yaml’ -> ‘/etc/crowdsec/local_api_credentials.yaml’
‘./config/online_api_credentials.yaml’ -> ‘/etc/crowdsec/online_api_credentials.yaml’
‘./config/config.yaml’ -> ‘/etc/crowdsec/config.yaml’
‘./config/prod.yaml’ -> ‘/etc/crowdsec/prod.yaml’
‘./config/dev.yaml’ -> ‘/etc/crowdsec/dev.yaml’
‘./config/acquis.yaml’ -> ‘/etc/crowdsec/acquis.yaml’
‘./config/profiles.yaml’ -> ‘/etc/crowdsec/profiles.yaml’
‘./config/api.yaml’ -> ‘/etc/crowdsec/api.yaml’
‘./config/simulation.yaml’ -> ‘/etc/crowdsec/simulation.yaml’
[11/26/2020:01:40:47 PM][INF] crowdsec_wizard: Installing crowdsec binaries
‘./cmd/crowdsec/crowdsec’ -> ‘/usr/local/bin/crowdsec’
‘./cmd/crowdsec-cli/cscli’ -> ‘/usr/local/bin/cscli’
[11/26/2020:01:40:47 PM][INF] crowdsec_wizard: Restoring configuration
Error: unknown command "update" for "cscli"
Run 'cscli --help' for usage.
FATA[0000] While executing root command : unknown command "update" for "cscli" 
Error: unknown command "backup" for "cscli"
Run 'cscli --help' for usage.
FATA[0000] While executing root command : unknown command "backup" for "cscli" 
[11/26/2020:01:40:47 PM][INF] crowdsec_wizard: Restoring saved database
[11/26/2020:01:40:47 PM][INF] crowdsec_wizard: Finished, restarting
Job for crowdsec.service failed because the control process exited with error code. See "systemctl status crowdsec.service" and "journalctl -xe" for details.
[11/26/2020:01:40:47 PM][ERR] crowdsec_wizard: Failed to restart crowdsec
[root@ip-172-31-72-35 crowdsec-v1.0.0-rc2]# systemctl restart crowdsec
Job for crowdsec.service failed because the control process exited with error code. See "systemctl status crowdsec.service" and "journalctl -xe" for details.
[root@ip-172-31-72-35 crowdsec-v1.0.0-rc2]# journalctl -xe
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit user-0.slice has finished shutting down.
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal polkitd[588]: Registered Authentication Agent for unix-process:21076:60919312 (system bus name :1.21308 [/usr/bin/pkttyagent
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal systemd[1]: Cannot add dependency job for unit rpcbind.socket, ignoring: Unit is masked.
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal systemd[1]: Starting Crowdwatch agent...
-- Subject: Unit crowdsec.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit crowdsec.service has begun starting up.
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: - using env:        export GIN_MODE=release
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: - using code:        gin.SetMode(gin.ReleaseMode)
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] POST   /v1/watchers              --> github.com/crowdsecurity/crowdsec/pkg/apiserver/controller
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] POST   /v1/watchers/login        --> github.com/appleboy/gin-jwt/v2.(*GinJWTMiddleware).LoginHa
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] GET    /v1/refresh_token         --> github.com/appleboy/gin-jwt/v2.(*GinJWTMiddleware).Refresh
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] POST   /v1/alerts                --> github.com/crowdsecurity/crowdsec/pkg/apiserver/controller
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] GET    /v1/alerts                --> github.com/crowdsecurity/crowdsec/pkg/apiserver/controller
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] GET    /v1/alerts/:alert_id      --> github.com/crowdsecurity/crowdsec/pkg/apiserver/controller
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] DELETE /v1/alerts                --> github.com/crowdsecurity/crowdsec/pkg/apiserver/controller
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] DELETE /v1/decisions             --> github.com/crowdsecurity/crowdsec/pkg/apiserver/controller
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] DELETE /v1/decisions/:decision_id --> github.com/crowdsecurity/crowdsec/pkg/apiserver/controlle
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] GET    /v1/decisions             --> github.com/crowdsecurity/crowdsec/pkg/apiserver/controller
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal crowdsec[21082]: [GIN-debug] GET    /v1/decisions/stream      --> github.com/crowdsecurity/crowdsec/pkg/apiserver/controller
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal systemd[1]: crowdsec.service: control process exited, code=exited status=1
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal systemd[1]: Failed to start Crowdwatch agent.
-- Subject: Unit crowdsec.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit crowdsec.service has failed.
-- 
-- The result is failed.
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal systemd[1]: Unit crowdsec.service entered failed state.
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal systemd[1]: crowdsec.service failed.
Nov 26 13:41:27 ip-172-31-72-35.ec2.internal polkitd[588]: Unregistered Authentication Agent for unix-process:21076:60919312 (system bus name :1.21308, object path /org/
[root@ip-172-31-72-35 crowdsec-v1.0.0-rc2]#

Hi @indranilkamulkar,

Could you please tell us from which crowdsec version you tried to upgrade ?
For information, upgrading from v0.3X to v1.X doesn’t work with wizard.sh, you need to do a migration as explained in v1 documentation (this documentation will soon be pushed in official doc).

1 Like

Hi, do you plan to have repo (debian in my case) to be able to install / upgrade crowdsec with system command (apt) ?
Last time I upgraded cs-custom-blocker I had to reconfig the database connection.

Maybe this will be more stable after 1.0 release ?

Hello @Sich

Yes, the crowdsec package for debian is currently in our roadmap.
For bouncers, we didn’t start the packaging for them, but we are currently adding an upgrade.sh scripts so you will be able to upgrade your bouncer without loosing data.

1 Like

upgrade.sh that we can run with some cron is fine for me.
As long as that don’t break config and that I don’t have to recheck all servers each time crowdsec get updated :slight_smile:

We are sorry but you will not be able to use the upgrade.sh scripts in a cron for the moment :confused:
You will have to download the latest release of the bouncer, and run the upgrade.sh script to install the new bouncer binary.

We are currently focus on packaging crowdsec itself, bouncers will come next, stay tuned :slight_smile:

1 Like

The upgrade.sh script is available from cs-firewall-bouncer v0.0.4 and cs-custom-bouncer v0.0.4 :slight_smile:

Note: Those bouncers must be used with crowdsec v1.X

1 Like