The container starts up and creates the config.yaml and online_api_credentials.yaml files and then exits with fatal errors. The config file it creates seems to be incomplete. If I create config.yaml from the GitHub example, it then errors out with other missing files and folders. The container is not creating all of the required files and folders it needs on first start to run properly. I do not see /etc/config/ exposed as a volume in the Dockerfile. I downloaded the Dockerfile and tried to build it locally, but it errors out so I was not able to test building it myself and adding that to see if it resolves the issue. Is there something I am missing?
As explained on gitter, the issue is this volume mount "${docker}/crowdsec/config:/etc/crowdsec/". It’s remove all the files present in /etc/crowdsec/ container folder.
What you need to do is to mount the config file directly :
I just made a working example with Caddy and example app, here is the docker compose file
version: '3'
services:
#the application itself : static html served by apache2.
#the html can be found in ./app/
app:
image: httpd:alpine
restart: always
volumes:
- ./app/:/usr/local/apache2/htdocs/
networks:
crowdsec_test:
ipv4_address: 172.20.0.2
#the reverse proxy that will serve the application
caddy:
image: caddy
container_name: caddy
ports:
- '8000:80'
links:
- app
volumes:
- ./caddy/Caddyfile:/etc/caddy/Caddyfile
- logs:/var/log/caddy
networks:
crowdsec_test:
ipv4_address: 172.20.0.3
#crowdsec : it will be fed caddy's logs
#and later we're going to plug a firewall bouncer to it
crowdsec:
image: crowdsecurity/crowdsec:latest
restart: always
environment:
#this is the list of collections we want to install
#https://hub.crowdsec.net/author/crowdsecurity/collections/caddy
COLLECTIONS: "crowdsecurity/caddy"
GID: "${GID-1000}"
DISABLE_ONLINE_API: "true"
depends_on:
- 'caddy'
volumes:
- ./crowdsec/acquis.yaml:/etc/crowdsec/acquis.yaml
- logs:/var/log/caddy
- crowdsec-db:/var/lib/crowdsec/data/
- crowdsec-config:/etc/crowdsec/
networks:
crowdsec_test:
ipv4_address: 172.20.0.4
#metabase, because security is cool, but dashboards are cooler
dashboard:
#we're using a custom Dockerfile so that metabase pops with pre-configured dashboards
build: ./crowdsec/dashboard
restart: always
ports:
- 3000:3000
environment:
MB_DB_FILE: /data/metabase.db
MGID: "${GID-1000}"
depends_on:
- 'crowdsec'
volumes:
- crowdsec-db:/metabase-data/
networks:
crowdsec_test:
ipv4_address: 172.20.0.5
volumes:
logs:
crowdsec-db:
crowdsec-config:
networks:
crowdsec_test:
ipam:
driver: default
config:
- subnet: 172.20.0.0/24
Using Docker compose, if you specify an empty folder for an empty volume, it will mount properly the folder without overwriting it and so in our case make the crowdsec config folder persistent.
Ps: I used DISABLE_ONLINE_API: "true" to avoid registering to the Central API because it a testing environment. Avoid using it on prod environment so you will share your signals to the community and have community signals back.
The volume mount only needs to be empty with Docker for Windows. The documentation also states for a volume to be persistent for portability it needs to be declared in the Dockerfile.
The current Dockerfile does not have a volume declaration. I believe this is what the issue is. The container is not setup to use a volume for /etc/crowdsec.
I tried to build the current Dockerfile as is from Github with no modifications and it fails out with the error below. I was going to add a volume to the Dockerfile to test, but the current Dockerfile does not build.
crowdsec-/wizard.sh
Removing intermediate container 425b0ea1c4dd
---> 01ffced50dec
Step 7/21 : RUN cd crowdsec-v* && ./wizard.sh --docker-mode && cd -
---> Running in 1231d6185cf5
/bin/sh: cd: line 1: can't cd to crowdsec-v*: No such file or directory
The command '/bin/sh -c cd crowdsec-v* && ./wizard.sh --docker-mode && cd -' returned a non-zero code: 2
ERROR: Service 'crowdsec' failed to build : Build failed
@samcro1967 The build didn’t went well because there is no .git repo, it seems you didn’t use git to clone the repository.
/go/src/crowdsec/platform/linux.mk:5: Building for linux
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
@he2ss You are correct. Looking back at my history I did a wget instead of a git clone. I am now able to build locally. Adding a volume declaration to the Dockerfile had no effect.
I restarted Docker daemon as well with no effect. I spun up another Ubuntu 20.04 VM and installed docker and get the same results. I am at a complete loss here. This is the only container I have encountered that does not copy required files to a mounted volume when it is first spun up. I just spun up adguardhome-sync this morning and it created the required files in the mounted volume.
I will abandon docker for this and look to move to bare metal since the docker image does not appear to function correctly.
@samcro1967 I confirm that the docker image works properly, as we already have users using it and even us internally.
As explained in the first reply of this post, from what I understood, the issue is that the target volume /etc/crowdsec/ is overwritten and crowdsec isn’t able to retreive it own configuration files that are on the image.
