How GeoIP databases (GeoLite2-{City,ASN}) are updated?

[g00g1]

I am curious whether my CrowdSec instance uses outdated GeoIP databases. Does CrowdSec pulls it daily/weekly from hub-data.crowdsec.net? If so, does this website update those files? If not, how can I update mmdb without reloading/restarting CrowdSec instance? I don’t want to delete my buckets due to this frequent updates.

[iiAmLoz]

Currently we provide monthly updates via the CDN, we are having internal talk about making this shorter EG: bi-weekly, however, there a massive cost implication on our end.

You can setup to download the mmdb files directly from MaxMind to the /var/lib/crowdsec/data folder but they must be named the same as our files. Once these are downloaded you can run systemctl reload crowdsec this will dump all current buckets to temporary files and read them back into memory once the reload is complete. However, this will mean in the meantime whilst crowdsec is reloading it will miss any new log entries.

[g00g1]

@iiAmLoz

You can setup to download the mmdb files directly from MaxMind to the /var/lib/crowdsec/data folder but they must be named the same as our files.

If following this path, should I be concerned that CrowdSec overwrites my own mmdb with ones downloaded from CrowdSec own CDN? If yes, can you please give some insights on how can I prevent this in CrowdSec-native way? I mean I could do chattr +i, but this hack is pretty awful IMO and I would prefer not to use this workaround if there are any better alternatives.

@iiAmLoz

Once these are downloaded you can run systemctl reload crowdsec this will dump all current buckets to temporary files and read them back into memory once the reload is complete.

Oh, so CrowdSec actually does not loose buckets? I am not sure if this works on my CrowdSec instance setup. I have prometheus enabled and collect metrics exposed by CrowdSec. Whenever I reload crowdsec.service, I see that sum(cs_buckets) metric is reset to zero. Is this a bug or this feature not working perfectly for me?

@iiAmLoz

However, this will mean in the meantime whilst crowdsec is reloading it will miss any new log entries.

Sad to hear but I guess it is not very critical if reload takes a few seconds.

[g00g1]

@iiAmLoz

Once these are downloaded you can run systemctl reload crowdsec this will dump all current buckets to temporary files and read them back into memory once the reload is complete. However, this will mean in the meantime whilst crowdsec is reloading it will miss any new log entries.

I have checked the source code and this is not true - var tmpFile string is never set with a value in reloadHandler unlike at debugHandler