I’ve gone through all the unparsed issues noted in this forum, but nothing so far has fixed an issue which I’m particularly concerned about. VSFTPD is not being parsed. Of course, I’ve installed gettext and restarted crowdsec service. It’s been running for a few weeks now. Not sure if it’s a permissions thing or what. Any help would be appreciated. Release: 1.0.4
INFO[0000] Buckets Metrics:
+-------------------------------------------+---------------+-----------+--------------+--------+---------+
| BUCKET | CURRENT COUNT | OVERFLOWS | INSTANCIATED | POURED | EXPIRED |
+-------------------------------------------+---------------+-----------+--------------+--------+---------+
| crowdsecurity/http-bad-user-agent | - | 3 | 79 | 83 | 76 |
| crowdsecurity/http-crawl-non_statics | 1 | - | 705 | 738 | 704 |
| crowdsecurity/http-path-traversal-probing | - | - | 3 | 3 | 3 |
| crowdsecurity/http-probing | - | - | 183 | 190 | 183 |
| crowdsecurity/http-sensitive-files | - | - | 4 | 4 | 4 |
+-------------------------------------------+---------------+-----------+--------------+--------+---------+
INFO[0000] Acquisition Metrics:
+-----------------------------+------------+--------------+----------------+------------------------+
| SOURCE | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+-----------------------------+------------+--------------+----------------+------------------------+
| /var/log/apache2/access.log | 1022 | 1022 | - | 1018 |
| /var/log/apache2/error.log | 24 | - | 24 | - |
| /var/log/auth.log | 9839 | - | 9839 | - |
| /var/log/kern.log | 146 | - | 146 | - |
| /var/log/syslog | 31369 | - | 31369 | - |
+-----------------------------+------------+--------------+----------------+------------------------+
INFO[0000] Parser Metrics:
+----------------------------------+-------+--------+----------+
| PARSERS | HITS | PARSED | UNPARSED |
+----------------------------------+-------+--------+----------+
| child-crowdsecurity/apache2-logs | 1070 | 1022 | 48 |
| child-crowdsecurity/http-logs | 3066 | 1462 | 1604 |
| child-crowdsecurity/sshd-logs | 1025 | - | 1025 |
| crowdsecurity/apache2-logs | 1046 | 1022 | 24 |
| crowdsecurity/cdn-whitelist | 3 | 3 | - |
| crowdsecurity/dateparse-enrich | 1022 | 1022 | - |
| crowdsecurity/geoip-enrich | 1022 | 1022 | - |
| crowdsecurity/http-logs | 1022 | 417 | 605 |
| crowdsecurity/iptables-logs | 293 | - | 293 |
| crowdsecurity/non-syslog | 1046 | 1046 | - |
| crowdsecurity/rdns | 3 | 3 | - |
| crowdsecurity/seo-bots-whitelist | 3 | 3 | - |
| crowdsecurity/sshd-logs | 205 | - | 205 |
| crowdsecurity/syslog-logs | 41354 | 41354 | - |
| crowdsecurity/whitelists | 1022 | 1022 | - |
| vsftpd-logs | 916 | - | 916 |
+----------------------------------+-------+--------+----------+
INFO[0000] Local Api Metrics:
+--------------------+--------+------+
| ROUTE | METHOD | HITS |
+--------------------+--------+------+
| /v1/alerts | GET | 5 |
| /v1/alerts | POST | 3 |
| /v1/alerts/146 | GET | 1 |
| /v1/watchers/login | POST | 11 |
+--------------------+--------+------+
INFO[0000] Local Api Machines Metrics:
+----------------------------------+----------------+--------+------+
| MACHINE | ROUTE | METHOD | HITS |
+----------------------------------+----------------+--------+------+
| 173ac883192e107fc2f5f1e6586d8251 | /v1/alerts | POST | 3 |
| 173ac883192e107fc2f5f1e6586d8251 | /v1/alerts | GET | 5 |
| 173ac883192e107fc2f5f1e6586d8251 | /v1/alerts/146 | GET | 1 |
+----------------------------------+----------------+--------+------+
Hello,
From your config / metrics, it seems that you have a “custom” vsftpd log parser, and that it doesn’t succeed to parse any of the 916 lines of logs that were parsed (see the line with vsftpd-logs in your parser metrics).
Would you mind sharing some logs and/or the parser so we can see what’s going on ? 
Cheers,
Sure, but which logs do you want me to post. Seems like just posting vsftp logs wouldn’t provide any pertinent info. Also I definitely don’t want to post my vsftp logs here.
As a parser… not sure what you want me to share…
Hello,
What I meant is that, given that there are already some tests for vsftpd logs, this parser is assumed to work. Given this, there are at least two suspects :
- [1] you modified your parser (and doesn’t work anymore ?)
- [2] your logs are not standard
To check [1] maybe you should check the output of cscli parsers list : it will tell you if the parser is up-to-date (but I guess not) and will point you to the path of the parser’s yaml configuration file.
For [2], if you don’t want to share a log sample (that would allow to ensure that your logs are parsed by the upstream parser), you should check your config to tell if you’re using a custom format ?
I would advise trying to install the upstream parser if yours differs. You can find it here if you want to check the diffs with yours.
I have no problem sending the ftp logs to you directly, but I don’t want to publish them on a public server.
Testing to see if responding directly to your email goes anywhere.
cscli parsers list