Captcha that does not work

1

Hello to all,

I’m coming to see you because I just discovered crowdsec and it looks really great! I can’t wait to use it! Currently however, despite the fact that everything seems to be correct on paper, I realize that despite the feedback from cscli which tells me that my ip was correctly blocked via captcha, I can still access my site without any problems, and I have on the other side a vm with kali linux which also has the same ip and nekto continues to run without encountering apparent problems …

There must be a subtlety I didn’t make.

Here is the feedback from cscli decisions and cscli collections list

±----±---------±------------------±-------------------------------------±--------±--------±—±-------±-------------------±---------+
| ID | SOURCE | SCOPE:VALUE | REASON | ACTION | COUNTRY | AS | EVENTS | EXPIRATION | ALERT ID |
±----±---------±------------------±-------------------------------------±--------±--------±—±-------±-------------------±---------+
| 501 | crowdsec | Ip:xxxx.xx.xx | crowdsecurity/http-crawl-non_statics | captcha | 55 | 3h59m44.326934702s | 61 |
±----±---------±------------------±-------------------------------------±--------±--------±—±-------±-------------------±---------+
root:/home/arawaks/cs-php-bouncer#

NFO[10-03-2022 10:37:11 AM] Ignoring file /etc/crowdsec/parsers/s02-enrich/geoip-enrich.yaml of type parsers

NAME :package: STATUS VERSION LOCAL PATH

crowdsecurity/modsecurity :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/modsecurity.yaml
crowdsecurity/naxsi :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/naxsi.yaml
crowdsecurity/postfix :heavy_check_mark: enabled 0.2 /etc/crowdsec/collections/postfix.yaml
crowdsecurity/sshd :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/sshd.yaml
crowdsecurity/base-http-scenarios :heavy_check_mark: enabled 0.3 /etc/crowdsec/collections/base-http-scenarios.yaml
crowdsecurity/iptables :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/iptables.yaml
crowdsecurity/nginx :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/nginx.yaml
crowdsecurity/whitelist-good-actors :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/whitelist-good-actors.yaml
crowdsecurity/wordpress :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/wordpress.yaml
crowdsecurity/linux :heavy_check_mark: enabled 0.2 /etc/crowdsec/collections/linux.yaml
crowdsecurity/mysql :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/mysql.yaml
crowdsecurity/vsftpd :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/vsftpd.yaml
crowdsecurity/apache2 :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/apache2.yaml
crowdsecurity/dovecot :heavy_check_mark: enabled 0.1 /etc/crowdsec/collections/dovecot.yaml

Another question while I think about it, I have 2 subdomains, will the security policies applied by crowdsec be also applied to the subdomains ?

I’m currently on debian 11, if you want more information I’m at your disposal, thanks in advance to all. And thank you for this really great software!

2

Hello @arawaks,

Welcome to the community!

The CrowdSec agent does only the detection part.
To have a remediation (ban, captcha …) you must install a bouncer.
Currently, only few bouncers support the captcha remediation:

  • The wordpress bouncer
  • The cs-php-bouncer (mostly for apache)
  • Soon the NGINX bouncer

For your other question, CrowdSec will trigger alert on the logs it reads, so i think yes :slight_smile:

3

Thanks for your answer :slight_smile: I have already installed cs-php-bouncer, and I just saw that this bouncer has not interrogated the api since this morning, which could explain why it does not block anything.

4

Yes indeed. Do you need help for this problem or did you manage to solve this ?

5

I need help please, because even in the logs I do not see what can cause this problem

6

Can you give me the step that you did to install the bouncer please?

7

I relied on this PHP Bouncer | CrowdSec

On the other hand, I forgot to tell you something that I think is important, impossible to install crowdsec-firewall-bouncer-iptables, I do not know if it is related, maybe not at all, but when I tried to install it I have an error with dpkg and the installation is not done, ditto if I try to remove the few packages that are broken and that are installed

At the level of the active boucer here is what I have

NAME IP ADDRESS VALID LAST API PULL TYPE VERSION

crowdsec-php-bouncer-fGK86f5Y :heavy_check_mark: 2022-03-10T08:45:27Z
crowdsec-php-bouncer-9eHZqI64 :heavy_check_mark: 2022-03-10T11:14:43Z
FirewallBouncer-1646934798 :heavy_check_mark: 2022-03-10T17:53:18Z
CustomBouncer-1646934864 :heavy_check_mark: 2022-03-10T17:54:25Z
FirewallBouncer-1646934865 :heavy_check_mark: 2022-03-10T17:54:25Z

And one last thing, the whole thing is installed on a vps on which I have a backup, so if you feel that it may be necessary to start from 0 it is possible, as I had made a backup just before the crowdsec installation.

Even if in principle it could be good to try to fix the mistakes I made here so that I try to understand them, I let you decide what you think is best.

8

Hello @arawaks ,

Before investigating, what is your webserver?

9

Hello,

I am running Apache 2.4 :slight_smile:

I managed to fix the broken packages problem by running a secondary installation method that was available in the documentation. However I still can’t run cs-firewall-bouncer for example, it refuses to run without me being able to identify in the logs the reason of this refusal.

I guess it could be related to my original problem, what’s strange is that when I redid the installation this morning, I followed (again the documentation to the letter) despite that impossible to make a ban effective, or even a captcha.

10

Hello,

For the PHP bouncer, do you still have the output of ./install.sh --apache ? Did you run the chown command mentionned ? (cf. PHP Bouncer | CrowdSec)

Can you please paste the output of the cs-firewall-bouncer installation please ? And the content of the log file ?

11

The output tells me that at priopri everything went well, I have the message that appears telling me that the bouncer was correctly installed. I had not paid attention to the fact that apparently it does not find a folder that was supposed to be called vendor.

cp: cannot stat ‘vendor/’: No such file or directory

crowdsec-php-bouncer installed successfully!

Please set the owner of ‘/usr/local/php/crowdsec/’ to www-data or to your webserver owner.

You can do it with:

sudo chown www-data /usr/local/php/crowdsec/

Add the “php_value auto_prepend_file ‘/usr/local/php/crowdsec/crowdsec-php-bouncer.php’” to your .htacess file.

And reload your webserver.

As for the rights, yes I have put it.

And as for the journal, which one do you mean exactly? When I did a logctl I came across this:

crowdsec-firewall-bouncer[19288]: time=“11-03-2022 15:59:45” level=fatal msg=“Get localhost:8080/v1/decisions/stream?startup=true: dial tcp [::1]:8080: connect: connection refused”
Mar 11 15:59:45 aseaction systemd[1]: crowdsec-firewall-bouncer.service: Main process exited, code=exited, status=1/FAILURE
Mar 11 15:59:45 aseaction systemd[1]: crowdsec-firewall-bouncer.service: Failed with result ‘exit-code’.
Mar 11 15:59:45 aseaction systemd[1]: Failed to start The firewall bouncer for CrowdSec.

It seems that there is a connection problem, for an unknown reason, and I just looked at the ip I’m currently using was not blocked by crowdsec during previous tests. And I have no active decisions

I also had ufw installed, but I took care to deactivate it, so it’s not blocking anything

12

For the PHP bouncer, did you installed composer ? if the vendor directory doesn’t exist it means that the composer install failed. And this might be the reason of the fail.

For the firewall bouncer, it means that it can’t connect to [::1]:8080.
Can you paste the configuration of your firewall bouncer please? (don’t forget to hide senstive information)

13

Okay, I will try to reinstall compose then, do you know what could have caused this bad installation? Because I followed the documentation but it’s strange .

Here is what is in my configuration file.

mode: iptables
pid_dir: /var/run/
update_frequency: 10s
daemonize: true
log_mode: file
log_dir: /var/log/
log_level: info
api_url: http://localhost:8080/
api_key: xxxxxxxxx
disable_ipv6: false
deny_action: DROP
deny_log: false
supported_decisions_types:

  • ban
    #to change log prefix
    #deny_log_prefix: "crowdsec: "
    #to change the blacklists name
    #blacklists_ipv4: crowdsec-blacklists
    #blacklists_ipv6: crowdsec6-blacklists
    #if present, insert rule in those chains
    iptables_chains:
  • INPUT

- FORWARD

- DOCKER-USER

14

No :confused: maybe you run the install script as root whereas composer is not available as root ?

For the api_url , can you try to put http://127.0.0.1:8080 and retry please?

15

No, no I installed the bouncer as a normal user.

And no, unfortunately it still doesn’t work.

crowdsec-firewall-bouncer[19834]: time=“2022-03-11T17:08:35Z” level=info msg=“crowdsec-firewall-bouncer v0.0.22-668797c45c6fa8e0a59d48cb75fb6b61fa535734”
Mar 11 17:08:37 aseaction.crowdsec-firewall-bouncer[19834] : time=“11-03-2022 17:08:37” level=fatal msg=“Get http://localhost:8080/v1/decisions/stream?startup=true : dial tcp [::1]:8080 : connect : connection refused”
Mar 11 17:08:37 aseaction systemd[1] : crowdsec-firewall-bouncer.service : Main process quit, code=exited, status=1/FAILURE
Mar 11 17:08:37 aseaction systemd[1] : crowdsec-firewall-bouncer.service : Failed with result ‘exit-code’.
Tue 11 17:08:37 aseaction systemd[1]: Failed to start the firewall bouncer service for CrowdSec.

16

For the PHP bouncer, what is output of composer install if you run it directly ?

For the firewall bouncer, i don’t get why your bouncer try to request [::1]:8080 when you specified 127.0.0.1:8080.

Does your crowdsec local api is running ? can you run a cscli decisions list for example to see if cscli can interact correctly with LAPI.

17

I just reinstalled composer, hopefully correctly this time, it tells me this when I give it the command

Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Nothing to install, update or remove
Generating autoload files
16 packages you are using are looking for funding.
Use the composer fund command to find out more!

I think I found out why, in my host file it said to listen in the wrong place, that was the basic configuration.

Anyway, after commenting the problematic line so that it doesn’t listen where it shouldn’t, the error message changes.

wall-bouncer[20530]: time=“11-03-2022 17:30:40” level=fatal msg=“API error: access forbidden”
Mar 11 17:30:40 aseaction systemd[1]: crowdsec-firewall-bouncer.service: Main process exited, code=exited, status=1/FAILURE
Mar 11 17:30:40 aseaction systemd[1]: crowdsec-firewall-bouncer.service: Failed with result ‘exit-code’.
Mar 11 17:30:40 aseaction systemd[1]: Failed to start The firewall bouncer for CrowdSec.
lines 155078-155134/155134 (END)

And here is for cscli decisions list

cscli decisions list
No active decisions

EDIT: I managed to start the firewall, the problem apparently came from the api among other things.

Sources : Crowdsec packages for OpenWrt - #38 by jmarcet - For Developers - OpenWrt Forum

● crowdsec-firewall-bouncer.service - The firewall bouncer for CrowdSec
     Loaded: loaded (/etc/systemd/system/crowdsec-firewall-bouncer.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2022-03-12 07:45:02 UTC; 5min ago
    Process: 24776 ExecStartPre=/usr/local/bin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml -t (code=exited, status=0/SUCCESS)
    Process: 24799 ExecStartPost=/bin/sleep 0.1 (code=exited, status=0/SUCCESS)
   Main PID: 24781 (crowdsec-firewa)
      Tasks: 8 (limit: 2302)
     Memory: 12.6M
        CPU: 12.255s
     CGroup: /system.slice/crowdsec-firewall-bouncer.service
             └─24781 /usr/local/bin/crowdsec-firewall-bouncer -c /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

I’ll see if it solved the initial problem and if the ban works properly, I’ll keep you informed

Good news! The ban works! On the other hand impossible to make the captcha work! I post you my configuration file, knowing that I also go on this site, Protect your PHP websites with CrowdSec - The open-source & collaborative IPS but without success, crowdsec refuse to restart if I add it to my file as it is

name : crawler_captcha_remediation
filters :
  - Alert.Remediation == true && Alert.GetScenario() in ["crowdsecurity/http-crawl-non_statics", "crowdsecurity/http-bad-user-agent"]
decisions:
  - type: captcha
    duration : 4h
on_success : break
---


name : default_ip_remediation
debug : true
filters:
 - Alert.Remediation == true && Alert.GetScope() == "Ip
decisions :
 - type : ban
 - type : captcha
   duration : 4h
# notifications :
# - slack_default # Define the webhook in /etc/crowdsec/notifications/slack.yaml before enabling it.
# - splunk_default # Define the url and the splunk token in /etc/crowdsec/notifications/splunk.yaml before enabling it.
# - http_default # Define the required http parameters in /etc/crowdsec/notifications/http.yaml before enabling it.
# - email_default # Set the required http parameters in /etc/crowdsec/notifications/email.yaml before enabling this.
on_success : break

+--------+----------+-------------------+----------------------------------------------------+---------+---------+----------------------------+--------+--------------------+----------+
|   ID   |  SOURCE  |    SCOPE:VALUE    |                       REASON                       | ACTION  | COUNTRY |             AS             | EVENTS |     EXPIRATION     | ALERT ID |
+--------+----------+-------------------+----------------------------------------------------+---------+---------+----------------------------+--------+--------------------+----------+
| 126336 | cscli    | Ip:xx.xx.xx.xx    | manual 'captcha' from                              | captcha |         |                            |      1 | 3h56m28.45019278s  |      183 |
|        |          |                   | '37b2622f83ae4143a7167b3796444bfawFs3SJkaT0B6sRvg' |         |         |

It’s ok, I found where my error came from, I had to reinstall cs-php to make it work. I put the post in resolution, thanks to you @alteredCoder for the time you gave me.

18

Great that you manage to solve it :slight_smile:

19

@alteredCoder So unfortunately yes and no, I have self-sabotaged I think! :joy:

I take the liberty of reopening the post, I have reinstalled everything in the meantime because following the multiple problems I had I wanted to understand if these errors were caused by me or not.

Everything works fine and I didn’t encounter any problems when launching the firewall, so I think that the installation I have now is more stable than the other one.

Nevertheless, I still have problems with the php bouncer, it doesn’t download correctly and the vendor folder doesn’t install… This is problematic and I would like to know if I am the only one to have this problem ? If not, I think it would be nice if the information could be sent back.

Fortunately, I still have my old installation that I can mount on disk at any time, so I was able to recover the cs-php installation folder that I had and that works correctly with the famous vendor folder !

Unfortunately, I’m back to my basic problem. The captcha is not displayed, but when I simulate an attack, it is correctly blocked by the firewall with the captcha as decision.

| 138805 | crowdsec | Ip:185.xxx.xxx.xxx | crowdsecurity/http-crawl-non_statics | captcha | CH | 59898 AllSafe Sarl | 45 | 1h0m40.867003526s | 41 |

A more or less similar problem seems to have been encountered by a user IP got blocked by crowdsec instead of captcha · Issue #2 · crowdsecurity/cs-php-bouncer · GitHub

The logs don’t seem to show any particular error (but if needed I’ll send it to you). I also commented on the network interface which could be a problem but the listening error doesn’t appear in the logs anyway.

I also uninstalled / reinstalled several times the cs-php, I made sure that the rights for apache were correctly assigned and I added the line that it is recommended to add in the htaccess by adding an allowoverride my vhost file to be sure that it reads it.

For the moment still nothing, and obviously I make sure that the cache of the navigators do not mislead me. So now I confess I don’t know what to do and why this bouncer (at least with me) seems to cause so much trouble.

Here is a result of my last attack simulation, we see below that the firewall blocks it very quickly, in a case like this one I know that the ban pass in priority, so it seems normal not to see captcha, but even when I put in captcha manually I have nothing. As the feedback below attests.

| 161915 | cscli | Ip:185.159.157.19 | manual ‘captcha’ from | captcha | 1 | 3h59m56.088731087s | 51 |

If you have any suggestions or ideas, I’d love to hear from you!

Oh and last thing, I have installed the necessary dependencies to make the bouncer work via composer as indicated in this link:

And I have just tried to reprodure the whole thing with a vm and I have again while trying to install the bouncer via git this message :

impossible to evaluate ‘vendor/’: No file or folder of this type

20

Hello @arawaks ,

Does your firewall bouncer is installed on the same machine than the php bouncer?