Intro

Hi, in this post, we are going to protect our web server against undesired path enumeration. There are several bots out there trying to exploit known vulnerabilities or abuse default configurations.

This post is the continuation of Secure your ssh server with fail2ban. Make sure you have at least fail2ban installed and running.

After I enabled access logs for my web server, I began checking them. There were a lot of funny and scary requests, as you can see in the image. That’s what led me to this solution.

Web server access logs

Creating the fail2ban filter for lighttpd

First of all, since I’m using lighttpd, we need to create a filter for our purpose. This involves telling fail2ban how to analyze these logs, specifically to detect and block strange requests based on 403 and 404 errors. If you’re using Apache or Nginx, this step is not necessary; fail2ban on Debian 12 comes with these filters already defined for them. However, from now on, I will specifically address lighttpd.

Ensure you have the accesslog module enabled.

~$ lighty-enable-mod accesslog

Now, let’s create the filter in /etc/fail2ban/filter.d/lighttpd-botsearch.conf with the following content:

# Fail2Ban filter to match web requests for selected URLs that don't exist
#

[INCLUDES]

# Load regexes for filtering from botsearch-common.conf
before = botsearch-common.conf

[Definition]

failregex = ^<HOST> \S+ - \[[^\]]+\] "(GET|POST|HEAD) \/[^\"]+" (403|404) \d+ ".*" ".*"$

ignoreregex = 

datepattern = ^%%d/%%b/%%Y:%%H:%%M:%%S

# DEV Notes:
# Based on nginx-botsearch filter
#
# Author: Diego Arias

Creating the fail2ban jail for lighttpd

The next step is to create a jail for lighttpd. We will create a new file at /etc/fail2ban/jail.d/lighttpd.conf with the following content:

[lighttpd]
backend = polling
enabled = true
port = http,https
filter = lighttpd-botsearch
logpath = /var/log/lighttpd/access.log
bantime = 3d
findtime = 10m
maxretry = 5

Besides defining the filter, logpath, etc., this rule is basically saying that if a host gets 5 responses with a 403/404 status code within a 10-minute window, block that host for three days.

Restart both services and check that they come up correctly.

~$ systemctl restart lighttpd.service fail2ban.service

~$ systemctl status lighttpd.service fail2ban.service

Check for banned IPs

After some hours, it’s time to check if any IP address has been banned.

~$ fail2ban-client status lighttpd

Lighttpd banned ips

It’s working, awesome!