Updating Fail2ban and adding new rules

Introduction

The last stable version of fail2ban contained in Raspbian stretch  is 0.9.6-2. However, newer fail2ban versions include useful functionalities, such as the possibility to use a persistent datastore to save the IPs banned. Raspbian buster is expected to include version 0.10.2-2 but this will not happen until 2019. For this reason, I decided to update the software manually. In this short guide you will see how to install the last stable version of fail2ban and how to create new filters.

Updating Fail2ban

This tutorial assumes that fail2ban is already installed and working in a Raspberry running Raspbian Stretch version. Find in this link a basic guide to install and configure it. First of all, uninstall your current fail2ban version by doing:

apt-get purge fail2ban

Do not use option autoremove to avoid deleting orphan packages that are still needed (such as pynotify). Once done, download the last fail2ban release from project Github. As of June 2018, latest release is 0.10.3.1. Use these commands (updating the wget path with the last available and stable release):

mkdir /tmp/fail2ban
cd /tmp/fail2ban

wget https://github.com/fail2ban/fail2ban/archive/0.10.3.1.tar.gz -I /tmp/
tar -xvfj /tmp/0.10.3.1.tar.gz -C /tmp/
cd /tmp/fail2ban-0.11.0
sudo python setup.py install

This will install Fail2Ban into the python library directory. The executable scripts are placed into /usr/bin, and configuration in /etc/fail2ban. To check if everything is correctly installed just type:

fail2ban-client -h

Usage: fail2ban-client [OPTIONS] <COMMAND>
Fail2Ban v0.10.3.fix1 reads log file that contains password failure report and bans the corresponding IP addresses using firewall rules.
...

You should see in the second line of the output that the new version of the software is installed. Before starting the service, there are a few configuration changes that should be done.

Configuring Fail2ban

It is recommended to create local configuration files to introduce customisation. To do so, make a copy by typing:

cp /etc/fail2ban/fail2ban.conf /etc/fail2ban/fail2ban.local
cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

The first file, fail2ban.conf, is the main configuration file. In this case I have only modified the path where I want the database to be placed (dbfile) and the purge age to be used (dbpurgeage). I use this software to protect the access to my domestic server, so I have decided not to purge (never) the IPs banned. This configuration has the disadvantage that, after some time, myiptables will have plenty of entries. I will see in a few months if it slows the Raspberry Pi or not.

The second file, jail.conf, configures the jails to be activated. It has been heavily refactored since release 0.9 so it is worth to review all the options and make customization in the local copy. In my case, I have included in the ignoreip list my local IPs, increased the bantime/ findtime and reduced the maxretry. I only use ssh and some apache jails. Moreover, as explained in this post, I have created a new action to send IFTTT notifications to my mobile phone when a ban is produced.

Note that, in order to activate some jails, you should first configure your apache server to produce the expected logs. For instance, and by default, apache does not log the 404/file not found error. To log them do the following change in /etc/apache2/apache.conf

...
LogLevel notice core:info
...

In my case, I was receiving lots of HTTP Request version 1.0. Since none of my devices uses it, I have created this rewrite rule to send a 404 error to HTTP 1.0 requests.

RewriteEngine On RewriteCond %{THE_REQUEST} HTTP/1\.0$ RewriteRule ^ - [L,R=404]

Creating a service for Fail2ban

Following the python installation will not automatically create a service script. To enable fail2ban as an automatic service, simply copy the script file located in/tmp/fail2ban/fail2ban-0.11.0/files/fail2ban.service.in to/lib/systemd/system and rename it to fail2ban.service. Depending on your system version, you will probably have to do some changes to the service file. In my case, the service file once adapted is:

[Unit]
Description=Fail2Ban Service
Documentation=man:fail2ban(1)
After=network.target iptables.service firewalld.service ip6tables.service ipset.service
PartOf=iptables.service firewalld.service ip6tables.service ipset.service

[Service]
Type=simple
ExecStartPre=/bin/mkdir -p /var/run/fail2ban
ExecStart=/usr/local/bin/fail2ban-server -xf start
# if should be logged in systemd journal, use following line or set logtarget to sysout in fail2ban.local
# ExecStart=/usr/local/bin/fail2ban-server -xf --logtarget=sysout start
ExecStop=/usr/local/bin/fail2ban-client stop
ExecReload=/usr/local/bin/fail2ban-client reload
PIDFile=/var/run/fail2ban/fail2ban.pid
Restart=on-failure
RestartPreventExitStatus=0 255

[Install]
WantedBy=multi-user.target

Test if the service is created properly by starting and stopping it:

sudo systemctl start fail2ban.service
sudo systemctl stop fail2ban.service

Have a look at the logs generated in /var/log/fail2ban.log to ensure that everything is working correctly. Once checked, tell the system to load the service automatically at boot time with this command:

sudo systemctl enable fail2ban.service

To know more about this topic, have a look at the following article. A great source of information with examples regarding systemctl can be found in this page.

Creating a new filter in Fail2ban

I receive every day hundreds of HTTP Requests from all over the world trying to find security vulnerabilities in my server. Some of them are detected by the already activated jails. For instance, the apache-noscript  will block recurring requests searching for scripts that don’t exist. However, some other requests are not captured by any filter. For this reason I have created a new one that will search certain expressions in the access.log file. To do so, just create a new file in folder /etc/fail2ban/filter.d/ with the desired name and include on it the regular expressions searched:

touch /etc/fail2ban/filter.d/apache-block-scan.conf

# Fail2Ban configuration file# Fail2Ban configuration file##[INCLUDES]before = apache-common.conf
[Definition]
failregex = ^<HOST>.*"(GET|POST|HEAD) \/.* HTTP.*" 301 .*$
            ^<HOST>.*"(GET|POST|HEAD) \/.* HTTP.*" 404 .*$
            ^<HOST>.*"POST \/ HTTP.*" 200 .*$

datepattern = ^[^\[]*\[({DATE})
              {^LN-BEG}

ignoreregex =

As can be noticed, this rule will search GET or POST requests with a 301 (Moved Permanently) or 404 (Not Found) error because, as previously commented, I redirect all the unwanted requests to these errors. Moreover, it will consider POST requests to the document root as failed expressions, since there is no post option there.

You can test the rule by using fail2ban-regex utility. For instance, to check if a certain log line will be detected by the regular expressions included in a filter do:

fail2ban-regex '203.171.239.246 - - [19/Jun/2018:13:31:45 +0200] "POST /xw.php HTTP/1.1" 301 573 "-" "Mozilla/5.0"' /etc/fail2ban/filter.d/apache-block-scan.conf

Running tests
=============

Use   failregex filter file : apache-block-scan, basedir: /etc/fail2ban
Use      datepattern : Default Detectors
Use      single line : 203.171.239.246 - - [19/Jun/2018:13:31:45 +0200] "...


Results
=======

Failregex: 1 total
|-  #) [# of hits] regular expression
|   1) [1] ^<HOST>.*"(GET|POST|HEAD) \/.* HTTP.*" 301 .*$
`-

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [1] ^[^\[]*\[(Day(?P<_sep>[-/])MON(?P=_sep)ExYear[ :]?24hour:Minute:Second(?:\.Microseconds)?(?: Zone offset)?)
`-

Lines: 1 lines, 0 ignored, 1 matched, 0 missed
[processed in 0.04 sec]

Once satisfied with the results, and in order to activate the rule, include it in your local jail file:

vi /etc/fail2ban/jail.local

[apache-block-scan]

enabled = true
port = http,https
filter = apache-block-scan
logpath = %(apache_access_log)s
maxretry = 2
findtime = 7d
action = %(action_)s

And finally reload fail2ban's jails to start blocking requests by doing:

sudo service fail2ban reload