A CrowdSec Primer: A Modern Replacement for Fail2Ban

If you’ve been around security for a while you’ve probably used—or are still using—Fail2Ban. It reminds me of Nikto or Netcat in a way in that it’ll always have a place of respect in my heart, but while Nikto and Netcat were supplanted in many ways by other tools, Fail2Ban still seems like the best tool out there for managing abusive users on services like SSH.

With CrowdSec that appears to be changing.

Introduction

CrowdSec is more than just a Fail2Ban replacement though, and makes it a bit confusing. It calls itself:

Their team reached out to me after I featured them on Unsupervised Learning.

An open-source and crowd-powered software enabling you to detect & block attacks.

…which is definite in the Fail2Ban space, but when I had a call with their team they were thinking much bigger than that. Here are some of their described features:

From their site…

allows you to detect attacks and respond at all required levels (detect where your logs are, block at CDN or application level)
is easy to install and maintain with no technical requirement. The installer even comes with a wizard duh!
is designed to be integrated with other solutions and components (ie. use CrowdSec to read your mod_security logs and automatically block attackers at your CDN level)
is about sharing : meta-data about the attack/attacker you detect is sent to a central API, and malevolent IPs are shared with all users.
is a lightweight : it runs standalone, doesn’t require much ram or CPU
can work with cold logs: you can run it on “cold” logs and see what could have happened
comes with out of the box dashboards, because we know visualisation is key

Some other features I liked were:

They’re a French company, so privacy is a huge focus. According to them, the only three things that leave your box are: a timestamp, the offending IP, and the policy they violated, and
Their tool is built on Go.


Download and install

You can install this and be up and running in like two minutes.

Download

git clone https://github.com/crowdsecurity/crow...

Install

sudo ./wizard.sh -i

As the GIF shows above, it basically walks you through a wizard to select what types of daemons/logs you want to monitor—which it also auto-detects—and you’re done.

Components

The system consists of three main components:

The CrowdSec Service, which is basically the persistent service that monitors logs, tracks attacks, etc.
The Command Line Tool, which is the cli interface for interacting with the service.
Bouncers, which are the integrations with other tools that allow actions to take place.

You can also do configuration by editing files, which we’ll talk about later.

So basically the service does all the monitoring, the cscli tool is how you do configuration, ban stuff, get metrics, etc., and the bouncers are how the system interacts with other tools to actually do things, like blocking someone in SSH or Cloudflare, etc.

Usage

So there are a few different ways you can interact with the tool. My favorite is through the cscli command, which has these options:

cscli metrics

This gives you the primary outputs of what’s been seen and acted upon within the system.

cscli metrics

The output of the ‘cscli metrics’ command

This shows you all kinds of stuff, like the number of attacks within the various collections (modules), such as—in this box’s case:

http-backdoors-attempts
http-bad-user-agent
http-crawl-non_statistics
http-path-tranversal-probing
http-probing
http-sensitive-files
http-xss-probing
ssh-bf
ssh-bf-user-enum

You also get to see which logs are being looked at, plus the metrics on how those logs were parsed.

cscli ban list

This command shows you the current contents of who’s been banned, along with what got them banned.

A look at a hosts current banned IPs

This shows you things like the IP that got banned, the number of events that were seen from them, the number of times they’ve been banned, the country they came from, as well as the IP their IP belongs to.

Configuration

In addition to the cscli method of making changes to configuration, you can also do so old-school Linux style. The main config for editing scenarios, for example, is in:

vi /etc/crowdsec/config/profiles.yaml

Editing the YAML config files for detection scenarios

The bad thing is also that it’s YAML.

The good thing about this is it’s just YAML. And you get to create your own scenarios as well.

An example of a custom configuration.

Integrations

What makes this tool more like a platform than a utility is its numerous integrations with other tools.

Some of the integrations you can do with other technologies

So the system doesn’t just detect attacks using its view into your logs, it can also trigger various actions once something is detected, such as:

Blocking people in Cloudflare
Running your own arbitrary scripts
Executing a block in netfilter/iptables
Denying an IP in Nginx
Blocking in WordPress
Etc.

This tells me they’re thinking big and long-term with this thing, and not just as a replacement for a local banning tool.

Summary

So the bottom line—at the very miniumum—is that we seem to have a modern replacement for Fail2Ban, and over time that may turn into something more.

Notes


Oct 9, 2020 — Because I mentioned them in my newsletter, and I’m super enthused about them, I’m pursuing a sponsorship engagement with them for Unsupervised Learning. But I don’t really do sponsorships like other places. I consider it more like “bi-directional positivity”, i.e., I’m going to say what I’m going to say about them regardless, and if I can get them to support the show then that’s great. They get zero creative control, and I don’t even promise to mention them any number of times, or in any format. That is 100% up to me, based on what I think is the best experience for readers and listeners. It’s completely up to me if and how I choose to raise them in the show, and they get no input into what’s said. I feel strongly that this is the only type of paid product discovery that works well with a direct-support monetization model, and with a show that’s free from outside influences. You can read more about my approach here. More
The other thing about this whole sponsorship/money business is that I tend to feature a product for around 6-12 months, so if you’re reading this after 2021 they likely won’t be a sponsor anymore.

—

If you get value from this content, you can support it directly by becoming a member. Being a member gets you access to the newsletter every week instead of just twice a month, access to the UL Slack Channel, the UL Book Club, the UL Archives, and access to future member-only content.