Ticket #833 (assigned task)

Opened 8 years ago

Last modified 7 years ago

Intrusion Detection System

Reported by: lmacken Owned by: lmacken
Priority: major Milestone:
Component: Security Version:
Severity: High Keywords:
Cc: Blocked By:
Blocking: Sensitive:


A couple of weeks ago I did an initial deployment of an Intrusion Detection System in our infrastructure. It utilizes the prelude stack, and is currently powered by auditd and prelude-lml events. Audit gives us a ridiculous amount of power with regarding to monitoring *everything* that happens on a system. Prelude-lml, out of the box using it's pcre plugin, is able to watch a large variety of service logs, including many things we are running (asterisk, mod_security, nagios, cacti, PAM, postfix, sendmail, selinux, shadowutils, sshd, sudo). Prewikka is the web-based frontend (https://admin.fedoraproject.org/prewikka).

I created a new 'prelude' puppet module that contains the configuration for audit, auditsp-plugins, libprelude, prelude-manager, prewikka, prelude-correlator, and prelude-lml. Turning a node/servergroup into a sensor entails adding the following to your class definition: 'include prelude::sensor::audisp' My initial deployment entailed setting up the prelude-manager and correlator on a single box, and hooking up a single sensor (bastion).

So, we're now at the point where we can fine tune our audit rules before we further deploy this infrastructure. Some things we want to consider:

  • Creating security policies for each servergroup
  • Define what files/directories/activities we want to monitor on which machines
  • What events to we want to receieve email/sms for?

Change History

comment:1 Changed 8 years ago by lmacken

[X] Build EPEL-5 i386 and x86_64 packages
    [X] Build F-10 prelude stack
    [X] Build F8 audit package (1.7.4)
[X] Test the stack setup locally
[X] Put x86_64 packages in f-i repo
[_] Build i386 EL-5 packages and push them into puppet
[X] Create prelude puppet module
    [X] Setup an audit prelude sensor
    [X] Setup prelude-correlator
    [X] Setup prelude-lml to monitor log files
    [X] Setup prelude-manager
    [X] Setup prewikka
[X] Deployment
    [X] Setup prewikka and prelude mysql db on db1
    [X] Setup prelude-manager and prelude-correlator on noc1
    [X] Setup audit and prelude-lml sensor on bastion
[X] Rebuild against audit-1.7.5, which has an improved dispatcher
[X] Tweak audit rules
    [X] Fix displayname of nodes
    [X] start with the default stig.rules
    [X] Enable tracing and bypass rules
    [X] Change failure mode to printk1 from panic 2
    [X] Add 'ids' keys to the rules, for easy filtering
[_] Setup email notifications for critical issues
[_] Zabbiz, nagios, cacti integration
[_] File alteration monitoring
[_] Hook up prelude-lml to our rsyslog heirarchy on log1
    : This is a bit more complicated, since the log heirarchy is
    : sorted by month/day/, and there is no simple way to point
    : prelude-lml at them.

comment:2 Changed 8 years ago by lmacken

  • Status changed from new to assigned

comment:3 Changed 8 years ago by lmacken

So, here's an issue that I hit with prelude-lml in our environment

We've got rsyslogd setup on a central logging server where everything dumps it's logs to. However, the hierarchy that they are stored in is not very suitable for easily monitoring (/var/loghosts/$HOST/$YEAR/$MONTH/$DAY/*.log). Ideally, we want prelude-lml to watch these logs across all hosts, and our central logging server seems like the best place to do it, but due to the structure, it is very difficult to do so.

Suggestions welcome.

comment:4 Changed 8 years ago by lmacken

Sorry for the lack of updates, but progress is being made on this task.

Things that we need to do before deploying to the rest of our systems

  • Get our AVCs down to none at all
  • Update to the new audit daemon once it is released
  • Tweak prewikka filters/thresholds so it is manageable.
  • Setup a cron job to flush the prelude db occasionally of alerts that are older than 5 days.

comment:5 Changed 7 years ago by lmacken

So, we've been making good progress thus far...

  • SELinux AVCs are down to a tolerable level
  • I've been keeping our selinux-policy, audit, and prelude packages up to date
  • I've been flushing our heartbeat/alert db every now and then by hand. We need this to be a cronjob.

I'm going to sit down next week and setup a couple more prelude sensors, most likely on fedorapeople and fedorahosted.

comment:6 Changed 7 years ago by lcbruzenak

Can you share your prelude puppet module? It is exactly what I need. I have setup a few machines by hand with about the same setup you described, and I was about to look into puppet myself. It would be a great help if the puppet prelude module and any supporting files/scripts were available for reference. Thx, LCB.

Note: See TracTickets for help on using tickets.