Setting up Snorby in Security-Onion

This video has been created by IronGeek and it is originally available on his website.

I wanted to post it on the blog because setting up a Network Security Monitoring system with Security-Onion is rather easy compared with the days I had to deploy Sguil by hand. Doug Burks is doing a great job here:)

I have to say that I prefer using Sguil in a NSM system as Richard Bejtlich has explained many times, but Snorby may be useful for many people.







Here you can find Richard Bejtlich talking about NSM and Sguil.

Auditing logs: tracing e-mail transactions II

In the previous post Auditing logs: tracing e-mail transactions I talked about a script I wrote to make sense of the e-mail logs when doing troubleshooting or just investigating an incident. The downside, of course, is that we have to process all the logs every time the script is ran and this can be time consuming.

To solve this problem I decided to index the 'reduced' data with elasticsearch, that is a search engine based on lucene. As a result, I can make quick searches upon millions of unique e-mail transactions and get a result in a matter of seconds.


Structuring the e-mail transactions
First of all, I have to clarify that each e-mail transaction is a unique operation in a single e-mail server. Therefore, sending an e-mail may involve several e-mail transactions because it is traversing several e-mail servers throughout our infrastructure. At the end, each of these unique transactions are represented by the following data structure:

• from: A string representing the sender's e-mail address.

• to: Array of recipients. Each element in the array is a dictionary with the following fields:

1. recipent: one of the recipient's e-mail address
2. relay: the next hop in the delivery chain. It can be a server or local delivery.
3. status: A string that informs about the result of the operation.

 Since a particular e-mail can be stuck in the queue and re-delivered later, we may see the same recipient appearing several times, but with different status messages: ie. deferred and successfully sent.

• msgid: unique message identifier that represents a particular sent e-mail. Not to confuse with an e-mail transaction that is unique within an e-mail server.
• client: The previous hop in the e-mail chain. 
• date: String representing the time in which the transaction took place.
• host: the server that is executing the e-mail transaction.

Data retention policy
I have configured the index to automatically drop entries older than 90 days. In my opinion, we do not need to keep this data for a longer period of time, because the troubleshooting tends to involve transactions that took place in the previous days. Of course, In case it is needed, we still keep the raw logs for a longer period and they can be manually parsed or, simply, re-indexed.

Reducing and indexing the raw logs
The script  elasticsearch.py parses the raw logs and indexes them by using the REST API.

There is a very dirty hack here, since many syslog servers do not send the year in the log messages. The script assumes we are indexing logs from the current year and makes a dirty trick on January 1st. In case you want to index a log file from the previous year you can use the -y flag to force this case. Yes, it is as dirty as I said :)   Modern syslog servers like rsyslog or syslog-ng can be configured to output the year, but it may not be the case with older installations.

Searching
The script search.py connects to the elasticsearch server and sends the queries.

A query example would be the following one:

./search.py -f bob@foo.com -d '2011-12-15 00:00:00' -D '2011-12-17 00:00:00' -h smtp -r 10
transaction: pBFERI8l019936
client: bob.desktop [10.10.10.10]
from:   bob@foo.com
msgid:  20111215142718.GD23530@smtp.foo.com
date:   2011-12-15 15:27:19
to:     recipent='alice@bar.com' , relay='smtp.bar.com', status='Sent (Ok: queued as 5298B228B8B)'
host:   smtp

transaction: pBFDdOFC012829
client: bob.desktop [10.10.10.10]
from:   bob@foo.com
msgid:  20111215133924.GC23530@smtp.foo.com
date:   2011-12-15 14:39:25
to:     recipent='alice@bar.com' , relay='smtp.bar.com', status='Sent (Ok: queued as 16F49228B8B)'
host:   smtp

The above query searches all the e-mail transactions that match:

• The sender is bob@foo.com
• Between 2011-12-15 00:00:00 and 2011-12-17 00:00:00
• The transactions occur in the server called smtp
• Only output the 10 most recent transactions.

All the scripts can be found here.

Owning a Windows Domain with Metasploit's Incognito and Persistence Modules

Found via @armitagehacker on Twitter.

This video shows a demo that uses Armitage (Metasploit) to compromise a Windows Domain Controler.

The attacker gains access to an unpatched Windows web server  by exploiting the classic MS08-067. On the web server, the attacker is able to obtain the cached domain credentials of an administrator  and use them to compromise the domain controller.

The attacker also makes use of the persistence module to keep a foothold on the compromised system.

Auditing logs: tracing e-mail transactions

Every good admin knows that analyzing the log files plays a key role in not only security, but also in the day-to-day IT operations. Who does not read the logs... (sarcasm)?

One of the situations I have to face really often is investigate e-mail problems such: delays, e-mail not arriving, bounces, user account does not exist, etc.. or, simply, whether a user sent an e-mail or not.

If reading the logs created by Postfix or Sendmail is a pain 'per se', trying to understand what happened in a scenario with multiple intermediate servers is a nightmare. Therefore, doing log management is key to succeed (in this particular situation or any other that involves large quantities of data).


There is nothing fancy in my setup. I am just using a centralized syslog server to collect all the raw logs created by the e-mail servers. Then, we can trace the problem from one single place, that is good, but trying to understand what happened is still a pain.  We still cannot see the forest for the trees.


I have come up with a Python script ( search_email_transactions.py ) that parses Postfix and Sendmail logs. It searches all the e-mail transactions that match the  message id, the sender, the recipient and the date.

The output is self-explanatory:

# cat /var/log/all |  ./search_email_transactions.py -l -  -f ^me@foo.com  -i 4EDC80B6.8040107@smtp.foo.com

transaction: E324DA41CA
from:   me@foo.com
msgid:  4EDC80B6.8040107@smtp.foo.com
date:   Dec  5 09:28:42
to:     recipent='you@bar.com' , relay='smtp.bar.com[10.10.10.1]:25', status='sent (250 2.0.0 pB58ShMJ024096 Message accepted for delivery)'
host:   smtp1

In this example, it only outputs one unique transaction but in a scenario with multiple servers we should have a listing of all the transactions and all the servers involved, giving us the full picture.

Since it is time and CPU consuming, I am planning to modify this script to treat all the raw logs and dump all this information to a search engine like elasticsearch, making the troubleshooting faster while still having the raw logs in case I need to go deeper.

Understanding APT and Counter Espionage

Thanks to Irongeek I have found the following talk at SkyDogCon.

This is a really good talk you have to watch if you want to understand APT beyond any FUD and if you want  learn how to defend your network against it.

Hiding The Toolkit On Linux With LUKS

The idea behind this post is making the Incident  Response a little bit more complicated during a pen-test, by hiding our tools in "hidden volumes".


Ideally,  when the hard drive has unpartitioned space, the attacker can create a new partition and encrypt it with LUKS to hide the tools. By doing so, the main file system remains unchanged (no new files are written/modified that can trigger an alert in the HIDS) and it is difficult to spot unless the defenders keep track of the logs that will point to new file systems being mounted.


The procedure would be pretty easy:

• Mount the LUKS volume
• Execute the tool in backround
• Umount the LUKS volume in 'lazy mode'

Reading umount(8) , we can find the flag -l :

Lazy unmount. Detach the filesystem from the filesystem  hierarchy now, and cleanup all references to the filesystem as soon as it is not busy anymore.  (Requires kernel 2.4.11 or later.)

The following script works under the same principle, but it creates a file instead of a partition. In this case, it creates a 300MB file named "volume"  that will contain a ext3 file system encrypted with AES 256.


#! /bin/sh

function find_device_name {
        losetup -j $1 | awk '{print $1}' | cut -f1 -d ":"
}


function create_raw_vol {
        dd if=/dev/zero of=${1} bs=1M count=$2 &> /dev/null
}


function create_luks_volfile {
        local raw_vol_dev=""
        raw_vol_dev=`losetup -f`
        create_raw_vol $1 $2
        losetup $raw_vol_dev $1
        echo $3 |cryptsetup  -c aes -s 256 luksFormat `find_device_name $1`  &> /dev/null
        losetup -d `find_device_name $1`
}


function mount_luks_volfile {
        local raw_vol_dev=""
        raw_vol_dev=`losetup -f`
        losetup $raw_vol_dev $1
        echo $3 |  cryptsetup luksOpen `find_device_name $1` $2 &> /dev/null
}

function umount_luks_volfile {
        cryptsetup luksClose ${2} &> /dev/null
        losetup -d `find_device_name $1`

}

vol_name="./volume"
vol_size="300"
mapper_name="hidden_volume"
password="1234"


create_luks_volfile $vol_name $vol_size $password
mount_luks_volfile $vol_name $mapper_name $password
mkfs.ext3 /dev/mapper/${mapper_name}
sleep 5
umount_luks_volfile $vol_name $mapper_name

Fetching the SAM and System Files Without Shutting Down Windows

Via securitybydefault [Spanish]


The linked blog post explains how to fetch the SAM and System files from a Windows computer without shutting down the system.


Since both files are locked by other processes, they cannot be read. Therefore, the standard procedure would be shutting down Windows and running a live distribution to obtain a copy.


The article points to a talk given by Tim Tomes and Mark Bagget in Hack3rcon II, where they introduce a script they wrote to extract the files  by creating Shadow Copies.