Second Generation Honeynets JedHaile Mike Clark Rob McMillen Edward Balas Mike Davis Dragos Ruiu

Purpose An introduction to Second Generation Honeynet Technologies.

Agenda Inline Snort Sebek2 Virtual Honeynets This is the agenda we will be following for today. 9/19/2018

Honeynets Nothing more then one type of honeypot. High-interaction honeypot designed to capture in-depth information. Its an architecture, not a product or software. Populate with live systems.

How it works A highly controlled network where every packet entering or leaving is monitored, captured, and analyzed. Any traffic entering or leaving the Honeynet is suspect by nature. http://www.honeynet.org/papers/honeynet/

Honeynet Requirements Data Control Data Capture Data Collection (for distributed Honeynets) http://www.honeynet.org/alliance/requirements.html

Honeynet - GenI

Honeynet - GenII Easier to Deploy Harder to Detect Both Data Control and Data Capture on the same system. Harder to Detect Identify activity as opposed to counting connections. Modify packets instead of blocking.

Honeynet - GenII

Honeynet Tools Find all the latest Honeynet tools for Data Control, Capture, and Analysis at the Honeynet Tools Section. http://www.honeynet.org/papers/honeynet/tools/

Data Control Rob McMillen Good morning or afternoon. My name is Rob McMillen, and I will be covering the data control portion of today’s presentation. Let me start by telling you a bit about myself. I am an active duty Marine Corps Officer, currently stationed at Company L, Marine Cryptologic Support Battalion, and I love to play with computers during my spare time. I began playing with Honeynets while attending the Naval Postgraduate school in 1999, and I am the developer of the rc.firewall script and the current maintainer of snort_inline. Rob McMillen

No Data Control Here we have a Honeynet without any data control. I guess you could say this represents the standard system on a network, but if these systems get compromised, we do not have a means to mitigate the potential damage they may cause on the Internet. All network traffic is allowed to flow into and out off the Honeynet. If a honeypot is compromised, the attacker can use this system as a launching point to attack other systems on the Internet. It is essential to control the data that leaves a compromised Honeynet to ensure we are good net citizens.

Data Control Adding a choke point that is capable of limiting and possibly scrubbing every packet between our Honeynet and the world is one solution to our data control issue that reduces the risk involved with a Honeynet. We want every packet originating outside our Honeynet to enter the Honeynet without any restrictions. But we need to limit the number of connections our compromised Honeypot is allowed to initiate on the Internet. We can reduce the risk and potential damage even further by adding an Network Intrusion Prevention System (NIPS) that drops all known malicious packets originating from a compromised Honeypot. The addition of an NIPS will also let us increase the number of connections we allow honeypots to initiate. The biggest challenge is figuring out the limit settings. The more we let him do, the more we learn about his tactics, tools, and techniques. However, this freedom allows the attacker to cause more harm on the Internet. The combination of a choke point; the rc.firewall script – connection limiting; and snort_inline – packet scrubber - gives us a data control solution. Better yet, we are working on a bootable cdrom that combines these tools and makes it easier to deploy.

IPTABLES Packet Handling

rc.firewall Configure the gateway mode Sets default policies Allows ALL traffic to our Honeynet Limits traffic from our Honeynet Alerts on certain traffic Sets restrictions on the management interface The first portion of our data control solution is the rc.firewall script. This script uses the variables set by the user to configures an iptables based firewall to: Configure the gateway mode: NAT Bridge Sets Default Policies: Drop Allows all traffic into our Honeynet Limits traffic initiated by our Honeynet Alerts on certain traffic. New inbound connection per protocol: TCP, UDP, ICMP, Other New outbound connections per protocol. When outbound limits are reached per protocol. Sets restrictions on the management interface.

rc.firewall (data control) ### Set the connection outbound limits for different protocols. SCALE="day" TCPRATE="15" UDPRATE="20" ICMPRATE="50" OTHERRATE="15" iptables -A FORWARD -p tcp -i $LAN_IFACE -m state --state NEW -m limit --limit ${TCPRATE}/${SCALE} --limit-burst ${TCPRATE} -s ${host} -j tcpHandler -m limit --limit 1/${SCALE} --limit-burst 1 -s ${host} -j LOG --log-prefix "Drop TCP after ${TCPRATE} attempts“ -s ${host} -j DROP Due to the amount of time we have, I will focus on the connection limiting portion of the script, which directly affects data control. As you can see, the first thing we have to do is set the SCALE and the different protocol limits. These limits are dependent on the amount of risk you are willing to take. Remember, the more we let out the more we learn. Ideally, there are enough connections to retrieve toolkits and start probes, but not enough to start attacks. Next, we have the real magic behind the connection limiting. There are a set of three rules ( like the ones listed in the bottom of the slide) for each protocol. These are the rules that are responsible for the connection limiting and logging once the limit is reached. As you can see, these rules are added to the FORWARD chain (chain responsible for forwarding traffic through the firewall). The look for tcp traffic coming into the LAN interface, and our statefull firewall is looking for NEW connections. If we have a NEW connection, the limit is checked. If it is not reached, the packet is sent to the tcpHandler. Here, the packet is accepted or sent to the QUEUE (the packet scrubber). If the limit is reached, the packet goes to the next rule where it is logged and dropped. The –limit-burst portion requires a bit of explanation. Think of a leaky bucket capable of holding a certain number of new connection packets. Our rate tells us how many packets the bucket leaks per ${SCALE}. The –limit-burst tells us how many packets the bucket can hold in the first place. For example, lets say we set our TCPRATE to 25 and SCALE to day. Our honeypot will be able to establish 25 new tcp connections as fast as it possibly can. Once the bucket is full, it only leaks 25 new connections per our ${SCALE}:day: ~1 packet an hour.

snort_inline iptables -A FORWARD -i $LAN_IFACE -m state --state RELATED,ESTABLISHED -j QUEUE Next we have our packet scrubber: snort_inline. Snort_inline is basically Snort. The only difference is that it accepts packets from iptables, via the use of a kernel module named ip_queue, and tells iptables to drop or accept the specific packet. Additionally, snort_inline is capable of replacing portions of the packet payload as the packet traverses the firewall. The decisions are based on the snort rule set with the help of a few new rule types. Let’s start by looking at the firewall rule that sends the packet to snort_inline. As you can see, it is similar to the rules we saw on the previous slide with the exception of the target: QUEUE. This target tells iptables to pass the packet to a user space application via the use of the ip_queue kernel module. The ip_queue kernel module copies the packet from kernel space to user space and hands it to an application that has registered with the module to receive these packets (only one module can receive packets at any given time). In our case the application is snort_inline. The difference between a packet from ip_queue and one from a sniffer is the absence of a pcap header (contains such things as captured length and time) and the absence of a link layer. Following the packet through the firewall we see it arrive at the inbound network interface. From here, iptables decides it needs to go to ip_queue when the packet meets the criteria set in the firewall rule that contains the QUEUE target. Iptables hands the packet to ip_queue which then creates a copy and copies it from kernel space to user space. Ip_queue then gives the packet to snort_inline, where a pcap header is added and it is sent via the snort process (to include plugins). If none of the rules match, snort_inline tells ip_queue to accept this packet. Ip_queue then tells iptables to accept the packet and it leaves the oubound interface on its way to its destination. If one of the rules hit, the packet is handled according to the rule types I will explain on the next slide. One thing to note. If you take a look at the rc.firewall script, you will notice that not all traffic is sent via the QUEUE. The main reason for this is due to the expensive copy from kernel space to user space. If all traffic traversed snort_inline, throughput would suffer. Also, we are only interested in dropping malicious traffic originating from our Honeynet; therefore, we only send traffic from our Honeynet through snort_inline.

snort_inline reject tcp $HONEYNET any <> $EXTERNAL_NET 80 (msg: "REJECT";) drop tcp $HONEYNET any <> $EXTERNAL_NET 80 (msg: "DROP TCP";) sdrop tcp $HONEYNET any <> $EXTERNAL_NET 80 (msg: "SDROP";) alert tcp $HONEYNET any <> $EXTERNAL_NET 80 (msg: "Modifying HTTP GET"; content:"GET"; replace:“BET";) Next we have the rule types. In addition to the existing snort rule types, such as log and alert, we have added three new rule types and a keyword. The new rules work as follows: Reject: drops the matching packet and sends a reset of icmp host unreach depending on the protocol. *** This does not work in bridge mode at this time. Drop: drops the matching packet and alerts like a normal snort alert rule. Sdrop: drops the matching packet and does not log. Think of it as a silent drop. Finally, we have the replace keyword. In conjunction with the content keyword, we can search the packet for a specific word Thanks to Lance, we have a script that will convert all of your alert rules to drop rules. This script, named convert.sh, comes with the snort_inline distribution that can be downloaded from snort-inline.sf.net The Honeynet Project also officers a statically compiled version of snort_inline with a rule set and configuration file which will make it a lot easier to deploy.

Future Get ip_queue functionality in other OS. Create a generic way to get packets to SI. Better way to do connection limiting. Integrate snort_inline into snort. Feel free to contact me at (rvmcmil@cablespeed.com) if you have ideas or suggestions. Currently, this functionality we have talked about is limited to the linux opearting system due to the dependency on iptables’ ip_queue kernel module. It would be great to get such functionality with other firewalls. Another possible avenue of approach would be a generic way of getting the packets from the system and sending them to snort_inline. Another way of doing connection limiting would be great. The current method is OK, but ideally, we would prefer something that gives you x number of connections per a given time instead of x number of connections followed by x number of connections evenly spread during the given time. Finally, I am working with the snort team to get snort_inline integrated into the snort source, which would make the our lives easier. If you have any ideas, suggestions, or complaints, feel free to contact me at this e-mail address.

Summary In summary, we talked about a solution to data control. This solution consisted of a firewall and packet scrubber placed in a location where all traffic travels between the Honeynet and the world. This firewall performs connection limiting and can send packets to a packet scrubber to ensure we are being good net citizens. Since we are using a bridge as our data control mechanism, we can place the device between the honeypots and a production system in order to make the Honeynet look as if it is part of the production network. The data control mechanism is virtually invisible to any attacker because the bridge does not decrement the time to live of the packets.

Resources http://www.honeynet.org/papers/gen2/ http://bridge.sourceforge.net/ http://www.honeynet.org/papers/gen2/rc.firewall http://snort-inline.sourceforge.net Contact: rvmcmil@cablespeed.com

An Introduction to Sebek2 Edward Balas ebalas@iu.edu

What is Sebek A forensic tool used to record activities on a honeypot Designed to record keystrokes even when intruder uses secure communication channel. Provides ability to recover files copied to or from a honeypot with SCP. Operates as a hidden kernel module on a honeypot.

Who developed Sebek Sebek is a collaborative effort within the Honeynet Project. Originally based on the adore rootkit and coded by Mike Clark for the linux OS. The second version, written by Edward Balas, included a number of improvements that made it more covert and comprehensive. This second version is currently being ported to a number of OSs by members of the team

Sebek ports under development Solaris Job de Hass Windows Mike Davis OpenBSD Dragos Ruiu

Sebek Design There are 2 components: The client, which collects the data on the honeypot The server, which records and processes the data exported by the honeypots.

Sebek Client Two fundamental issues for the client collection of forensic data covert export of the data to a collector

Forensic data collection Client code uses a kernel module to replace the function that implements the read system call. When data of interest is observed by this module, it is copied and queued for export. all data recorded in newest version A variety of techniques are used to hide the existence of the kernel module. These are similar to those used by modern rootkits. All actions take place in kernel space outside the view of users.

Data export The kernel module creates its own ethernet frames and then sends them directly to the ethernet driver, bypassing firewall filters, and packet sniffing. A special source MAC OUI is used in the ethernet frames created by Sebek. When a honeypot running Sebek receives packets with the source OUI, it silently ignores them and prevents users for seeing them. It is impossible for a user on the honeypot to detect the Sebek traffic being exported.

The data that Sebek gathers In the past Sebek has used a number of heuristics to determine what data if found interesting. The newest and yet to be released version gathers all data that is seen in the “sys_read” function call. This ensures that there are no blind spots as there are in the current version. Once the data is exported and gathered on the collector it is the responsibility of a tool called sbdump.pl to do the interpretation.

The Data Export Format Sebek data is colon delimited and has the following fields: Timestamp, in seconds since epoch Process ID for process making read request. User ID for process making read request. Command Name for the process. File Descriptor Number. TTY name. Length of Data Data

Sebek: The Server Side The Sebek server code involves two applications: sebeksniff sbdump.pl. sebeksniff can extract Sebek records from an packet capture file or can sniff directly from a LAN. Records are stored in a file based on the Source IP address of the honeypot. sbdump.pl attempts make viewing the data easy, it includes ability to recover SSH copied files.

Where Can I get Sebek Sebek can be downloaded from http://project.honeynet.org/papers/honeynet/tools/ For questions or comments contact Edward Balas ebalas@iu.edu

Using Sebek Example shows a user logging into a honeypot with SSH and then copying a file to the honeypot First we will look at the raw data recorded Second we will how how sbdump.pl can process the data for the user.

Raw Data: the df command 1052534859:840:0:sshd:7::c:2:d 1052534859:842:0:bash:0:pts:c:2:f 1052534859:840:0:sshd:7::c:2:f 1052534861:842:0:bash:0:pts:c:2: 1052534861:840:0:sshd:7::b:2: 1052534861:840:0:sshd:7::b:68:Filesystem 1K-blocks Used Available Use% Mounted on 1052534861:840:0:sshd:7::b:396:/dev/sda5 505605 131737 34776 4 28% / /dev/sda1 101089 12566 83304 14% /boot /dev/sda3 679912 16428 628948 3% /home none 127956 0 127956 0% /dev/shm /dev/sda2 1873772 834856 943732 47% /usr /dev/sda6 505605 434100 45401 91% /var

Raw Data Observation It is difficult to read the raw data Redundancy of data for related key strokes Redundancy for same data getting piped through multiple processes Control chars and non-text data

What can sbdump.pl do? Run as sbdump.pl -c 10.0.0.1 [edb@sumatra sebek]$ sbdump.pl -c ./10.0.0.1 02:47:39-2003/05/10 [0:bash:842:pts:0]df 02:47:44-2003/05/10 [0:bash:842:pts:0]who 02:47:47-2003/05/10 [0:bash:842:pts:0]last 02:48:28-2003/05/10 [0:bash:842:pts:0]scp 10.0.0.2:/tmp/svs_thinking.gif . 02:48:55-2003/05/10 [0:ssh:886:pts:3]SSH-2.0-OpenSSH_3.1p1 02:49:36-2003/05/10 [0:bash:842:pts:0]scp edb@10.0.0.2:/tmp/svs_thinking.gif . 02:49:36-2003/05/10 [0:ssh:888:pts:3]SSH-2.0-OpenSSH_3.1p1 02:49:40-2003/05/10 [0:ssh:888:pts:4]thepasswd=this 02:49:40-2003/05/10 [0:scp:887:pts:7]C0644 194614 svs_thinking.gif

Things to notice The -c argument attempts to get all interactive terminal “character” data, it is not always 100% accurate as you can see by some of the scp data that came through. We can see that the intruder first checked the disk for capacity, then checked to see who was logged and the last time folks logged in, after that came a failed file transfer attempt then the good attempt Lets take a closer look at the file transfer…

Recovering SCP files Run sbdump.pl with -s arg Following points descovered: A file called svs_thinking.gif was copied to the box from a remote site, we know from the eariler run of sbdump that it was 10.0.0.2. Password used to authenticate was “thepasswd=this” edb@sumatra sebek]$ ./sbdump.pl -s ./10.0.0.1 02:49:40-2003/05/10 SCP (local)<-remote svs_thinking.gif 194614 bytes 02:49:40-2003/05/10 SCP: passwd thepasswd=this 1

The file…

Limits Data export mechanism uses UDP and thus does not provide reliable stream transport. Heuristics used in Sebek to detect interesting data are not 100% fool proof, some tuning may be necessary. Same heuristics rely on static signatures for interesting data, thus providing target for evasion techniques. All heuristics have been removed from the client for the newest version, however sbdump.pl is still vulnerable to evasion.

Michael A. Davis Lead Developer SecurityProfiling, Inc. Windows port of Sebek Michael A. Davis Lead Developer SecurityProfiling, Inc.

The Problem 100% Sebek UNIX functionality on Win32 while maintaining interoperability with Sebek UNIX.

Requirements Stealth Logging of GUI and Console I/O Filter sebek packets from ALL IP network drivers. Undetectable Logging of GUI and Console I/O Example: cmd.exe via IIS. Secure configuration Discuss a scenario explaining why there needs to be both a console logger and GUI logger.

The Solution In memory patching of syscalls Hook the console and GUI subsystems Spy on all the read/write calls performed by the subsystem. Determine any undocumented structures by reverse engineering Use a private key for configuration changes Patch NDIS RegisterProtocol function. All NDIS drivers will be filtered. All Console applications, not just cmd.exe, will be monitored. Discuss the undocumented structures in the CSRSS subsystem and the work Blake Watts did

Stealth Hook Registry functions(ZwQueryKey, ..) Hook File System functions(ZwQueryDirectoryObject, …) Hide all packets we transmit or receive(NDIS hooking) Use a special configuration application to configure the driver. Configuration is stored in the actual driver Discuss how the private key will be the filename of the configuration tool. And how only a binary with the name will be able to actually see the SEBEK kernel driver in the registry and on the file system.

Advantages Not easily detectable. Full “view” of the system. Kernel drivers are not easily detectable to the average windows hacker. Being in the kernel allows us full control of the system.

Disadvantages Never 100% undetectable. May degrade system performance. Undocumented structures etc lead to a longer development cycle. Kernel drivers can never be 100% undetectable. Although there are many tricks available to obtain almost 100% undetectability. Because we filter calls such as ZwQueryDirectoryObject(), the system will perform slightly slower. Undocumented structures like that in the CSRSS subsystem lead to longer development times and problems when MS releases new OS’s, Service Packs, etc.

Current Version Filters network traffic for specific OUI Will hide other sebek packets from remote hosts Console monitoring (cmd.exe etc) Initial version of the configuration application Can change OUI to filter Stores configuration in the driver

Future Enable stealth features Transmit monitored data to remote host Hook registry and file system functions Transmit monitored data to remote host Once keystroke have been logged send them to the UNIX based archiver Monitoring GUI applications

A complete port of sebek is possible but difficult and time consuming Summary A complete port of sebek is possible but difficult and time consuming

Resources Download Authors Contributors http://project.honeynet.org Michael A. Davis (mike@datanerds.net) Contributors Blake Watts (blake@blakewatts.com) Thanks Blake for all of his hard work. Ask for help on the project (especially beta testers)

Virtual Honeynets Mike Clark

What is a Virtual Honeynet? Full OS Run within another OS

Virtualization Software VmWare User Mode Linux Others

Why use a Virtual Honeynet? Physical Concerns Portability Easy

Types of Virtual Honeynets Self-Contained Hybrid

Disadvantages Everything on 1 system Detection Can’t support every OS/hardware

<project@honeynet.org> http://www.honeynet.org <project@honeynet.org> Feel free to contact the Honeynet Project with any idea, questions, or concerns you may have. If you are interested in learning more about information security in general, you can find books written by member of the Honeynet Project at http://www.honeynet.org/book/books.html Best of luck! --- The Honeynet Project ---