1. Second Generation Honeynets
JedHaile
Mike Clark
Rob McMillen
Edward Balas
Mike Davis
Dragos Ruiu

2. Purpose
An introduction to Second Generation Honeynet Technologies.

3. Agenda Inline Snort Sebek2 Virtual Honeynets
This is the agenda we will be following for today.
9/19/2018

4. Honeynets Nothing more then one type of honeypot.
High-interaction honeypot designed to capture in-depth information.
Its an architecture, not a product or software.
Populate with live systems.

5. How it works
A highly controlled network where every packet entering or leaving is monitored, captured, and analyzed.
Any traffic entering or leaving the Honeynet is suspect by nature.

6. Honeynet Requirements
Data Control
Data Capture
Data Collection (for distributed Honeynets)

7. Honeynet - GenI

8. Honeynet - GenII Easier to Deploy Harder to Detect
Both Data Control and Data Capture on the same system.
Harder to Detect
Identify activity as opposed to counting connections.
Modify packets instead of blocking.

9. Honeynet - GenII

10. Honeynet Tools
Find all the latest Honeynet tools for Data Control, Capture, and Analysis at the Honeynet Tools Section.

11. Data Control Rob McMillen
Good morning or afternoon. My name is Rob McMillen, and I will be covering the data control portion of today’s presentation.
Let me start by telling you a bit about myself. I am an active duty Marine Corps Officer, currently stationed at Company L, Marine Cryptologic Support Battalion, and I love to play with computers during my spare time. I began playing with Honeynets while attending the Naval Postgraduate school in 1999, and I am the developer of the rc.firewall script and the current maintainer of snort_inline.
Rob McMillen

12. No Data Control
Here we have a Honeynet without any data control. I guess you could say this represents the standard system on a network, but if these systems get compromised, we do not have a means to mitigate the potential damage they may cause on the Internet.
All network traffic is allowed to flow into and out off the Honeynet. If a honeypot is compromised, the attacker can use this system as a launching point to attack other systems on the Internet.
It is essential to control the data that leaves a compromised Honeynet to ensure we are good net citizens.

13. Data Control
Adding a choke point that is capable of limiting and possibly scrubbing every packet between our Honeynet and the world is one solution to our data control issue that reduces the risk involved with a Honeynet.
We want every packet originating outside our Honeynet to enter the Honeynet without any restrictions. But we need to limit the number of connections our compromised Honeypot is allowed to initiate on the Internet.
We can reduce the risk and potential damage even further by adding an Network Intrusion Prevention System (NIPS) that drops all known malicious packets originating from a compromised Honeypot. The addition of an NIPS will also let us increase the number of connections we allow honeypots to initiate.
The biggest challenge is figuring out the limit settings. The more we let him do, the more we learn about his tactics, tools, and techniques. However, this freedom allows the attacker to cause more harm on the Internet.
The combination of a choke point; the rc.firewall script – connection limiting; and snort_inline – packet scrubber - gives us a data control solution. Better yet, we are working on a bootable cdrom that combines these tools and makes it easier to deploy.

14. IPTABLES Packet Handling

15. rc.firewall Configure the gateway mode Sets default policies
Allows ALL traffic to our Honeynet
Limits traffic from our Honeynet
Alerts on certain traffic
Sets restrictions on the management interface
The first portion of our data control solution is the rc.firewall script. This script uses the variables set by the user to configures an iptables based firewall to:
Configure the gateway mode:
NAT
Bridge
Sets Default Policies: Drop
Allows all traffic into our Honeynet
Limits traffic initiated by our Honeynet
Alerts on certain traffic.
New inbound connection per protocol: TCP, UDP, ICMP, Other
New outbound connections per protocol.
When outbound limits are reached per protocol.
Sets restrictions on the management interface.

16. rc.firewall (data control)
### Set the connection outbound limits for different protocols. SCALE="day" TCPRATE="15" UDPRATE="20" ICMPRATE="50" OTHERRATE="15"
iptables -A FORWARD -p tcp -i $LAN_IFACE -m state --state NEW
-m limit --limit ${TCPRATE}/${SCALE} --limit-burst
${TCPRATE} -s ${host} -j tcpHandler
-m limit --limit 1/${SCALE} --limit-burst 1 -s ${host}
-j LOG --log-prefix "Drop TCP after ${TCPRATE} attempts“
-s ${host} -j DROP
Due to the amount of time we have, I will focus on the connection limiting portion of the script, which directly affects data control. As you can see, the first thing we have to do is set the SCALE and the different protocol limits. These limits are dependent on the amount of risk you are willing to take. Remember, the more we let out the more we learn. Ideally, there are enough connections to retrieve toolkits and start probes, but not enough to start attacks.
Next, we have the real magic behind the connection limiting. There are a set of three rules ( like the ones listed in the bottom of the slide) for each protocol. These are the rules that are responsible for the connection limiting and logging once the limit is reached.
As you can see, these rules are added to the FORWARD chain (chain responsible for forwarding traffic through the firewall). The look for tcp traffic coming into the LAN interface, and our statefull firewall is looking for NEW connections. If we have a NEW connection, the limit is checked. If it is not reached, the packet is sent to the tcpHandler. Here, the packet is accepted or sent to the QUEUE (the packet scrubber). If the limit is reached, the packet goes to the next rule where it is logged and dropped.
The –limit-burst portion requires a bit of explanation. Think of a leaky bucket capable of holding a certain number of new connection packets. Our rate tells us how many packets the bucket leaks per ${SCALE}. The –limit-burst tells us how many packets the bucket can hold in the first place. For example, lets say we set our TCPRATE to 25 and SCALE to day. Our honeypot will be able to establish 25 new tcp connections as fast as it possibly can. Once the bucket is full, it only leaks 25 new connections per our ${SCALE}:day: ~1 packet an hour.

17. snort_inline iptables -A FORWARD -i $LAN_IFACE -m state
--state RELATED,ESTABLISHED -j QUEUE
Next we have our packet scrubber: snort_inline. Snort_inline is basically Snort. The only difference is that it accepts packets from iptables, via the use of a kernel module named ip_queue, and tells iptables to drop or accept the specific packet. Additionally, snort_inline is capable of replacing portions of the packet payload as the packet traverses the firewall. The decisions are based on the snort rule set with the help of a few new rule types.
Let’s start by looking at the firewall rule that sends the packet to snort_inline. As you can see, it is similar to the rules we saw on the previous slide with the exception of the target: QUEUE. This target tells iptables to pass the packet to a user space application via the use of the ip_queue kernel module. The ip_queue kernel module copies the packet from kernel space to user space and hands it to an application that has registered with the module to receive these packets (only one module can receive packets at any given time). In our case the application is snort_inline. The difference between a packet from ip_queue and one from a sniffer is the absence of a pcap header (contains such things as captured length and time) and the absence of a link layer.
Following the packet through the firewall we see it arrive at the inbound network interface. From here, iptables decides it needs to go to ip_queue when the packet meets the criteria set in the firewall rule that contains the QUEUE target. Iptables hands the packet to ip_queue which then creates a copy and copies it from kernel space to user space. Ip_queue then gives the packet to snort_inline, where a pcap header is added and it is sent via the snort process (to include plugins). If none of the rules match, snort_inline tells ip_queue to accept this packet. Ip_queue then tells iptables to accept the packet and it leaves the oubound interface on its way to its destination. If one of the rules hit, the packet is handled according to the rule types I will explain on the next slide.
One thing to note. If you take a look at the rc.firewall script, you will notice that not all traffic is sent via the QUEUE. The main reason for this is due to the expensive copy from kernel space to user space. If all traffic traversed snort_inline, throughput would suffer. Also, we are only interested in dropping malicious traffic originating from our Honeynet; therefore, we only send traffic from our Honeynet through snort_inline.

18. snort_inline reject tcp $HONEYNET any <>
$EXTERNAL_NET 80 (msg: "REJECT";)
drop tcp $HONEYNET any <>
$EXTERNAL_NET 80 (msg: "DROP TCP";)
sdrop tcp $HONEYNET any <>
$EXTERNAL_NET 80 (msg: "SDROP";)
alert tcp $HONEYNET any <>
$EXTERNAL_NET 80 (msg: "Modifying HTTP GET";
content:"GET"; replace:“BET";)
Next we have the rule types. In addition to the existing snort rule types, such as log and alert, we have added three new rule types and a keyword. The new rules work as follows:
Reject: drops the matching packet and sends a reset of icmp host unreach depending on the protocol. *** This does not work in bridge mode at this time.
Drop: drops the matching packet and alerts like a normal snort alert rule.
Sdrop: drops the matching packet and does not log. Think of it as a silent drop.
Finally, we have the replace keyword. In conjunction with the content keyword, we can search the packet for a specific word
Thanks to Lance, we have a script that will convert all of your alert rules to drop rules. This script, named convert.sh, comes with the snort_inline distribution that can be downloaded from snort-inline.sf.net The Honeynet Project also officers a statically compiled version of snort_inline with a rule set and configuration file which will make it a lot easier to deploy.

19. Future Get ip_queue functionality in other OS.
Create a generic way to get packets to SI.
Better way to do connection limiting.
Integrate snort_inline into snort.
Feel free to contact me at if you have ideas or suggestions.
Currently, this functionality we have talked about is limited to the linux opearting system due to the dependency on iptables’ ip_queue kernel module. It would be great to get such functionality with other firewalls.
Another possible avenue of approach would be a generic way of getting the packets from the system and sending them to snort_inline.
Another way of doing connection limiting would be great. The current method is OK, but ideally, we would prefer something that gives you x number of connections per a given time instead of x number of connections followed by x number of connections evenly spread during the given time.
Finally, I am working with the snort team to get snort_inline integrated into the snort source, which would make the our lives easier.
If you have any ideas, suggestions, or complaints, feel free to contact me at this address.

20. Summary
In summary, we talked about a solution to data control. This solution consisted of a firewall and packet scrubber placed in a location where all traffic travels between the Honeynet and the world. This firewall performs connection limiting and can send packets to a packet scrubber to ensure we are being good net citizens.
Since we are using a bridge as our data control mechanism, we can place the device between the honeypots and a production system in order to make the Honeynet look as if it is part of the production network. The data control mechanism is virtually invisible to any attacker because the bridge does not decrement the time to live of the packets.

21. Resources http://www.honeynet.org/papers/gen2/
Contact:

22. An Introduction to Sebek2
Edward Balas

23. What is Sebek A forensic tool used to record activities on a honeypot
Designed to record keystrokes even when intruder uses secure communication channel.
Provides ability to recover files copied to or from a honeypot with SCP.
Operates as a hidden kernel module on a honeypot.

24. Who developed Sebek
Sebek is a collaborative effort within the Honeynet Project.
Originally based on the adore rootkit and coded by Mike Clark for the linux OS.
The second version, written by Edward Balas, included a number of improvements that made it more covert and comprehensive.
This second version is currently being ported to a number of OSs by members of the team

25. Sebek ports under development
Solaris
Job de Hass
Windows
Mike Davis
OpenBSD
Dragos Ruiu

26. Sebek Design There are 2 components:
The client, which collects the data on the honeypot
The server, which records and processes the data exported by the honeypots.

27. Sebek Client Two fundamental issues for the client
collection of forensic data
covert export of the data to a collector

28. Forensic data collection
Client code uses a kernel module to replace the function that implements the read system call.
When data of interest is observed by this module, it is copied and queued for export.
all data recorded in newest version
A variety of techniques are used to hide the existence of the kernel module. These are similar to those used by modern rootkits.
All actions take place in kernel space outside the view of users.

29. Data export
The kernel module creates its own ethernet frames and then sends them directly to the ethernet driver, bypassing firewall filters, and packet sniffing.
A special source MAC OUI is used in the ethernet frames created by Sebek.
When a honeypot running Sebek receives packets with the source OUI, it silently ignores them and prevents users for seeing them.
It is impossible for a user on the honeypot to detect the Sebek traffic being exported.

30. The data that Sebek gathers
In the past Sebek has used a number of heuristics to determine what data if found interesting.
The newest and yet to be released version gathers all data that is seen in the “sys_read” function call.
This ensures that there are no blind spots as there are in the current version.
Once the data is exported and gathered on the collector it is the responsibility of a tool called sbdump.pl to do the interpretation.

31. The Data Export Format
Sebek data is colon delimited and has the following fields:
Timestamp, in seconds since epoch
Process ID for process making read request.
User ID for process making read request.
Command Name for the process.
File Descriptor Number.
TTY name.
Length of Data
Data

32. Sebek: The Server Side
The Sebek server code involves two applications:
sebeksniff
sbdump.pl.
sebeksniff can extract Sebek records from an packet capture file or can sniff directly from a LAN.
Records are stored in a file based on the Source IP address of the honeypot.
sbdump.pl attempts make viewing the data easy, it includes ability to recover SSH copied files.

33. Where Can I get Sebek
Sebek can be downloaded from
For questions or comments contact Edward Balas

34. Using Sebek
Example shows a user logging into a honeypot with SSH and then copying a file to the honeypot
First we will look at the raw data recorded
Second we will how how sbdump.pl can process the data for the user.

35. Raw Data: the df command
:840:0:sshd:7::c:2:d
:842:0:bash:0:pts:c:2:f
:840:0:sshd:7::c:2:f
:842:0:bash:0:pts:c:2:
:840:0:sshd:7::b:2:
:840:0:sshd:7::b:68:Filesystem K-blocks Used Available
Use% Mounted on
:840:0:sshd:7::b:396:/dev/sda
4 28% /
/dev/sda % /boot
/dev/sda % /home
none % /dev/shm
/dev/sda % /usr
/dev/sda % /var

36. Raw Data Observation It is difficult to read the raw data
Redundancy of data for related key strokes
Redundancy for same data getting piped through multiple processes
Control chars and non-text data

37. What can sbdump.pl do? Run as sbdump.pl -c 10.0.0.1
sebek]$ sbdump.pl -c ./
02:47: /05/10 [0:bash:842:pts:0]df
02:47: /05/10 [0:bash:842:pts:0]who
02:47: /05/10 [0:bash:842:pts:0]last
02:48: /05/10 [0:bash:842:pts:0]scp :/tmp/svs_thinking.gif .
02:48: /05/10 [0:ssh:886:pts:3]SSH-2.0-OpenSSH_3.1p1
02:49: /05/10 [0:bash:842:pts:0]scp .
02:49: /05/10 [0:ssh:888:pts:3]SSH-2.0-OpenSSH_3.1p1
02:49: /05/10 [0:ssh:888:pts:4]thepasswd=this
02:49: /05/10 [0:scp:887:pts:7]C svs_thinking.gif

38. Things to notice
The -c argument attempts to get all interactive terminal “character” data, it is not always 100% accurate as you can see by some of the scp data that came through.
We can see that the intruder first checked the disk for capacity, then checked to see who was logged and the last time folks logged in, after that came a failed file transfer attempt then the good attempt
Lets take a closer look at the file transfer…

39. Recovering SCP files Run sbdump.pl with -s arg
Following points descovered:
A file called svs_thinking.gif was copied to the box from a remote site, we know from the eariler run of sbdump that it was
Password used to authenticate was “thepasswd=this”
sebek]$ ./sbdump.pl -s ./
02:49: /05/10 SCP (local)<-remote svs_thinking.gif bytes
02:49: /05/10 SCP: passwd thepasswd=this 1

40. The file…

41. Limits
Data export mechanism uses UDP and thus does not provide reliable stream transport.
Heuristics used in Sebek to detect interesting data are not 100% fool proof, some tuning may be necessary.
Same heuristics rely on static signatures for interesting data, thus providing target for evasion techniques.
All heuristics have been removed from the client for the newest version, however sbdump.pl is still vulnerable to evasion.

42. Michael A. Davis Lead Developer SecurityProfiling, Inc.
Windows port of Sebek
Michael A. Davis
Lead Developer
SecurityProfiling, Inc.

43. The Problem
100% Sebek UNIX functionality on Win32 while maintaining interoperability with Sebek UNIX.

44. Requirements Stealth Logging of GUI and Console I/O
Filter sebek packets from ALL IP network drivers.
Undetectable
Logging of GUI and Console I/O
Example: cmd.exe via IIS.
Secure configuration
Discuss a scenario explaining why there needs to be both a console logger and GUI logger.

45. The Solution In memory patching of syscalls
Hook the console and GUI subsystems
Spy on all the read/write calls performed by the subsystem.
Determine any undocumented structures by reverse engineering
Use a private key for configuration changes
Patch NDIS RegisterProtocol function. All NDIS drivers will be filtered.
All Console applications, not just cmd.exe, will be monitored.
Discuss the undocumented structures in the CSRSS subsystem and the work Blake Watts did

46. Stealth Hook Registry functions(ZwQueryKey, ..)
Hook File System functions(ZwQueryDirectoryObject, …)
Hide all packets we transmit or receive(NDIS hooking)
Use a special configuration application to configure the driver.
Configuration is stored in the actual driver
Discuss how the private key will be the filename of the configuration tool. And how only a binary with the name will be able to actually see the SEBEK kernel driver in the registry and on the file system.

47. Advantages Not easily detectable. Full “view” of the system.
Kernel drivers are not easily detectable to the average windows hacker.
Being in the kernel allows us full control of the system.

48. Disadvantages Never 100% undetectable. May degrade system performance.
Undocumented structures etc lead to a longer development cycle.
Kernel drivers can never be 100% undetectable. Although there are many tricks available to obtain almost 100% undetectability.
Because we filter calls such as ZwQueryDirectoryObject(), the system will perform slightly slower.
Undocumented structures like that in the CSRSS subsystem lead to longer development times and problems when MS releases new OS’s, Service Packs, etc.

49. Current Version Filters network traffic for specific OUI
Will hide other sebek packets from remote hosts
Console monitoring (cmd.exe etc)
Initial version of the configuration application
Can change OUI to filter
Stores configuration in the driver

50. Future Enable stealth features Transmit monitored data to remote host
Hook registry and file system functions
Transmit monitored data to remote host
Once keystroke have been logged send them to the UNIX based archiver
Monitoring GUI applications

51. A complete port of sebek is possible but difficult and time consuming
Summary
A complete port of sebek is possible but difficult and time consuming

52. Resources Download Authors Contributors http://project.honeynet.org
Michael A. Davis
Contributors
Blake Watts
Thanks Blake for all of his hard work.
Ask for help on the project (especially beta testers)

53. Virtual Honeynets
Mike Clark

54. What is a Virtual Honeynet?
Full OS
Run within another OS

55. Virtualization Software
VmWare
User Mode Linux
Others

56. Why use a Virtual Honeynet?
Physical Concerns
Portability
Easy

57. Types of Virtual Honeynets
Self-Contained
Hybrid

58. Disadvantages Everything on 1 system Detection
Can’t support every OS/hardware

59. <project@honeynet.org>
Feel free to contact the Honeynet Project with any idea, questions, or concerns you may have. If you are interested in learning more about information security in general, you can find books written by member of the Honeynet Project at
Best of luck!
--- The Honeynet Project ---