Rootkits Brent Boe Vasanthanag Vasili

Rootkits: What is a Rootkit? A rootkit is a set of tools used for (covertly) maintaining root access to a system Rootkits allow attackers the ability to circumvent protection mechanisms limiting root access Provide a much higher layer of stealth than normal “Trojan programs” by hiding processes and files

Rootkits: What is NOT a Rootkit? A rootkit is not an exploit used to gain root access Rootkits can only work if the attacker can gain administrative access (Typical) Attacker sequence of events Locate vulnerability on target host Run exploit to gain root access Install Rootkit Remove Evidence Locate next vulnerable host

Rootkit Functionality Maintain Access SSH (is for script kiddies) Reverse shell (a bit unusual if servers initiate connections) Covert channel backdoor – a signal system buried in an arbitrary field of a completely innocuous protocol. Destroy evidence Disable shell history (e.g. Linux - unset HISTFILE; export HISTFILE=/dev/null) Kill syslog daemon and freeze the system log Modify log files Attack other systems Local attack tools - Password Cracking, Capture root and access and obtain access to machines Remote attack tools - Scanners and Autorooters DOS tools –Conduct DOS attack on remote server Clean the host system of previous infections More than one rootkit can cause system instability and compromise the rootkit

What does a Rootkit hide? The attacker’s files The attacker’s processes (eg: sniffers, PW crackers) The attacker’s user account Unusual environment variables (network cards in promiscuous mode) Specific network connections to and from compromised machines

Necessary Background User Space Kernel Space The Kernel Space is more privileged than the User Space The lower a rootkit can go, the more likely it is to avoid detection and defeat Host Intrusion Prevention Systems User Space Kernel Space

Necessary Background The Intel x86 based chips use “rings” for access control with Ring 0 being the most permissive and Ring 3 being the most restrictive User programs run in Ring 3 Kernel programs run in Ring 0 Rings 1 and 2 are unused Ring 0

Types of Rootkits Binary Rootkits Kernel Rootkits System call Rootkits Library Rootkits Virtual Machine Rootkits Database Rootkits Runtime Kernel Patches User Space Kernel Space Kernel Space and User Space

Binary Rootkits These rootkits are collections of subverted popular system binaries (or executables). Trojaned to perform action conducive to attacker (eg: hide malicious process) Binary files usually precompiled for particular platform for user to choose & utilize correct one Attacker deploys kit after breaking In via installation script which places binaries over original ones & saves old copies On Linux, the attacker may choose to directly modify the source code on the target machine and recompile the binary.

Some trojaned binaries: inetd, rlogin, rshd, sendmail, sshd, telnetd may contain magic password that provide access to attacker for remote access. ps to hide processes from causal viewing by system admin. netstat provides connection hiding ls, dir provide file hiding login,su,ping provide local access

Binary Rootkit Detection Before the system is infected, compute the checksums of the binaries CRC checksums Cryptographic checksums Better to store the checksums on separate media (i.e. CD-ROM) so an advanced attacker cannot modify the files In practice, if a file (legitimately) changes frequently, this may lead to frequent checksum recomputations and false positives. Checksum computation is used by the program Tripwire

Kernel Rootkits First reported in 1997 Loadable Kernel Modules hook into system kernel and modifies selected sys_call addresses stored in the system call table Replaces the addresses of the legitimate sys_calls with the addresses of the sys_calls that are to be installed by the hacker’s LKM Eg: KNARK ( targeting Linux2.2 Kernel)

Kernel Rootkits User Space Kernel Space LKM Use Loadable Kernel Modules (LKMs) for Linux or Device Drivers for Windows Full kernel access User Space Kernel Space LKM

Kernel rootkit redirecting the system call table Redirects the references to system call table to new location. New system call table is installed in new loc. New system call table contains the address of malicious sys_call functons Redirecting can be done by overwriting the pointer to the original system call table with the address of a new system call table that is created by the hacker

Kernel Rootkit Detection Look for strange/inappropriate modules/device drivers Keep in mind the binaries that would help examine this information may be compromised too. /lib/modules Prevent LKMs altogether by disallowing module loading Sometimes a compile time option StMichael Monitors various portions of the kernel for modifications. When “rootkit activity” is detected, attempts to restore to a previous good state

Necessary Background When a process wants to communicate with the kernel it uses the system call table The process throws a specific interrupt to pass control to the kernel Windows – push the index of the system call in eax. Throw interrupt x2e Linux – push the index of the system call into eax. Throw interrupt x80

System Call and Library Rootkits Replaces the standard system library for relaying kernel information to a user process The user library (libc) provides an interface to the system call table. The advantage – no binaries need to change Duplicate LKM functionality without entering the kernel space Very easy to hide processes and files T0rn8 kit most prominent one

System Call and Library Rootkit Detection System calls like truss, strace, and ltrace can be used to trace the execution path of the system calls Some integrity tools generate checksums against the system call tables.

Virtual Machine Based Rootkits (VMBR) A VMBR moves the targeted system into virtual machine. Instead of moving the attack code lower into the kernel space, it pushes the user higher into the user space The previous (unhooked) OS runs over a virtual machine (as the guest software) The guest is not allowed to interact with states outside of its Virtual Machine The attacker has the liberty to run anything on the machine Any anti-rootkit software run inside of the virtual machine will not detect any modifications to it’s state

Steps of VMBR installation Modify the Boot Sequence to load the Virtual Machine Monitor (VMM) first Modify it after shutdown after all monitoring processes have exited. Interfere with the disk controller’s write – so that only the rootkit can store disk blocks Working at this low level to avoid interference with monitoring software Overwrite the master boot record so the VMBR loads first Reboot and … The target system is now running as a guest, you can interfere with them, but they can’t interfere with you

VM Rootkit Detection Detecting a VM rootkit can be quite difficult (from the inside of the guest software) Possible to detect a rootkit using instructions that reveal information about the kernel state (or the emulated kernel state) redpill – uses the sidt instruction to store the interrupt descriptor table register. Since the VMM needs to move the emulated interrupt descriptor table, the ITDR will begin at a much higher address then it normally would. Easiest way to detect a VM rootkit; boot from an alternate media.

Database Rootkits A database can be considered a type of operating system Users Processes Executables Jobs Symbolic Links

Database Rootkits 1st Generation Rootkits 2nd Generation Rootkits Change the data dictionary (modify a view, procedure, and change synonyms) For example, change ALL_USERS to be select * from sys.user$ u where u.name != ‘HACKER’; 2nd Generation Rootkits Change the binary of the database so that all sys.user$ variables become sys.aser$ Remove the ‘Hacker’ entry from sys.user$ The system is now using sys.aser$ internally, but all integrity checks use sys.user$ 3rd Generation Rootkits For Oracle, Direct SGA (System Global Area) Manipulation – directly modify the contents of the database through modifying the memory the database is stored in

Database Rootkit Detection Examine the internal views for obvious changes Examine the internal system variables for any changes or new, unrecognized variables

Runtime Kernel Patching Modifying the memory of the kernel while it resides in memory. Simply modify a few bytes here, a FAR JMP there to execute the rootkit code, and you’re done. A technique called detour patching totally that can totally circumvent executing code by modifying the control flow at runtime Very difficult to detect Very difficult to pull off successfully Need extremely specific details about the target machine

General Rootkit Detection Behavioral Detection Look for suspect behaviors, such as writes to the memory containing important system call tables Look for a change in the number, order, and frequency of calls Signature Detection –search for unique byte patterns Can be defeated through code obfuscation techniques System Integrity Scans Scan the kernel for inappropriate FAR JMP instructions Detect unauthorized changes to loaded OS components in memory Offline analysis of drives

Sony BMG Rootkit Scandal Sony BMG Music Entertainment was sued in 2005 for surreptitious distribution of rootkit software on audio compact discs. It used a software called Extended Copy Protection (XCP) designed to help prevent unlimited copying and unauthorized redistribution of the music on the disc. XCP interferes with the normal way in which the Microsoft windows OS plays CDs This causes the system vulnerable to malicious code CD ROMS were inoperable due to the change in the registry settings caused by the software

Conclusion Many rootkits practice “offense in depth,” and are by no means limited to only one of the techniques listed here. Control of a system is determined by who can operate closer to hardware, or in the case of equal activity levels, who can best predict the actions of the other The best way to fight rootkits is to prevent them from getting on your system in the first place – Intrusion Detection Systems, Host Intrusion Prevention Systems.

References Beck, M et al. Linux Kernel Programming. 3rd ed. London: Addison Wesley, 2002. Cesare, Silvio. “Runtime Kernel Patching.” 03 Mar 2007. < http://www.uebi.net/silvio/runtime-kernel-kmem-patching.txt > Chuvakin, Anton. An Overview of Unix Rootkits. iDefense Labs: Feb 2003. < www.rootsecure.net/content/downloads/pdf/unix_rootkits_overview.pdf > Hoglund, Greg, Jamie Butler. Rootkits: Subverting the Windows Kernel. Addison Wesley Professional: Upper Saddle River, NJ, 22 July  2005. King, Samuel T. et al. SubVirt: Implementing malware with virtual machines. Mar 01 2007. < www.eecs.umich.edu/virtual/papers/king06.pdf > Kornbrust, Alexander. “Oracle Rootkits 2.0”. Black Hat 2006 USA, Las Vegas, NV. 02 Aug 06. < http://www.red-database-security.com/wp/oracle_rootkits_2.0.pdf >

References Levine, John G. et al. “A Methodology to Characterize Kernel Level Rootkits Exploits that Overwrite the System Call Table”. IEEE. 2004. <http://ieeexplore.ieee.org/iel5/9051/28706/01287894.pdf > Locally checks for signs of a rootkit. 01 Mar 2007. 28 Feb 2007.      <http://www.chkrootkit.org/> Red-database-Security in the news/press. 23 Jan 2007. Red-Database-Security GmbH. 1 Mar 2007.     < http://www.red-database-security.com/wp/db_rootkits > Rootkit. 5 March 2007. Wikimedia Foundation Inc.26 Feb 2007.     <http://en.wikipedia.org/wiki/Rootkit> Rootkits how to combat them. 1996 - 2007.  Kaspersky lab. 29 Feb 2007.     <http://www.viruslist.com> What is a rootkit? . 2 Mar 2007.    <http://www.tech-faq.com/rootkit.shtml> Zaytsev, Oleg. Rootkits, Spyware/Adware, Keyloggers and Backdoors: Detection and Neutralization. A-List Publishing, Sep 1 2006.