Lecture 15: UNIX Forensics 6/25/2003 CSCE 590 Summer 2003

Collecting Volatile Data Trusted toolkit Book recommends renaming trusted tools –Avoids accidents –i.e. tls, tnetstat Same considerations as before – is a tradeoff Unplugging from network causes many UNIX systems to start writing error messages to the log and the console

w – what Find out who is logged in and what they’re doing USER field: username logged in TTY field – control terminal assigned to the session –ttyn – n >= 0 integer. Console logon –ptsn or ttypn may signify a network connection FROM field – where they’ve logged in from (hyphen for at the console) field – local starting time of connection

w - what IDLE field – length of time since last process was run JCPU field – total time used by all processes connected to that terminal PCPU field – processor time time used by process listed in WHAT column WHAT field – process currently running on that terminal

Running Processes ps –eaf (System V) ps –wwwaux (BSD) Either on Linux and newer operating systems Note the START field – when a process started If the COMMAND field of a process contains bizarre characters, may indicate the presence of a buffer overflow attack Bizarre characters or strangely formatted fields may also indicate a trojaned copy of ps that isn’t compiled quite right for your OS

lsof List of every open file used by a process On UNIX an open file can be: –Regular file –Directory –Block special file (disk drive) –Character special file (tty terminal) –A library –A stream –Network file (Internet socket, NFS file or UNIX domain socket)

lsof No options gives full list of all open files and sockets +d dirname: search for all open files and directories directly within in dirname (does not recurse or follow symbolic links) +D dirname: above + recursion -I: all open Internet files and connections, alternative to netstat -D r: option tells lsof to not create a cache file (possible system change in live response)

lsof Output COMMAND – command that was run PID – process ID listed from /proc USER – user associated with process FD – either File Descriptor of file (followed by r, w, u mode + some other possible codes) or: –cwd – current working directory of process –ltx – shared library text (code and data) –Mxx hex memory-mapped type number xx –mem memory-mapped file –pd parent directory –rtd root directory –txt program text (code and data)

lsof Output TYPE – type of node associated with the file DEVICE – device numbers for character special, block special, regular, directory, or NFS files SIZE – size of file or file offset in bytes NODE – inode number of file NAME – name of mount point or file system on which file resides

Loadable Kernel Modules Instead of trojaning executables like ps, rootkits (collections of trojaned system utilities) may actually use LKMs to hide their tracks Insidious for live response: –UNIX kernel is a single program –LKMs can be dynamically linked into the kernel after it is booted –These change the way the operating system works –Use the kernel to hide your tracks – it lies to your trusted tools which have to use system calls to interface with the kernel. This is the point where the kernel can lie to you knark, adore, heroin

kstat LKM Detector Checks memory (/dev/kmem) for info about host, including LKMs Options: –-P: displays all processes –-p pid: display process with process id pid –-M: display kernel’s LKMs’ linked list –-m addr: display info about module at addr –-s: display info about system call table

Finding knark with kstat kstat –s –sys_fork 0xc284652c WARNING! Should be at 0xc0108c88 –LKM rootkits typically change system call table and generates warnings –Older kstat used System.map file, now compiled in –Use warnings as a signature of particular rootkits –knark changes 7 system calls: sys_fork, sys_read, sys_execve, sys_kill, sys_ioctl, sys_settimeofday, sys_clone

Other LKM Rootkits adore: –7 calls as well –sys_fork, sys_kill, sys_clone, sys_write, sys_close, sys_mkdir, sys_getdents rkit –Only change one system call –sys_setuid

kstat -P Shows all processes running Including those hidden by LKM rootkits Compare against output of ps and lsof to see if anything is missing compared to output of kstat –P

Other LKM Rootkit Detectors Chkrootkit Does regular rootkits as well Excellent refernce links on web site – rkscan – Carbonite –LKM itself that does an lsof/ps at the kernel level. “Freezes” the status of every process in Linux's task_struct – &subcontent=/resources/proddesc/carbonite.htm

Finding Sniffers Use ifconfig to collect information about network interface configuration Also look for interfaces in promiscuous mode (sniffing) –Usually interfaces only pay attention to packets addressed to them or to the broadcast address –PROMISC tells the to pay attention to all packets going by on the wire –Some systems (Solaris) do not set promiscuous flag

MAC Times Collect those time stamps Using ls: –ls –alRu (atime access times) –ls –alRc (ctime change times) –Ls –alR (mtime modification times) Could also use our sleuth tools

System Logs Can be volatile if the system is going to remain up If you are going to shut it down for imaging, these might be able to wait –Check /var/log or /var/adm Logs of interest (platform dependent) –utmp – (w utility reads info from here) –wtmp – (last utility reads from here) –lastlog – (lastlog utility) –Process accounting logs (lastcomm utility) –Messages –Web logs, history files, ftp Xferlog –Syslogs (/etc/syslog.conf for locations)

Configuration Files You may want to collect these during initial response: –/etc/passwd –/etc/shadow –/etc/groups –/etc/hosts –/etc/hosts.equiv –/etc/ssh_known_hosts and /etc/ssh_known_hosts2 –~/.rhosts and ~/.shosts –~/.ssh/known_hosts and ~/.ssh/known_hosts2

Configuration Files More configuration files: –/etc/hosts.allow –/etc/hosts.deny –/etc/syslog.conf –/etc/rc* –/etc/inetd.conf –Crontab files in /var/spool/cron (and anacron) –Iptables, ipchains rules –It might be good to grab all of /etc

Other System Configuration df –k uname –a uptime date who arp –a md5sums

Memory Grab memory –# dd if=/proc/kmem | nc forensic.rig.net –# dd if=/proc/kcore | nc forensic.rig.net –# dd if=/dev/mem | nc forensic.rig.net –# dd if=/dev/kmem | nc forensic.rig.net –# dd if=/dev/kcore | nc forensic.rig.net Also might check for crash dumps and core dumps