1. Streamlining IP Registrant Identification, Notification, and Blocking for Threats in the Wild Tom Jagatic and Merri Beth Lavagnino Indiana University

2. Copyright Tom Jagatic and Merri Beth Lavagnino, 2005. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

3. Overview I.Who We Are II.Motivation and Preparation III.Recurrent Incident Management System (RIMS) Description IV.Future Work V.Questions?

4. I. Who We Are

5. Indiana University Indiana University has eight campuses: –the original campus in Bloomington, which is a residential campus (38,589 students); –an urban campus in Indianapolis, which also includes the IU Medical Center (29,860 students); –and six regional campuses in the Indiana cities of Gary, South Bend, Fort Wayne, Kokomo, Richmond, and New Albany. Total students: More than 98,000

6. University IT Policy Office (ITPO) Reporting to the VP for IT, responsible for… Information technology policy development, dissemination, interpretation, and education. Computer accounts management. Coordinating response to incidents of abuse, misuse, or inappropriate use of Indiana University information technology resources. Interacting with computer security engineers to develop and administer security education and awareness programs, and to assist with investigation of computer security incidents. Administration of the University IT Security Office (ITSO).

7. University IT Security Office (ITSO) Highly capable in specific technologies Detection (netflow, etc.) Creates auto-processes that provide vulnerable or likely compromised host lists daily to ITPO Incident Response Strategic prevention (firewall, border filters, etc.) Consults with central computing dept or departmental technicians on security issues and options Works with the central computing department on infrastructure security (security CDs, device registration, etc.)

8. Vice President for Information Technology and Research Michael McRobbie Chief IT Security and Policy Officer Mark Bruhn Admin. Assistant L. McNabb Admin. Assistant L. McNabb Disaster Recovery Marge Abels, Program Coordinator Disaster Recovery Marge Abels, Program Coordinator IT Security Office IT Security Officer Tom Davis M. Abels D. Greenberg N. Johnson A. Korty S. Krulewitch D. Monnier M. Rainey J. Sweeny M. Abels D. Greenberg N. Johnson A. Korty S. Krulewitch D. Monnier M. Rainey J. Sweeny REN-ISAC Doug Pearson REN-ISAC Doug Pearson Center Support Stacie Wiegand Center Support Stacie Wiegand IT Policy Office Deputy IT Policy Officer Merri Beth Lavagnino Accounts Administration Laura Klein, Manager Accounts Administration Laura Klein, Manager Incident Response IUB C. Conklin T. Grubb R. Hasty IUB C. Conklin T. Grubb R. Hasty IUPUI B. Hanes IUPUI B. Hanes IUB T. Jagatic IUB T. Jagatic IUPUI J. Abels R. Whitt IUPUI J. Abels R. Whitt Data/Policy Administration Data/Policy Administration S. Wiegand

9. Incident Response Works in web-based incident response application and database (RT -- Request Tracker) Works to locate reported compromised devices Administers tactical filters (dhcp lease blocks, disabling data jacks and usernames, etc.) Interacts with department technicians and individual users about issues with specific devices Reviews and works through lists from ITSO Coordinates large responses with computing dept units and department technicians Works to identify specific misbehaving individuals

10. Incident Response Statistics

11. Incident Response Staffing Three dedicated FTE: –Information Security Technician (Tier 1) –Coordinator/Senior Security Analyst (Tier 2 Technical) –Digital Copyright Compliance Coordinator Plus Manager: –Deputy IT Policy Officer (Tier 2 Policy) Plus others as needed: –Security Engineers in the ITSO (Tier 3 Technical) –Chief IT Security and Policy Officer (Tier 3 Policy) –Other central computing staff (network, messaging, support units, communications staff, etc.)

12. II. Motivation and Preparation

13. We Needed Something to Deal with… Increase in “mass” incidents vs. individual incidents Increase in ITSO’s capability to detect malicious activity Lack of interoperability of existing tools Difficulty to track repeat compromises/ multiple compromises of same host Manual isolation processes no longer fast enough and scalable to effectively manage volume and contain threats in the wild

14. Individual Incident Reports: NOT to be automated From outside or inside sources Concerning one IP number Two kinds: –NON-BEHAVIORAL incidents are things that machines are doing that need to be fixed, such as a virus, worm, spam bot, etc. –BEHAVIORAL incidents are people-initiated, such as chain mail, account misuse, malicious activity, copyright violation, etc.

15. “Mass” Incident Reports: Wanted to automate From security engineer proactive detection work Generally as a result of current vulnerabilities, viruses, worms, etc. or degradation of service Blaster, Nachi, Sobig, Spam bots, Gaobot, Sasser, IRC bots, etc.

16. Preparation Little by little we’ve been addressing pieces of the problem…

17. Building Blocks Central tracking system for all incidents of abuse or misuse of IU IT resources –All emails coming in to abuse, security, itpo, itso, it-incident, copyright, etc. addresses Log host (syslogged data) Network tools –Datajacks database –DHCP registration and administration tool

18. DHCP Registration Tool A first attempt to connect to the network on specific subnets redirects to the DHCP registration web page Enter netid and password to authenticate Enter support provider contact Agree to Appropriate Use statements Automatically grabs MAC Address Creates database of information

26. Recurrent Incident Management System (RIMS) After several basic building blocks were in place, we began working on RIMS to automate the manual processes We realize RIMS will phase out as newer solutions are implemented!! But this is a heck of a lot better today than it was three years ago…

27. III. RIMS Description (Recurrent Incident Management System)

28. What is RIMS? A system to effectively manage recurrent incidents—identification, notification, blocking, and reporting RIMS is a symbiotic system with our overall incident response tracking system RT (Request Tracker, Best Practical) This approach has the following advantages: –One common repository to perform raw-text incident searches (statistics are coarse) –If granular data/statistics are needed, RIMS is not cluttered with unrelated (or potentially misclassified) incidents. The data sent into RIMS is very clean.

29. Overview Incident data sent into RIMS from various sources throughout the day –Trusted sources –Intrusion detection sensors Near real-time (every 3 hours) updates repository of registrant data (60 days) Upon loading of incident data into RIMS, registrant data is associated by date, time, and IP

30. Overview cont’d At a scheduled time (typically in the AM upon full receipt of IDS sensor data) sends summary reports into RT (Incident Tracking System) Incidents are evaluated and notifications sent Blocking occurs Notifications and blocking have the potential to be fully automated (presently human review is performed)

31. Sources of Registrant Data DHCP: full dhcpd handshake log –Focus on the DHCPACKs Dialup: radius/tacacs records VPN: radius records –Make some assumptions about orphaned records These three record types comprise a vast majority of incidents It should be emphasized, this system is designed to associate date, time, ip  user (not the other way around). The above conditions don’t apply in reverse.

33. Identifying a registrant The process of identifying a user (DHCP) –Date, time, and ip  MAC Address  user –Interface with the MAC address registration database Dialup and VPN –Much cleaner –Network IDs are explicitly logged –Information such as calling number, duration, client IP (needed for remote VPN), … Static IPs (not yet, but coming soon) –Effort underway—IP Security and Accountability Project

34. VPN subtlety There are two classes of VPN users –VPN wireless –VPN remote Users on the IU wireless network require use of the VPN These users of course, use DHCP. We treat VPN wireless users, just as any other DHCP host VPN remote users have their VPN account disabled Better to block machines than accounts

35. Gathering person info Connecting to an LDAP directory of people data, the following information is collected –Email address –Full name –Affiliation (faculty, staff, student) –Campus –Department

37. The lookup process I’ve got an ip, where do I look? Simply given an IP address, there’s no clear indication which records contain registrant info Changing network configurations make classifying IPs by record source difficult Our approach doesn’t attempt to make an initial classification Seven passes –Dialup records –VPN records –DHCP records (querying registrant databases from two campuses) –Network names from DHCP records –Identify a campus (needed to know where to block) –If all else fails, perform a hostname lookup (sometimes DNS names are helpful) –Query LDAP people directory

38. IDS Sensor detects Date: Thu, 02 Dec 2004 00:20:00 EST To: [removed] From: [removed] Subject: [removed] From [removed] Thu Dec 2 00: 20:03 2004 Return-Path: [removed] X-Original-To: [removed] Delivered-To: [removed] * compromised * korgo * Snort sensor * 445/tcp with payload "/x.exe" * Hi * block: yes ASAP * [removed] Date-Time IP 12/01-10:53:31.181928 10.1.72.144 12/01-13:34:05.393195 10.1.81.247 12/01-15:53:04.607407 10.1.82.21 12/01-17:04:52.154014 10.1.73.12 12/01-21:28:44.173896 10.1.74.177 Date: Thu, 02 Dec 2004 03:20:06 EST To: [removed] From: [removed] Subject: Korgo hosts From [removed] Thu Dec 2 03: 20:09 2004 Return-Path: [removed] X-Original-To: [removed] Delivered-To: [removed] * compromised * Korgo worm * Snort Sensor * 445/tcp with payload of "/x.exe" * Hi * Block ASAP * [removed] Date Time IP 2004-12-01 00:00:33 10.2.231.44 2004-12-01 00:03:42 10.2.232.112 2004-12-01 11:21:49 10.2.233.36 2004-12-01 15:18:56 10.3.224.41 2004-12-01 16:06:03 10.1.82.21 2004-12-01 17:08:56 10.1.73.12 2004-12-01 19:59:28 10.2.231.142

43. Notification

44. Blocking (e.g. DHCP)

47. Ad-hoc registrant lookups

48. Ad-hoc query pretty printed % event-query.pl | event-pp.pl 2005-04-03 12:00:00 10.1.15.6 ***** DHCP Host Summary ***** Date of activity : 2005-04-03 12:00:00 Assigned IP : 10.1.15.6 MAC address : 1e:a7:de:ad:be:ef Date assigned : 2005-04-02 21:10:08 Computer name : FOOBAR Network : WCC-staff Registrant : Tom Jagatic Email : tjagatic@indiana.edu Status : STUDENT Dept code : n/a Campus code : BL

49. Performance When database loads (to populate the registrant records), performance takes a hit Lookup performance, overall is acceptable A lookup of 10,000 IPs –22.480 user, 1.030 system, 1:23.04 elapsed –This example: 120 lookups per second –Detailed profiling hasn’t been performed, yet latency with other systems undoubtedly a factor

50. Volume of registrant records Kept for 60 days (our retention practice) Upon every load, a purge for old records also occurs DHCP (IUB/IUPUI): –19,640,856 dhcp ACKs in 60 days –40,350 distinct MAC addresses on an average day VPN (IUB/IUPUI): –463,104 records –16,478 distinct users Dialup (IUB/IUPUI): –1,102,933 records –12,636 distinct users

51. Statistics/Reporting Sensors - 1 detect per IP for a given day Fewer users than IPs identified Most typical where IP turnover rapid –VPN –Dialup Counting users is more appropriate than counting IPs

52. MS04011 LSASS Exploit v. Time 2004-09-20 to 2005-04-01

53. LSASS Exploit Detects v. Networks 2004-09-20 to 2005-04-01

54. LSASS Exploit Repeat Offenders: 2004-09-20 to 2005-04-01 User Record Type Num Days Observed Days Diff User1dialup212 User2dialup221 User3dialup391 User4vpn312 User5vpn324 User6dialup3127 User7vpn355 User8vpn316 User9dialup311 User10dialup351 User11vpn326 User12vpn4121 User13dhcp416 User14dialup471 User15dialup440 User16vpn48 User17dialup4193 User18vpn414 User19vpn416 User20dialup47 User21vpn416 User22vpn410 User23dialup462 User24vpn410

55. Design Objected oriented Perl Classes –IRTools::Config - global configuration –IRTools::Log - abstraction for DHCP, Dialup, and VPN lookups –IRTools::Log::Import - used for formatting data prior to loading into DB –IRTools::RI - profile, report, event, block, notify record manipulation –IRTools::Web - future UI class for web MySQL

56. Recurrent Incident Tables Profile –Creation time, description, data source, data coverage, purpose, review, active, warning message, block message, LSP message Report –Creation time, timezone, inclusion criteria, thresholds used, confidence level, Incident Response Action (Block/Notify), User Action (Rebuild/Clean), priority, reporting office, sender Event –Date and Time, record source (campus), record type (dhcp,dialup, vpn, other), record begin, record end, client IP, IP, hostname, network, network id, email, full name, title, dept, campus, MAC address, NetBIOS name, calling number, false positive (Y/N)

57. IV. Future Work

58. Further work Admin Web Interface Self-Service Web Interface –Users can unblock themselves, provided they are not repeaters Reporting for Local Service Providers –Hinges on another project (IPSAP) Incorporation of authentication logs –Ability to associate Network id  ip

59. Further work cont’d. Boolean logic for notification –If detect for profile A and B, only then notify Further enhancements to IDS data interface (SMTP not ideal) API (likely SOAP) DMCA copyright identification/notification Ad hoc and standardized reporting interface

60. V. Questions?

61. Tom Jagatic tjagatic at iu.edu Merri Beth Lavagnino mbl at iu.edu Indiana University IT Policy Office http://www.itpo.iu.edu/