Copyright Joel Rosenblatt 2010 Copyright Joel Rosenblatt 2010. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author. Copyright (c) 2010 The Trustees of Columbia University in the City of New York

GULP Grand Unified Logging Program Educause Security Professionals Conference April 14, 2010 Joel Rosenblatt Manager Computer & Network security Columbia University, CISO Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Columbia Network Environment Large research university Decentralized management structure Over 90,000 network nodes Over 55,000 MAC addresses active on average Decentralized computer support No sniffing traffic or scanning machines allowed “Free Love” IP address assignments No university wide, corporate like, firewalls 80,000 email accounts Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Initial problems to solve We wanted to offer pain free use of our network to visiting people We needed to reduce the overhead of registering machines Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Free Love The solution is …. Copyright (c) 2010 The Trustees of Columbia University in the City of New York

What is “Free Love” From http://www.educause.edu/ir/library/pdf/erm0266.pdf “Free Love” and Secured Services, by Vace Kundakci “Free love” allowed all computers, whether public or private, wired or wireless, in residence halls, at the libraries, in faculty and staff offices, or anywhere else on campus to connect directly to the network, and thereby to the world, without further ado. Copyright (c) 2010 The Trustees of Columbia University in the City of New York

NEW Problems to solve How do you answer the question… Who is using a certain IP address? Who is using a certain MAC address? When was a certain IP address being used by a certain user? Copyright (c) 2010 The Trustees of Columbia University in the City of New York

GULP The NEW solution is … Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Grand Unified Logging Program - GULP Problem – How do you know who is using an IP/MAC address without registration? GULP processes the logs from 12+ different services that require authentication It processes information from DHCP and the ARP cache to associate MAC address with IP address GULP correlates all information A user can be tracked by IP, MAC, or UNI – even if the IP is not on the Columbia network The data is kept for 28 days and then purged Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Basic GULP workflow Pull all logs that associate an authenticated user, process, timestamp and IP address Dump information into a database Pull information from the network that associates IP address, MAC address and time (DHCP and ARP cache) Add network information into appropriate records in the database Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Some technical stuff … We are currently pulling logs from servers Future enhancement will be a push process We use cron to run scripts to pull different logs at different times depending on service We use a Perl parser designed for each log to extract the relevant data A script runs overnight to correlate the user>IP>MAC mapping Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Network security vs Public Safety What machine used that IP address at 3:00pm Was the machine with MAC address XX connected to the network yesterday How many MACs used that jack Who used that IP address at 3:00pm Did the person named John Doe log in to the network yesterday How many people used that IP address – and when Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Nifty Web interface Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Sample GULP for UNI Joel Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Gulp for IP 68.197.237.122 Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Question No one has seen this student for 10 days, can you tell me anything? Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Lost person procedure Look up ID of missing person using GULP Analyze login records for location and times Work with Public Safety to establish if this information matches up with missing person report Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Question A (faculty, staff, student) received this anonymous email from Yahoo – can you tell me who sent it? Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Procedure to track down some anonymous email senders Get IP address of email sender from headers (this does not work easily with Gmail) Pop into GULP See what comes up We have found that, quite often, the offender will fire off the nasty email, then login to our systems to check on their own email, once they authenticate, GULP has them Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Question We got a call from LE that someone is applying for Credit Cards using the identities of employees, can you help? Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Procedure to help Law Enforcement find Bad Guys Get some data from LE – in this case, we got the IP address that the applications were being submitted from Pop into GULP and see what you get P.S. The person is currently in jail Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Question (Department that runs their own network – I know you have them ) We can’t find this machine anywhere. All I know is the IP address, can you help? Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Procedure to find lost computers Take the IP address and pop into GULP The user or users of that computer will be displayed – then it is a simple matter of calling them and asking where they are Copyright (c) 2010 The Trustees of Columbia University in the City of New York

GULP data mining Use GULP data to discover compromised passwords Use GULP data to satisfy Audit requirements Use GULP data to expose MAC spoofers Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Compromised Password Discovery Create a daily process that looks at the last few days of GULP data (we use 48 hours) Look at the location information of the logins (We use ASN data) If a user logs in from “x” locations or more (we use 6) in the time period, there is a strong possibility that the password has been compromised Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Audit requirements One of the things that Auditors often ask is how do you monitor the logins of employees to sensitive systems GULP is the perfect answer – you know who logged in from where and can even setup an “off hour” filter to look for unusual logins Copyright (c) 2010 The Trustees of Columbia University in the City of New York

MAC spoofers GULP correlates User, IP and MAC Using some additional information, you can look for multiple MAC addresses being use by the same ID from the same Jack or location (We have written some additional tools, but that is a different presentation ) Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Summary GULP is a powerful and useful tool for bringing together disparate pieces of information. GULP can be used in a “free love” or a managed environment. Once you have GULP, it will quickly become the “go to” tool for any question that involves WHO or WHERE Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Questions? Copyright (c) 2010 The Trustees of Columbia University in the City of New York

Joel Rosenblatt Joel at columbia.edu 212 854 3033 Copyright (c) 2010 The Trustees of Columbia University in the City of New York