1. Incident Handling in Academia What to do when you have been hacked!

2. The Presenters  Scott Fendley –BS Comp Science – U of AR 1999 –MS Comp Science – U of AR 2004 –Security Analyst, Dept of Computing Services –Volunteer Incident Handler, SANS Institute  David Merrifield –Associate Director of Computing Services

3. Session Description  Explores how to handle the attacks on your Internet infrastructure.  Discusses a time-tested 6 step procedure for Incident Handling.  Touches on the legal issues relevant to all Academic Institutions (K12 or Higher Ed)  Dealing with Law Enforcement and handling Evidence  Employee Monitoring vs Student Monitoring

4. Disclaimers, Disclaimers, Disclaimers  I am not a lawyer. Consult your nearest legal counsel if you choose to handle incidents on your campus or have questions.  The majority of this information is the basis of my procedures at the University of Arkansas, but your mileage may vary.

5. Foundation of Incident Handling  An Action Plan for dealing with intrusions, cyber-theft, denial of service and other security-related events  Events can be of a electronic nature or of a physical nature.

6. Definitions  Incident – an adverse event in an information system, and/or network, or the threat of the occurrence of such event. –Ex: unauthorized use of another user’s account –Execution of malicious code –Unauthorized use of system privileges  Event – Any observable occurrence in a system and or/network. –Ex: Packet Traces –System Boot Sequences –Anything that you can record in your IH notebook

7. Incident Handling Metaphor  Incident Handling is like First Aid.  The Handler is under pressure and mistakes can be costly  Practice is a key. Skills degrade without use.  Use pre-designed forms and procedures, and call on others for help.

8. Emergency Action Plan  Remain Calm.  Communicate with your management, and coordinate with your co-workers to keep things focused.  Use formalized language. –EX: Whiskey Five Yankee Mic, We have a bogey on your nine. –Explicit meaning, no room for interpretation is less likely to cause mistakes.

9. Emergency Action Plan  REMAIN CALM (still!) Do not hurry. Mistakes can be costly.  Notes, logs and other evidence are crucial –If the perpetrator is ever found and arraigned, how can you testify if your notes are not organized and detailed?  Failure to take notes is the most common mistake.  Consult your legal counsel for how long you should keep your logs.  Quality not Quantity

10. Emergency Action Plan  Take good notes. –Remember what your English teacher taught you. –The 4 W’s Who? What? When? Where? –Extra Credit for the 5 th W and the H Why? How?

11. Emergency Action Plan (1)  Notify your manager of your progress  Do you have easy access to your School’s phone directory? Pager numbers? Home numbers?  If you are over your head, do not hesitate to ask for help –FBI Field Office –handlers@incidents.orghandlers@incidents.org –Local Law Enforcement –Trained Computer Forensic Investigators

12. Emergency Action Plan (2)  Enforce a “need to know” policy.  Do not tip your hand to potential insider threats.  Use out of band communications. (Don’t email people about IH discussions.) –Telephones –Faxes –Personal Visits  PGP Keys

13. Emergency Action Plan (3)  Contain the problem. (stop the bleeding) –Pull the network plug? –Pull the power plug? –Forensic Evidence Quandary.

14. Containment Micro Example  Call the user and say “Take your hands off the keyboard and move away from the computer.”  Stand up go to the back of the computer and unplug the network (and/or modem).  Don’t touch anything, we’ll be right there.  Fax instructions/forms for them to fill out.

15. Emergency Action Plan (4)  Make a backup of the affected system(s) as soon as is practical. Use new, unused media.  Make a binary, or bit-by-bit backup.  Failure to make a backup is the second most common error.  Chain of custody of the evidence.

16. Emergency Action Plan (5)  Get rid of the problem. Identify what went wrong if you can. Take steps to correct the deficiencies that allowed the problem to occur.  Nuke the computer or just scrub it?  Get back in business using clean backups and monitor the system to make sure it can resume functioning.

17. Emergency Action Plan (6)  Learn from this experience.  Share your experience with others. –Sys-admin List for K12 –Arktech List for Universities and Colleges –Another useful list is Unisog@sans.org for all Educational entities.Unisog@sans.org  Review the incident from start to completion.  Identify areas of improvement  Engineers versus Mathematicians

18. Seven Deadly Sins of IH  Failure to report or ask for help  Incomplete/non-existent notes  (Accidental) Mishandling/destroying evidence  Failure to create working backups.  Failure to contain or eradicate  Failure to prevent re-infection  Failure to apply lessons learned

19. Emergency Action Plan Summary  Remain calm, don’t hurry.  Notify your oranizations’s management, apply need to know, use out of band communications.  Take good notes (even if you aren’t/can’t prosecute).  Contain the problem  Back up the system(s), collect evidence  Eradicate the problem and get back to business  Lessons Learned

20. Six Steps of Incident Handling  Preparation  Identification  Containment  Eradication  Recovery  Lessons Learned

21. Preparation  Update your organization’s disaster recovery plan to include Incident Handling  Establish visibility and a compensation plan for the team. (Slush fund for food and caffeine for long weekends or evenings of mitigating an emergency.)  Checklists!  Emergency Communications Plan

22. Preparation Key Points  Password Access  Conduct training for incident handlers (War Games)  Establish guidelines for inter-departmental cooperation.  Build relationships with techies and sys admins  Develop interfaces with law enforcement agencies in your area.

23. Preparation - Jump Bag  Small tape recorder –Blank Tapes  Binary Backup Utils –Safe Back –Ghost –Encase  Forensic Software –TCT –Autopsy –Encase  Small Hub and cables  Laptop (extra batteries)  CD’s with clean binaries –Sysinternals –Foundstone –Windows Resource Kit  Call List, Phone book  Cell Phone (batteries)  Fresh Blank Media (CD-Rs Floppys, Zip, etc)

24. Preparation in a nutshell  Policy  People  Data  Software/Hardware  Communications  Supplies  Transportation  Space  Power and environmental controls  Documentation

25. Identification  Fire Alarm Analogy –Who can pull a fire alarm? –Who authorizes re-entry?  Maintain situation awareness  Provide current “intelligence”  Correlate information (mailing lists are great sources for newest worms/viruses or attacks)

26. Signs of an incident  Intrusion Detection system alarm  Suspicious entries in system or networking accounting  Discrepancies in logs  (Un) successful logon attempts  Unexplained, new user accounts  Unexplained processes or services running  Notification via abuse@ address or phone call  Poor system performance  Unusual time of usage.

27. Identification  Initial Assessment  “Efficient handling of errors is part of the process”  Be careful to maintain a provable chain of custody.  Use the tape record if at all possible to keep notes for you on what commands you run and actions you do.  Make law enforcement sign for any evidence you hand off to them. Assign a value to it.

28. Containment  This is where we cross the threshold in which we begin to actively modify the system.  Keep the system pristine  Pull the system off the network (or perhaps the subnet off the network).  Load your binaries, set the path  Backup the system

29. Containment  Safely store any backup disks/tapes so that they will not be lost and/or stolen. Multiple copies are best with volatile media types.  Keep a low profile.  Analyze a copy of the backup  Report to management on progress  Are you sure you backed up the media in question?

30. Containment  Acquire logs and other sources of information.  Firewalls, IDS Logs  Logs from other systems nearby

31. Containment  Consult with system owners (departmental technical staff)  Change passwords  Determine possible other systems that have potentially had passwords breached.  Packet sniffers are easy to install.

32. Eradication  Is your schools policy to nuke the computer and reinstall with a secured OS, or just clean and secure?  Improve your defenses  Perform vulnerability analysis and system audits.  Locate the most clean backup and carefully install it.

33. Recovery  Restore from backups if required  Be sure you do not restore the malware  Secured system?  Validate the system and create baselines  Test that everything on the system is working as expected with the owner.  Place the final decision on the system owner of when to restore operations.  Monitor the systems

34. Follow-up / Lessons Learned  Develop a follow-up report –Start as soon as possible –Include any forms you used in identification step –Details, details, details!  Lessons Learned Meeting  Executive Summary Report  Recommended Changes to procedures?  Additions to jump kit

35. Legal Issues to Academia  HIPAA –Privacy Rule (2002) –Security Rule (2005)  FERPA (Buckley Amendment)  DMCA  Patriot Act

36. Monitoring  Monitoring employees  Student Privacy  Student-employees?

37. Law Enforcement Contacts  University Police  City Police or County Sheriff  FBI (Field office in LR)  Secret Service  Department of Homeland Security  Infraguard Arkansas

38. More Information  http://www.sans.org/ http://www.sans.org/  http://www.securityfocus.com/ http://www.securityfocus.com/  http://www.foundstone.com/ http://www.foundstone.com  http://www.sysinternals.com/ http://www.sysinternals.com/  http://www.incidents.org/ http://www.incidents.org/  http://ists.dartmouth.edu/ http://ists.dartmouth.edu/

39. Questions?  Contact me at scottf@uark.edu or call me at 479-575-2022.scottf@uark.edu  Also, talk to those in the state and across the nation for specific questions. –Sys_admin@lists.state.ar.usSys_admin@lists.state.ar.us –Unisog@sans.orgUnisog@sans.org