1. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE USC CSci530 Computer Security Systems Lecture notes Fall 2009 Dr. Clifford Neuman University of Southern California Information Sciences Institute

2. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE CSci530: Security Systems Lecture 7, October 9 2009 (Following Mid-term exam) Introduction to Malicious Code Dr. Clifford Neuman University of Southern California Information Sciences Institute

3. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Classes of Malicious Code How propagated Trojan Horses –Embedded in useful program that others will want to run. –Covert secondary effect. Viruses –When program started will try to propagate itself. Worms –Exploits bugs to infect running programs. –Infection is immediate.

4. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Classes of Malicious Code The perceived effect Viruses –Propagation and payload Worms –Propagation and payload Spyware –Reports back to others Zombies –Controllable from elsewhere

5. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Activities of Malicious Code Modification of data –Propagation and payload Spying –Propagation and payload Advertising –Reports back to others or uses locally Propagation –Controllable from elsewhere Self Preservation –Covering their tracks

6. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Defenses to Malicious Code Detection –Virus scanning –Intrusion Detection Least Privilege –Don’t run as root –Separate users ID’s Sandboxing –Limit what the program can do Backup –Keep something stable to recover

7. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Trojan Horses A desirable documented effect –Is why people run a program A malicious payload –An “undocumented” activity that might be counter to the interests of the user. Examples: Some viruses, much spyware. Issues: how to get user to run program.

8. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Trojan Horses Software that doesn’t come from a reputable source may embed trojans. Program with same name as one commonly used inserted in search path. Depending on settings, visiting a web site or reading email may cause program to execute.

9. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Viruses Resides within another program –Propagates itself to infect new programs (or new instances) May be an instance of Trojan Horse –Email requiring manual execution –Infected program becomes trojan

10. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Viruses Early viruses used boot sector –Instruction for booting system –Modified to start virus then system. –Virus writes itself to boot sector of all media. –Propagates by shared disks.

11. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Viruses Some viruses infect program –Same concept, on start program jumps to code for the virus. –Virus may propagate to other programs then jump back to host. –Virus may deliver payload.

12. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Recent Viruses Spread by Email Self propagating programs –Use mailbox and address book for likely targets. –Mail program to targeted addresses. –Forge sender to trick recipient to open program. –Exploit bugs to cause auto execution on remote site. –Trick users into opening attachments.

13. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Viruses Phases Insertion Phase –How the virus propagates Execution phase –Virus performs other malicious action Virus returns to host program

14. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Analogy to Real Viruses Self propagating Requires a host program to replicate. Similar strategies –If deadly to start won’t spread very far – it kills the host. –If infects and propagates before causing damage, can go unnoticed until it is too late to react.

15. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE How Viruses Hide Encrypted in random key to hide signature. Polymorphic viruses changes the code on each infection. Some viruses cloak themselves by trapping system calls.

16. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Macro Viruses Code is interpreted by common application such as word, excel, postscript interpreter, etc. May be virulent across architectures.

17. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Worms Propagate across systems by exploiting vulnerabilities in programs already running. –Buffer overruns on network ports –Does not require user to “run” the worm, instead it seeks out vulnerable machines. –Often propagates server to server. –Can have very fast spread times.

18. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Delayed Effect Malicious code may go undetected if effect is delayed until some external event. –A particular time –Some occurrence –An unlikely event used to trigger the logic.

19. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Zombies/Bots Machines controlled remotely –Infected by virus, worm, or trojan –Can be contacted by master –May make calls out so control is possible even through firewall. –Often uses IRC for control.

20. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Spyware Infected machine collect data –Keystroke monitoring –Screen scraping –History of URL’s visited –Scans disk for credit cards and password. –Allows remote access to data. –Sends data to third party.

21. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Some Spyware Local Might not ship data, but just uses it –To pop up targeted ads –Spyware writer gets revenue for referring victim to merchant. –Might rewrite URL’s to steal commissions.

22. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Theory Can not detect a virus by determining whether a program performs a particular activity. –Reduction from the Halting Problem But can apply heuristics

23. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Defenses to Malicious Code Detection –Signature based –Activity based Prevention –Prevent most instances of memory used as both data and code

24. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Defenses to Malicious Code Sandbox –Limits access of running program –So doesn’t have full access or even users access. Detection of modification –Signed executables –Tripwire or similar Statistical detection

25. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Root Kits Hide traces of infection or control –Intercept systems calls –Return false information that hides the malicious code. –Returns fall information to hide effect of malicious code. –Some root kits have countermeasures to attempts to detect the root kits. –Blue pill makes itself hyper-root

26. Copyright © 1995-2009 Clifford Neuman - UNIVERSITY OF SOUTHERN CALIFORNIA - INFORMATION SCIENCES INSTITUTE Best Detection is from the Outside Platform that is not infected –Look at network packets using external device. –Mount disks on safe machine and run detection on the safe machine. –Trusted computing can help, but still requires outside perspective