LECTURE 6 MALICIOUS SOFTWARE NETW4005 COMPUTER SECURITY A LECTURE 6 MALICIOUS SOFTWARE

Content 6.1 Malicious Software 6.2 Malware Technology 6.3 Viruses 6.4 Worms 6.5 Bots 6.6 Rootkits

6.1 Malicious Software Programs that exploit system vulnerabilities. Known as malicious software or malware Malicious software can be divided into three categories: 1. Program fragments that need a host program E.g. viruses, logic bombs, and backdoors 2. Independent self-contained programs E.g. worms, bots 3. Replicating or not Sophisticated threat to computer systems

6.2 Malware Terminology Name Description Virus Virus attaches itself to a program and propagates copies of itself to other programs Worm Worm program that propagates copies of itself to other computers Logic Bomb Logic bomb triggers action when condition occurs Trojan Horse Trojan horse program that contains unexpected additional functionality Backdoor Backdoor program modification that allows unauthorized access to functionality Mobile Code Mobile code software that can be shipped unchanged to a heterogeneous collection of platforms and execute with identical semantics

6.3 Viruses Piece of software that infects programs Modifying them to include a copy of the virus So it executes secretly when host program is run Specific to operating system and hardware Taking advantage of their details and weaknesses A typical virus goes through phases of: 1. Dormant (Idle) 2. Propagation (Copies itself) 3. Triggering (Being activated) 4. Execution (Running – Damaging)

6.3.1 Virus Structure A computer virus has three parts: 1) Infection mechanism: The means by which a virus spreads, enabling it to replicate. The mechanism is also referred to as the infection vector. 2) Trigger: Event or condition determining when the payload is activated or delivered. 3) Payload: What the virus does, besides spreading. The payload may involve damage or may involve benign but noticeable activity.

6.3.2 Virus Classification Boot Sector Virus Description Boot Sector Virus Infects a master boot record or boot record. Spreads when a system is booted from the disk containing the virus. File Infector Infects files that the OS consider to be executable. Macro Infects files with macro code that is interpreted by an application. Encrypted The virus creates a random encryption key, stored with the virus, and encrypts the remainder of the virus. When an infected program is invoked, the virus uses the stored random key to decrypt the virus. When the virus replicates, a different random key is selected.

Stealth virus Polymorphic Virus A form of virus explicitly designed to hide itself from detection by antivirus software. Polymorphic Virus A virus that mutates with every infection, making detection by the “signature” of the virus impossible. Metamorphic It rewrites itself completely at each iteration, increasing the difficulty of detection. It may change their behavior as well as their appearance.

6.3.4 Virus Countermeasures Prevention - ideal solution but difficult Best approach is to be able to do the following: 1. Detection - determine & locate virus 2. Identification - identify the specific virus that infected 3. Removal - remove all traces of the virus from the infected program If detect but can’t identify or remove, must discard and replace infected program

6.3.5 Anti-Virus Evolution Virus & Antivirus technology have both evolved Early viruses simple code, easily removed As become more complex, nowadays. Four generations of Antivirus software: 1. First: Signature scanners to identify a virus 2. Second: Heuristics rules used to search virus infections 3. Third: Identify virus by its actions 4. Fourth: Packages consisting of a variety of antivirus techniques.

6.4 Worms A worm is a program that can replicate itself and send copies from computer to computer across network connections. using email, remote exec, remote login Has phases like a virus: Dormant, Propagation, Triggering, Execution Propagation phase: searches for other systems, connects to it, copies self to it and runs Concept of worm was introduced in John Brunner’s novel “Shockwave Rider” in 1975. First known worm was implemented by Xerox Palo Alto labs in 1980’s

6.4.1 Worm Technology The state of the art in worm technology includes the following: Multiplatform: Can attack in variety of platforms. Multi-exploit: Exploiting web servers, browsers, e-mail, file sharing & other networking machines to attack. Ultrafast spreading: Accelerating the speed of a worm. Polymorphic: Takes multiple copies and act differently. Metamorphic : Have a repertoire of behavior patterns Transport vehicles: Ideal for spreading other attack tools Zero-day exploit: A worm should exploit an unknown vulnerability that is only discovered by the general network community when the worm is launched.

6.4.2 Worm Countermeasures Overlaps with Anti-Virus techniques. Antivirus software can be used to detect worms Worms also cause significant network activity Worm defense approaches include: Signature-based worm scan filtering (Worm signature) Filter-based worm containment (Worm Content) Payload-classification-based worm containment (Anomaly detection) Threshold Random Walk (TRW) scan detection (Random Scan) Rate limiting and Rate halting (Limit Traffic & Blocks outgoing traffic) There is considerable overlap in techniques for dealing with viruses and worms. Once a worm is resident on a machine, antivirus software can be used to detect it. In addition, because worms propagation generates considerable network activity, the monitoring of that activity can lead form the basis of a worm defense. Have classes: Signature-based worm scan filtering: generates a worm signature, which is then used to prevent worm scans from entering/leaving a network/host. Filter-based worm containment: focuses on worm content rather than a scan signature. The filter checks a message to determine if it contains worm code. Payload-classification-based worm containment: examine packets to see if they contain a worm using anomaly detection techniques Threshold random walk (TRW) scan detection: exploits randomness in picking destinations to connect to as a way of detecting if a scanner is in operation Rate limiting: limits the rate of scanlike traffic from an infected host. Rate halting: immediately blocks outgoing traffic when a threshold is exceeded either in outgoing connection rate or diversity of connection attempts. Rate halting can integrate with a signature- or filter-based approach so that once a signature or filter is generated, every blocked host can be unblocked; as with rate limiting, rate halting techniques are not suitable for slow, stealthy worms.

6.4.3 Proactive Worm Containment (PWC) PWC scheme is host based software. PWC monitors the rate of frequency of outgoing connection attempts and the diversity of connections to remote hosts. When such a surge is detected, the software immediately blocks its host from further connection attempts. PWC system consists of a PWC manager & PWC agents in hosts.

PWC operates as follows 1) A PWC agent monitors outgoing traffic for scan activity, If a surge is detected, the agent: a) Issues an alert to local system; b) Blocks all outgoing connection attempts; c) Transmits the alert to the PWC manager; d) Starts a relaxation analysis. 2) PWC manager receives an alert, and propagates the alert to all other agents. 3) The host receives an alert, and performs the following actions: a) blocks all outgoing connection attempts from the specific alerting port b) starts a relaxation analysis.

6.4.4 Network Based Worm Defense (NBWD)

The key element of a NBWD is worm monitoring software. Two types of monitoring software are needed: 1) Ingress Monitors (Located at Border router, External firewall) 2) Egress Monitors (Located at individual LANs, External border router, Switch, External Firewall) The two types of monitors can be collocated. It is designed to catch the source of a worm attack by monitoring outgoing traffic.

NBWD architecture works as follows: 1. Sensors deployed at various network locations detect a potential worm. 2. and send alerts to a central server that correlates / analyzes incoming alerts. 3. forwards info to a protected environment, where worm is sandboxed for analysis 4. protected system tests the suspicious software against an appropriately instrumented version of the targeted application to identify the vulnerability. 5. protected system generates one or more software patches and tests these. 6. system sends the patch to the application host to update the targeted application.

6.5 Bots A bot (robot), also known as a zombie or drone. It is a program that secretly takes over hundreds or thousands of Internet-attached computer and then uses that computer to launch attacks that are difficult to trace to the bot's creator. The collection of bots often is capable of acting in a coordinated manner; referred to as a botnet. A botnet exhibits three characteristics 1) The bot functionality 2) A remote control facility 3) A spreading mechanism to propagate the bots and construct the botnet. Some uses of bots include: Distributed denial-of-service attacks, spamming, sniffing traffic, keylogging, spreading new malware, installing advertisement add-ons, attacking irc chat networks, manipulating online polls/games.

6.6 Rootkits Set of programs installed for admin access Malicious and stealthy changes to host O/S May hide its existence Subverting report mechanisms on processes, files, registry entries etc May be: Persistent or memory-based User or kernel mode Installed by user via trojan or intruder on system Range of countermeasures needed

Summary Malicious Software Malware Technology Viruses Worms Bots Rootkits