Speaker ： Hong-Ren Jiang A Novel Testbed for Detection of Malicious Software Functionality 1

Outline Abstract Introduction Related Work Detection Theory Host-Based Detection Network-Based Detection Laboratory Setup Physical Level Virtual Level Experiments and Results Discussion and Further Work Conclusion 2

Abstract 3 This paper presents a novel open-source testbed for behavioural software analysis. designed to meet current trends in the malware community by allowing controlled access to the Internet in the analysis phase. A novel way of using honeypot technology is proposed to build a testbed that is able to analyse current threats.

Introduction (1/2) 4 Malicious code may be spread by introducing malicious hidden functionality. From the more than files they investigated,approximately 15% contained malicious code. However, attackers are constantly improving their code to evade the traditional detection algorithms.

Introduction (2/2) 5 Sandboxing is a relatively new approach to behavioural malware detection. However, a trend among malware authors, presented by Symantec,is to use staged downloads of malware. A modern behavioural analysis environment designed to detect unwanted functionality in downloaded applications.

Related Work 6 Virtual machine technology is proposed as a flexible solution for building the laboratory environment.such as Norman Sandbox and Panda TruPrevent. This paper contributes to existing malware analysis documentation in the following areas: A powerful open-source software analysis testbed is provided The testbed design is based on a number of updated malware detection theories The testbed allows for safe analysis of software,also with malicious content, utilizing staged download techniques.

Detection Theory 7 Most traditional anti-virus systems use signature- based detection,the major drawback being the inability to detect new, unknown malicious code. When new instances of malicious software are found, new signature files have to be written and distributed to the detection tools.

Host-Based Detection(1/2) 8 This section presents techniques that can be used to observe changes in the host operating system,during or immediately after installation of an application. Wang et al.suggest complementing the traditional approaches with a concept they refer to as Auto-Start Extensibility Points. an overwhelming majority of all spyware programs infect a system in such a way that they are automatically started upon reboot and the launch of most commonly used applications.

Host-Based Detection(2/2) 9 Software using the auto-start extensibility points is distinguished by two categories: Standalone applications that are automatically run by registering as an OS auto-start extension, such as a Windows NT service Extensions to existing applications that are automatically run, or extensions to popular applications commonly run by users, like web browsers Wang et al.introduce a concept called cross-view diff to detect stealth software. Cross-time diff aims at comparing the state of a running system with a previous snapshot of the same system. such as Tripwire.

Network-Based Detection(1/4) 10 Network-based detection refers to techniques used to discover the presence of malicious entities by studying properties related to network activity. Router Access Lists is to update routers with records of hosts or network segments of which are allowed or denied access to network resources. In a behavioural analysis scenario, access control lists can be utilized to help detect usage of address spoofing.

Network-Based Detection(2/4) 11 Intrusion detection systems (IDS) are utilities used to inspect both network flow and the content of each packet sent on a network. Mukherjee et al.suggest using IDSs to detect the presence of malware or intruders on a network.Backdoors can be detected and payload fields. Polymorphic worms have proved to make IDS-based detection more difficult. recent research has demonstrated that even such obfuscation techniques are possible to handle

Network-Based Detection(3/4) 12 Spitzner is one of the leaders of the honeypot development. A honeypot is a security resource whose value lies in being probed, attacked, or compromised. Honeypots can be used in production environments to help raising alerts of unauthorized activity on the network and to slow down worms. Honeypots are categorized into three different classes: low-interaction, medium-interaction and highinteraction honeypots.

Network-Based Detection(4/4) 13 Malware enabled to communicate with its environment utilizes different techniques to establish the communication. Skaggs et al.suggest using remote vulnerability scanners to detect the presence of malware on a host.

Laboratory Setup(1/3) 14 Sherlock is configured as a firewall Marple is a “gatekeeper” Horatio acts as a remote vulnerability scanner Drew is a network sniffer Bond is the victim where malicious software is installed.

Laboratory Setup(2/3) 15

Laboratory Setup(3/3) 16

Physical Level 17 It is critical that the physical level has been subject to system hardening so that the probability of unwanted infections is reduced to a minimum. The probability of malware being able to infect both Windows and Linux systems is relatively low, though greater than nil. To further reduce the risk of unwanted infections,the Linux system hardening tool Bastille 、 Administrative tools 、 Samhain.

Virtual Level 18 three roles were identified and included in the behavioural detection environment; a victim, an eavesdropper and an attacker. The victim is the host to be analyzed and where the object to be scrutinized should be installed. The eavesdropper should silently listen in to all traffic to be able to determine if any malicious or unwanted data is sent. The attacker should actively probe the victim for vulnerabilities by simulating attacker behaviour.

Experiments and Results(1/2) 19

Experiments and Results(2/2) 20

Discussion and Further Work 21 look at one example of stealth enabled software. It is therefore difficult to say something about the general detection capabilities of the testbed. None of the existing tools surveyed include aspects from all malware detection approaches. In the future, the quality of the testbed should be validated by running several tests with different categories of malware.

Conclusion 22 This paper has presented a testbed for behavioural analysis of MS Windows software where current malware trends and detection theories are considered. It is demonstrated that analysis environments can be designed without any license cost allowing organizations and students with limited resources to contribute to this important area of research.