1. 1 Real Time Polymorphic Shellcode Detection Evgeny Pinchuk (evgenyp@radware.com) Radware SOC Team

2. 2 Introduction Techniques for detecting buffer overruns –Protocol inspection for anomalies –Exploitation payload detection What’s a shellcode Pattern matching Definition of polymorphism –In order to execute encrypted code, we must decrypt it first. Differences between AV and IDS/IPS –Speed –Accuracy of executed code

3. 3 Polymorphic vs. Regular Regular Shellcode NOP SledShellcodePaddingReturn Address Polymorphic Shellcode NOP SledDecipher EngineShellcode*PaddingReturn Address * Ciphered shellcode

4. 4 Current techniques for detection Counting NOP (or fake NOP) instructions –CPU consuming (making it not RT) –High false positives rate Spectrum Analysis –High false positives –Beatable by four bytes encryption Code emulation –CPU consuming (making it not RT) Data Mining –Involves network learning mechanisms –High false positives rate –Preferred solution

5. 5 The End Lets open the discussion !!!