BugBench: A Benchmark for Evaluating Bug Detection Tools Shan Lu, Zhenmin Li, Feng Qin, Lin Tan, Pin Zhou and Yuanyuan Zhou University of Illinois, Urbana-Champaign.

Slides:

Advertisements

Similar presentations

UW-Madison Computer Sciences Multifacet Group© 2011 Karma: Scalable Deterministic Record-Replay Arkaprava Basu Jayaram Bobba Mark D. Hill Work done at.

Advertisements

A Performance Comparison of DRAM Memory System Optimizations for SMT Processors Zhichun ZhuZhao Zhang ECE Department Univ. Illinois at ChicagoIowa State.

Idempotent Code Generation: Implementation, Analysis, and Evaluation Marc de Kruijf ( ) Karthikeyan Sankaralingam CGO 2013, Shenzhen.

Dec 5, 2007University of Virginia1 Efficient Dynamic Tainting using Multiple Cores Yan Huang University of Virginia Dec

® 1 Stack Value File : Custom Microarchitecture for the Stack Hsien-Hsin Lee Mikhail Smelyanskiy Chris Newburn Gary Tyson University of Michigan Intel.

LIFT: A Low-Overhead Practical Information Flow Tracking System for Detecting Security Attacks - F. Qin, C. Wang, Z. Li, H. Kim, Y. Zhou, Y. Wu (UIUC,

Hastings Purify: Fast Detection of Memory Leaks and Access Errors.

CORK: DYNAMIC MEMORY LEAK DETECTION FOR GARBAGE- COLLECTED LANGUAGES A TRADEOFF BETWEEN EFFICIENCY AND ACCURATE, USEFUL RESULTS.

SAFECode Memory Safety Without Runtime Checks or Garbage Collection By Dinakar Dhurjati Joint work with Sumant Kowshik, Vikram Adve and Chris Lattner University.

1 Refining Buffer Overflow Detection via Demand-Driven Path-Sensitive Analysis Wei Le and Mary Lou Soffa University of Virginia sotesty.cs.virginia.edu.

CPSC 388 – Compiler Design and Construction

Mark and Sweep Algorithm Reference Counting Memory Related PitFalls

SAFECode SAFECode: Enforcing Alias Analysis for Weakly Typed Languages Dinakar Dhurjati University of Illinois at Urbana-Champaign Joint work with Sumant.

CSE 451: Operating Systems Section 1. Why are you here? 9/30/102.

Korey Breshears. Overview  What are automated security tools?  Why do we need them?  What types of tools are there?  What problems do these tools.

Introduction The Approach ’ s Overview A Language of Pointers The Type System Operational Semantics Type Safety Type Inference The Rest of C Experiments.

Stack buffer overflow

TaintCheck and LockSet LBA Reading Group Presentation by Shimin Chen.

October 2003 What Does the Future Hold for Parallel Languages A Computer Architect’s Perspective Josep Torrellas University of Illinois

Efficient and Flexible Architectural Support for Dynamic Monitoring YUANYUAN ZHOU, PIN ZHOU, FENG QIN, WEI LIU, & JOSEP TORRELLAS UIUC.

Decomposing Memory Performance Data Structures and Phases Kartik K. Agaram, Stephen W. Keckler, Calvin Lin, Kathryn McKinley Department of Computer Sciences.

Memory Subsystem Performance of Programs using Coping Garbage Collection Authers: Amer Diwan David Traditi Eliot Moss Presented by: Ronen Shabo.

Yuanyuan ZhouUIUC-CS Architectural Support for Software Bug Detection Yuanyuan (YY) Zhou and Josep Torrellas University of Illinois at Urbana-Champaign.

LIFT: A Low-Overhead Practical Information Flow Tracking System for Detecting Security Attacks Feng Qin, Cheng Wang, Zhenmin Li, Ho-seop Kim, Yuanyuan.

Exploiting Load Latency Tolerance for Relaxing Cache Design Constraints Ramu Pyreddy, Gary Tyson Advanced Computer Architecture Laboratory University of.

PathExpander: Architectural Support for Increasing the Path Coverage of Dynamic Bug Detection S. Lu, P. Zhou, W. Liu, Y. Zhou, J. Torrellas University.

Quarantine: A Framework to Mitigate Memory Errors in JNI Applications Du Li ， Witawas Srisa-an University of Nebraska-Lincoln.

MemTracker Efficient and Programmable Support for Memory Access Monitoring and Debugging Guru Venkataramani, Brandyn Roemer, Yan Solihin, Milos Prvulovic.

Bell: Bit-Encoding Online Memory Leak Detection Michael D. Bond Kathryn S. McKinley University of Texas at Austin.

1 RISE: Randomization Techniques for Software Security Dawn Song CMU Joint work with Monica Chew (UC Berkeley)

U NIVERSITY OF M ASSACHUSETTS A MHERST Department of Computer Science 2006 Exterminator: Automatically Correcting Memory Errors Gene Novark, Emery Berger.

Qin Zhao (MIT) Derek Bruening (VMware) Saman Amarasinghe (MIT) Umbra: Efficient and Scalable Memory Shadowing CGO 2010, Toronto, Canada April 26, 2010.

Flexible Reference-Counting-Based Hardware Acceleration for Garbage Collection José A. Joao * Onur Mutlu ‡ Yale N. Patt * * HPS Research Group University.

Jiang Lin 1, Qingda Lu 2, Xiaoning Ding 2, Zhao Zhang 1, Xiaodong Zhang 2, and P. Sadayappan 2 Gaining Insights into Multi-Core Cache Partitioning: Bridging.

Backwards-Compatible Array Bounds Checking for C with Very Low Overhead Dinakar Dhurjati and Vikram Adve ICSE 2006 Itay Polack

School of Electrical Engineering and Computer Science University of Central Florida Anomaly-Based Bug Prediction, Isolation, and Validation: An Automated.

Computer Science Detecting Memory Access Errors via Illegal Write Monitoring Ongoing Research by Emre Can Sezer.

Title of Selected Paper: IMPRES: Integrated Monitoring for Processor Reliability and Security Authors: Roshan G. Ragel and Sri Parameswaran Presented by:

Chapter 14 Memory API Chien-Chung Shen CIS, UD

Application-Aware SoftWare AnomalyTreatment (SWAT) of Hardware Faults Byn Choi, Siva Hari, Man-Lap (Alex) Li, Pradeep Ramachandran, Swarup Sahoo, Sarita.

CNIT 127: Exploit Development Ch 4: Introduction to Heap Overflows

Measuring Memory using valgrind CSCE 221H Parasol Lab, Texas A&M University.

U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science Yi Feng & Emery Berger University of Massachusetts Amherst A Locality-Improving.

Protecting C Programs from Attacks via Invalid Pointer Dereferences Suan Hsi Yong, Susan Horwitz University of Wisconsin – Madison.

Detecting Atomicity Violations via Access Interleaving Invariants

Soyeon Park, Shan Lu, Yuanyuan Zhou UIUC Reading Group by Theo.

Automatic CPU-GPU Communication Management and Optimization Thomas B. Jablin,Prakash Prabhu. James A. Jablin, Nick P. Johnson, Stephen R.Breard David I,

PRES: Probabilistic Replay with Execution Sketching on Multiprocessors Soyeon Park and Yuanyuan Zhou (UCSD) Weiwei Xiong, Zuoning Yin, Rini Kaushik, Kyu.

Content Coverity Static Analysis Use cases of Coverity Examples

Presented by: Daniel Taylor

Bugs (part 2) CPS210 Spring 2006.

Reducing Memory Interference in Multicore Systems

YAHMD - Yet Another Heap Memory Debugger

Debugging Memory Issues

ECE Dept., Univ. Maryland, College Park

Zhichun Zhu Zhao Zhang ECE Department ECE Department

Seminar in automatic tools for analyzing programs with dynamic memory

malloc(): Dynamic Memory Management

Checking Memory Management

Jihyun Park, Changsun Park, Byoungju Choi, Gihun Chang

PerfView Measure and Improve Your App’s Performance for Free

Mid Term II Review.

José A. Joao* Onur Mutlu‡ Yale N. Patt*

Introduction to Static Analyzer

Chien-Chung Shen CIS/UD

CETS: Compiler-Enforced Temporal Safety for C

Sunita·ain.com n.com. sunita·ain.comsunita·ain.com In.comIn.com.

How Memory Leaks Work with Memory Diagram

Makefiles, GDB, Valgrind

Sampling Dynamic Dataflow Analyses

Presentation transcript:

BugBench: A Benchmark for Evaluating Bug Detection Tools Shan Lu, Zhenmin Li, Feng Qin, Lin Tan, Pin Zhou and Yuanyuan Zhou University of Illinois, Urbana-Champaign

Content of This Talk Share our experience Bug/application characteristics analysis BugBench has been used by  Our previous work [Micro’04, ISCA’04, HPCA’05]  Other research groups: UCSD, Purdue, NCSU, etc.

Current Benchmark Suite NameProgramSourceLOCCrash Latency Bug Type NCOMncompress-4.2.4Red Hat1.9KN/AStack smash POLYpolymorph-0.4.0GNU0.7K9040K InstStack smash & Global buffer overflow GZIPgzip-1.2.4GNU8.2K15K InstGlobal buffer overflow COMP129.compressSPEC952.0KN/AGlobal buffer overflow GO099.GoSPEC9529.6KN/AGlobal buffer overflow MANman-1.5h1Red Hat4.7K29.5M InstGlobal buffer overflow BCbc-1.06GNU17.0K189K InstGlobal buffer overflow SQUDsquid-2.3squid93.5K0Global buffer overflow CALBcachelibUIUC6.6KN/AUninitialized read CVScvs GNU114.5KN/ADouble free YPSVypserv-2.2Linux NIS11.4KN/AMemory leak PFTPproftpd-1.2.9ProFTPD68.9KN/AMemory leak SQUD2squid-2.4squid104.6KN/AMemory leak HTPDhttpd Apache224KN/AData race MSQL1msql-4.1.1MySQL1028KN/AData race MSQL2msql MySQL514KN/AAtomicity MSQL3msql-4.1.1MySQL1028KN/AAtomicity PSQLpostgresql-7.4.2PostgreSQL559KN/ASemantic HTPD2httpd Apache224KN/ASemantic memory related multi-thread related semantic Other type of bugs: In searching …

Functionality miss stack buffer overflow miss moderate global-buffer overflow miss stack buffer overflow miss 1 Byte global-buffer overflow Failed to apply Valgrind Purify CCured NameCatch Bug?Related Memory Object Type ValgrindPurifyCCured NCOMNo YesStack POLYVaryYes Stack & global buffer GZIPYes Global buffer COMPNo Yes GONoYes MANYes BCYes Heap buffer SQUDYes N/A

Heap Usage Ratio [Heap/(Heap+Stack)] 99% 0% 76.6%23.9% 85.1% Overhead Valgrind: 6.4X (NCOM) ~ 119X (BC) Purify: 28% (POLY) ~ 76X (BC) CCured: 4% (POLY) ~ 3.7X (GZIP) Mem. Access Freq. (# per Instruction) BC NCOM Memory Alloc Freq. (# per MInst) % 4% 28% 69% 1.35X

Experience Summary Building benchmark is a time-consuming and long-term work  Motivate automatic tools to extract bugs Bug/application characteristics are important for selecting applications Need cooperation from entire community