Presentation is loading. Please wait.

Presentation is loading. Please wait.

BugBench: A Benchmark for Evaluating Bug Detection Tools Shan Lu, Zhenmin Li, Feng Qin, Lin Tan, Pin Zhou and Yuanyuan Zhou University of Illinois, Urbana-Champaign.

Published byGloria Bryan Modified over 9 years ago

Similar presentations

Presentation on theme: "BugBench: A Benchmark for Evaluating Bug Detection Tools Shan Lu, Zhenmin Li, Feng Qin, Lin Tan, Pin Zhou and Yuanyuan Zhou University of Illinois, Urbana-Champaign."— Presentation transcript:

1 BugBench: A Benchmark for Evaluating Bug Detection Tools Shan Lu, Zhenmin Li, Feng Qin, Lin Tan, Pin Zhou and Yuanyuan Zhou University of Illinois, Urbana-Champaign

2 Content of This Talk Share our experience Bug/application characteristics analysis BugBench has been used by  Our previous work [Micro’04, ISCA’04, HPCA’05]  Other research groups: UCSD, Purdue, NCSU, etc.

3 Current Benchmark Suite NameProgramSourceLOCCrash Latency Bug Type NCOMncompress-4.2.4Red Hat1.9KN/AStack smash POLYpolymorph-0.4.0GNU0.7K9040K InstStack smash & Global buffer overflow GZIPgzip-1.2.4GNU8.2K15K InstGlobal buffer overflow COMP129.compressSPEC952.0KN/AGlobal buffer overflow GO099.GoSPEC9529.6KN/AGlobal buffer overflow MANman-1.5h1Red Hat4.7K29.5M InstGlobal buffer overflow BCbc-1.06GNU17.0K189K InstGlobal buffer overflow SQUDsquid-2.3squid93.5K0Global buffer overflow CALBcachelibUIUC6.6KN/AUninitialized read CVScvs-1.11.4GNU114.5KN/ADouble free YPSVypserv-2.2Linux NIS11.4KN/AMemory leak PFTPproftpd-1.2.9ProFTPD68.9KN/AMemory leak SQUD2squid-2.4squid104.6KN/AMemory leak HTPDhttpd-2.0.49Apache224KN/AData race MSQL1msql-4.1.1MySQL1028KN/AData race MSQL2msql-3.23.56MySQL514KN/AAtomicity MSQL3msql-4.1.1MySQL1028KN/AAtomicity PSQLpostgresql-7.4.2PostgreSQL559KN/ASemantic HTPD2httpd-2.0.49Apache224KN/ASemantic memory related multi-thread related semantic Other type of bugs: In searching …

4 Functionality miss stack buffer overflow miss moderate global-buffer overflow miss stack buffer overflow miss 1 Byte global-buffer overflow Failed to apply Valgrind Purify CCured NameCatch Bug?Related Memory Object Type ValgrindPurifyCCured NCOMNo YesStack POLYVaryYes Stack & global buffer GZIPYes Global buffer COMPNo Yes GONoYes MANYes BCYes Heap buffer SQUDYes N/A

5 Heap Usage Ratio [Heap/(Heap+Stack)] 99% 0% 76.6%23.9% 85.1% Overhead Valgrind: 6.4X (NCOM) ~ 119X (BC) Purify: 28% (POLY) ~ 76X (BC) CCured: 4% (POLY) ~ 3.7X (GZIP).85.48.69.65.55.52.5.62 Mem. Access Freq. (# per Instruction) BC NCOM Memory Alloc Freq. (# per MInst) 769 0 138 480 18% 4% 28% 69% 1.35X

6 Experience Summary Building benchmark is a time-consuming and long-term work  Motivate automatic tools to extract bugs Bug/application characteristics are important for selecting applications Need cooperation from entire community

Download ppt "BugBench: A Benchmark for Evaluating Bug Detection Tools Shan Lu, Zhenmin Li, Feng Qin, Lin Tan, Pin Zhou and Yuanyuan Zhou University of Illinois, Urbana-Champaign."

Similar presentations

UW-Madison Computer Sciences Multifacet Group© 2011 Karma: Scalable Deterministic Record-Replay Arkaprava Basu Jayaram Bobba Mark D. Hill Work done at.

UW-Madison Computer Sciences Multifacet Group© 2011 Karma: Scalable Deterministic Record-Replay Arkaprava Basu Jayaram Bobba Mark D. Hill Work done at.
A Performance Comparison of DRAM Memory System Optimizations for SMT Processors Zhichun ZhuZhao Zhang ECE Department Univ. Illinois at ChicagoIowa State.

A Performance Comparison of DRAM Memory System Optimizations for SMT Processors Zhichun ZhuZhao Zhang ECE Department Univ. Illinois at ChicagoIowa State.
Idempotent Code Generation: Implementation, Analysis, and Evaluation Marc de Kruijf ( ) Karthikeyan Sankaralingam CGO 2013, Shenzhen.

Idempotent Code Generation: Implementation, Analysis, and Evaluation Marc de Kruijf ( ) Karthikeyan Sankaralingam CGO 2013, Shenzhen.
Dec 5, 2007University of Virginia1 Efficient Dynamic Tainting using Multiple Cores Yan Huang University of Virginia Dec. 5 2007.

Dec 5, 2007University of Virginia1 Efficient Dynamic Tainting using Multiple Cores Yan Huang University of Virginia Dec
® 1 Stack Value File : Custom Microarchitecture for the Stack Hsien-Hsin Lee Mikhail Smelyanskiy Chris Newburn Gary Tyson University of Michigan Intel.

® 1 Stack Value File : Custom Microarchitecture for the Stack Hsien-Hsin Lee Mikhail Smelyanskiy Chris Newburn Gary Tyson University of Michigan Intel.
LIFT: A Low-Overhead Practical Information Flow Tracking System for Detecting Security Attacks - F. Qin, C. Wang, Z. Li, H. Kim, Y. Zhou, Y. Wu (UIUC,

LIFT: A Low-Overhead Practical Information Flow Tracking System for Detecting Security Attacks - F. Qin, C. Wang, Z. Li, H. Kim, Y. Zhou, Y. Wu (UIUC,
Hastings Purify: Fast Detection of Memory Leaks and Access Errors.

Hastings Purify: Fast Detection of Memory Leaks and Access Errors.
CORK: DYNAMIC MEMORY LEAK DETECTION FOR GARBAGE- COLLECTED LANGUAGES A TRADEOFF BETWEEN EFFICIENCY AND ACCURATE, USEFUL RESULTS.

CORK: DYNAMIC MEMORY LEAK DETECTION FOR GARBAGE- COLLECTED LANGUAGES A TRADEOFF BETWEEN EFFICIENCY AND ACCURATE, USEFUL RESULTS.
SAFECode Memory Safety Without Runtime Checks or Garbage Collection By Dinakar Dhurjati Joint work with Sumant Kowshik, Vikram Adve and Chris Lattner University.

SAFECode Memory Safety Without Runtime Checks or Garbage Collection By Dinakar Dhurjati Joint work with Sumant Kowshik, Vikram Adve and Chris Lattner University.
1 Refining Buffer Overflow Detection via Demand-Driven Path-Sensitive Analysis Wei Le and Mary Lou Soffa University of Virginia sotesty.cs.virginia.edu.

1 Refining Buffer Overflow Detection via Demand-Driven Path-Sensitive Analysis Wei Le and Mary Lou Soffa University of Virginia sotesty.cs.virginia.edu.
CPSC 388 – Compiler Design and Construction

CPSC 388 – Compiler Design and Construction
Mark and Sweep Algorithm Reference Counting Memory Related PitFalls

Mark and Sweep Algorithm Reference Counting Memory Related PitFalls
SAFECode SAFECode: Enforcing Alias Analysis for Weakly Typed Languages Dinakar Dhurjati University of Illinois at Urbana-Champaign Joint work with Sumant.

SAFECode SAFECode: Enforcing Alias Analysis for Weakly Typed Languages Dinakar Dhurjati University of Illinois at Urbana-Champaign Joint work with Sumant.
CSE 451: Operating Systems Section 1. Why are you here? 9/30/102.

CSE 451: Operating Systems Section 1. Why are you here? 9/30/102.
Korey Breshears. Overview  What are automated security tools?  Why do we need them?  What types of tools are there?  What problems do these tools.

Korey Breshears. Overview  What are automated security tools?  Why do we need them?  What types of tools are there?  What problems do these tools.
Introduction The Approach ’ s Overview A Language of Pointers The Type System Operational Semantics Type Safety Type Inference The Rest of C Experiments.

Introduction The Approach ’ s Overview A Language of Pointers The Type System Operational Semantics Type Safety Type Inference The Rest of C Experiments.
Stack buffer overflow http://en.wikipedia.org/wiki/Stack_buffer_overflow.

Stack buffer overflow
TaintCheck and LockSet LBA Reading Group Presentation by Shimin Chen.

TaintCheck and LockSet LBA Reading Group Presentation by Shimin Chen.
October 2003 What Does the Future Hold for Parallel Languages A Computer Architect’s Perspective Josep Torrellas University of Illinois

October 2003 What Does the Future Hold for Parallel Languages A Computer Architect’s Perspective Josep Torrellas University of Illinois
Efficient and Flexible Architectural Support for Dynamic Monitoring YUANYUAN ZHOU, PIN ZHOU, FENG QIN, WEI LIU, & JOSEP TORRELLAS UIUC.

Efficient and Flexible Architectural Support for Dynamic Monitoring YUANYUAN ZHOU, PIN ZHOU, FENG QIN, WEI LIU, & JOSEP TORRELLAS UIUC.

Similar presentations

Ads by Google