1. Barmak Meftah VP, Engineering Fortify Software
Software Security at the Source

2. Why Benchmarks?
A good and hopefully objective gauge for comparing bug detection tools or for that matter any piece of infrastructure software
Should have 3 attributes:
Measure what end-users care about (results that can be easily consumed, performance, and accuracy of output)
Scenarios that users care about and is close to real life situations (TPC benchmarks are a great example)
Start with known bugs (maybe open source projects)