1. Introduction to Benchmarking (II): The current situation in other imaging communities Introduction to Benchmarking (II): The current situation in other imaging communities

2. The term benchmarking was first used by cobblers to measure people's feet for shoes. They would place someone's foot on a "bench" and mark it out to make the pattern for the shoes. Benchmarking is used to measure performance using a specific indicator (cost per unit of measure, productivity per unit of measure, cycle time of x per unit of measure or defects per unit of measure) resulting in a metric of performance that is then compared to others. cobblersindicatorcobblersindicator

3. General View Of Benchmarking Computing Performance Bioimaging Bioinformatics Data Mining Business 3DEM

4. What to Meassure? Performance Quality & Time & Cost Results

5. CHALLENGE PUBLIC DATASET

6. BIOIMAGING

11. PATTERN RECOGNITION & DATA MINING & COMPUTATIONAL LEARNING

14. TBD: 1. neural information processing systems (NIPS) 2. BCI competitions

15. NEUROSCIENCE

18. COMPUTATIONAL/MOLECULAR BIOLOGY

20. PUBLIC DATASET

24. In the nut shell

25. It aims to establish the state-of-the-art in microarray data analysis, as well as identify progress and highlight promising directions for future efforts. CAMDA

26. Every year, thousands of papers are published that describe new algorithms to be applied to medical images. But very few provide a fair and direct comparison of the newly proposed algorithm with the state of the art. We believe that such comparisons can help to substantially improve the performance of medical image analysis algorithms. Grand Challenge in Medical Imaging Analysis

27. Its purposes are: To encourage research on algorithms that scale to commercial sizes To provide a reference dataset for evaluating research […] Million Song Dataset

28. To exploit the competitive nature of many researchers and drive progress forwards at a faster pace. PASCAL

29. KEY POINTS FOR A CHALLENGE To engage the community To generate or collect some datasets To identify the tasks under evaluation To define the figures of merit To plan a schedule

30. Precedents in 3DEM