1. 1 The Red Team Gwen Jacobs Ed Lazowska

2. 2 What biologists want … z Can I evaluate an experimental design? z Can I store the results? z Can I visualize the results? z Can I reproduce the results? z Can I make inquiries? z Can I share and build upon data, tools, results?

3. 3 z Can I store the results? y Data validation / quality control x Partial data x Errors in the data x Flamingly false data x Synonyms and homonyms x Context in which the data was gathered y Storing/retrieving combinatorial structures y Shared repositories

4. 4 z Can I visualize the results? y Multi-dimensional data visualization is the challenge y Need time as a variable

5. 5 z Can I reproduce the results? y Jill’s talk goes here

6. 6 z Can I make inquiries? y Data mining y Non-parametric statistics y Content-based image retrieval y Standards: yes or no?

7. 7 z Can I share data, tools, results? y Ontologies / semantics y Dealing with synonyms/homonyms y Standards: yes or no? x Yes: Can’t we all just get along? x No: Standards impede innovation; what we need is technologies that would allow ontologies to interoperate – schema mapping etc. (cont’d …)

8. 8 y How to make the best algorithms known y How to make tools that are usable by other than the developer, and that can interoperate y Data integration / federation y Searching the intergalactic knowledge base

9. 9 What can we do? z Fundamentally change the structure of the biomedical enterprise y Make computing explicit y Improve the peer review of computational work y Adequately fund the Roadmap y Fund algorithm and tool development where there is a clear biological driver y Create alternative funding models for hardening software x New panels, new panelists

10. 10 z Define “challenge problems” y “Here are 3 large databases, here are 3 tough questions, whoever’s first wins” x Use your own tools Tests tool capabilities x Have someone else use your tools Tests tool usability

11. 11 z Support training y Hourglass model x Broad at the undergraduate level x Narrow and deep at the graduate level x Broadening again post-graduate y Undergraduate x Less specialized x More concept-focused x CS students should have a serious minor (e.g., biology) x Bio students should have lots of computation (programming, data structures, algorithms, statistics, a smattering of databases and visualization)

12. 12 z Support creation of robust software by a non-R01 process y Need for software development and algorithm development needs to be explicitly recognized in R01’s y Separate mechanism needed to fund the hardening of software tools that are of value to the community y Also may need to explicitly support algorithm and tool development (community infrastructure)

13. 13 z Focus on tools usable by others z Figure out how to mandate reproducible research – openly publish y data y tools y papers

14. 14 z Need progress on simulation y Hierarchical / multi-level y Hybrid z Computer scientists and biologists have mismatched goals y CS people seek a general solution y Biologists want a specific application addressed Dangling observations