1. Tamara Munzner University of British Columbia Department of Computer Science Outward and Inward Grand Challenges VisWeek08 Panel: Grand Challenges for Information Visualization 20 Oct 2008

2. 2 Grand Challenges: Definitions n grand challenges in other fields n physics: build atom bomb n astro: man on the moon n biology: cure cancer n “outward” grand challenges n high impact, broadly understandable, inspiring n clear milestone to judge success n concrete driving problems to galvanize field

3. 3 Infovis Outward Grand Challenge: TPT n total political transparency n goal: reduce government corruption through civilian oversight n data: campaign contributions, voting records, redistricting, earmarks, registered lobbyists, military procurement contracts, street repair records, real estate assessment records,... n available in theory, not understandable in practice - yet n infovis-complete set of problems n implication: need open software for open data n concern not only for truth, but also for justice n capability for analysis equally distributed in society

4. 4 Inward GC: Towards Science n not ready to solve this or any other outward grand challenge n “inward” grand challenge for infovis: building it into a science n how can we accelerate the transition from a collection of papers to a body of work that constitutes a science? n need synthesis at scales larger than a single paper n textbooks n need common framework unifying all vis work n guide for doing good science within single paper n guide for creating papers that can interlock usefully others n some current thoughts as concrete example...

5. 5 Validation Methods - How To Choose? n unsatisfying flat list of validation methods when writing recent paper [Process and Pitfalls in Writing Infovis Papers. Munzner. Chapter (p. 134-153) in Information Visualization: Human-Centered Issues and Perspectives. Springer LNCS 4950, 2008.] n algorithm complexity analysis n implementation performance (speed, memory) n quantitative metrics n qualitative discussion of result pictures n user anecdotes (insights found) n user community size (adoption) n informal usability study n laboratory user study n field study with target user population n design justification from task analysis n visual encoding justification from theoretical principles n how to choose?

6. 6 Separating Design Into Levels n multiple levels domain problem characterization data/operation abstraction design encoding/interaction technique design algorithm design n three separate design problems n not just the encoding level n each level has unique threats to validity n evocative language from security via software engineering n dependencies between levels n outputs from level above are inputs to level below n downstream levels required for validating some upstream threats

7. 7 problem data/op abstraction encoding/interaction algorithm Problem Characterization n you assert there are particular tasks of target audience that would benefit from infovis tool support n did you get the problem right? n threat: your target users don’t actually do this n immediate validation: you observe/interview target population n vs. assumptions or conjectures n downstream validation: adoption rates n you build tool, they choose to use it to address their needs

8. 8 Abstraction Design n for chosen problem, you abstract into operations on specific data type n often need to derive/transform data type from raw data n ex: choose coast-to-coast train route n abstraction: path following on node-link graph with initial node positions (lat, lon) and two sets of weights on edges (cost, beauty) n can your abstraction solve the problem? n threat: bad choice of abstraction not felicitous for solving problem n downstream validation: observe whether useful with field study problem data/op abstraction encoding/interaction algorithm

9. 9 Encoding/Interaction Design n for chosen abstraction, you design visual encoding, interaction techniques n path following ex: n visual encoding: maximize angular resolution, minimize edge bends, maintain quasi-geographic constraints n interaction: rearrange nodes as selected to make chosen path central n can your encoding/interaction communicate your abstraction? n threat: design not effective for achieving operations n immediate validation: justify that choices do not violate known perceptual/cognitive principles n downstream validation: use system to do assigned tasks, measure human time/error costs problem data/op abstraction encoding/interaction algorithm

10. 10 Algorithm Design n for chosen encoding/interaction, you design computational algorithm n is your algorithm better than previous approaches? n threat: algorithm slower than previous ones n immediate validation: analyze computational complexity n downstream validation: after implementation, measure wallclock time problem data/op abstraction encoding/interaction algorithm

11. 11 Matching Validation To Threats threat: wrong problem validate: observe target users threat: bad data/operation abstraction threat: ineffective encoding/interaction technique validate: justify design threat: slow algorithm build system validate: measure system time validate: measure human time/errors for operation validate: document human usage of deployed system validate: observe adoption rates n common problem: mismatches between design+threat and validation n ex: cannot validate claim of good encoding design with wallclock timings n guidance from model: n explicit separation into levels with linked threat and validation for each

12. 12 Interlocking Between Papers problem assumption data/operation abstraction assumption encoding/interaction technique assumption algorithm n common problem: difficult to make connections between individual papers at different levels n ex: read paper on specific graph layout algorithm, do I know what visual encoding approach is it good for? n guidance from model: n explicitly state upstream assumptions