1. HUMAN COMPUTATION IN VISUALIZATION: USING PURPOSE DRIVEN GAMES FOR ROBUST EVALUATION OF VISUALIZATION ALGORITHMS VISUAL ANALYTICS AND IMAGING LAB DEPT OF COMPUTER SCIENCE NAFEES AHMED, ZIYI ZHENG, KLAUS MUELLER

2. VISUAL ANALYTICS AND IMAGING LAB  HUMAN COMPUTATION  Relevance in visualization  AN Example: Disguise  Findings  Future thoughts

3. human Computation WHAT & WHY?

4. VISUAL ANALYTICS AND IMAGING LAB ?

5. ?

6. ? Crowd sourcing VizWiz Soylent

7. VISUAL ANALYTICS AND IMAGING LAB ? PURPOSE DRIVEN games ESP Game Tag a tune verbosity fold.it Ete-RNA Phylo Eye-wire

8. HUMAN COMPUTERS in VISUALIZATION WHY DOES IT MAKE SENSE?

9. VISUAL ANALYTICS AND IMAGING LAB VISUALIZATION Creates Understandable and Meaningful pictures from data In analyzing pictures and patterns still far better than any computers Human brain and vision

10. VISUAL ANALYTICS AND IMAGING LAB Image Data VISUALIZATION DESIGN RAW Data Data Analysis Prepared Data Focus Data Geometric Data Image Data FilteringMappingRendering Design/Select Candidate Visualize Data Evaluate Finalized Method Not Satisfactory Satisfactory RAW Data VISUALIZATION rendering

11. VISUAL ANALYTICS AND IMAGING LAB Evaluation of visualization User evaluation controlled environment Cost Small number of samples Crowd sourced Noisy data Cost

12. evaluation: using an online game Design, implementation and findings

13. VISUAL ANALYTICS AND IMAGING LAB problem Transparency perception ability to perceive the ordering given the blended color Evaluate blending algorithm In terms of transparency perception

14. VISUAL ANALYTICS AND IMAGING LAB Porter & Duff Hue Preserving Local blending Local blending with blurring Algorithms [Chuang 09][Porter 84] [Wang 09]

15. VISUAL ANALYTICS AND IMAGING LAB ARCADE | survival

16. VISUAL ANALYTICS AND IMAGING LAB ARCADE | survival

17. VISUAL ANALYTICS AND IMAGING LAB INTRUDER OVER COLLECTOR HIT INTRUDER COLLECTOR OVER INTRUDER HIT INTRUDER SCORE HEALTH SCORE HEALTH ANY STATE HIT COLLECTOR SCORE HEALTH ANY STATE NO ACTION SCORE HEALTH INTRUDER EXPLODES ANY STATE CLICK EMPTY SPACE SCORE HEALTH correctness

18. VISUAL ANALYTICS AND IMAGING LAB correctness PLAYERS try to Optimize score Score only improves with correct guessing of order Each click gives us feedback about human perception of transparency ordering

19. VISUAL ANALYTICS AND IMAGING LAB observations 1.Playability within 15 days of launch… 261 players 30,000+ data-points generated.

21. VISUAL ANALYTICS AND IMAGING LAB observations 2.Data collection speed 14.6 clicks per minute 1000 players x 24hr playtime = 21 million data points!

22. VISUAL ANALYTICS AND IMAGING LAB observations 3.cost once developed and deployed almost FREE!

23. VISUAL ANALYTICS AND IMAGING LAB observations 4.Data quality Provides much less noisy data than other web-based methods. trying to optimize score, not earning! Extent of data-points collected

24. VISUAL ANALYTICS AND IMAGING LAB observations

25. VISUAL ANALYTICS AND IMAGING LAB observations 5.limitations TIME TO DEVELOP ensuring correctness and fun MAKING IT POPULAR

26. Future thoughts Ideas, motivations

27. VISUAL ANALYTICS AND IMAGING LAB Future thoughts INTEGRATION WITH A LEARNING AGENT FOR ADAPTIVE DESIGN OF NEW ALGORITHMS

28. VISUAL ANALYTICS AND IMAGING LAB Future thoughts OPTIMIZATION OF VISUALIZATION PARAMETERS (transfer functions/ view angles)

29. THANK you VISUAL ANALYTICS AND IMAGING LAB DEPT OF COMPUTER SCIENCE NAFEES AHMED nuahmed@cs.stonybrook.edu www.kuntal.name Ziyi zheng zizhen@cs.stonybrook.edu www.cs.sunysb.edu/~zizhen/ Klaus mueller mueller@cs.stonybrook.edu www.cs.sunysb.edu/~mueller/ http://vail.cewit.stonybrook.edu/Projects/HPU/Disguise