1. Dist FuncIntroVAAppsATGWrap-up 1/25 Visual Analytics Research at Tufts Remco Chang Assistant Professor Tufts University

2. Dist FuncIntroVAAppsATGWrap-up 2/25 Problem Statement The growth of data is exceeding our ability to analyze them. The amount of digital information generated in the years 2002, 2006, 2010: – 2002: 22 EB (exabytes, 10 18 ) – 2006: 161 EB – 2010: 988 EB (almost 1 ZB) 1: Data courtesy of Dr. Joseph Kielman, DHS 2: Image courtesy of Dr. Maria Zemankova, NSF

3. Dist FuncIntroVAAppsATGWrap-up 3/25 Problem Statement The data is often complex, ambiguous, noisy. Analysis of which requires human understanding. – About 2 GB of digital information is being produced per person per year – 95% of the Digital Universe’s information is unstructured 1: Data courtesy of Dr. Joseph Kielman, DHS 2: Image courtesy of Dr. Maria Zemankova, NSF

4. Dist FuncIntroVAAppsATGWrap-up 4/25 Example: What Does Fraud Look Like? Financial Institutions like Bank of America have legal responsibilities to report all suspicious activities Data size: approximately 200,000 transactions per day (73 million transactions per year) Problems: – Automated approach can only detect known patterns – Bad guys are smart: patterns are constantly changing – No single transaction appears fraudulent – Few experts: fraud detection is considered an “art” – Data is messy: lack of international standards resulting in ambiguous data Current methods: – 10 analysts monitoring and analyzing all transactions – Using SQL queries and spreadsheet-like interfaces – Limited to the time scale (2 weeks)

5. Dist FuncIntroVAAppsATGWrap-up 5/25 WireVis: Financial Fraud Analysis In collaboration with Bank of America – Looks for suspicious wire transactions – Currently beta-deployed at WireWatch – Visualizes 7 million transactions over 1 year Uses interaction to coordinate four perspectives: – Keywords to Accounts – Keywords to Keywords – Keywords/Accounts over Time – Account similarities (search by example)

6. Dist FuncIntroVAAppsATGWrap-up 6/25 WireVis: Financial Fraud Analysis Heatmap View (Accounts to Keywords Relationship) Strings and Beads (Relationships over Time) Search by Example (Find Similar Accounts) Keyword Network (Keyword Relationships) R. Chang et al., Scalable and interactive visual analysis of financial wire transactions for fraud detection. Information Visualization,2008. R. Chang et al., Wirevis: Visualization of categorical, time-varying data from financial transactions. IEEE VAST, 2007.

7. Dist FuncIntroVAAppsATGWrap-up 7/25 What is Visual Analytics? Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces [Thomas & Cook 2005] Since 2004, the field has grown significantly. Aside from tens to hundreds of domestic and international partners, it now has a IEEE conference (IEEE VAST), an NSF program (FODAVA), and a forthcoming IEEE Transactions journal.

8. Dist FuncIntroVAAppsATGWrap-up 8/25 Individually Not Unique Analytical Reasoning and Interaction Visual Representation Production, Presentation Dissemination Data Representation Transformation Validation and Evaluation Data Mining Machine Learning Databases Information Retrieval etc Tech Transfer Report Generation etc Quality Assurance User studies (HCI) etc Interaction Design Cognitive Psychology Intelligence Analysis etc. InfoVis SciVis Graphics etc

9. Dist FuncIntroVAAppsATGWrap-up 9/25 In Combinations of 2 or 3… Analytical Reasoning and Interaction Visual Representation Production, Presentation Dissemination Data Representation Transformation Validation and Evaluation Data Mining Machine Learning Databases Information Retrieval etc InfoVis SciVis Graphics etc

10. Dist FuncIntroVAAppsATGWrap-up 10/25 In Combinations of 2 or 3… Analytical Reasoning and Interaction Visual Representation Production, Presentation Dissemination Data Representation Transformation Validation and Evaluation Interaction Design Cognitive Psychology Intelligence Analysis etc. Tech Transfer Report Generation etc

11. Dist FuncIntroVAAppsATGWrap-up 11/25 Extending Visual Analytics Principles Global Terrorism Database – Application of the investigative 5 W’s Bridge Maintenance – Exploring subjective inspection reports Biomechanical Motion – Interactive motion comparison methods Where When Who What Original Data Evidence Box R. Chang et al., Investigative Visual Analysis of Global Terrorism, Journal of Computer Graphics Forum, 2008.

12. Dist FuncIntroVAAppsATGWrap-up 12/25 Extending Visual Analytics Principles Global Terrorism Database – Application of the investigative 5 W’s Bridge Maintenance – Exploring subjective inspection reports Biomechanical Motion – Interactive motion comparison methods R. Chang et al., An Interactive Visual Analytics System for Bridge Management, Journal of Computer Graphics Forum, 2010. To Appear.

13. Dist FuncIntroVAAppsATGWrap-up 13/25 Extending Visual Analytics Principles Global Terrorism Database – Application of the investigative 5 W’s Bridge Maintenance – Exploring subjective inspection reports Biomechanical Motion – Interactive motion comparison methods R. Chang et al., Interactive Coordinated Multiple-View Visualization of Biomechanical Motion Data, IEEE Vis (TVCG) 2009.

14. Dist FuncIntroVAAppsATGWrap-up 14/25 Human + Computer A Mixed-Initiative Perspective So far, our approach is mostly user-driven Human vs. Artificial Intelligence Garry Kasparov vs. Deep Blue (1997) – Computer takes a “brute force” approach without analysis – “As for how many moves ahead a grandmaster sees,” Kasparov concludes: “Just one, the best one” Artificial Intelligence vs. Augmented Intelligence Hydra vs. Cyborgs (1998) – Grandmaster + 1 computer > Hydra (equiv. of Deep Blue) – Amateur + 3 computers > Grandmaster + 1 computer 1 How to systematically repeat the success? – Unsupervised machine learning + User – User’s interactions with the computer 1. http://www.collisiondetection.net/mt/archives/2010/02/why_cyborgs_are.php ComputerTranslationHuman

15. Dist FuncIntroVAAppsATGWrap-up 15/25 Examples of Human + Computer Computing CAPCHA – RE-CAPCHA – General Crowd-Sourcing Adaptive / Intelligent User Interfaces (IUI) User assisted clustering / searching

16. Dist FuncIntroVAAppsATGWrap-up 16/25 Simple Example Distance Function

17. Dist FuncIntroVAAppsATGWrap-up 17/25 Application 1: Find Important Features Data set: X, 178x13 3 classes add 10 random number columns as extra features

18. Dist FuncIntroVAAppsATGWrap-up 18/25 1 st Step: Success Trying to separate circled green dots from all blue dots

19. Dist FuncIntroVAAppsATGWrap-up 19/25 Result Recall the structure of data set Weight vector: – Randomly generated features gets low weights 0.0960.1500.06200.0180.0110.0250.0390.0370.0470.0910.1860.127 0.0380.01100.01700.0460000 Original Wine Dataset, each instance has 13 feature values 10 Randomly generated feature values for every instance

20. Dist FuncIntroVAAppsATGWrap-up 20/25 Visual Analytics for Political Science

21. Dist FuncIntroVAAppsATGWrap-up 21/25 Aggregate Temporal Graph 1000 simulations 60 time steps in each simulation (time step == a node) (edge == transition) Merged time steps if two states are the same

22. Dist FuncIntroVAAppsATGWrap-up 22/25 Aggregate Temporal Graph

23. Dist FuncIntroVAAppsATGWrap-up 23/25 Gateways and Terminals Each of the yellow vertices is a Gateway to the vertex set of {A}. That is, every maximal path leaving a yellow vertex eventually passes through A. Vertex G is a Gateway to each of the yellow vertices, or Terminals. That is, every maximal path leaving G passes eventually through each of the yellow vertices.

24. Dist FuncIntroVAAppsATGWrap-up 24/25 Applications of Aggregate Temporal Graphs A generalizable representation of problems involving parameter spaces that are too large to explore as a whole, but which are composed of related individual parts can be examined independently Collaborative Analysis – Each analyst’s trail is a simulation – Each configuration state is a node Web Analytics – Each visit is a simulation – Each configuration of a page is a node

25. Dist FuncIntroVAAppsATGWrap-up 25/25 Conclusion Analytical Reasoning and Interaction Visual Represent ation Production, Presentatio n Disseminati on Data Representat ion Transformat ion Validation and Evaluation Visual Analytics is a growing new area that is looking to address some pressing needs – Too much (messy) data, too little time By combining strengths and findings in existing disciplines, we have demonstrated that – There are some great benefits – But there are also some difficult challenges

26. Dist FuncIntroVAAppsATGWrap-up 26/25 Questions? Thank you!

27. Dist FuncIntroVAAppsATGWrap-up 27/25 Backup Slides

28. Dist FuncIntroVAAppsATGWrap-up 28/25 (2) Investigative GTD Where When Who What Original Data Evidence Box R. Chang et al., Investigative Visual Analysis of Global Terrorism, Journal of Computer Graphics Forum (Eurovis), 2008.

29. Dist FuncIntroVAAppsATGWrap-up 29/25 WHY ? WHY ? This group’s attacks are not bounded by geo-locations but instead, religious beliefs. Its attack patterns changed with its developments. (2) Investigative GTD: Revealing Global Strategy

30. Dist FuncIntroVAAppsATGWrap-up 30/25 Domestic Group A geographically- bounded entity in the Philippines. The ThemeRiver shows its rise and fall as an entity and its modus operandi. (2) Investigative GTD: Discovering Unexpected Temporal Pattern

31. Dist FuncIntroVAAppsATGWrap-up 31/25 What is in a User’s Interactions? Types of Human-Visualization Interactions – Word editing (input heavy, little output) – Browsing, watching a movie (output heavy, little input) – Visual Analysis (closer to 50-50) VisualizationHuman Output Input Keyboard, Mouse, etc Images (monitor)

32. Dist FuncIntroVAAppsATGWrap-up 32/25 Discussion What interactivity is not good for: – Presentation – YMMV = “your mileage may vary” Reproducibility: Users behave differently each time. Evaluation is difficult due to opportunistic discoveries.. – Often sacrifices accuracy iPCA – SVD takes time on large datasets, use iterative approximation algorithms such as onlineSVD. WireVis – Clustering of large datasets is slow. Either pre-compute or use more trivial “binning” methods.

33. Dist FuncIntroVAAppsATGWrap-up 33/25 Discussion Interestingly, – It doesn’t save you time… – And it doesn’t make a user more accurate in performing a task. However, there are empirical evidence that using interactivity: – Users are more engaged (don’t give up) – Users prefer these systems over static (query-based) systems – Users have a faster learning curve We need better measurements to determine the “benefits of interactivity”