1. Visualisation Network-of-Experts Malvern, UK NOV 4th 2008 Dr. Amy Vanderbilt www.DrAmyVanderbilt.com DrV@DrAmyVanderbilt.com Evaluation of Interactive Visualisation of Network Dynamics

2. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com Evaluation of Interactive Visualisation of Network Dynamics 2 WE DO NOT HAVE THE ANSWERS…BUT WE INTEND TO FIND THEM Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041 QUESTION: what types of visualizations are most effective for discovering, simulating and predicting network dynamics;  Network Dynamics - What do we mean by Network Dynamics  The Need for Evaluation - Why and how the time has come to design portable and repeatable experiments to compare various types of visualizations for these tasks;  Types of Evaluations - What those experiments might entail  Those Pesky Humans – the human element and their effect on visualization research  IST-085 / RTG -041 - Interactive Visualization of Network Dynamics: mission and plans (shameless advertising)

3. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com Network Dynamics 3 CHANGE OVER TIME Definition and Perspective Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041 The changes taking place within a network and the trends within that change.  Changes in structure and topology  Changes in behavior [ network behavior!]  Growth and attrition [changes in the # of nodes]  Changes in density [changes in the # of links and/or links and nodes]  Changes in centrality and other per-node or per-link measures  And more! netmorph.org/.../randalk/results/simple-demos/

4. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com The Need for Evaluation 4 WE CANNOT VISUALIZE REALITY UNTIL WE CAN VISUALIZE DYNAMICS Why Now Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041  Visual Complexity is indexing 623 projects (here are thumbnails of a few) & that is only the beginning  So many visualizations confuse users and prove that the penultimate visualization utility is not yet realized  Network analysis until now has focused on the study and visualization of static snapshot networks  But networks (of any kind) rarely stand still http://www.visualcomplexity.com/vc/

5. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com The Need for Evaluation 5 A STRONG BASE OF RESEARCH IS IN PLACE TO TAKE THIS STEP Why Now Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041  Network dimensions and measures change continually over time  And networks interact with other networks which are also changing  Layered networks then take another dimension of inter-layer dynamics  The military and commercial community has not yet found the solution they seek for network analysis because they have not yet found the solution that visualizes the network environment [the dynamic reality of layered networks interacting and changing over time]  Network dynamics is the next step in visualization towards meeting this need  And hundreds of candidate visualizations are soon to arrive (just as they did for static visualization)  How are we to know what aspects of these visualizations are most effective

6. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com The Need for Evaluation 6 WE NEED TO TEST WORLDWIDE ON DIVERSE DATA SETS AND POPULATIONS Why Portable and Repeatable? Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041  To test on only one population or culture, or on limited data sets is insufficient to determine the efficacy of visualizations of network dynamics  A truly effective visualization will work well on any domain and with any user  But how can we test on multiple domains, data sets and user types?  We need a portable, repeatable test that can be quickly and easily carried out anywhere in the world

7. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com Types of Evaluations 7 EACH OPTION COMES WITH PROS AND CONS Ideas for Portable and Repeatable Evaluations Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041 Here are some starter ideas for what a portable, repeatable evaluation might require or entail: Element Content (Interaction) Varied Dimension Rotating Zooming Sorting Varied Scale Delivery Web Laptop PDA Users Military Personnel Conference Attendees Students  For portability and repeatability we may have to sacrifice by not testing some forms of interaction: KAVEs Glove based interaction Virtual Reality etc

8. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com Those Pesky Humans 8 THE USER IS AN INTEGRAL PART OF THE VISUALIZATION SYSTEM The User as a Critical System Element Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041  But we cannot forget the USER!  For any visualization system to be useful and effective:  It must convey information to the user and  The user must then be able to do something with that information  Ideally, the user will do that something FASTER or BETTER in some way

9. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com Those Pesky Humans 9 VISULAIZATIONS MUST BE TAILORED TO THE USER BY THE USER The Undeniable Fact Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041 For all these years that we have studied information visualization and network analysis, there is one thing we just cannot get past: FOR THE HUMAN IN THE LOOP, VISUALIZATION OF INFORMATION IS A PERSONAL ARTFORM  Two analysis given the same data and the same question to ask will use two different visualizations and methods to reach the same conclusion  So that means there is more than one path to the answer…Yay! BUT that means that analysts have to be able to tailor their visualizations to suit their own tastes and styles  What this means for our experiment is: how can we conclude what is generally effective when everyone wants something different?

10. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com IST-085 / RTG -041 10 VISUALIZATION OF INFORMATION REMAINS A TOP PRIORITY FOR NATO COUNTRIES History Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041 This group emerged out of a long history of groups: 1994DRG Panel 11 Visualising non-Visual Information Workshop  1996 - 1998DRG Panel 8 - RSG30 and the formation of Network of Experts  1998 – 2001 IST013  2001 – 2004IST021  2005 - 2008IST059

11. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com IST-085 / RTG -041 11 IST-085 IS EXPERIMENTAL & FOUNDED IN THE FRAMEWORK AND SURVEY OF IST-059 Plans Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041 IST-085 will take some new directions:  Develop the required experiments to provide insight into what characteristics of interactive visualizations are most likely to aid the military user in determining and predicting the types of change happening within a network, given various influence factors.  Produce a report detailing the results of these studies.  If feasible, the Group will mount a demonstration giving the opportunity for hands on exploration of interactive visualizations of network dynamics to show the experimental design and reported results.  Extend the network dynamics capability of the framework and survey of IST059/RTG025  Continue to foster the activities of the visualization network of experts originally created by RTG-002.

12. Amy K. C. S. Vanderbilt, Ph.D. TITLE (USA) 571-723-5645DrV@DrAmyVanderbilt.com IST-085 / RTG -041 12 INTERESTED??...CONTACT AMY VANDERBILT AT DRV@DRAMYVANDERBILT.COM Shameless Advertising Outline Network Dynamics The Need for Evaluation Types of Evaluations Those Pesky Humans IST-085 / RTG -041 Upcoming Events  2008 MEMBER SEARCH  2009 MEETINGS  Spring – Paris – Group Meeting  Fall – State College PA – N/X Meeting  2010 MEETINGS  Spring – TBD – Group Meeting  Fall – Las Vegas NV – NATO Sponsored Workshop  …JOIN US!