Temporal Search and Replace: A novel tool to simplify event sequences in large complex temporal datasets Allan Fong Hanseung Lee Rongjian Lan University of Maryland Department of Computer Science

Outline  Background  Our Contributions  Temporal Search and Replace Features  User Interface & Interaction Description  Demo  Search Algorithm  Conclusions and Future Work

Background  Motivation:  Simplify the visualization of large complex temporal datasets  Related works:  Search and replace research for graphics [Kurlander 1992] [Yeh 2006]  Psychology research  Miller’s 7 +/- 2 and chunking [Miller 1956]  Temporal chunking (clustering) [Farrell 2012]  Gestalt perception (202)

Our Contributions  Goal:  Develop and integrate a Temporal Search and Replace tool into EventFlow  Contributions:  Replace capabilities in EventFlow  Search capability for repeating sequences  Search capability for no ordered sequences  Search capability using wildcard notations  Customers/End Users:  Megan Monroe and Catherine Plaisant (UMD HCIL)  Sheila Weiss (UMD Baltimore)

Temporal Search and Replace Features TypeShape & ColorVisualization Point EventColored Triangle Interval EventColored Interval Shape Any Point EventBlack Triangle Any Interval EventBlack Interval Shape Any EventBlack Diamond RepetitionBlack Rectangle No OrderRed Oval ≥4≥4 {, =,,,, } ≥2 = {, …}, One time stamp (e.g., one day)  Journal Published: 5/1/10 Two time stamps (e.g., start time and an end time)  Professor: 9/5/09 - 9/30/11

Visualizing Nested Constraints Reverse (BFS) Breadth-First Search is used to plot inner constraint boxes & ovals first Dynamic add / remove constraints are easily done using the following property All nodes on the same level are disjoint sets A parent node contains the range of all its child nodes Indices: Constraints: Events:

User Interface & Interaction

Demo

Search Algorithm  Repetition We extend the original EventFlow fixed-length pattern search algorithm(Temporal Pattern Search - TPS) to support flexible pattern searching. Search index is repeatedly reset so a pattern can be matched multiple times  No Order Instead of permuting the events and do sequential event matching, keep matching greedily until each event in the constraint is matched for once.  Wildcard Finding any event after the current time in the record

Embedded Constraints  Using tree structure to store the embedded constraints  Using stack to store the active constraints Indices: Constraints: Events:

Conclusions and Future Work  Introduce a novel temporal event search and replace tool  Extend EventFlow’s search algorithm to support repetition, no order and wildcard constraints.  Conduct user study with 9 participants  Future work includes:  Add rules that better suit the purpose of simplification of temporal events.  Polish the user interface to reduce the stress of query formulation.  Pattern-level find and replace.

Acknowledgements  Megan Monroe, Catherine Plaisant, and Ben Shneiderman  Sheila Weiss  Kent Norman  And for all the participants of the usability study!

Thank you for your attention! Question?

Backup Slides

Usability Study  9 participants (5 males, 4 females, mean age 26.7, std 2.5)  Background review, Training session, Usability testing  Search for event sequence and replace  Search for people who had both book and newspaper publications where order is not important  Search for people who have at least 10 conference publications  Search and replace all publication events Very noticeable reduction in visual clutter after using Tool Repetition more intuitive than No Order and Wildcard searches

Recommended Improvements RecommendationLevel of Impact Implementation Difficulty R1Set a default replacement glyph for users that they can easily editHighEasy R2Set default interval glyphs to span the entire search event sequenceHighEasy R3Make it easier to select and drag edges of interval markersMedium R4Reduce space between two event sequence boxesHighEasy R5Add visualization of where replacement events are snapping to in the original sequenceMediumHigh R6 Add features to let users search and incrementally replace sequencesHigh R7Combine the two-steps search and replace process into one stepMediumEasy R8Map the flow of user clicks for all tasks to systematically eliminate unnecessary clicksHigh R9Activate event constraint windows with a button rather than the click and draft option or make the click and drag response less sensitive MediumHigh R10Create a generic wildcardMediumHigh R11Include summary of the number of replacements foundHighMedium R12Add time constraints for event sequences, such as searching between two time periodsHigh R13Make interval glyph more representative of intervals, particularly in the overview panelLowMedium R14Include additional ranking features, such as ranking by interval duration or ranking by the number of events in an interval High R15Automate the alignment of event sequences after a searchLowMedium