1. Visual Analytics for Big Video Visualization
Robert S. Laramee
Visual and Interactive Computing Group
Department of Computer Science
Swansea University 1

2. Overview Data Visualization Motivation: Rugby Sport Video Analysis
Visualization Using Glyphs
Glyph Design and Sorting
System Interface
Knowledge Assisted Ranking
Conclusions and Acknowledgments 2

3. Visualization and the Visual Cortex
Visualization exploits our powerful visual system
2 million nerve fibers coming from optic nerves
Several billion neurons devoted to analyzing visual information (30% cortex)
8% for touch, 3% for hearing (Discover, 1993, Ware, 2013)
Enables massively parallel processing of the visual field, i.e., incoming color, motion, texture, shapes etc. 3

4. Benefits of Data Visualization4

5. Visual Analytics
”Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces.”
–Illuminating the Path: The Research and Development Agenda for Visual Analytics, Thomas and Cook, 2005 5

6. Motivation: Massive Multimedia Data from Videos
Case Study-Sport Videos:
Rugby Sport Video Analysis
Watching video is time consuming
Trying to find most important events
Trying to look for patterns and relationships
Trying to compare games 6

7. Visualization Using Glyphs
Data Visualization Using Glyphs: a special class of visualization
Discrete: The underlying data is represented pictorially
Glyphs by themselves are not novel
Sorting: we combine visualization and computer vision techniques, add sorting and interaction capabilities 7

8. Challenges of Sport Video Data8

9. Glyphs For Visualizing Rugby Events9

10. Glyphs For Visualizing Rugby Events10

11. Visualization Pipeline11

12. System Interface 12

13. Hierarchical Sorting 13

14. Glyph Sorting Results and Demo
2 matches
Gain (Y) vs Event (X) Plotted
Events are:
Kick Reception,
Restart Reception,
Lineout,
Turnover,
Scrum,
Penalty
Purple highlights indicate points scored 14

15. Knowledge-Assisted Ranking
Rugby Sport Video Analysis
Watching video is time consuming (linear search)
Trying to find most important events (manual search), looking for patterns, relationships, game comparisons
Analysis: Trying to rank events according to their importance
Challenging when incorporating several data dimensions 15

16. Enhanced System Overview
Glyph-Based Visualization: shows events sorted by ranking and Y axis (Gain)
Model Visualization: Analysis how current model parameters and accuracy correspond to ranking input
Ranking Input: ranking model can be exported to primary axis in glyph-based visualization
Glyph Control Panel: interaction enables control of axes within glyph-based visualization 16

17. Visual Analytic Pipeline
Tacit Knowledge: A priori knowledge that domain experts have
Partial Knowledge: Knowledge domain experts have about most important influential data attributes (or sort keys)
Tacit Knowledge does not scale up to large number of events, but choosing a few representative events is easy
System Knowledge: No a priori knowledge - derived from tacit knowledge + partial knowledge + sorting functions 17

18. Event Ranking Using Regression
Approach inspired by card sorting (Rugg and McGeorge 1997)
User-centered approach to categorize a set of items in to groups,
e.g., symbols in cartography (Roth et al., 2011), online course sites (Doubleday, 2013)
“Cards” are glyphs
Let: e1,e2,…,en be events
Let: es1<es2<…<esn an ordering
Define: y = Eb where E is an n x m matrix, and bj are the weights or contributions of each sort key
Goal: estimate weights of b and ranking function using regression 18

19. Event Ranking Models
User provides sample event ranking (small number e.g., 9)
System compares manual ranking with ranking predicted by regression model
Each polyline is an event
Contribution of each data attribute within model is depicted by gauges on each axis
Last axis encodes ranking confidence
Each regression model may discover a different set of key performance indicators
Performance of each model is evaluated, best model is chosen 19

20. Refining Model Parameters
User may rank event based on an ad hoc requirement, intuition, guesses
Thus user may refine model parameters by applying weighting parameters to sort keys
Enables user to explore new sorting strategies and understand impact on predicted ranking
e.g. remove a sort key 20

21. Interactive Brushing All views are coordinated and linked.
Glyphs corresponding to brushed polylines are highlighted (in focus)
Context glyphs can be scaled down
Users can select a glyph and replay original video of event 21

22. Comparing Two Matches Comparison of 2 matches: 81-7, 16-17
Expert user starts by selecting representative glyphs based on Gain
Events are input into ranking input and confirm by watching corresponding video
Analysts then visually assess resulting model
Phases are removed from attribute ranking
Large cluster of scoring events is observed in 1st match and absent in 2nd
Two poor kicks + turnovers discovered quickly 22

23. Knowledge Assisted Ranking Results and Demo
Analyst: “Using the software has enabled us to discover new key performance indicators that we wouldn’t have recognized before…”
Coach: “The system here is a good way at grouping clips…”
Player: “The software is useful as it allows you to break up the gaim by what you want to see…” 23

24. Acknowledgements Thank you for your attention! Any questions?
We thank the following:
David H..S. Chung, Rhodri Bown, Min Chen, Iwan W. Griffiths, Phillip A. Legg, Adrian Morris, Matthew L. Parry,
The Welsh Assembly Government (WAG)
For more information, please see:
David Chung, Phillip Legg, Matthew
Parry, Iwan Griffiths, Rhodri Bown,
Robert S Laramee, and Min Chen,
Knowledge-Assisted Ranking: A Visual
Analytic Application for Sport Event
Data, IEEE Computer Graphics &
Applications (IEEE CG&A), forthcoming 24

25. Empirical User Study Investigate difficulty of formalizing a ranking.
5 participants
Task: identify and rank set of events by most important positive outcome
Task 1: 5 events
Task 2: 10 events
Color map emphasizes worst and best events
Assess confidence
Task 3: identify most important set of influential attributes
Task 4: specify model 25