1. Visual Analytics of User Behavior Project Description: Analyze and predict user behavior in a virtual world to inform dynamic modifications to the environment to create a richer virtual experience. Major Accomplishments: –Identified virtual environmental differences that affect the user’s exploration affinity level –Extended game engine technology with visualization toolset to represent user position, velocity, time, map coverage, interaction events –Utilized deterministic nature of the engine to mine additional data of interest after original play session 1

2. Visual Analytics of User Behavior Support Complimentary grant support: NSF EAGER (EArly Grants for Exploratory Research): “Identifying and Integrating Creative Patterns of User Behavior and Experience in Virtual Worlds” Grant description: A new interdisciplinary methodology for both the analysis of user’s experiences in virtual worlds and how this analysis can influence the behavior of the world to produce more effective user experience. This is of particular value to virtual worlds which don’t have a single “winning condition” such as a game, but instead might support a wide variety of interactive behavior, all of which are valid, but can differ widely in simple analysis. 2

3. Visual Analytics of User Behavior Background Present state of knowledge in virtual world analysis –network analysis (connectivity, load, latency) –econometrics on virtual economies –profiling game play activities Primarily driven by game companies –during development (Microsoft Labs, Halo series) –for an ongoing MMOG (Blizzard, World of Warcraft) –over a game network service (Steam, Xbox Live) 3

4. Visual Analytics of User Behavior Background Currently, virtual world design is modified based on anecdotal observation of users during sessions and interviews afterwards. This won’t be possible with massive multi-user system. As virtual world becomes more complex, the ability to understand behavior of user and system diminishes. Multi-user system will need more robust analysis methods. These methods can modify world behavior so that different interaction types can have successful experiences. 4

5. Visual Analytics of User Behavior Methodology New methods: Cultural Analytics –“the use of computational methods for the analysis of patterns in visual and interactive media.” –Data mining, knowledge exploration, and information visualization as applied to cultural artifacts and experiences such as paintings, cartoons, or virtual worlds. Logging, visualizing, designing –Record events in the world and telemetry on the user –Record images of user and user’s view –Visualize spatial, temporal, and narrative patterns –Explore mechanisms to dynamically modify the virtual world based on behavior patterns 5

6. Visual Analytics of User Behavior Logging Event logging –Server-side code hooks fire when an event occurs –Events logged as time-stamped “triples” (subject-verb-object) Object / user interactions (Player1 activates Object5) World state changes User telemetry logging –Data is polled from client at set rate (1/sec) and logged on server User input (trackball direction, velocity) User avatar position / orientation User camera position / orientation / type 6

7. The Real-Fake Exhibition –[April 1 – May 28, 2011] –California State University, Sacramento University Library Gallery Student Population: 27,000 –The Scalable City was featured Video monitoring system installed –To track physical interactions Virtual World monitoring system enabled –To track events, world state and archive screen renders 7 Visual Analytics of User Behavior Exhibition

8. The Real-Fake Exhibition –The Scalable City Approx. 73 hours of playtime recorded 528,546 lines of tracking data 2,290,000 surveillance images (41 GB) 8 Visual Analytics of User Behavior Exhibition

9. Database of analytics data –Events categorized in triples (Subject – Verb – Object) 9 SubjectVerbObject HousePieceactivatesCity PlayercompletesGlobeMode ServerdeactivatesHousePiece entersLot exitsRoad joinsSimulation pauses resets starts unpauses Visual Analytics of User Behavior Exhibition

10. Database of analytics data –User data polled and logged for each player every second 10 MeasureDescription TimestampTime of recording Player IDServer’s identification integer for the player Trackball Input(x,y) movement of the trackball since last poll Num Input EventsNumber of unique input events since last poll Current CityThe city that the player currently occupies Player PositionWorld space location of the player Player DirectionDirection the player is facing Player Activity StateMeasure of current player activity [Very Active, Active, Inactive] Camera IDIndicates camera settings, based on activity levels Camera PositionCurrent world space camera position Camera Relative PositionPosition relative to the player’s position Camera OrientationIndicates where the camera is looking Visual Analytics of User Behavior Exhibition

11. Image data –View of User Video feed captures user interaction with physical interface Tracks bystanders experiencing but not interacting Still images archived at one frame / second –User’s view Screen renders presented to the user are sampled and archived ( one frame / second ) Long periods of inactivity disable archiving 11 Visual Analytics of User Behavior Image Data

12. View of User footage –A dimension of user behavior data typically disregarded Social influences come into play Were others waiting during a user’s play session? Did the user watch someone else interact first? Did some play sessions have multiple users taking turns? 12 Visual Analytics of User Behavior Image Data

13. User’s View (Screen Images) –What was the player seeing in the virtual world? –Visual experience influences behavior 13 Visual Analytics of User Behavior Image Data

14. Image Analysis –Color distribution –Object recognition –Feature analysis 14 Visual Analytics of User Behavior Image Data

15. User’s View (Screen Images) –Chroma Key Rendering prototype for simplification of object recognition –Requires two simultaneous render modes Real-time performance not to par for use in exhibition 15 Visual Analytics of User Behavior Image Data

16. Database of analytics data –679 unique play sessions identified –Additional data mined after original sessions Facilitated by deterministic nature of engine Example: city completion level 16 Visual Analytics of User Behavior Exhibition Data

17. Findings from Exhibition –Trackball analysis Users tended to go left instead of right 17 Visual Analytics of User Behavior Data

18. Findings from Exhibition –Number of Cities visited by each user 77 % visited just one city 18 Visual Analytics of User Behavior Data

19. Findings from Exhibition –Starting city type drives exploration affinity to other cities Starting in curly road pattern makes users much less likely to travel to other cities 19 Visual Analytics of User Behavior Data Frequency of multi-city exploration per starting city type

20. Findings from Exhibition (position) 20 Visual Analytics of User Behavior Data

21. Findings from Exhibition (camera position) 21 Visual Analytics of User Behavior Data

22. Visualization Toolset –Flash based gui integrated into virtual world –Avatar visual representation controls –Indicators for position, velocity & time data –Lighting / Time of Day controls –Playback controls –Real time data visualizations 22 Visual Analytics of User Behavior Visualization

23. Visualization Toolset 23 Visual Analytics of User Behavior Visualization

24. Visualization Toolset 24 Visual Analytics of User Behavior Visualization

25. Visualization Toolset 25 Visual Analytics of User Behavior Visualization

26. Visualization Toolset 26 Visual Analytics of User Behavior Visualization

27. Virtual world behavior driven by analysis –Real-time analysis will customize and enhance experience on a per-user basis –Categorize user on the fly Timid Speedy Immersed –Loading based on Navigation prediction Preload assets based on paths users typically take at certain decision points. 27 Visual Analytics of User Behavior Future Work

28. Virtual world behavior driven by analysis –Analytics Camera Navigation assistance Based on previous user patterns –Resetting behavior Reset to environments conducive to more fulfilling experience 28 Visual Analytics of User Behavior Future Work