Heraklion, Crete, Greece June 2015 Fusality: An Open Framework for Cross-platform Mirror World Installations Nicholas F. Polys Benjamin Knapp Matthew Bock Christina Lidwin Dane Webster Nathan Waggoner Ivica Bukvic

Heraklion, Crete, Greece June 2015 Institute for Creativity, Arts and Technology (ICAT) The building is the instrument Mirror Worlds of public, gallery and performance spaces (NSF, VA) The CreateCube – Optical tracking – 128 channel sound – HMD and Tablet views 3/virginia-tech-icat-vr-research-oculus-rift

Heraklion, Crete, Greece June 2015 One Dimension (Milgram et al.)

Heraklion, Crete, Greece June 2015 Scope Mirror Worlds – the fusion of real and virtual realities – Sensor Systems and Smart Buildings – Online and local users have visual and audio displays – Distributed and local users share the space and its events Design dimension: – Reflecting events across participant *alities

Heraklion, Crete, Greece June 2015 Requirements Work with 3D content pipelines taught in School of Visual Arts Work with the WWW ecology and other rendering engines Enable faculty and visiting artists to quickly prototype and build Mirror Worlds installations at VT’s Center for the Arts

Heraklion, Crete, Greece June 2015 Approach Leverage Web3D Architecture – X3D / X3DOM – Websockets – node.js, socket.io – Start w TCP/IP for lossless communication Support adaptable message content, worlds and user roles – ‘Mirror worlds channels’ via node.js – Position, orientation, … gestures and UI events – JSON encoding of messages Build base 3D model of Moss Arts Center building with common naming and coordinate system – Express.js for spawning new mirror worlds as channels

Heraklion, Crete, Greece June 2015 Design

Heraklion, Crete, Greece June 2015 Realization

Heraklion, Crete, Greece June 2015

Further Development

Heraklion, Crete, Greece June 2015 Next Steps Examining levels of event logging, replay, feedback User roles and distributed event management Examine new X3DOM efforts for WebRTC and DIS and custom event representations Cluster rendering to thin CreateCube clients (e.g. nVidia GRID w/ Instant Reality and Unity, Paraview)

Heraklion, Crete, Greece June 2015 Future Research Fidelity and Presence Sensors and displays in Ubiquitous Computing Scalability of MMOGs Embodiment and Affect Occupancy and traffic patterns Collaborative decision making / social communication

Heraklion, Crete, Greece June 2015 Questions? Collaborations Welcome!