1. Heraklion, Crete, Greece 18 - 21 June 2015 Fusality: An Open Framework for Cross-platform Mirror World Installations Nicholas F. Polys Benjamin Knapp Matthew Bock Christina Lidwin Dane Webster Nathan Waggoner Ivica Bukvic

2. Heraklion, Crete, Greece 18 - 21 June 2015 Institute for Creativity, Arts and Technology (ICAT) http://www.icat.vt.edu/ The building is the instrument Mirror Worlds of public, gallery and performance spaces (NSF, VA) http://www.icat.vt.edu/impact/project/mirror-worlds http://www.icat.vt.edu/impact/project/mirror-worlds The CreateCube http://www.icat.vt.edu/content/cube-0 http://www.icat.vt.edu/content/cube-0 – Optical tracking – 128 channel sound – HMD and Tablet views http://www.theverge.com/2015/3/13/820419 3/virginia-tech-icat-vr-research-oculus-rift

3. Heraklion, Crete, Greece 18 - 21 June 2015 One Dimension (Milgram et al.)

4. Heraklion, Crete, Greece 18 - 21 June 2015 Scope Mirror Worlds – the fusion of real and virtual realities – Sensor Systems and Smart Buildings – Online and local users have visual and audio displays – Distributed and local users share the space and its events Design dimension: – Reflecting events across participant *alities

5. Heraklion, Crete, Greece 18 - 21 June 2015 Requirements Work with 3D content pipelines taught in School of Visual Arts Work with the WWW ecology and other rendering engines Enable faculty and visiting artists to quickly prototype and build Mirror Worlds installations at VT’s Center for the Arts

6. Heraklion, Crete, Greece 18 - 21 June 2015 Approach Leverage Web3D Architecture – X3D / X3DOM – Websockets – node.js, socket.io – Start w TCP/IP for lossless communication Support adaptable message content, worlds and user roles – ‘Mirror worlds channels’ via node.js – Position, orientation, … gestures and UI events – JSON encoding of messages Build base 3D model of Moss Arts Center building with common naming and coordinate system – Express.js for spawning new mirror worlds as channels

7. Heraklion, Crete, Greece 18 - 21 June 2015 Design

8. Heraklion, Crete, Greece 18 - 21 June 2015 Realization

9. Heraklion, Crete, Greece 18 - 21 June 2015

10. Further Development

11. Heraklion, Crete, Greece 18 - 21 June 2015 Next Steps Examining levels of event logging, replay, feedback User roles and distributed event management Examine new X3DOM efforts for WebRTC and DIS and custom event representations Cluster rendering to thin CreateCube clients (e.g. nVidia GRID w/ Instant Reality and Unity, Paraview)

12. Heraklion, Crete, Greece 18 - 21 June 2015 Future Research Fidelity and Presence Sensors and displays in Ubiquitous Computing Scalability of MMOGs Embodiment and Affect Occupancy and traffic patterns Collaborative decision making / social communication

13. Heraklion, Crete, Greece 18 - 21 June 2015 Questions? Collaborations Welcome! npolys@vt.edu