Virtual Museum Ramón Oliva, Miguel Pasenau & Eugeni Casadesús 2012, January 24th Virtual Reality, Immersive Interaction, Usability and Presence (RVA) course Master in Computing ( LSI – UPC)

2012, January 24th / 2 Contents Goal Implementation Results Conclusions

2012, January 24th / 3 Contents Goal Implementation Results Conclusions

2012, January 24th / 4 Goal Virtual Museum: 3 rooms User can move around: – Collision detection Objects exposed, selectable for inspection

2012, January 24th / 5 Environment & tools Cave or Power-Wall: – Limited space – User gestures to move around scene  Kinect VRJuggler: – Targets Cave, Power-Wall and desktop computer as well Kinect interface: – Natural user gestures – NITE + OpenNI + FAAST Bullet: collision detection

2012, January 24th / 6 Used tools gMeshSim ( PFC-EI) GiD b VRJuggler Bullet 2.78 openNI Nite FAAST

2012, January 24th / 7 Contents Goal Implementation Results Conclusions

2012, January 24th / 8 Museum floor plan triangles

2012, January 24th / 9 Objects triangles triangles triangles triangles triangles triangles triangles triangles triangles

2012, January 24th / 10 User movements Start & finish navigation: left arm up Forward, backward, left, right: – Segway: lean forward / backward: moves fw / bw – Rotate torso left / right: rotates left / right – Left arm up / down: starts / ends movement Selection – Using wand – Stretching right arm to ‘grasp’ object in front Inspection: – Zoom in / out Help – Raising right hand

2012, January 24th / 11 VRJuggler approach User stays fixed, scene moves Init(): – Initialize devices – Build scene: read ply models: museum walls, pedestals & objects – Create bullet physics: Museum, objects and pedestals containers

2012, January 24th / 12 VRJuggler approach preFrame(): – Update physics – Gets and process kinect and other events data Draw(): – Navigation mode: draw museum and objects – Inspection mode: draw selected object zoomed

2012, January 24th / 13 Actual state Museum, pedestals and objects are rendered Collision detection is enabled User movements are enabled: – Forward and backward, left and right turning – Grasp object for inspection Help mode Inspection mode: – Scale object up and down – Automatic rotation Kinect integration: – Gestures recognition and application responds

2012, January 24th / 14 Actual state Museum, pedestals and objects are rendered Collision detection is enabled User movements are enabled: – Forward and backward, left and right turning – Grasp object for inspection Help mode Inspection mode: – Scale object up and down – Automatic rotation Kinect integration: – Gestures recognition and application responds

2012, January 24th / 15 VRJuggler issues No x64 alone build & install Only mixed x32 and x64 Linux build and install Using x32 build: – Units in feet and inches, not meters User centred, scene moving No restrictions or definition of movements No event filtering possible No seamless platform transition

2012, January 24th / 16 Bullet + VRJuggler issues User centred, scene moving Translations  whole scene is translated the hard way Rotations  whole scene is rotated the hard way Collisions: – User: forward ray – Scene: bbox and cylinders moving with scene – Object selection = collision wand ray - object

2012, January 24th / 17 Kinect issues Unstable FAAST 0.10 Using.jconf: lot of noise Gesture recognition using FAAST: – Send keys / events to window with focus – Depends on sensor position / orientation Some delay

2012, January 24th / 18 Contents Goal Implementation Results Conclusions

2012, January 24th / 19 Results: navigation mode

2012, January 24th / 20 Results: inspection mode

2012, January 24th / 21 Results: help

2012, January 24th / 22 Results Source code at:

2012, January 24th / 23 Contents Goal Implementation Results Conclusions

2012, January 24th / 24 Conclusions Developing a RV application: – Hard to do – Still a juggling art VRJuggler: – Hard to build and install: documentation so-so – Only x32 is known to work Bullet: easy to install and to start with Kinect: difficult to incorporate to VRJuggler: – Gesture analysis and interpretation – Complicated device and proxies structure – “Easy” using FAAST patch

2012, January 24th / 25 Future work Solve problems with power-wall + kinect: – Inverse camera parameters – Inverse movements Use transformations instead of modifying coordinates Avoid noise depending on sensor position and orientation Interactive museum floor plan editor

2012, January 24th / 26 References Bullet documentation VRJuggler documentation FAAST documentation skeleton.html skeleton.html

Virtual Museum Virtual Reality, Immersive Interaction, Usability and Presence (RVA) course Master in Computing ( LSI – UPC) Ramón Oliva, Miguel Pasenau & Eugeni Casadesús 2012, January 24th