HoloSync: Exploring Discoverable Conversational Interfaces for Model State Control ALI SIDDIQUI

QUESTION How can a conversational UI improve the control of model state information in holographic models?

WHAT IS MODEL STATE? Object Data Layers Dynamic Model Introduction Question Model State The Aim Object Data Analysis Layers Prototypes Dynamic Model Evaluation Implications

OBJECTIVES Conversational UI Model State Control Introduction Question The Aim Analysis Prototypes Conversational UI Model State Control Evaluation Implications

WHY VOICE?

PRECEDENTS - HoloLens UI Style SketchUp Viewer Microsoft’s 3D Viewer Introduction UI Style Question Model State Icons & Text Virtual Buttons 2D Panel The Aim SketchUp Viewer Analysis Prototypes Evaluation Implications Microsoft’s 3D Viewer

Speech To Text Feedback PRECEDENTS - Voice Introduction UI Style Question Speech To Text Feedback Natural Language Voice Feedback Model State The Aim Cortana Alexa Analysis Prototypes Evaluation Implications Google Assistant Siri

WORKFLOW Literature Reviews Visual Studio Unity 3D Microsoft HoloLens Introduction Question Model State Literature Reviews Visual Studio Unity 3D The Aim Analysis Prototypes Evaluation Implications Microsoft HoloLens User Testing

Basic voice commands that trigger an event. PROTOTYPE 1.1 Introduction Question Basic voice commands that trigger an event. User : “Context On” “Trees Off” “Alignment On” Model State The Aim Analysis Prototypes Evaluation Implications

PROTOTYPE 1.2 Getting voice feedback from the UI. User: Introduction Getting voice feedback from the UI. User: “How do I view the floors?” UI: “Try saying: only show the floors.” Question Model State The Aim Analysis Prototypes Evaluation Implications

PROTOTYPE 1.3 Introduction Adding conversational commands through keyword recognition. User: “only show me the floors and the glass.” Keywords: only, floors, glass. Question Model State The Aim Analysis Prototypes Evaluation Implications

Adding discoverability to inform the user of more information. PROTOTYPE 1.4 Introduction Question Model State Adding discoverability to inform the user of more information. The Aim Analysis Prototypes Evaluation Implications

Conversational Language ACHIEVEMENTS Introduction Question Model State Conversational Language “Talk” with the UI Model State Control The Aim Analysis Prototypes Evaluation Implications

LIMITATIONS User commands need to be object specific Introduction Question User commands need to be object specific Interacting with individual objects within layers Limited to predefined keywords Model State The Aim Analysis Prototypes Evaluation Implications

Intelligent Voice Feedback MOVING FORWARD Introduction Question Adaptable Script Synonym Recognition Intelligent Voice Feedback Model State The Aim Analysis Prototypes Evaluation Implications

QUESTION How can a conversational UI improve the control of model state information in holographic models?

Quicker Than Current Apps IMPLICATIONS Introduction Question Easy to Learn Improves Over Time Quicker Than Current Apps Model State The Aim Analysis Prototypes Evaluation Implications