1. Expanding User Interaction in Virtual Reality at VR Medialab
June 18th 2004
Henrik Hansen & Thorkild Sørensen

2. Program Background and Motivation Project Outline
Test Application Introduction and Demonstration
Usability Tests
VR Chemistry Application
Results and Conclusion

3. Program Background and Motivation Project Outline
Test Application Introduction and Demonstration
Usability Tests
VR Chemistry Application
Results and Conclusion

4. Project Background Project Inspiration
The project was proposed by our self
Inspired by previous experience and demonstration sessions in the VR arenas
The first hand impression was that user interaction was very limited and problematic
Previous experience with PDA based applications and speech dialog systems lead to the idea of using these in VR context
Previous work in the field, also indicated that these interaction possibilities are interesting in VR

5. Project Motivation Problems in VR Interaction:
Forced to seek compromises
Specialized devices without standards
Low level of direct interaction in most application
Imprecise devices with few input options and no feedback
Mainly for selection and navigation
Two user scenarios are the most common at VR Medialab
Example of interaction in a VR application
Shows advantages and disadvantages of the existing devices

6. Airflow and noise simulation in a prototype car
Interaction Example
Airflow and noise simulation in a prototype car
Upræcis menu selection
Menuen bruges for meget
Begrænsede muligheder i menuen
Kun on/off knapper ingen sliders/dropdown eller noget

7. PDA 2D GUI in VR Advantages Disadvantages
Users are familiar with 2D widgets
Metaphors exists for a lot of tasks
Very flexible, can be tailored to any application
Does not occupy precious screen space in the virtual world
Well suited for alpha-numeric input
Disadvantages
Not an immersive input/output device
Requires the user to look away from the VR display
Only one person can see feedback given on the device
Devices are two-hand operated

8. Speech Interfaces in VR
Advantages
Hands and Eye Free
”Beyond the Here and Now”
Alpha-numeric Input
Naturalness/Immersiveness
Introduce New Functionality
Disadvantages
High Cognitive Load
Speech Recognition Errors

9. Program Background and Motivation Project Outline
Test Application Introduction and Demonstration
Usability Tests
VR Chemistry Application
Results and Conclusion

10. Project Outline Purpose of the project:
Enable the use of 2D GUIs on a PDA and speech interfaces at VR Medialab, hence in VR++
Demonstrate the capabilities of these interaction possibilities
Conduct usability tests with the new possibilities
Analysis
- Investigate possibilities (based on previous work and own ideas)
- Basic Requirements
Analysis
Integration
Test
Application

11. Integration in VR++
Speech:
2D GUI:

12. Program Background and Motivation Project Outline
Test Application Introduction and Demonstration
Usability Tests
VR Chemistry Application
Results and Conclusion

13. Test Application (1) Purpose: Type of application:
Demonstrate the capabilities of the new interaction possibilities
Serve as test platform during development
Application for usability tests
Type of application:
Include all VR interaction types
Existing vs. New

14. Test Application (2)
Simple VR application including both old and new interaction possibilities:
Wanda, Speech and PDA GUI
Objects:
4 colored rooms
Objects in 4 shapes, 10 colors and 5 sizes
Commands:
Select, deselect, navigate, add, remove, change attributes

15. Test Application Architecture

16. Test Application Demonstration
Demonstration of 2D GUI and speech interaction in the Test Applications
Upræcis menu selection
Menuen bruges for meget
Begrænsede muligheder i menuen
Kun on/off knapper ingen sliders/dropdown eller noget

17. Program Background and Motivation Project Outline
Test Application Introduction and Demonstration
Usability Tests
VR Chemistry Application
Results and Conclusion

18. Potential Test Objectives
The tests can be divided into two groups:
VR in general:
Are 2D GUIs on a PDA and speech usable in a VR context?
Is it difficult for the user to map the 3D world to a 2D representation?
Can a speech interface be combined with a handheld device?
Will a speech interface increase the immersiveness of an application?
PDA navigation compared to Wanda navigation
Etc.
Test Application specific:
General usability test of the user interfaces for the two new interaction possibilities in the Test Application
Tidsbegrænsninger gjorde at vi måtte vælge
Vores holdning er at test application skal testes først inden man kan bruge den til generelle test af SUI & GUI i VR
Ydermere er Test Applicationen ikke brugbar til alle test som den er nu.
De endelige test er en Navigation test og en generel usability test a SUI.

19. Speech Usability Test
18 test persons performed tasks with speech in the Test Application and gave feedback
The goal is to get valuable feedback for a potential next iteration

20. Navigation Test
The purpose of the test was to compare two different PDA navigation possibilities with the existing Wanda navigation.
Test persons are the same as in the speech usability test

21. Navigation Demonstration
Demonstration of Wanda navigation in the navigation scenario
Upræcis menu selection
Menuen bruges for meget
Begrænsede muligheder i menuen
Kun on/off knapper ingen sliders/dropdown eller noget

22. Navigation Test Results
Alternative ways of testing PDA navigation

23. Program Background and Motivation Project Outline
Test Application Introduction and Demonstration
Usability Tests
VR Chemistry Application
Results and Conclusion

24. VR Chemistry Application
2D GUI for a ”real” VR application
VR Chemistry is an education tool for high school chemistry teachers and students

25. VR Chemistry Demonstration
Demonstration of a 2D GUI for the VR Chemistry Application
Upræcis menu selection
Menuen bruges for meget
Begrænsede muligheder i menuen
Kun on/off knapper ingen sliders/dropdown eller noget

26. Program Background and Motivation Project Outline
Test Application Introduction and Demonstration
Usability Tests
VR Chemistry Application
Results and Conclusion

27. Status
Generic modules for 2D GUIs and speech interfaces are integrated into the VR++ framework
The capabilities of the new interaction possibilities are demonstrated with the Test Application
Initial Usability tests have been conducted

28. Conclusion Our opinion:
We see speech interfaces as a promising interaction paradigm in VR, both because it can increase the immersiveness and it can solve many interaction tasks
2D GUIs are not an immersive device, hence they are not an obvious choice for VR applications. However in the current state of VR interaction we also find 2D GUIs very functional and effecient

29. Thank you for your time! Any Questions? Upræcis menu selection
Menuen bruges for meget
Begrænsede muligheder i menuen
Kun on/off knapper ingen sliders/dropdown eller noget