1. Mixed Reality: Are Two Hands Better Than One?
Aaron Kotranza
John Quarles
Benjamin Lok
University of Florida
In this work we investigated the importance of two handed interaction in Mixed Reality.

2. Motivation: Mixed Environments for Review and Generation of Engineering Design (MERGED)
Users handle tracked real objects to manipulate collocated virtual counterparts
Our motivation is MERGED: a mixed reality environment for the evaluation of engineering designs.
(click)
In this form of mixed reality, users handle tracked real objects to manipulate collocated virtual objects –
here a user handles a drill and pliers to attach a metal plate to an assembly.

3. Problem Statement Engineering assembly using MERGED
Mixed Reality: interact using two hands
How important is this:
Increase task performance?
Make interaction more similar to performing the real-world task?
Need to track (and have an avatar for) both hands?
We want to use MERGED to evaluate engineering assembly designs.
(click)
Since we are using Mixed Reality, we can easily afford two handed interaction
Our goal is to answer the question: how important is two handed interaction? Does it matter that we can interact in MR using two hands?
We evaluate ‘importance’ through the following:
-Does using two hands increase task performance?
-Will using two hands make the interaction of Mixed Reality more similar to how you interact when performing the real world task?
If two handed interaction is indeed important, than we want all users to interact using both hands – for this do we need to track and have an avatar for both hands?
While we mention engineering assembly, we believe that these questions and their answers generalize to the simulation of any hands-on tasks in Mixed Reality.

4. User Study Evaluate MR for an assembly task
We conducted a user study to evaluate MR interaction for an assembly task.
(start video)
The task was to place these pipes into this connector array in a layout specified by the proposed design sheet (here).
(design sheet pops up)
The proposed design sheet shows the finished assembly from a different perspective, so the task is both physical and cognitive.
(goal pops up)
Here is the finished assembly
(task)
And the task performance

5. Population n=18 VR2006 attendees (nVR2006=11)
Experienced VR users
UF advanced graphics students (nUF=7)
Novice VR users
The study population consisted of 18 participants
(click)
11 VR2006 conference attendees, who were experienced VR users
7 University of Florida students from a graduate advanced graphics class, who were novice VR users
For statistical calculations, the populations were not combined, but the same results and trends were found in both.

6. Focus Some participants used two hands, but others only used one hand
Divide into conditions post priori
Use results to explore our questions
We noticed that some participants used two hands, but others only used one hand to perform the task.
(click)
This was used to divide participants into one handed and two handed conditions, post priori
The focus of this talk is using the results of this study to explore our questions of how important is two handed interaction to Mixed Reality
One handed (n1H=6) Two handed (n2H=12)

7. ≈ Hypotheses Using two hands will:
Increase performance
Afford interaction more similar to performing the real-world task
For most users to interact using two hands
Need to track and provide avatars for two hands ≈
We expect that interacting using two hands will
(click)
-increase performance
-and afford interaction that is more similar to performing the real-world task
In other words, with two hands, performing the mixed reality task will approximate performing the real task at a high degree of fidelity
We also hypothesize that for the majority of users to choose to interact with two hands, we need to track and provide avatars for both hands

8. Measures Performance Speed Errors Completion time
Collisions between pipes
Misplacements of pipes
Collision
To measure the importance of two handed interaction we first look at performance
(click)
We look at speed – completion time
And Errors, as measured by collisions and misplacements
Misplacement

9. Measures Similarity of interaction to real-world task
Do both conditions use the same actions to perform the task?
Same actions of (pre-study) participants performing the real-world assembly?
We also examine the similarity of mixed reality interaction to performing the real-world task
(click)
Did both one and two handed groups use the same actions to perform the task?
And are these the same actions used by pre-study participants performing the same task but with all real objects, without the HMD.

10. Results No significant difference in performance Speed Errors
Completion Time (p = 0.66)
Errors
Collisions (p = 0.64)
Misplacements (p = 0.66)
Speed
One-handed (1H)
Two-handed (2H)
Mean
176 sec
223 sec
Std. Dev. 57 85
There was no significant improvement in performance for the two handed condition over the one handed condition
(click)
We expected the completion time to be lower for the two handed condition, it was actually slightly higher, but not significantly
We expected fewer collisions when interacting using two hands, because two hands would make the pipes less unwieldy. This was the case, but not significantly
Misplacements were also lower for two handed, but not significantly
Errors
One-handed (1H)
Two-handed (2H)
Mean
4.3 collisions
2.5 collisions
Std. Dev. 5
1.5
1.0 pieces
0.4 pieces
Std. Dev
0.5 2

11. Why? One-handed condition: Found ways to compensate
Different actions, same raw performance
So, why was no performance benefit shown for using two hands?
(click)
Participants interacting using only one hand found ways to compensate, by using actions different from those interacting using two hands.
These compensating actions allowed participants in the one-handed condition to be just as fast as those in the two-handed condition.

12. Results Two-handed interaction ≈ real-world task
One-handed was fundamentally different
Participants who used two hands interacted similarly to performing the real-world task, using the same set of actions to perform the task.
(click)
But, participants using only one hand interacted fundamentally differently, using a different set of actions
(click to start video)
Instead of using two hands to position a pipe over the board and place it into the array, they placed one leg of the pipe into the connector and then pivoted the pipe so it fit into the connectors.
(wait for it)
They also rotated the pipes into the correct orientation for placing, while the pipes were still on the table. This way the table acted as a second stabilizing hand, allowing for efficient rotation.

13. Is two-handed interaction important?
Performance
No significant improvement
Similarity
Need to use two hands
Time is not an accurate measure
“Encourage” two handed interaction
Avatars for both hands?
Our system provided only one
Now we can answer our question – is interacting using two hands important to mixed reality?
(click)
With respect to performance, there was not a significant improvement from using two hands
But our similarity of interaction measure says that it is critical for users to interact using two hands in order for mixed reality interaction to approximate performing the real-world task
From this, we can conclude that commonly used performance metrics such as completion time are not always good measures of performance when it comes to interaction!
Here, raw time showed that performance is the same when interacting with either one or two hands, but this did not give us an accurate picture of what was going on.
From watching videos of the interaction, it is apparent that interaction was significantly different when using two hands than when using only one hand.
Since we have concluded that two handed interaction is critical to mixed reality, we want to encourage users to interact using two hands.
For this do we need to track both hands and provide two avatars?
In the study we only provided one avatar, so if we found that most users chose to interact using two hands anyway, then we can conclude that we don’t need to track both hands.

14. Results 67% interacted with two hands (n2H=12) Regardless of:
Prior VR experience
Quantity of real objects
Comfort level with HMD
And we found that two-thirds, 12 out of 18 participants used both hands to interact.
(click)
This was regardless of prior experience with VR, and the quantity of real objects.
However, the comfort level with the HMD did impact the participants’ choice to use two hands. Half of the participants who used only one hand did so because they needed their spare hand to steady the HMD.

15. Answers Two handed interaction is critical to MR Most users:
Big difference in similarity of interaction
Need users to interact using two hands
Most users:
Interact using two hands
Regardless of avatars
Answering our hypotheses…
(click)
We have concluded that two-handed interaction is critical for MR to afford interaction similar to real-world task performance
Because of this, we want users to choose to interact using two hands when presented with an MR system
We found that the majority of users will choose to interact using both hands, even if the MR system does not provide an avatar for both hands.
To system designers, this means that with respect to encouraging two handed interaction, we do not need to track both hands.

16. Conclusion
If a system goal is to provide interaction as similar as possible to the real world
Training systems
Must afford two-handed interaction
MR easily affords two handed interaction
We concluded that being able to interact using two hands is critical because it makes the interaction similar to that of the real world. So, when does this come into play…
(click)
If the goal of a system is to provide interaction as similar as possible to the real world, which is the case for many training systems, then
Our results suggest that the system must afford two handed interaction
and MR is a technology that easily affords two handed interaction

17. Acknowledgements Funding People University of Florida
IEEE VR2006 Organizers and participating attendees
University of Florida study participants