3DUI – Submission #120 LOP-cursor Fast and Precise Interaction with Tiled Displays Using One Hand and Levels of Precision Henrique Debarba, Luciana Nedel,Anderson Maciel Federal University of Rio Grande do Sul (UFRGS) Institute of Informatics (INF)

Contributions Levels of Precision Cursor (LOP-cursor) Technique for high precision and fast pointing Two-legged cursor metaphor Simultaneous control over 2 cursors

Introduction LOP-cursor Evaluation Conclusion -Motivation -Related Work LOP-cursor -LOP-cursor technique -two-legged cursor Evaluation -Comparative -In depth LOP Conclusion -Conclusions -Future works

Motivation Large and very high resolution displays Mobile Devices Usefull Are becomming cheaper Ordinary interaction techniques may not be adequate Mobile Devices First computational device really pervasive (Ballagas 2006) Equiped with a wide range of sensors and displays Very popular

Motivation at&t

Motivation Operations Centre of Rio de Janeiro

Motivation Large and very high resolution displays Mobile Devices Usefull Are becomming cheaper Ordinary interaction techniques may not be adequate Mobile Devices First computational device really pervasive (Ballagas 2006) Equiped with a wide range of sensors and displays Very popular

Motivation BIG TV AMD eyefinity

Motivation Large and very high resolution displays Mobile Devices Usefull Are becomming cheaper Ordinary interaction techniques may not be adequate Mobile Devices First computational device really pervasive (Ballagas 2006) Equiped with a wide range of sensors and displays Very popular

Motivation Large and very high resolution displays Mobile Devices Usefull Are becomming cheaper Ordinary interaction techniques may not be adequate Mobile Devices First computational device really pervasive (Ballagas 2006) Equiped with a wide range of sensors and displays Very popular

Motivation Large and very high resolution displays Mobile Devices Usefull Are becomming cheaper Ordinary interaction techniques may not be adequate Mobile Devices First computational device really pervasive (Ballagas 2006) Equiped with a wide range of sensors and displays Very popular

Motivation Large and very high resolution displays Mobile Devices Usefull Are becomming cheaper Ordinary interaction techniques may not be adequate Mobile Devices First computational device really pervasive (Ballagas 2006) Equiped with a wide range of sensors and displays Very popular

Motivation There are more registered cellphone lines than inhabitants in Brazil (over 200 Million)

Related Work Pointing using optical flow Sweep (Ballagas et al, 2005) Interaction techniques in large display environments using hand-held devices (Jeon et al, 2006) Pointing using pattern recognition Point & Shoot (Ballagas et al, 2005) Touch projector (Boring et al, 2010 and 2011) Pointing using external tracking of the device Lightsense (Olwal, 2006)

Related Work Pointing with levels of precision Hybridpointing (Forlines et al, 2006) ARC-pad (McCallum and Irani, 2009)

LOP-cursor shapes Rectangle/Control canvas Mobile Device Touchscreen Ring Leg 2 Cursor Arrow Leg 1 Cursor Line Connects Rectangle and Ring

LOP-cursor shapes

Overview Levels of Precision Cursor (LOP-cursor) 2-Leg cursor metaphor First Level Rough pointing RayCasting Second Level Precise pointing Touchscreen

Overview Levels of Precision Cursor (LOP-cursor) 2-Leg cursor metaphor First Level Rough pointing RayCasting Second Level Precise pointing Touchscreen

Overview Levels of Precision Cursor (LOP-cursor) 2 Leg cursor metaphor First Level Rough pointing RayCasting Second Level Precise pointing Touchscreen

Overview Levels of Precision Cursor (LOP-cursor) 2-Leg cursor metaphor First Level Rough pointing RayCasting Second Level Precise pointing Touchscreen

Overview Levels of Precision Cursor (LOP-cursor) 2 Leg cursor metaphor First Level Rough pointing RayCasting Second Level Precise pointing Touchscreen

3 STATES 1 Free-pointing 2 3 Hold-control-canvas + Free-pointing Pin-control-canvas + Free-pointing

3 STATES - Free-pointing 1 Free-pointing

3 STATES - Hold-control-canvas 2 Hold-control-canvas + Free-pointing

3 STATES - Pin-control-canvas Pin-control-canvas + Free-pointing

Pinch/Stretch to rescale reduce size Stretch Increase size

Selection But, how can I trigger a selection if touch screen is being used for pointing?

Selection Method Click on the back of the device

Device orientation Obtained using Gyroscope, Accelerometer and Magnetometer. Gyro – rate of change in three axis Acc. – corrects drift around X and Z Mag. – corrects drift around Y Sensors fusion (Madgwick et al, 2011) Magnetometer correction z y x ADD slide - redundancia

Device Position Fixed position Calibration rather than tracking Omitir??

Evaluation Comparative evaluation In depth LOP-cursor evaluation Orientation RayCasting vs ARC pad vs Constrained LOP-cursor Additional test with unconstrained LOP-cursor In depth LOP-cursor evaluation Task 1 – point and select Task 2 – point, select, drag and dock Task 3 – 2-legged classification

Evaluation Comparative evaluation In depth LOP-cursor evaluation Orientation RayCasting vs ARC pad vs Constrained LOP-cursor Additional test with unconstrained LOP-cursor In depth LOP-cursor evaluation Task 1 – point and select Task 2 – point, select, drag and dock Task 3 – 2-legged classification

Evaluation Comparative evaluation In depth LOP-cursor evaluation Orientation RayCasting vs ARC pad vs Constrained LOP-cursor Additional test with unconstrained LOP-cursor In depth LOP-cursor evaluation Task 1 – point and select Task 2 – point, select, drag and dock Task 3 – 2-legged classification

Comparative Evaluation ORayCasting

Evaluation Comparative evaluation In depth LOP-cursor evaluation Orientation RayCasting vs ARC pad vs Constrained LOP-cursor Additional test with unconstrained LOP-cursor In depth LOP-cursor evaluation Task 1 – point and select Task 2 – point, select, drag and dock Task 3 – 2-legged classification

Comparative Evaluation ARC pad Dizer q eh nossa implementacao referenciar McCallum and Irani, UIST 2009

Comparative Evaluation Our implementation of the ARC pad

Evaluation Comparative evaluation In depth LOP-cursor evaluation Orientation RayCasting vs ARC pad vs Constrained LOP-cursor Additional test with unconstrained LOP-cursor In depth LOP-cursor evaluation Task 1 – point and select Task 2 – point, select, drag and dock Task 3 – 2-legged classification

Comparative Evaluation CLOP-cursor Comentar que eu soh estou mostrando que temq clicar em baixo

Evaluation Comparative evaluation In depth LOP-cursor evaluation Orientation RayCasting vs ARC pad vs Constrained LOP-cursor Additional test with unconstrained LOP-cursor In depth LOP-cursor evaluation Task 1 – point and select Task 2 – point, select, drag and dock Task 3 – 2-legged classification

Comparative Evaluation LOP-cursor

Comparative Evaluation Experiment design Within subject 3 Techniques - Counterbalanced Orientation RayCasting vs ARC pad vs Constrained LOP-cursor 4 Target sizes – 1 – 2 – 4 – 8 cm 11 participants Interacting 150 cm away from the display Asked to favor accuracy over time LOP-cursor additional evaluation 6 Target sizes – 0.5 – 1 – 2 – 4 – 8 – 16 cm 8 participants

Comparative Evaluation Error rate Mean time

LOP-cursor Aditional Evaluation LOP-cursor - LOP-First level and LOP-Second level User preference for each size Error rate

Comparative Evaluation LOP-cursor Preferred by 7 users ARC pad Preferred by 1 user

In depth LOP-cursor evaluation Design 3 tasks Task 1 – point and select Task 2 – point, select, drag and dock Task 3 – 2-legged classification 4 target sizes – 0.3 – 0.8 – 2 – 4 cm 11 participants Interacting 150 cm away from the display Asked to favor accuracy over time

Task 1 - Selection

Task 2 - Select + Drag + Dock

Task 3 - Two-legged classification

Results Mean time System Usability Scale (SUS) questionnaire Scored 77 SD 12.8

Conclusions Ordinary smartphones are valuable gestural input devices to interact with large and high resolution displays Defining a raw position with raycasting, and than a fine tune with touch input allows fast and precise interaction on selection and docking tasks

FUTURE WORKS Control over multiple computers investigate 2-legged metaphor Are users able to use cursors simultaneously? How detrimental to interaction is attention switch between cursors? Which tasks may benefit from the use of the 2-legged cursor?

Screen Replication

3D selection

! Thank you hgdebarba@inf.ufrgs.br

? Questions hgdebarba@inf.ufrgs.br

Offset

Offset

Offset

Offset

Pinch/Stretch to zoom in/out zoom out

Assess limitations of position calibration How detrimental to the user experience is the use of a calibrated position? How much can users move away from calibrated position without significantly affect pointing performance? Tirar

Device Position Fixed position Calibration rather than tracking 2 1 Omitir?? 1

Device Orientation Magnetometer correction Projecting on XZ plane Redundancy control Magnetometer is updated only when gyroscope confirms that a rotation change actually occurred z y x