Interaction with Surfaces

Aims Last week focused on looking at interaction with keyboard and mouse This week ◦ Surface Interaction ◦ Gestures

Surface What do we mean by surface?

Touch Screens Very durable – only input device to survive at Disney World Arm fatigue can be a problem ◦ Tilt the screen (lighting) ◦ Surface to rest arms 4

Types of Surfaces Lots of different types: ◦ Resistive – two layers when press down they collide ◦ Capacitive - layer that stores electrical charge, touches the monitor with his or her finger, some of the charge is transferred to the user ◦ Surface acoustic wave system - two transducers and reflector can sense when wave has been disturbed

Modelling User Interaction Would be nice if you could calculate the time required to point at an object.

Fitts’ Law (1954) Predictive Model of time required to point at an object – nice formulae Can determine optimal location and size of buttons Time required to complete hand movement was dependant on the distance users had to move (D) The target Size (W) 7

Fitts’ Law If it took 2 seconds to select a target that was 10 cm away might expect it to double if 20 cm away to 4 seconds not the case Increasing size of the target from 1cm 2 to 2cm 2 enables users to point more rapidly Start and Stop times are constant MT = a + b log 2 (D/W+1) 8

Fitts Law and users Know your users! Fitt’s law well established for adults Needs refinement for other users such as children and elderly Age can effect speed and accuracy 9

Study performed using mouse Repeating action of Selecting two targets 1= Young adults 2= 5 year olds 3= 4 year olds Study performed using mouse Repeating action of Selecting two targets 1= Young adults 2= 5 year olds 3= 4 year olds Juan Pablo Hourcade, Benjamin B. Bederson, Allison Druin, and Francois Guimbretiere Differences in pointing task performance between preschool children and adults using mice. ACM Trans. Comput.-Hum. Interact. 11, 4 (December 2004),

Social Interaction Even with this knowledge design for interaction problematic when you have multi-user interaction Example multi-player games on large displays (Video)

From GUIs to NUIs Natural User Interface Focus on natural way of interacting with devices Concept around for long time Newton Message Pad Apple 1989 Hand writing recognition AWFUL!

Gestures Natural ways to interact Vision recognition used to capture hand movement and gestures in front of a white board Widely researched area in HCI Wii Accelerometer based gesture recognition 13

State-Transition Model Direct touch Idle Active Finger lands Finger lifted Finger moving in air Finger moving on device

Gestures The actual shape 15 Filtered to interpret a circle

Gestures Shaking – basic note entry Tilting - subdividing note durations in the stored sequence. Tapping - tap for lower and higher notes. Tossing – used to pass music to another device 16 Weinberg, G., Godfrey, M., and Beck, A ZOOZbeat: mobile music recreation. In Proceedings of the 28th of the international Conference Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA, April , 2010). CHI EA '10. ACM, New York, NY,

Gestures Issues ◦ Training usually required ◦ Remembering gestures = Gulf of execution ◦ Nothing happens = Gulf of evaluation  Need to understand cause movement constraints are reached ◦ No real standards – Apple defined gestures ? ◦ Can programme your own 17

What’s this gesture? 18

What’s this gesture? 19

Gestures Surface established gestures 20

Gestures Surface user defined gestures 21

What about Fat Fingers User touches device large part of finger contacts surface (system not realise it) Can’t see the pixels that are being targeted – finger is in the way Video

Gulf of Competence Gulf occurs when learn something one way and then have to relearn it another For example mouse to keyboard shortcuts Mouse to gesture interaction

Summary Gestures How many could a child remember? Would they need training? Is this part of the system or one you created? Need to discuss issues for your assignment One of major input methods to consider 24