Tangible User Interfaces (TUI’s)

What are Tangible User Interfaces? Physical WorldDigital world TUI’s 2

GUI vs TUI (Ishii 2008) 3

Precursors E.g. Marble answering machine, Durrell Bishop, Royal College of Art, Interaction Design,

Early Work Fitzmaurice et al – Graspable user interfaces Imro Imro Ishii et al Tangible bits 5

Recent work E.g. Lumino, Baudish et al, m/watch?v=tyBbLqViX7g m/watch?v=tyBbLqViX7g E.g. Portico, Avrahami et al,

Benefits of TUI’s (Ishii 2008) 1.Double interaction loop - immediate tactile feedback 2.Persistency of tangibles 3.Coupled input/output space 4.Special vs generic purpose 5.Space-multiplexed vs time- multiplexed input Also fun + engaging! 7

Sense position & orientation on touch technology Add intelligent drawing support How can they be best combined with multi-touch surfaces for enjoyable and productive interaction? Can we connect these physical drawing tools to the digital space? 8

Our Approach Design – Tangible hardware Implementation – Recognizer – Drawing application Usability evaluation Capacitive Touch (CapTUI)Infrared Touch (TanGeo) – Ryan Tan & Bryan Chen – Rachel Blagojevic – Jacky Zhen 9

CapTUI Technology – Capacitive Small touch screens e.g. iPad, smart phones etc… Touch detection via electrical pulse from fingers/conductive material 10

Design: Tangible Hardware 11

Final Design 12

Implementation: Tangible Recognition Tangible ID – 3 point (min) unique patterns Valid patternsInvalid patterns 13

Implementation: Tangible Recognition Learning phase Recognition phase – Touch point detection – Match point distances to saved tangible ID’s – No way of knowing which part of the touch point is in contact (+/- error) 14

Implementation: Drawing Application Beautification Ink-to-edge snapping Corner snapping & Length visualization 15

Implementation: Drawing Application Visual drawing guides Tangible outline Angle visualization 16

First iteration: Video Second iteration: Demo 17

Evaluation First iteration: usability – Simple drawing tasks Second iteration: comparative study – Recognizable vs non recognizable drawing tools on screen 18

Usability Evaluation Can users construct simple drawings using the tangibles? Is the system usable? 10 participants 5 simple drawing tasks 19

Usability Evaluation: Results First exploration – Technology works – Is usable for simple drawings Tangible detection problems – Stability – Consistent circuit – Finger to tangible contact – Friction with screen – Comfortable drawing Tangible outline helpful – recognition indicator Drawing guides needed 20

Comparative Study Does CapTUI assist users to easily draw precise geometric drawings Recognizable vs non recognizable drawing tools on screen 12 Participants 21

Comparative Study: Results CapTUI rated significantly higher than Paint overall Visual guides helpful for precise drawing – significantly lower average angle error. Participants enjoyed using CapTUI significantly more Participants believed that CapTUI produces significantly more tidy drawings than Paint. Making fine grained movements with the tangibles difficult Tangible design still needs work – consistent detection – accurate positioning 22

Tangeo Technology – Infrared Table tops e.g. Microsoft Surface 2.0 PixelSense - Touch detection via infrared reflection for each pixel Image processing on detected pixels Identifies finger/blobs/tags 23

Design: Tangible Hardware 24

Final Design 25

Implementation: Tangible Recognition Learning phase Recognition phase – Detection via custom tags – Use tag location to get tangible outline – Use thresholds for blob sizes 26

Implementation: Drawing Application 27

Implementation: Drawing Application Visual Guides – Tangible outline – Angle visualisation – Length visualisation – Ink beautification corner snapping ink-to-edge snapping 28

Usability Evaluation Can users construct simple geometric drawings using Tangeo? Is the system usable? 2 phase cycle 8 participants 4 drawing tasks 29

Usability Evaluation: Results Enjoyable / easy to use the tangibles Good recognition Visual guides helpful and easy to understand Drawing accuracy – less positive perception Add stylus for drawing 30

References Marble answering machine - Crampton Smith, G. The Hand That Rocks the Cradle. I.D., May/June 1995, pp Fitzmaurice G. W., H. Ishii, and W. Buxton Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95), ACM Press/Addison- Wesley Publishing Co., New York, NY, USA, Ullmer B. and H. Ishii The metaDESK: models and prototypes for tangible user interfaces. In Proceedings of the 10th annual ACM symposium on User interface software and technology (UIST '97). ACM, New York, NY, USA, Ishii H., B. Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p , March 22-27, 1997, Atlanta, Georgia, United States Ishii H., Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and embedded interaction (TEI '08). ACM, New York, NY, USA, xv-xxv. Baudisch P., T. Becker, and F. Rudeck Lumino: tangible building blocks based on glass fiber bundles. In ACM SIGGRAPH 2010 Emerging Technologies (SIGGRAPH '10). ACM, New York, NY, USA, Article 16, 1 pages. Avrahami D., J. Wobbrock, and S. Izadi Portico: tangible interaction on and around a tablet. In Proceedings of the 24th annual ACM symposium on User interface software and technology (UIST '11). ACM, New York, NY, USA, Blagojevic R., X. Chen, R. Tan, R. Sheehan, and B. Plimmer Using tangible drawing tools on a capacitive multi-touch display. In Proceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and Computers (BCS-HCI '12). British Computer Society, Swinton, UK, UK, Zhen, J. S., R. Blagojevic and B. Plimmer (2013). Tangeo: Geometric Drawing with Tangibles on an Interactive Table- Top. CHI Paris France, ACM. WIP: in press. Shaer O. and E. Hornecker (2010) "Tangible User Interfaces: Past, Present and Future Directions", Foundations and Trends® in Human-Computer Interaction: Vol. 3: No 1-2, pp