Introduction to UbiComp Hao-Hua Chu UbiComp Lab Graduate Institute of Networking & Multimedia Department of Computer Science and Information Engineering National Taiwan University

What is Ubiquitous (Pervasive) Computing? Bring computing into our everyday activities in the physical world What is the most natural way to access & interact with the bits in the digital world? PCs with keyboard & mouse, monitors? Mobile phones with tiny keypad & tiny screen? Digital TVs? Game consoles? Digital Watches? Combination of all of them? Ubicomp not useful for some of my students

Natural interaction with the bits (digital world) Maybe not through any of these unfamiliar digital access devices! What if familiar physical objects can also become our interface to the digital world? Desk, wall, floor, chair, fridge, papers, bed, As if computing devices have disappeared … Interaction with the digital world is through interaction in the physical world. Access digital domain through physical world

Ubicomp Applied to Healthcare What is the natural way of delivering online/Internet care to elders at home? PC-like devices (Notebook PCs, PDAs, cell phones, etc.)? Elders’ ADLs do not naturally involve PC-like devices But naturally involve everyday objects in their physical surrounding Example: good-dietary service Create an information website? Built into a dining table Elders are not going to sit in front of a computer, they are not look at small screen on cell phone or PDAs with small keypad

Invisible and Disappearing Computer (Weiser) Why invisible and disappearing”? Technology makes lives easier or more difficult (for ordinary people)? “Usability “ and “simplicity” Better with nothing-never-nowhere computing Are you going to spend more (frustration) time “operating machines” or enjoy living?

Invisible and Disappearing Computer (Weiser) Unless necessary, don’t make people operate complex machines to get digital services. Project Aura (CMU) “The most precious resource in a computer system is .. User Attention. Today's systems distract a user in many explicit and implicit ways, thereby reducing his/her effectiveness.” Are you going to spend more (frustration) time “operating machines” or enjoy living?

Smart Everyday Objects How to make computing invisible and disappearing? Hide computing into everyday objects Embed computing into familiar everyday objects computing, sensing, and networking capabilities Everyday objects understand you & environment (context-aware computing). Context = 5Ws (who – when – where – what – how)

Smart Everyday Objects (Examples) Posture Chair (MIT media lab) Bionic Running Shoes (Adidas) Moving Portrait (MIT media lab)

More Examples of Smart Objects History table cloth (EU Equator) Emotional Décor (NYU) I/O Brush (MIT) History table cloth (memory)

Surface Computing (Microsoft Research) A new way that is so familiar to us. A new way that isn’t really new.

Maybe not so smart objects Casio Color Wrist Camera Dada MP3 player shoes LG Internet fridge Credit Card Barbie Doll (from Accenture)

Would you buy it for twice the price? Please Vote

LED Display Cloth

A Wearable Display for Team Sports (U. of Sydney)

Topobo (MIT)

Blendie (MIT) A “sensitive” blender that can express emotion.

Case Studies Diet-aware Dining Table (PERVASIVE 2006) Geta Sandals (Personal & Ubiquitous Computing) Object Locator Ring & Watch

Case Study 1: Diet-aware Dining Table Detect what and how much you eat from the table Sensing to recognize behavior Combine weight sensor and RFID sensors to track food transfer among containers Interaction Natural user eating behaviors become system input (no need to operate any devices). How do you design a user interface without affecting one’s appetite? We are exploring using computers that not only can sense our behavior, but also influence and change our behavior for better living. We have experiment this concept by constructing a persuasive smart object – a lunch trary. The tray can sense a child’s eating behavior using sensors underneath the tray. Over the tray, a projector projects an interactive game. The game takes inputs of eating actions and tries to persuade the child to eat all foods placed on the tray. The game is about coloring a cartoon character. Each bowl of food corresponds to one color. As a child eats on type of food, a part of the cartoon character is colored. To make the cartoon character fully colored, the child needs to eat all the foods on the tray. We are working with behavior psychologist … food preferences are developed at young age. We dislike a food because we never try it, or eat enough to become accustomed to its taste.

Diet-Aware Dining Table: Single Interaction Example Bob pours tea from the tea pot to personal cup, and drinks it Put on tea pot. RFID tag appears Weight increases ∆w3 Pour tea! |∆w3 - ∆w1 | ≈ ∆w2 Pick up tea pot. RFID tag disappears Weight decreases ∆w1 Pour tea? Weight increases ∆w2.

Case Study 2: Geta Sandals Shoes (Slippers) that track where you walk Track people’s locations with minimal infrastructure in the deployed environment Footstep-based localization Error accumulation (1~10%) Location-aware RFID tags d1 d2

Case Study 3: Object Locator Ring (Watch) Track locations of everyday things Where did I put these everyday things? Glasses, cell phones, wallets, keys, remote controls, … Track locations of everyday things RFID reader on ring RFID tags on everyday objects Ultrasonic Indoor location systems (MIT Cricket)

How does it work? Assumption: Picking up phone: Carrying the phone: Most objects are moved by hands Picking up phone: Phone presence (RFID tag) detected by the ring antenna Carrying the phone: Continuous presence (on-hand) Non-presence (pocket-it) Dropping the phone: Phone presence (RFID tag) not detected by ring antenna Location of the object?

Part II: Persuasive Objects (Smart environment, smarter people)

Persuasive Computing Consider a frequent “ where are my (misplaced) things?” person … Possible solutions Build an object location tracker Build a beeper that remind misplaced objects Which one is more effective use of technology?

Can digital technology be used to help us … Eat healthy Exercise more Drive safely Quit smoking Become more tidy Control spending Dress style

Persuasive computing (objects) Baby Think It Over Textrix VR Bike Slot Machine Smart Tachograph (ETH) 20

Case Studies Playful tray (UbiComp 2007) Playful toothbrush (CHI 2008) Mug-Tree (UbiComp 2007) Nutritional-aware kitchen (CHI 2007, PERSUASIVE 2008) Clean-your-room poster ChroMirror (CHI 2008) Mug-Forest (New)

Case Study 1: Playful Tray Encourage good eating habit in young children Sense to recognize behavior Weight sensor underneath the tray to sense eating actions Eating actions as game input Play to engage behavior change Interactive games: coloring cartoon character or penguin fishing User study on 4 children (autistic)

Case Study 2: Playful Toothbrush Encourage proper and thorough brushing for young children Bad teeth for young children with cognitive impairments) Sense to recognize behavior Webcam to detect brushing motions Brushing actions are game input Play to engage behavior change Start with a mirror image of dirty teeth. Physical brushing maps to virtual plaque removal User study on 13 kindergarten children Brush extension Web camera for tracking brushing motions Camera #2 to record videos for later human analysis Brushing game

Case Study 3: Mug-Tree Encourage healthy habit of drinking fluid regularly Sense to recognize behavior Tilt sensor to detect drinking actions Drinking actions are game inputs Play to engage behavior change Game metaphor: hydrating/dehydrating body -> watering/drying a tree

Case Study 4: Nutrition-aware Kitchen Raise awareness of nutritional fact to everyday family cooks Sense to recognize behavior Combine weight and camera sensors to detect cooking actions (change food ingredients) Voice input for food ingredient label Food ingredients -> meal calories Play to engage behavior change Overcooking calories -> overweight on a family member Imbalance the seesaw board -> big boulder sliding down

Case Study 4: Nutrition-aware Kitchen Raise awareness of nutritional fact to everyday family cooks User study on 3 family cooks Figure 7. User interface of Calorie-aware Kitchen, including (a) overview of calorie in the system; (b) recommended calorie needs and current used calories; (c) a calorie-aware game with a beloved family member to bring enjoyment of calorie control (c) (a) (b) recommended calorie needs current calorie in use in total calorie of finished course name of ingredients, total calorie in this container, and the most recent calorie change

Case Study 5: Clean-your-room poster Persuade young children to put things back where they belong after using them Sense to recognize behavior RFID sensors to check if things were put back in the shelf Play to engage behavior change Misplaced things -> trash in the virtual world Using child's sympathy for the animal to persuade the child to put things back to where they belong.

Case Study 6: ChroMirror Persuade people to explore more colorful dressing Sense to recognize dress & dressing color Camera and Computer Vision Play to engage behavior change Easily and playfully explore & experiment with how different clothing color look on people

Case Study 7: Mug-Forest Can a mobile phone do more than a notebook PC? Mobility (always with a user), small form factor, Can a mobile phone be extended for non-verbal communication?

Case Study 7: Mug-Forest Use social pressure for persuasion Sense to recognize drinking action Accelerometer (in a phone) to detect drinking action Camera to detect water level Play to use “social pressure” to engage behavior change Computer-mediated human persuasion, not computer persuasion

Questions & Answers Check out our projects & papers NTU Ubicomp Lab http://mll.csie.ntu.edu.tw