Computer Science Department California Polytechnic State University San Luis Obispo, CA, U.S.A. Franz J. Kurfess CPE/CSC 486: Human-Computer Interaction

2 © Franz J. Kurfess Usage of the Slides  these slides are intended for the students of my CPE/CSC 486 “Human-Computer Interaction” class at Cal Poly SLO ◆ some of them are based on the “Interaction Design” textbook ◆ ◆ if you want to use them outside of my class, please let me know ◆ some of them are based on other sources, which are identified at the bottom of the respective slide

3 © Franz J. Kurfess Course Overview ❖ Introduction ❖ Cognitive Foundations ❖ Input-Output Devices ❖ Interaction Spaces ❖ Interaction Styles ❖ Interaction with Mobile Devices ❖ Speech-Based Interaction ❖ User Assistance ❖ Natural User Interfaces ❖ Case Studies ❖ Project Presentations

4 © Franz J. Kurfess Chapter Overview Interaction with Mobile Devices ❖ Motivation ❖ Objectives ❖ Key Terms ❖ Summary

5 © Franz J. Kurfess Sources ❖ Pen Lister: Mobile and Emerging Devices, 2009  see Deviceshttp:// Devices ❖ Stephen Brewster: Multimodal Interaction, 2009 

6 © Franz J. Kurfess Logistics ❖ Term Project  mid-quarter project displays Thu, May 3  possibility of external visitors ❖ Research Activity  status update

7 © Franz J. Kurfess Agenda ❖ student presentations/papers  discussion  sign-up for those who already have a topic ❖ project: finalize teams, topics ❖ hand out: presentation/paper specifications

8 © Franz J. Kurfess Evaluation Criteria ❖ recall  important I/O devices and their properties ❖ recognition  which of a list of I/O devices has certain properties ❖ analysis  according to given criteria, how well does a device perform a certain task ❖ evaluation  is a particular I/O device suitable for a certain task  what is the performance of a device for a task  criteria to measure performance

9 © Franz J. Kurfess Shell’s Advocate ❖ In a mock debate with your neighbor, one person plays an advocate for command line interfaces (“shells”), the other for graphical user interfaces.  Even if you’re not personally convinced that your position (e.g. as advocate of the shell) is the better one, you must do your best to defend it (“devil’s advocate”).  As a starting point, you could use an exaggerated statement like “With today’s technology, shells are completely obsolete, and should not even be offered as alternatives to graphical user interfaces.”

10 © Franz J. Kurfess Batch Systems ❖ Why were batch systems historically the first user interfaces for computers? ❖ Are there circumstances under which batch systems are still appropriate today?

11 © Franz J. Kurfess Motivation

12 © Franz J. Kurfess Objectives

13 Mobile Devices Usage Capabilities Advantages Limitations

14 © Franz J. Kurfess Mobile Devices - Usage ❖ often closer proximity to users  with the user most of the time  easy to carry ❖ often multi-purpose devices  mobile phone, music player, camera, hand-held computer ❖ essential professional or personal device  connectivity (phone, text messaging, , Web)  organization (calendar, to do list, contacts, directions)  pleasure (music, photos, videos, e-books)

15 © Franz J. Kurfess Mobile Devices - Capabilities ❖ I/O capabilities  input  control and navigation (buttons; no mouse; cursor keys or limited pointing device; touch screen; gyroscope or accelerometer)  text (keyboard missing or small)  speech (microphone/head set)  output  visual: small screen  audio: small speaker, headphones  haptic: vibration ❖ computational capabilities  limited memory, processing ❖ connectivity  wired (USB)  wireless (cellular, Wi-Fi, Bluetooth, Infrared)

16 © Franz J. Kurfess Mobile Devices - Advantages ❖ quick & easy access ❖ multiple functions in one device

17 © Franz J. Kurfess Mobile Devices - Limitations ❖ input and output constraints  buttons, keyboard, navigation  screen size ❖ functional constraints  available functions are often not very sophisticated ❖ proprietary or unusual interaction methods  touch gestures, (virtual) keyboard arrangement ❖ synchronization with other devices  computer, home phone, car, …

18 © Franz J. Kurfess Device Categories ❖ Electronic Organizers ❖ PDAs ❖ Mobile Phones ❖ Smartphones ❖ Tablets

19 © Franz J. Kurfess Functionality Overlap ❖ Phones  directory ❖ PDAs  calendar  address book  notes  apps

20 © Franz J. Kurfess Universal Device vs. Specialized Tools

21 © Franz J. Kurfess Survival of Specialized Devices

22 Interaction with Mobile Devices user needs and requirements task analysis interaction flow screen design prototyping evaluation

23 Mobile Interaction Design Factors context interaction method format content source simplicity

24 © Franz J. Kurfess Context ❖ mobile interaction is embedded in the real world  users move  may be involved in other tasks  possibly involving hands  may require attention by the user  e.g. driving ❖ more challenging interaction experience  users are less focused on the interaction with the device  devices have more interaction constraints ❖ “head down” interaction

25 © Franz J. Kurfess Content Source ❖ original server ❖ intermediate server with specific mobile formats

26 © Franz J. Kurfess Interaction Method ❖ content transmission  push  pull  prefetch

27 © Franz J. Kurfess Format ❖ Web-generic ❖ mobile Web page ❖ mobile app

28 © Franz J. Kurfess Simplicity ❖ Occam’s Razor  the law of parsimony, economy or succinctness  among competing hypotheses, select the one with the fewest assumptions  offers the simplest explanation  may not be the most accurate one ❖ Einstein on Simplicity  “ Everything should be made as simple as possible, but not one bit simpler. ” Everything should be made as simple as possible, but not one bit simpler.

29 Designing for Mobile Devices Users Usage Scenarios Purpose Constraints

30 Output Constraints

31 © Franz J. Kurfess Small Screen ❖ visibility constraints  less display space (“screen estate”) ❖ interaction constraints  number and size of buttons, menus, and other interaction elements  more scrolling and paging  visual cues to indicate more content ❖ mobility advantages  device can be carried around  device can be placed in a convenient position for interaction  reading  phone call

32 © Franz J. Kurfess Readability ❖ Which text display is easier to read? The one in a few wide lines with a primarily horizontal arrangement, or the one arranged vertically?

33 © Franz J. Kurfess Single Window Mode ❖ most mobile devices do not allow the display of multiple windows  screen size constraints  performance constraints  interaction constraints ❖ consequences  switching within or between applications is more cumbersome  no simultaneous viewing  comparisons  correlations  information transfer

34 Input Constraints keyboard size, number of keys, arrangement of keys limited haptic feedback requires visual attention touch typing is much more challenging screen navigation simple for touch-based devices limited for others cursor keys, 4-way rocker switches pen inconvenient hand-writing recognition still somewhat problematic

35 © Franz J. Kurfess Touch-Based Interaction

36 Input Constraints keyboard screen navigation pen voice

37 © Franz J. Kurfess Keyboard Constraints ❖ size, number of keys, arrangement of keys ❖ limited haptic feedback ❖ requires visual attention  touch typing is much more challenging

38 © Franz J. Kurfess Screen Navigation Constraints ❖ simple for touch-based devices ❖ limited for others  cursor keys, 4-way rocker switches

39 © Franz J. Kurfess Pen Constraints ❖ inconvenient ❖ hand-writing recognition still somewhat problematic

40 © Franz J. Kurfess Voice Constraints ❖ audibility range ❖ background noise ❖ disturbance of others ❖ speech recognition

41 © Franz J. Kurfess Design Guidelines for Mobile Devices 1. Reduce the amount of content. 2. Single column layout works best. 3. Adjust the navigation method. bread crumbs vs. menu top or bottom placement 4. Minimize text entry. 5. Consider multiple mobile versions. iPhone vs. iPad; touch vs. keyboard; computing power 6. Touchscreen or not? 7. Utilize built-in functionality. phone calls, localization, QR codes

42 © Franz J. Kurfess Outlook: “Head Up” Interaction ❖ “hands-free” input  gestures  voice ❖ “eyes free” output  sound and tactile feedback  auditory widgets (“audicons”)  alerts, meaningful sounds  may be annoying for others  unless ear/head phones are used  tactile widgets (“tacticons”, “tactons”)  vibrations  surface variations  requires direct contact with the user

43 © Franz J. Kurfess Post-Test

44 © Franz J. Kurfess Evaluation ❖ Criteria

45 © Franz J. Kurfess Important Concepts and Terms ❖ batch system ❖ command-line interface ❖ contextual task analysis ❖ desktop ❖ direct manipulation ❖ forms ❖ full-screen interface ❖ goal ❖ graphical user interface (GUI) ❖ heuristic evaluation ❖ hierarchical menu ❖ human-machine interface ❖ intelligent agent ❖ interaction style ❖ menu ❖ mouse ❖ natural language ❖ networked menu ❖ system language ❖ task ❖ task analysis ❖ usability ❖ user-centered design ❖ user interface design ❖ user language ❖ user requirements ❖ What You See Is What You Get” (WYSIWYG) ❖ WIMP ❖ window

46 © Franz J. Kurfess Chapter Summary

47 © Franz J. Kurfess