1. Computing for Social Needs Jennifer Mankoff UC Berkeley

2. Inspiration in the World: Finding the Right Combination  Hard, real problems  Hard HCI problems  Low intuition about users  Success hard to test  Technology not always a good solution  Hard computer science problems

3. Examples  Word prediction (+ & -)  Augmented canes (+ & -)

4. Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

5. Approaches To Research in CSN  Design: For users  Method: For designers/evaluators  Tool: For programmers/designers

6. Approaches To Research in CSN  Design: For users  Identify need  Investigate solutions  Prototype, test & iterate  Method: For designers/evaluators  Tool: For programmers/designers

7. Approaches To Research in CSN  Design: For users  Method: For designers/evaluators  Identify model or theory  Test against circumstances or population  Iterate  Tool: For programmers/designers

8. Approaches To Research in CSN  Design: For users  Method: For designers/evaluators  Tool: For programmers/designers  Identify repeating need or use of technology  Abstract out  Test for reusability

9. Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

10. Design Example: Nutrition  Need: Healthier diets  Assumptions  Idea: Keep track of purchases, display advice

11. Design Example: Nutrition  Need: Healthier diets  Manage disease  America’s weight problem  Manage child health  Assumptions  Idea: Keep track of purchases, display advice

12. Design Example: Nutrition  Need: Healthier diets  Assumptions  People don’t really know what they consume  Receipts contain enough information for us to estimate nutrition  Idea: Keep track of purchases, display advice

13. Design Example: Nutrition  Need: Healthier diets  Assumptions  Idea: Keep track of purchases, display advice

14. Nutrition: Hard HCI Problems  Formative evaluation: testing perception  Interface design  Summative evaluation in real-use setting

15. Nutrition: Formative Eval  Survey shoppers  Background research

16. Nutrition: Formative Eval  Survey shoppers  Perceived calcium consumption  Perceived need for supplements  Calcium consumption in receipts  Background research

17. Nutrition: Formative Eval  Survey shoppers  Background research  Use of shopping receipts in bookkeeping  Interest in nutrition  % of time eating out  Impact of coupons, advice on shopping behavior

18. Nutrition: Hard HCI Problems  Formative evaluation: testing perception  Interface design  While at home  Continual  Peripheral  While shopping  While entering data  Summative evaluation in real-use setting “Was that ‘Apple cider’ Or ‘Apple scraper’

19. Nutrition: Hard HCI Problems  Formative evaluation: testing perception  Interface design  Summative evaluation in real-use setting  Measures change in awareness  Measures change in behavior

20. Nutrition: Hard Computer Science Problems  Recognition  OCR  Who eats what  Quantities, ingredients  Ambiguity

21. Nutrition: Hard Computer Science Problems  Recognition  OCR  Who eats what  Quantities, ingredients  Ambiguity

22. Nutrition: Hard Computer Science Problems  Recognition  Ambiguity  Resolving imperfect recognition automatically  Resolving imperfect recognition with user’s help

23. Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

24. Method Example: Comparative Accessibility  Need: Increased accessibility in all interfaces  Assumptions  Idea: Develop metrics for interpreting simulated testing results

25. Method Example: Comparative Accessibility  Need: Increased accessibility in all interfaces  More inclusive  Increase quality of life  Assumptions  Idea: Develop metrics for interpreting simulated testing results

26. Method Example: Comparative Accessibility  Need: Increased accessibility in all interfaces  Assumptions  Can’t test every interface with every type of disability  Can simulate disability sufficiently for testing  Idea: Develop metrics for interpreting simulated testing results

27. Method Example: Comparative Accessibility  Need: Increased accessibility in all interfaces  Assumptions  Idea: Develop metrics for interpreting simulated testing results

28. Comparative Accessibility: Hard HCI Problems  Can a novice simulating disability give feedback on an interface designed for experts in that disability?  How should heuristics include accessibility?  How do disabilities impact GOMS models?  How do disabilities impact Fitts’ law?

29. Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

30. Tool Example: Mouse predictions  Need: Access to any application  Assumptions  Idea: Recognize problems, predict targets, and use that to make the mouse do the right thing

31. Tool Example: Mouse predictions  Need: Access to any application  Equal access  Increased independence  Assumptions  Idea: Recognize problems, predict targets, and use that to make the mouse do the right thing

32. Tool Example: Mouse predictions  Need: Access to any application  Assumptions  Low vision or motor impairment  No access to application code  Access to OS (e.g. app can be installed)  Idea: Recognize problems, predict targets, and use that to make the mouse do the right thing

33. Tool Example: Mouse predictions  Need: Access to any application  Assumptions  Idea: Recognize problems, predict targets, and use that to make the mouse do the right thing.

34. Mouse predictions: Hard HCI problems  Existing motion models only account for averages  Existing user models inaccurate  UI for compensation unclear

35. Mouse predictions: Hard HCI problems  Existing motion models only account for averages  Minimum jerk model: X(t) = X 0 + (X 0 – X f ) (15  4 - 6  5 - 10  3 )  Fitts’ law: MT = a + b log(A/W)  Existing user models inaccurate  UI for compensation unclear

36. Mouse predictions: Hard HCI problems  Existing motion models only account for averages  Existing user models inaccurate  KLM  extra cognitive cycles  No model of fatigue  UI for compensation unclear

37. Mouse Prediction: Other Models  Velocity  Thrashing ( = target)  Spasming  Overshooting  Other characteristics?

38. Mouse predictions: Hard HCI problems  Existing motion models only account for averages  Existing user models inaccurate  UI for compensation unclear  “Beat Fitts’ law”  Feedback affects recognition

39. Mouse Predictions – UIs for Compensation  Gravity wells and area mouse  Mediation

40. Mouse Predictions: Hard Computer Science Problems  Recognition  Account for feedback  Account for fatigue  Ambiguity  Better interfaces for multiple targets?  Interface for multiple directions?  Appropriate balance of control and automation

41. Outline  Approaches to research in computing for social needs (CSN)  Example: Design  Example: Method  Example: Tool  Conclusions

42. Conclusions  Plenty of hard real problems  Plenty of hard HCI problems  Plenty of hard computer science problems  Research needed in designs, methods & tools

43. Thank You For More Information: jmankoff@cs.berkeley.edu http://www.cs.berkeley.edu/~jmankoff

44. Tool Example: Reconstruction of Mismatched Interfaces  Need: Adaptation to any set of input devices