2. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 1 User Studies Motivation January 30, 2007

3. How do we know whether security is usable?

4. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 3 Need to observe users We are not our users! (you may be surprised by what users really do)

5. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 4 Wireless privacy study Many users unaware that communications over wireless computer networks are not private How can we raise awareness? B. Kowitz and L. Cranor. Peripheral Privacy Notifications for Wireless Networks. In Proceedings of the 2005 Workshop on Privacy in the Electronic Society, 7 November 2005, Alexandria, VA.

6. Wall of sheep

7. Photo credit: Kyoorius @ techfreakz.org http://www.techfreakz.org/defcon10/?slide=38 Defcon 2001

8. Photo credit: http://www.timekiller.org/gallery/DefconXII/photo0003 Defcon 2004

9. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 8 Peripheral display Help users form more accurate expectations of privacy Without making the problem worse

11. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 10 Experimental trial Eleven subjects in student workspace Data collected by survey and traffic analysis Did they refine their expectations of privacy?

12. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 11 Results No change in behavior Peripheral display raised privacy awareness in student workspace But they didn’t really get it

13. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 12 Privacy awareness increased “I feel like my information /activity / privacy are not being protected …. seems like someone can monitor or get my information from my computer, or even publish them.”

14. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 13 But only while the display was on “Now that words [projected on the wall] are gone, I'll go back to the same.”

15. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 14 Security and privacy indicators

16. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 15 Evaluating indicators Case study: Privacy Bird

17. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 16 Platform for Privacy Preferences (P3P) 2002 W3C Recommendation XML format for Web privacy policies Protocol enables clients to locate and fetch policies from servers

18. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 17 Privacy Bird P3P user agent Free download http://privacybird.org/ Compares user preferences with P3P policies

21. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 20 Critique Privacy Bird Security people Can attackers spoof it? What if P3P policy contains lies? Can P3P policies be digitally signed? What about main-in- the-middle attacks? Usability people Green/red color blind problem Do people notice it in corner of browser? Do people understand privacy implications? Why a bird?

22. Typical security evaluation

23. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 22 Does it behave correctly when not under attack? No false positives or false negatives

24. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 23 Anti-phishing tools Y. Zhange, S. Egelman, L. Cranor, and J. Hong. Phinding Phish: Evaluating Anti-Phishing Tools. In Proceedings of NSSS 2006, forthcoming.

25. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 24 Does it behave correctly when under attack? Can attackers cause wrong indicator to appear?

26. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 25 Correct indicator Wrong indicator Attacker redirects through CDN

27. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 26 Can it be spoofed or obscured? Can attacker provide indicator users will rely on instead of real indicator?

28. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 27

29. Usability evaluation

30. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 29 C-HIP Model Communication- Human Information Processing (C-HIP) Model Wogalter, M. 2006. Communication- Human Information Processing (C-HIP) Model. In Wogalter, M., ed., Handbook of Warnings. Lawrence Erlbaum Associates, Mahwah, NJ, 51-61.

31. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 30 Do users notice it? If users don’t notice indicator all bets are off “What lock icon?” Few users notice lock icon in browser chrome, https, etc. C-HIP model: Attention switch, attention maintenance

32. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 31

33. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 32

34. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 33 Do users know what it means? Web browser lock icon: “I think that it means secured, it symbolizes some kind of security, somehow.” Web browser security pop-up: “Yeah, like the certificate has expired. I don’t actually know what that means.” C-HIP Model: Comprehension/Memory J. Downs, M. Holbrook, and L. Cranor. Decision Strategies and Susceptibility to Phishing. In Proceedings of the 2006 Symposium On Usable Privacy and Security, 12-14 July 2006, Pittsburgh, PA.

35. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 34 Netscape SSL icons Cookie flag IE6 cookie flag Firefox SSL icon

36. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 35 Privacy Bird icons Privacy policy matches user’s privacy preferences Privacy policy does not match user’s privacy preferences

37. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 36

38. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 37 Do users know what to do when they see it? C-HIP Model: Comprehension/Memory

39. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 38

40. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 39

41. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 40

42. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 41 Do users believe the indicator? “Oh yeah, I have [seen warnings], but funny thing is I get them when I visit my [school] websites, so I get told that this may not be secure or something, but it’s my school website so I feel pretty good about it.” C-HIP Model: Attitudes/Beliefs

43. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 42 Are users motivated to take action? May view risk as minimal May find recommended action too inconvenient or difficult C-HIP Model: Motivation

44. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 43 Do they actually do it? “I would probably experience some brief, vague sense of unease and close the box and go about my business.” C-HIP Model: Behavior

45. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 44

46. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 45 Do they keep doing it? Difficult to measure in laboratory setting Need to collect data on users in natural environment over extended period of time C-HIP Model: Behavior

47. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 46 How does it interact with other indicators? Indicator overload?

48. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 47

49. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 48 Summary: Security evaluation Does indicator behave correctly when not under attack? No false positives or false negatives Does indicator behave correctly when under attack? Can attackers cause wrong indicator to appear? Can indicator be spoofed or obscured? Can attacker provide indicator users will rely on instead of real indicator?

50. Usable Privacy and Security Carnegie Mellon University Spring 2007 Cranor/Hong http://cups.cs.cmu.edu/courses/ups-sp07/ 49 Summary: Usability evaluation Do users notice it? Do they know what it means? Do they know what they are supposed to do when they see it? Do they believe it? Are they motivated to do it? Will they actually do it? Will they keep doing it? How does it interact with other indicators?