1. C MU U sable P rivacy and S ecurity Laboratory http://cups.cs.cmu.edu/ Trust and Semantic attacks Ponnurangam Kumaraguru (PK) Usable, Privacy, and Security Mar 17, 2008

2. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 2 Who am I? Ph.D. candidate in the Computation, Organizations, and Society program in the School of Computer Science Research interests - Privacy, Security, Trust, Human Computer Interaction, and Learning Science

3. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 3 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

4. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 4 What is trust? No single definition Depends on the situation and the problem Many models developed Very few models evaluated

5. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 5 Trust in literature Economics (how trust affects transactions) Reputation Marketing (how to build trust) Persuasion HCI (what affects trust) Design Psychology (positive theory) Intimacy

6. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 6 Negative antecedents Risk Transaction cost Uncertainty … Trust Models Positive antecedents Benevolence Comprehensive information Credibility Familiarity Good feedback Propensity Reliability Usability Willingness to transact …

7. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 7 How do users make decisions? Interview design, 25 participants (11 - experts and 14 - non-experts) Measured the strategies and decision process of the users in online situations Results Non-experts wanted advice to help them make better trust decisions Non-experts used significantly fewer meaningful signals compared to experts P. Kumaraguru, A. Acquisti, and L. Cranor. Trust modeling for online transactions: A phishing scenario. In Privacy Security Trust, Oct 30 - Nov 1, 2006, Ontario, Canada.

8. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 8 Expert model Missed signals Meaningful signals Misleading signals Not deliberate states Unknown states States that affect well-being States that affect decision Signals

9. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 9 Non- expert model Missed signals Meaningful signals Misleading signals Not deliberate states Unknown states States that affect well-being States that affect decision Signals

10. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 10 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

11. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 11 Security Attacks: Waves Physical: attack the computers, wires and electronics  E.g. physically cutting the network cable Syntactic: attack operating logic of the computers and networks  E.g. buffer overflows, DDoS Semantic: attack the user not the computers  E.g. Phishing http://www.schneier.com/essay-035.html

12. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 12 Semantic Attacks “Target the way we, as humans, assign meaning to content.” System and mental model http://groups.csail.mit.edu/uid/projects/phishing/proposal.pdf

13. An email that we get

14. Features in the email Subject: eBay: Urgent Notification From Billing Department

15. Features in the email We regret to inform you that you eBay account could be suspended if you don’t update your account information.

16. Features in the email https://signin.ebay.com/ws/eBayISAPI.dll?SignIn&sid=veri fy&co_partnerid=2&sidteid=0

17. Website to collect information http://www.kusi.org/hcr/eBay/ws23/eBayISAPI.htm

18. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 18 What is phishing? Phishing is “a broadly launched social engineering attack in which an electronic identity is misrepresented in an attempt to trick individuals into revealing personal credentials that can be used fraudulently against them.” Financial Services Technology Consortium. Understanding and countering the phishing threat: A financial service industry perspective. 2005.

19. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 19 Phishing Attack Life Cycle Post Attack Fraud & Abuse Collection Attack Setup Planning Source:http://www.coopercain.com/User%20Data/A%20Leisurely%20Lunch%20Time%20Phishing%20Trip-show.ppt

20. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 20 A few statistics on phishing 73 million US adults received more than 50 phishing emails each in the year 2005 Gartner in 2006 found 30% users changed online banking behavior because of attacks like phishing Gartner in 2006 predicted $2.8 billion loss due to phishing in that year

21. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 21 Why phishing is a hard problem? Semantic attacks take advantage of the way humans interact with computers Phishing is one type of semantic attack Phishers make use of the trust that users have on legitimate organizations

22. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 22 Three strategies for usable privacy and security Invisible strategy Regulatory solution Detecting and deleting the emails User interface based Toolbars Training users

23. Our Multi-Pronged Approach Human side Interviews to understand decision-making PhishGuru embedded training Anti-Phishing Phil game Understanding effectiveness of browser warnings Computer side PILFER email anti-phishing filter CANTINA web anti-phishing algorithm Automate where possible, support where necessary

24. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 24 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

25. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 25 Why user education is hard? Security is a secondary task Users not motivated to taking time for education Non-existence of an effective method

26. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 26 To address the open questions Embedded training methodology Make the training part of primary task Create motivation among users Learning science Principles for designing training interventions

27. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 27 Approaches for training Posting articles FTC,… Phishing IQ tests Mail Frontier, … Classroom training (Robila et al.) Sending security notices http://www.ftc.gov/bcp/conline/pubs/alerts/phishingalrt.htm http://www.sonicwall.com/phishing/ http://pages.ebay.com/education/spooftutorial/

28. Security notices How to spot an email How to report spoof email Five ways to protect yourself from identity theft

29. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 29 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

30. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 30 Why learning science? Research on how people gain knowledge and learn new skills ACT-R theory of cognition and learning Declarative knowledge (knowing that) Procedural knowledge (knowing how) Learning science principles

31. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 31 Learning science principles Learning-by-doing More practice better performance Story-based agent Using agents in a story-based content enhances user learning Immediate feedback Feedback during learning phase results in efficient learning Clark, R.C., and Mayer, R.E. E-Learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. John Wiley & Sons, Inc., USA, 2002.

32. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 32 Learning science principles Conceptual-procedural Presenting procedural materials in between conceptual materials helps better learning Contiguity Learning increases when words and pictures are presented contiguously than isolated Personalization Using conversational style rather than formal style enhances learning Clark, R.C., and Mayer, R.E. E-Learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. John Wiley & Sons, Inc., USA, 2002.

33. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 33 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

34. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 34 Design constraints People don’t proactively read the training materials on the web People can learn from web-based training materials, if only we could get people to read them! (Kumaraguru et al.) P. Kumaraguru, S. Sheng, A. Acquisti, L. Cranor, and J. Hong. Teaching Johnny Not to Fall for Phish. Tech. rep., Cranegie Mellon University, 2007. http://www.cylab.cmu.edu/files/cmucylab07003.pdf.

35. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 35 Embedded training We know people fall for phishing emails So make the training available through the phishing emails Training materials are presented when the users actually fall for phishing emails Makes training part of primary task Creates motivation among users Applies learning-by-doing and immediate feedback principle

36. Embedded training example Subject: Revision to Your Amazon.com Information

37. Embedded training example Subject: Revision to Your Amazon.com Information Please login and enter your information http://www.amazon.com/exec/obidos/sign-in.html

38. Comic strip intervention

39. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 39 Design rationale What to show in the intervention? When to show the intervention? Analyzed instructions from most popular websites Paper and HTML prototypes, 7 users each Lessons learned Two designs Present the training materials when users click on the link

40. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 40 Study 1: Evaluation of interventions H1: Security notices are an ineffective medium for training users H2: Users make better decisions when trained by embedded methodology compared to security notices

41. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 41 Study design P. Kumaraguru, Y. Rhee, A. Acquisti, L. Cranor, J. Hong, and E. Nunge. Protecting People from Phishing: The Design and Evaluation of an Embedded Training Email System. CyLab Technical Report. CMU-CyLab-06-017, 2006. http://www.cylab.cmu.edu/default.aspx?id=2253 [to be presented at CHI 2007] Think aloud study Role play as Bobby Smith, 19 emails including 2 interventions, and 4 phishing emails Three conditions: security notices, text / graphics intervention, comic strip intervention 10 non-expert participants in each condition, 30 total

42. Intervention #1 - Security notices How to spot an email How to report spoof email Five ways to protect yourself from identity theft

43. Intervention # 2 - Comic strip

44. Applies personalization and story based principle Presents declarative knowledge

45. Intervention # 2 - Comic strip Applies personalization principle

46. Intervention # 2 - Comic strip Applies contiguity principle

47. Intervention # 2 - Comic strip Applies contiguity and conceptual-procedural principle Presents procedural knowledge

48. Intervention # 3 - Text / graphics

49. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 49 User involvement

50. PhishTraining Legitimate Spam

51. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 51 User study - results We treated clicking on link to be falling for phishing 93% of the users who clicked went ahead and gave personal information

52. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 52 User study - results

53. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 53 User study - results Significant difference between security notices and the comic strip group (p-value < 0.05) Significant difference between the comic and the text / graphics group (p-value < 0.05)

54. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 54 Lessons learned H1: Security notices are an ineffective medium for training users Supported H2: Users make better decision when trained by embedded methodology compared to security notices Supported

55. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 55 Open questions Previous studies measured only knowledge gain Users have specific knowledge than generalized knowledge (Downs et al.) What about knowledge retention and transfer?

56. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 56 Knowledge retention and transfer Knowledge retention (KR) The ability to apply the knowledge gained after a time period Knowledge transfer (KT) The ability to transfer the knowledge gained from one situation to another situation

57. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 57 Study design Setup Think aloud study Role play as Bobby Smith, business administrator Respond to Bobby’s email Experiment Part 1: 33 emails and one intervention Part 2 (after 7 days): 16 emails and no intervention Conditions Control: no intervention Suspicion: an email from a friend Non-embedded: intervention in the email Embedded: intervention after clicking on link

58. Sample of emails from study Email typeSenderSubject information Legitimate-no-linkBrandy AndersonBooking hotel rooms for visitors Legitimate-linkJoseph DicostaPlease check PayPal balance Phishing-no-accountWells FargoUpdate your bank information! Phishing-accounteBayReactivate your eBay account SpamEddie ArredondoFw: Re: You will want this job InterventionAmazonRevision to your Amazon.com information

59. Comic strip intervention

60. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 60 Hypotheses H1: Participants in the embedded condition learn more effectively than participants in the non-embedded condition, suspicion condition, and the control condition H2: Participants in the embedded condition retain more knowledge about how to avoid phishing attacks than participants in the non-embedded condition, suspicion condition, and the control condition

61. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 61 Hypotheses H3: Participants in the embedded condition transfer more knowledge about how to avoid phishing attacks than participants in the non-embedded condition, suspicion condition, and the control conditions

62. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 62 Study results We treated clicking on link to be falling for phishing 89% of the users who clicked went ahead and gave personal information

63. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 63 Results - Phishing account emails

64. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 64 Results - Legitimate link emails

65. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 65 Measuring retention Training on Amazon.com account revision phish Testing a week later on Citibank account revision phish Significant difference between embedded and other groups (p < 0.01) “I remember reading last time that thing [training material] said not click and give personal information.”

66. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 66 Measuring transfer Training on Amazon.com account revision phish Testing a week later on eBay account reactivation phish Significant difference between embedded and other groups (p < 0.01) “PhishGuru said not to click on links and give personal information, so will not do it, I will delete this email.”

67. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 67 A few observations “I was more motivated to read the training materials since it was presented after me falling for the attack.” “Thank you PhishGuru, I will remember that [the 5 instructions given in the training material].” “This [image in the email] looks like some spam.”

68. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 68 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

69. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 69 Ongoing work Test the system in real-world

70. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 70 Conclusion Educating users about security can be a reality rather than just a myth

71. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 71 Collect homework

72. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 72 Acknowledgements Members of Supporting Trust Decision research group Members of CUPS lab

73. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 73

74. C MU U sable P rivacy and S ecurity Laboratory http://cups.cs.cmu.edu/

75. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 75 Learning-by-doing principle Production rules are acquired and strengthened through practice More practice better performance Story-centered curriculum Cognitive tutors

76. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 76 Immediate feedback principle Feedback during knowledge acquisition phase results in efficient learning Corrects behavior Avoids floundering LISP tutors “yes” or “no” or detailed

77. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 77 Conceptual-Procedural principle A concept is a mental representation or prototype of objects or ideas A procedure is a series of clearly defined steps Presenting procedural materials in between conceptual materials helps better learning Studies Mathematics

78. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 78 Contiguity principle Learning increases when words and pictures are presented contiguously rather than isolated from one another Human learning process - creating meaningful relation between pictures and words Studies Vehicle braking system Geometry cognitive tutor

79. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 79 Personalization principle Using conversational style rather than formal style enhances learning To use “I,” “we,” “me,” “my,” “you,” and “your” in the instructional materials Studies Process of lightning formation Mathematics

80. C MU U sable P rivacy and S ecurity Laboratory http://www.cs.cmu.edu/~ponguru 80 Story-based agent principle Characters who help in guiding the users through the learning process Using agents in a story-based content enhances user learning Stories simulate cognitive process Experiments - Herman