C MU U sable P rivacy and S ecurity Laboratory Trust and Semantic attacks Ponnurangam Kumaraguru (PK) Usable, Privacy, and Security Mar 17, 2008

C MU U sable P rivacy and S ecurity Laboratory 2 Who am I? Ph.D. candidate in the Computation, Organizations, and Society program in the School of Computer Science Research interests - Privacy, Security, Trust, Human Computer Interaction, and Learning Science

C MU U sable P rivacy and S ecurity Laboratory 3 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

C MU U sable P rivacy and S ecurity Laboratory 4 What is trust? No single definition Depends on the situation and the problem Many models developed Very few models evaluated

C MU U sable P rivacy and S ecurity Laboratory 5 Trust in literature Economics (how trust affects transactions) Reputation Marketing (how to build trust) Persuasion HCI (what affects trust) Design Psychology (positive theory) Intimacy

C MU U sable P rivacy and S ecurity Laboratory 6 Negative antecedents Risk Transaction cost Uncertainty … Trust Models Positive antecedents Benevolence Comprehensive information Credibility Familiarity Good feedback Propensity Reliability Usability Willingness to transact …

C MU U sable P rivacy and S ecurity Laboratory 7 How do users make decisions? Interview design, 25 participants (11 - experts and 14 - non-experts) Measured the strategies and decision process of the users in online situations Results Non-experts wanted advice to help them make better trust decisions Non-experts used significantly fewer meaningful signals compared to experts P. Kumaraguru, A. Acquisti, and L. Cranor. Trust modeling for online transactions: A phishing scenario. In Privacy Security Trust, Oct 30 - Nov 1, 2006, Ontario, Canada.

C MU U sable P rivacy and S ecurity Laboratory 8 Expert model Missed signals Meaningful signals Misleading signals Not deliberate states Unknown states States that affect well-being States that affect decision Signals

C MU U sable P rivacy and S ecurity Laboratory 9 Non- expert model Missed signals Meaningful signals Misleading signals Not deliberate states Unknown states States that affect well-being States that affect decision Signals

C MU U sable P rivacy and S ecurity Laboratory 10 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

C MU U sable P rivacy and S ecurity Laboratory 11 Security Attacks: Waves Physical: attack the computers, wires and electronics  E.g. physically cutting the network cable Syntactic: attack operating logic of the computers and networks  E.g. buffer overflows, DDoS Semantic: attack the user not the computers  E.g. Phishing

C MU U sable P rivacy and S ecurity Laboratory 12 Semantic Attacks “Target the way we, as humans, assign meaning to content.” System and mental model

An that we get

Features in the Subject: eBay: Urgent Notification From Billing Department

Features in the We regret to inform you that you eBay account could be suspended if you don’t update your account information.

Features in the fy&co_partnerid=2&sidteid=0

Website to collect information

C MU U sable P rivacy and S ecurity Laboratory 18 What is phishing? Phishing is “a broadly launched social engineering attack in which an electronic identity is misrepresented in an attempt to trick individuals into revealing personal credentials that can be used fraudulently against them.” Financial Services Technology Consortium. Understanding and countering the phishing threat: A financial service industry perspective

C MU U sable P rivacy and S ecurity Laboratory 19 Phishing Attack Life Cycle Post Attack Fraud & Abuse Collection Attack Setup Planning Source:

C MU U sable P rivacy and S ecurity Laboratory 20 A few statistics on phishing 73 million US adults received more than 50 phishing s each in the year 2005 Gartner in 2006 found 30% users changed online banking behavior because of attacks like phishing Gartner in 2006 predicted $2.8 billion loss due to phishing in that year

C MU U sable P rivacy and S ecurity Laboratory 21 Why phishing is a hard problem? Semantic attacks take advantage of the way humans interact with computers Phishing is one type of semantic attack Phishers make use of the trust that users have on legitimate organizations

C MU U sable P rivacy and S ecurity Laboratory 22 Three strategies for usable privacy and security Invisible strategy Regulatory solution Detecting and deleting the s User interface based Toolbars Training users

Our Multi-Pronged Approach Human side Interviews to understand decision-making PhishGuru embedded training Anti-Phishing Phil game Understanding effectiveness of browser warnings Computer side PILFER anti-phishing filter CANTINA web anti-phishing algorithm Automate where possible, support where necessary

C MU U sable P rivacy and S ecurity Laboratory 24 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

C MU U sable P rivacy and S ecurity Laboratory 25 Why user education is hard? Security is a secondary task Users not motivated to taking time for education Non-existence of an effective method

C MU U sable P rivacy and S ecurity Laboratory 26 To address the open questions Embedded training methodology Make the training part of primary task Create motivation among users Learning science Principles for designing training interventions

C MU U sable P rivacy and S ecurity Laboratory 27 Approaches for training Posting articles FTC,… Phishing IQ tests Mail Frontier, … Classroom training (Robila et al.) Sending security notices

Security notices How to spot an How to report spoof Five ways to protect yourself from identity theft

C MU U sable P rivacy and S ecurity Laboratory 29 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

C MU U sable P rivacy and S ecurity Laboratory 30 Why learning science? Research on how people gain knowledge and learn new skills ACT-R theory of cognition and learning Declarative knowledge (knowing that) Procedural knowledge (knowing how) Learning science principles

C MU U sable P rivacy and S ecurity Laboratory 31 Learning science principles Learning-by-doing More practice better performance Story-based agent Using agents in a story-based content enhances user learning Immediate feedback Feedback during learning phase results in efficient learning Clark, R.C., and Mayer, R.E. E-Learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. John Wiley & Sons, Inc., USA, 2002.

C MU U sable P rivacy and S ecurity Laboratory 32 Learning science principles Conceptual-procedural Presenting procedural materials in between conceptual materials helps better learning Contiguity Learning increases when words and pictures are presented contiguously than isolated Personalization Using conversational style rather than formal style enhances learning Clark, R.C., and Mayer, R.E. E-Learning and the science of instruction: proven guidelines for consumers and designers of multimedia learning. John Wiley & Sons, Inc., USA, 2002.

C MU U sable P rivacy and S ecurity Laboratory 33 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

C MU U sable P rivacy and S ecurity Laboratory 34 Design constraints People don’t proactively read the training materials on the web People can learn from web-based training materials, if only we could get people to read them! (Kumaraguru et al.) P. Kumaraguru, S. Sheng, A. Acquisti, L. Cranor, and J. Hong. Teaching Johnny Not to Fall for Phish. Tech. rep., Cranegie Mellon University,

C MU U sable P rivacy and S ecurity Laboratory 35 Embedded training We know people fall for phishing s So make the training available through the phishing s Training materials are presented when the users actually fall for phishing s Makes training part of primary task Creates motivation among users Applies learning-by-doing and immediate feedback principle

Embedded training example Subject: Revision to Your Amazon.com Information

Embedded training example Subject: Revision to Your Amazon.com Information Please login and enter your information

Comic strip intervention

C MU U sable P rivacy and S ecurity Laboratory 39 Design rationale What to show in the intervention? When to show the intervention? Analyzed instructions from most popular websites Paper and HTML prototypes, 7 users each Lessons learned Two designs Present the training materials when users click on the link

C MU U sable P rivacy and S ecurity Laboratory 40 Study 1: Evaluation of interventions H1: Security notices are an ineffective medium for training users H2: Users make better decisions when trained by embedded methodology compared to security notices

C MU U sable P rivacy and S ecurity Laboratory 41 Study design P. Kumaraguru, Y. Rhee, A. Acquisti, L. Cranor, J. Hong, and E. Nunge. Protecting People from Phishing: The Design and Evaluation of an Embedded Training System. CyLab Technical Report. CMU-CyLab , [to be presented at CHI 2007] Think aloud study Role play as Bobby Smith, 19 s including 2 interventions, and 4 phishing s Three conditions: security notices, text / graphics intervention, comic strip intervention 10 non-expert participants in each condition, 30 total

Intervention #1 - Security notices How to spot an How to report spoof Five ways to protect yourself from identity theft

Intervention # 2 - Comic strip

Applies personalization and story based principle Presents declarative knowledge

Intervention # 2 - Comic strip Applies personalization principle

Intervention # 2 - Comic strip Applies contiguity principle

Intervention # 2 - Comic strip Applies contiguity and conceptual-procedural principle Presents procedural knowledge

Intervention # 3 - Text / graphics

C MU U sable P rivacy and S ecurity Laboratory 49 User involvement

PhishTraining Legitimate Spam

C MU U sable P rivacy and S ecurity Laboratory 51 User study - results We treated clicking on link to be falling for phishing 93% of the users who clicked went ahead and gave personal information

C MU U sable P rivacy and S ecurity Laboratory 52 User study - results

C MU U sable P rivacy and S ecurity Laboratory 53 User study - results Significant difference between security notices and the comic strip group (p-value < 0.05) Significant difference between the comic and the text / graphics group (p-value < 0.05)

C MU U sable P rivacy and S ecurity Laboratory 54 Lessons learned H1: Security notices are an ineffective medium for training users Supported H2: Users make better decision when trained by embedded methodology compared to security notices Supported

C MU U sable P rivacy and S ecurity Laboratory 55 Open questions Previous studies measured only knowledge gain Users have specific knowledge than generalized knowledge (Downs et al.) What about knowledge retention and transfer?

C MU U sable P rivacy and S ecurity Laboratory 56 Knowledge retention and transfer Knowledge retention (KR) The ability to apply the knowledge gained after a time period Knowledge transfer (KT) The ability to transfer the knowledge gained from one situation to another situation

C MU U sable P rivacy and S ecurity Laboratory 57 Study design Setup Think aloud study Role play as Bobby Smith, business administrator Respond to Bobby’s Experiment Part 1: 33 s and one intervention Part 2 (after 7 days): 16 s and no intervention Conditions Control: no intervention Suspicion: an from a friend Non-embedded: intervention in the Embedded: intervention after clicking on link

Sample of s from study typeSenderSubject information Legitimate-no-linkBrandy AndersonBooking hotel rooms for visitors Legitimate-linkJoseph DicostaPlease check PayPal balance Phishing-no-accountWells FargoUpdate your bank information! Phishing-accounteBayReactivate your eBay account SpamEddie ArredondoFw: Re: You will want this job InterventionAmazonRevision to your Amazon.com information

Comic strip intervention

C MU U sable P rivacy and S ecurity Laboratory 60 Hypotheses H1: Participants in the embedded condition learn more effectively than participants in the non-embedded condition, suspicion condition, and the control condition H2: Participants in the embedded condition retain more knowledge about how to avoid phishing attacks than participants in the non-embedded condition, suspicion condition, and the control condition

C MU U sable P rivacy and S ecurity Laboratory 61 Hypotheses H3: Participants in the embedded condition transfer more knowledge about how to avoid phishing attacks than participants in the non-embedded condition, suspicion condition, and the control conditions

C MU U sable P rivacy and S ecurity Laboratory 62 Study results We treated clicking on link to be falling for phishing 89% of the users who clicked went ahead and gave personal information

C MU U sable P rivacy and S ecurity Laboratory 63 Results - Phishing account s

C MU U sable P rivacy and S ecurity Laboratory 64 Results - Legitimate link s

C MU U sable P rivacy and S ecurity Laboratory 65 Measuring retention Training on Amazon.com account revision phish Testing a week later on Citibank account revision phish Significant difference between embedded and other groups (p < 0.01) “I remember reading last time that thing [training material] said not click and give personal information.”

C MU U sable P rivacy and S ecurity Laboratory 66 Measuring transfer Training on Amazon.com account revision phish Testing a week later on eBay account reactivation phish Significant difference between embedded and other groups (p < 0.01) “PhishGuru said not to click on links and give personal information, so will not do it, I will delete this .”

C MU U sable P rivacy and S ecurity Laboratory 67 A few observations “I was more motivated to read the training materials since it was presented after me falling for the attack.” “Thank you PhishGuru, I will remember that [the 5 instructions given in the training material].” “This [image in the ] looks like some spam.”

C MU U sable P rivacy and S ecurity Laboratory 68 Outline Trust Semantic attacks - Phishing User education Learning science Evaluating embedded training Ongoing work Conclusion

C MU U sable P rivacy and S ecurity Laboratory 69 Ongoing work Test the system in real-world

C MU U sable P rivacy and S ecurity Laboratory 70 Conclusion Educating users about security can be a reality rather than just a myth

C MU U sable P rivacy and S ecurity Laboratory 71 Collect homework

C MU U sable P rivacy and S ecurity Laboratory 72 Acknowledgements Members of Supporting Trust Decision research group Members of CUPS lab

C MU U sable P rivacy and S ecurity Laboratory 73

C MU U sable P rivacy and S ecurity Laboratory

C MU U sable P rivacy and S ecurity Laboratory 75 Learning-by-doing principle Production rules are acquired and strengthened through practice More practice better performance Story-centered curriculum Cognitive tutors

C MU U sable P rivacy and S ecurity Laboratory 76 Immediate feedback principle Feedback during knowledge acquisition phase results in efficient learning Corrects behavior Avoids floundering LISP tutors “yes” or “no” or detailed

C MU U sable P rivacy and S ecurity Laboratory 77 Conceptual-Procedural principle A concept is a mental representation or prototype of objects or ideas A procedure is a series of clearly defined steps Presenting procedural materials in between conceptual materials helps better learning Studies Mathematics

C MU U sable P rivacy and S ecurity Laboratory 78 Contiguity principle Learning increases when words and pictures are presented contiguously rather than isolated from one another Human learning process - creating meaningful relation between pictures and words Studies Vehicle braking system Geometry cognitive tutor

C MU U sable P rivacy and S ecurity Laboratory 79 Personalization principle Using conversational style rather than formal style enhances learning To use “I,” “we,” “me,” “my,” “you,” and “your” in the instructional materials Studies Process of lightning formation Mathematics

C MU U sable P rivacy and S ecurity Laboratory 80 Story-based agent principle Characters who help in guiding the users through the learning process Using agents in a story-based content enhances user learning Stories simulate cognitive process Experiments - Herman