C MU U sable P rivacy and S ecurity Laboratory Protecting People from Phishing: The Design and Evaluation of an Embedded Training System P. Kumaraguru, Y. Rhee, A. Acquisti, L. Cranor, J. Hong, E. Nunge

Phishing

Subject: eBay: Urgent Notification From Billing Department

Phishing We regret to inform you that you eBay account could be suspended if you don’t update your account information.

Phishing fy&co_partnerid=2&sidteid=0

Phishing website

C MU U sable P rivacy and S ecurity Laboratory 7 What is phishing? Phishing is “a broadly launched social engineering attack in which an electronic identity is misrepresented in an attempt to trick individuals into revealing personal credentials that can be used fraudulently against them.” Financial Services Technology Consortium. Understanding and countering the phishing threat: A financial service industry perspective

C MU U sable P rivacy and S ecurity Laboratory 8 Phishing is growing 73 million US adults received more than 50 phishing s a year in 2005 Gartner found approx. 30% users changed online banking behavior because of attacks like phishing in 2006 Gartner predicted $2.8 billion loss in 2006

C MU U sable P rivacy and S ecurity Laboratory 9 Why phishing is a hard problem? Semantic attacks take advantage of the way humans interact with computers Phishing is one type of semantic attack Phishers make use of the trust that users have on legitimate organizations

C MU U sable P rivacy and S ecurity Laboratory 10 Counter measures for phishing Silently eliminating the threat Regulatory & policy solutions filtering (SpamAssasin) Warning users about the threat Toolbars (SpoofGuard, TrustBar) Training users not to fall for attacks

C MU U sable P rivacy and S ecurity Laboratory 11 Why user education is hard? Security is a secondary task (Whitten et al.) Users are not motivated to read privacy policies (Anton et al.) Reading existing online training materials creates concern among users (Anandpara et al.)

C MU U sable P rivacy and S ecurity Laboratory 12 Our hypotheses Security notices are an ineffective medium for training users Users make better decision when trained by embedded methodology compared to security notices

C MU U sable P rivacy and S ecurity Laboratory 13 Design constraints People don’t proactively read the training materials on the web Organizations send “security notices” to train users and people don’t read security notices People can learn from web-based training materials, if only we could get people to read them! (Kumaraguru, 2006) P. Kumaraguru, S. Sheng, A. Acquisti, L. Cranor, and J. Hong. Teaching Johnny Not to Fall for Phish. Tech. rep., Cranegie Mellon University,

C MU U sable P rivacy and S ecurity Laboratory 14 Embedded training We know people fall for phishing s So make training available through the phishing s Training materials are presented when the users actually fall for phishing s

Embedded training example Subject: Revision to Your Amazon.com Information

Embedded training example Subject: Revision to Your Amazon.com Information Please login and enter your information

Comic strip intervention

C MU U sable P rivacy and S ecurity Laboratory 18 Design rationale What to show in the intervention? When to show the intervention? Analyzed instructions from most popular websites Paper and HTML prototypes, 7 users each Lessons learned Two designs Present the training materials when users click on the link

Comic strip intervention

Intervention #1 - Comic strip

Intervention #2 - Graphics and text

C MU U sable P rivacy and S ecurity Laboratory 24 Study design Think aloud study Role play as Bobby Smith, 19 s including 2 interventions, and 4 phishing s Three conditions: security notices, text / graphics intervention, comic strip intervention 10 non-expert participants in each condition, 30 total

Intervention #1 - Security notices

C MU U sable P rivacy and S ecurity Laboratory 26 Intervention #2 - Graphics and text

Intervention #3 - Comic strip

PhishTraining Legitimate Spam

C MU U sable P rivacy and S ecurity Laboratory 29 User study - results We treated clicking on link to be falling for phishing 93% of the users who clicked went ahead and gave personal information

C MU U sable P rivacy and S ecurity Laboratory 30 User study - results

C MU U sable P rivacy and S ecurity Laboratory 31 User study - results Significant difference between security notices and the comic strip group (p-value < 0.05) Significant difference between the comic and the text / graphics group (p-value < 0.05)

C MU U sable P rivacy and S ecurity Laboratory 32 Conclusion H1: Security notices are an ineffective medium for training users Supported H2: Users make better decision when trained by embedded methodology compared to security notices Supported

Latest comic strip design

C MU U sable P rivacy and S ecurity Laboratory 34 Ongoing work Measuring knowledge retention and knowledge transfer Knowledge retention is the ability to apply the knowledge gained from one situation to another same or similar situation after a time period Knowledge transfer is the ability to transfer the knowledge gained from one situation to another situation after a time period Is falling for phishing necessary for training?

C MU U sable P rivacy and S ecurity Laboratory 35 Coming up WWW 2007 CANTINA: A Content-Based Approach to Detecting Phishing Web Sites Learning to Detect Phishing s Our other research in anti-phishing Symposium On Usable Privacy and Security (SOUPS), July , 2007 at Carnegie Mellon University

C MU U sable P rivacy and S ecurity Laboratory 36 Acknowledgements Members of Supporting Trust Decision research group Members of CUPS lab

C MU U sable P rivacy and S ecurity Laboratory