1. Amir Herzberg and Ronen Margulies Bar Ilan University 1

2. Agenda Introduction: phishing, current defenses & user studies Psychology: principles of effective defense mechanisms Long-term user study & results Usability issues 2

3. Some Phishing Numbers Huge amount of attacks (antiphishing.org)antiphishing.org $3.2 billion lost in the US only in 2007 (Gartner) Some recent cyber hacks `spear phishing’ @ Lockheed Martin DigiNotar – stolen SSL certificates of CIA, MI6, Mossad, Google, Facebook, Skype and Yahoo 3

4. 4 Current Defenses: Passive Indicators Basic browser indicators Name of site & CA (from certificate) Warnings User-custom text/image for site (e.g. Yahoo!’s sign-in seal)

5. Previous Studies Short-term lab studies Awareness to study’s purpose  more cautious than real life Rather high detection rates, 63-95% [DTH06, WMG06, HJ08] Low detection rates 3-40% [DTH06, WMG06, SD*07] Unaware  less cautious than real life Very low detection rates, 0-8% [WMG06, SD*07, HJ08] 5

6. Goals, Method & Contribution Goals: Realistic evaluation of defense mechanisms Find effective mechanisms, detection and prevention Method: Long-term experiment, real-purpose system  Awareness is not a problem (More reliable) Results: Highly effective new mechanisms, best results when combined 82 % detection rates 93 % overall resistance rates

7. Agenda Introduction: phishing, current defenses & studies Psychology: principles of effective mechanisms Long-term user study & results Usability issues 7

8. Users Responses on the Web Click-whirr response: mindless response to repeating situation[ C08 ] [ KTW09 ]: click whirr responses allow phishing Automatic submission of credentials Automatic following of links: email, sites, homepage Most logins are not harmful  It’s easier to just skip checking passive indicators Especially since users’ primary goal isn’t security! Solutions? Forcing functions Negative training functions 8

9. Forcing Functions Forcing function prevents users from progressing with their task until taking a certain action Term from the human reliability field [ KTW09 ] suggested them for usable-security Method: site obligates users to take safe actions during each login With sufficient training, will become click whirr responses themselves Examples of forcing functions login mechanisms: Interactive custom indicators Login bookmarks 9

10. Interactive Custom Indicators Force users to click them in order to login Browser-side solution – Passpet [ YS06 ] Submits the password by clicking the custom pet image Server-side solution – site hides the password textfield until the user clicks his custom image Variation: several images on the login page 10

11. 11 Login Bookmarks User must click on bookmark to login Advantages: assures correct URL, SSL, prevention Suggested by Adida [ A07 ], not yet tested Bookmark contains token, used as 1 st authenticator Without a valid token, site prevents the login Password used as 2 nd authenticator Combining with interactive custom images Token enables displaying the correct image Provides “defense-in-depth”: prevention + detection Provides 2x2 (two-factor and two-sided) authentication

12. 12 Bookmark+ Interactive Image Login Ceremony AliceBrowsermysite.com types mysite.com/login.php GET /login.php You should login via your bookmark clicks bookmark secret token login.php + custom image You should login via your bookmark clicks image enables password submission submits password password

13. 13

14. 14

15. 15

16. 16

17. 17

18. 18

19. 19

20. 20

21. 21

22. Forcing Functions aren’t Enough How to defeat forcing functions? Bypass them with dangerous actions E.g.: follow a link to a spoofed login page instead of clicking the bookmark Needs training against dangerous actions Negative training functions: make users experience failure with dangerous actions Two mechanisms: “Non working” links in the site‘s email announcements “Non working” account-entrance button in the site‘s home page 22

23. Agenda Introduction: phishing, current defenses & studies Psychology: principles of effective mechanisms Long-term user study & results Usability issues 23

24. User Study Online exercise submission system ~ 400 computer science students Used the system regularly for 3 semesters Submitted exercises, received new grades emails Dozens – hundreds logins per user Each user was randomly assigned: A login method: image only, bookmark only, bookmark+image, bookmark+4 images, none An email method: no link, no link+warning, link 24

25. Negative Training Functions Bookmark & link users received “non-working” links Error message at the site’s login page Account-entrance button at the homepage Worked for non-bookmark users “Did not work” for bookmark users – same error message 25

26. Simulated Attacks All attacks invoked with low probabilities Spoofed sites allowed login Classic phishing attack Malicious bookmark replacement Spoofed home page attack Pharming attack (recent) browsers display an error page 26

27. Study Results – Detection Rates 27 Significant differences, best results when combined Interactive custom image is highly effective more than twice the detection rates of non-image users

28. Users’ Response to emails 28 Warnings don’t help The login bookmark is only effective when combined with “non working" links

29. Spoofed Home Page Attack Results Lower detection rates than other attacks Users might highly trust the home page of a familiar site Prevention gets higher importance Almost all bookmark users tried to enter the site's login page via its home page All but two stopped trying after 5 attempts or less login bookmark + “non working” account-entrance button = effective prevention 29

30. Additional Observations The login bookmark increases the detection rates Better detection rates for bookmark users than none users Better detection rates for bookmark+image than image only Modern browsers’ active warnings stopped 72% from entering spoofed pages Low false negative rates Only 1/8 of all users falsely reported a spoofed page, mostly once 30

31. Agenda Introduction: phishing, current defenses & studies Psychology: principles of effective mechanisms Long-term user study & results Usability issues 31

32. Usability Survey 72% want to use login bookmarks for high-value sites, 51% for medium-value sites Bookmark setup  not much of an objection Good willingness rates for interactive custom images 60% did not feel more protected, most did not understand the purpose of their mechanisms Contradiction with the good results  Users don’t need deep understanding for the mechanisms’ training to be effective  Mechanisms are adequate for the general public  Similar results for the general-public (?) 32

33. Conclusions Long-term user study measuring the effectiveness of forcing and negative training functions mechanisms Interactive custom images doubled the detection rates Login bookmarks + non-working links doubled the prevention rates Combining all mechanisms: best detection ( 82%) and overall resistance ( 93% ) rates Most users are willing to use the mechanisms, especially for high-value sites The mechanisms work in-spite many users did not understand their purpose 33

34. Thank you! 34