1. Mary Theofanos Visualization & Usability Group Information Access Division Information Technology Laboratory Usability Research in Support Of Cyber-Security

2. Comprehensive National Cyber-Security Initiative: Research and Development

3. Usability Research Goal: To enable policy makers to make better decisions

4. View of solution space of the security and usability equation Each point (solution) has a security level and a usability level Policy constrains solution space X Trivial usability solution x Trivial security solution x x x Ideally policy will permit the best solution X Optimum acceptable usability/security solution

5. Four Primary Research Areas 1. Passwords 2. Password Policies 3. Multi-Factor Authentication 4. Usability and Security Framework

6. Password Usability Research Goal: To enable decision makers to make better password policies Based on actual data Secure in practice not just secure in theory Takes into account user behavior

7. Research requires gathering data Instrument is a comprehensive survey of password usage and management Survey has been independently reviewed by experts in questionnaire design from Bureau of Labor Statistics Responses are anonymous to prevent misuse, but with some demographics Survey is for Federal employees only Currently being piloted across NIST Survey is low impact (estimated 15 minutes)

8. Password Survey: It’s a matter of perspective Usability View Password creation strategies Password management strategies Perception of policies/security Annoyance Factors Security View Authentication Compliance Training Needs

9. Password Survey Questions Password Survey How many passwords do you use? Do you use the same password on different accounts? How much time does it take to create a password? How do you keep track of your passwords?

10. To date we have: Over 550 responses Participants are passionate Over 95% use user name and password to authenticate Mean number of passwords 12 (range 1 to 210) Try to use same password for different accounts 70% “track on paper” in some form Have interference from multiple policies Need to make password policy explicit

11. Potential Impact Results could motivate password policies that are less onerous and more effective Wide spread participation will strengthen the results

12. Password Policy Quiz What are the minimum length and maximum lifetime? Are special characters required? Which special characters are allowed? Is white-space allowed? Are you allowed to write-down or store passwords online? Workplace password policies involve much more than length and lifetime.

13. Policies vary dramatically in length

14. Develop a useful password-policy language for studying and improving policies. Specifically, the language should enable us to (1) discuss and compare policies, and (2) assess how policy rules affect user behavior and security. Approach: 1. Develop a taxonomy of policy rules 2. Collect a corpus of representative policies 3. Analyze the corpus using its taxonomic structure 14 Goal

15. Step 1: Develop a Taxonomy Reduce policies to an unambiguous language: Users must change passwords before 90 days. Users must change passwords immediately if compromised. Users must change passwords immediately if directed by management. Users must change passwords immediately if found non- compliant. Users must change passwords immediately if shared. Users must create passwords with a character in the set of numbers, special characters (unspecified). Users must create passwords with length greater than or equal to 8 characters. Users must not create passwords in the set of passwords to an outside system. Users must not create passwords in the set of strings with a character repeated 5 or more times. Users must not create passwords in the set of their last 2 years of passwords. Users must not create passwords in the set of their last 8 passwords. … Benefits of a formal (EBNF) grammar: What is allowed, forbidden, and ambiguous is explicit. Specific statements can be pinpointed for discussion. Language differences no longer prevent comparisons. (Clarity first, then other usability issues)

16. Benefits of a formal (EBNF) grammar: What is allowed, forbidden, and ambiguous is explicit. Specific statements can be pinpointed for discussion. Language differences no longer prevent comparisons. (Clarity first, then other usability issues) 16

17. How many different rules? 41 policies 155 unique rules 449 total rules

18. Policy exploration and visualization

20. NIST

21. Policy exploration and visualization NISTCensus Users must create passwords with length greater than or equal to 8 characters.

22. A typical policy imposes 10+ rules on a user, and they are not the same from policy to policy. The only rule that over half the policies agreed on was an 8- character minimum. If users have 3-5 passwords, they will not be able to keep all the requirements straight. Nearly every policy had ill-formed requirements that make it impossible for a user to comply with certainty. 22 Basic findings

23. Potential applications Attach security rationale to regions of the space. What threat is this space addressing? How much does it help? Attach usability concerns. Does this rule make a reasonable demand on a user’s capabilities? Will the rule interfere with others? Check for consistency and ambiguity. Are the character sets and forbidden content completely specified? Explore and establish best practices more rigorously. Which rules appear frequently? Which are anomalies? Are they frequent within similar organizations? 23

24. Final remarks Goal: Develop a useful password-policy language for studying and improving policies. Specifically, the language should enable us to (1) discuss and compare policies, and (2) assess how policy rules affect user behavior and security. How do formal languages help with usability? Establish what the security requirements are. Provide a clear behavioral goal for users. Separate policy requirements from user education. 24

25. Multi-factor Authentication Which factors encourage adoption and which factors discourage adoption Literature search of relevant fields Establishing a database of publications Annotated bibliography of publications Test our hypothesis Currently running a diary study in conjunction with a PIV pilot

26. PIV study objectives Investigate and gain insight on the impact of this new system on worker performance Explore attitudes people have about this new two-factor authentication system that may affect uptake. Learn whether there are differences in users’ expectations and attitudes of systems prior to first use and ongoing use.

27. Scenarios of use Login using card and pin Digitally sign email Encrypt email Access web application to register visitors Data collection instruments Daily emails Periodic interviews Direct observation Pre and post surveys

28. Users concerns included: Forgetting card at home Leaving the card in the reader Session timeouts due to inactivity even though sitting at desk Forgetting to use card to login or unlock and using password instead Forgetting password

29. Ah, they’ll get used to it

30. Framework for Usability and Security Recognizes that security is not the primary task Recognizes that both disciplines must work together Development model Evaluation model

31. What is Usability ISO 13407:1999 “Usability: The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” How to measure Effectiveness Efficiency Satisfaction What to measure Users Goals Context of Use Definition Identifies How to Proceed:

32. Usability User Task User Evaluation Areas (UEAs) Attention Adoption Trust Conceptual Models Interaction Invisibility Impact & Side Effects Appeal Application Robustness

33. Usability and Security User Task User Evaluation Areas (UEAs) Attention Adoption Trust Conceptual Models Interaction Invisibility Impact & Side Effects Appeal Application Robustness

34. Usability and Security User Task

35. AEGIS Gather Participants Identify Assets Model System Assets and Context Value Assets WRT Security Properties Identify Threats Identify Vulnerabilities Identify RisksDesign Countermeasures Assess Cost/Benefits in Context Choose Acceptable Cost/Benefits/Risks Appropriate and Effective Guidance for Information Security* *Flechais, Sasse, & Hailes. 2003

36. Context of Use AEGIS Model Requirements Design Solution Evaluation Gather Participants Identify Assets Model System Assets and Context Value Assets WRT Security Properties Identify Threats Identify Vulnerabilities Identify RisksDesign Countermeasures Assess Cost/Benefits in Context Choose Acceptable Cost/Benefits/Risks User Centered Design User Appropriate and Effective Guidance for Information Security* *Flechais, Sasse, & Hailes. 2003

37. Context of Use Requirements Design Solution Evaluation Gather Participants Identify Assets Model System Assets &Context Value Assets WRT Security Properties Identify Threats Identify Vulnerabilities Identify Risks Design Countermeasures Assess Cost/Benefits in Context Choose Acceptable Cost/Benefits/Risks User Task Definition User Experience Usability Testing Summative Testing UCD and AEGISUser Centered Design

38. Context of Use Requirements Design Solution Evaluation Gather Participants Identify Assets Model System Assets &Context Value Assets WRT Security Properties Identify Threats Identify Vulnerabilities Identify Risks Design Countermeasures Assess Cost/Benefits in Context Choose Acceptable Cost/Benefits/Risks User Task Definition User Experience Usability Testing Summative Testing UCD and AEGIS

39. Context of Use Requirements Design Solution Evaluation Gather Participants Identify Assets Model System Assets &Context Value Assets WRT Security Properties Identify Threats Identify Vulnerabilities Identify Risks Design Countermeasures Assess Cost/Benefits in Context Choose Acceptable Cost/Benefits/Risks User Task Definition User Experience Usability Testing Summative Testing UCD and AEGIS Gather Participants Identify Assets Model System Assets & Context Value Assets WRT Security Properties Identify Threats Identify Vulnerabilities Identify Risks Design Countermeasures Assess Cost/Benefits in Context Choose Acceptable Cost/Benefits/Risks Assurance Cases EAU Testing

40. Propose : Action-Awareness Framework 1. What is the action that is required to meet the security requirements? Requires security model 2. What are the usability implications of that action? Requires usability model

41. Users are critical to cyber-security success Must understand user behavior Need good studies and data to support policies Need inter-disciplinary research We have the opportunity

42. Starting this fall: NIST sponsored NAS workshop last summer User Perception How do users perceive security – what is it? Security is not a perceived as a task Can it be learned? Inattentional Blindness Eye-tracking studies

43. Policies may weaken security But… Users rarely understand these threats. They are governed by multiple policies at work, through financial institutions, and for other online activities. The number of policies, ambiguities in them, and discrepancies among them are a cognitive burden. So… Users are forced to choose weak passwords or write them down. Policy violations become a matter of course. Threats that should be addressed by policy are not. Security is weakened. 43

44. Policies as password specifications Policies regulate behavior (or they try to). For instance: Users must not store passwords in writing anywhere. Users must create passwords with a character in the set of numbers. Users must not create passwords in the set of dictionary words. But they are not written in clear and unambiguous language. 44

45. Is the following a legal password: password2%(letters, number, and specials) ? password%(letters and specials) ? Password%(upper-case and lower-case letters and specials) ? |*@$$^^()%&(all specials) ? What constitutes a special character anyway? Department of Energy: Passwords contain a combination of letters, numbers, and at least one special character

46. Corporate and government policies of primary interest. Method: Visit every Federal agency on usa.gov. Search for “password policy” and “passphrase policy.” Discard all returned documents except official employee policies. Comparison with personal-site policies also of interest. Method: Visit websites with policies studied by Florencio and Herley (2010). Discard academic-site or other out-of-scope policies. Visit top-40 banking websites according to alexa.com. Search for password policies and change or creation instructions. 46 Step 2: Collect a Corpus

47. Workplace policies (22): Agency for Healthcare Research and Quality Ames Laboratory Argonne National Lab Brookhaven National Laboratory Census Bureau Department of Homeland Security Department of Commerce Department of Defense Department of Energy Federal Deposit Insurance Corporation Fermi National Accelerator Laboratory Lawrence Berkeley National Lab Marine Corps Enterprice Network Maui High Performance Computing Center National Aeronautics and Space Administration National Institute of Standards and Technology Peace Corps Small Business Administration SLAC National Accelerator Laboratory Department of State USDA Mainframes USDA Biosafety Level-3 Facilities

48. Personal policies (19): Google Facebook Yahoo Youtube Windows Live Amazon Weather Channel RockYou CBS Sports Vanguard Schwab Bank of America Citibank PayPal U.S. Bank Wachovia Online Services Discover Card TD Bank Financial Group HSBC

49. General statistics: How many different rules are there? Are any two policies the same? (i.e., the snowflake question) What rules appear frequently? How often are policies ambiguous or contradictory? Broader questions: Which rules constitute best practices? Which rules require user cooperation? What rules affect usability? What rules affect security? How? 49 Step 3: Analyze the Corpus

50. Are any two policies the same? No (they are each like unique little snowflakes). NIST (28) and the Census Bureau (22) share 14. DoC (28) shares 12 with NIST and 8 with Census.

51. What rules appear frequently? Users must create passwords with length greater than or equal to 8 characters. (23) Users must not communicate passwords to anyone. (15) Users must change passwords immediately if compromised. (10) Users must not create passwords with a substring in the set of dictionary words. (10) 73 rules appear only once.

52. How often are policies ambiguous or contradictory? Rules were flagged as ambiguous if they… Concerned special characters without defining them, Concerned “letters” without specifying case, Concerned vague prohibitions on “patterns” 34/41 policies (83%) contain an ambiguous rule.

53. Potential applications Attach security rationale to regions of the space. What threat is this space addressing? How much does it help? Attach usability concerns. Does this rule make a reasonable demand on a user’s capabilities? Will the rule interfere with others? Check for consistency and ambiguity. Are the character sets and forbidden content completely specified? Explore and establish best practices more rigorously. Which rules appear frequently? Which are anomalies? Are they frequent within similar organizations? 53