“We’re on the Same Page”: A Usability Study of Secure Using Pairs of Novice Users Scott Ruoti, Jeff Andersen, Scott Heidbrink, Mark O'Neill, Elham Vaziripour, Justin Wu, Daniel Zappala, Kent Seamons Internet Security Research Lab Brigham Young University 34th Annual ACM Conference on Human Factors in Computing Systems (CHI ‘16)

Security  was built without security  Security has since been bolted on  is still often insecure  In transit and at rest  was built without security  Security has since been bolted on  is still often insecure  In transit and at rest

End-to-end Encryption Alice Bob Sever Mallory

What’s the Hold Up?  Secure systems have largely been unusable  Only viable in corporate settings  How do we get the masses to adopt secure ?  We’ve mostly relied on expert-led adoption  Can be adopted in a grassroots style?  No previous study has examined this question  Secure systems have largely been unusable  Only viable in corporate settings  How do we get the masses to adopt secure ?  We’ve mostly relied on expert-led adoption  Can be adopted in a grassroots style?  No previous study has examined this question

Methodology  Brought in pairs of novice participants  Friends  Johnny ed Jane asking for help with his taxes  Johnny told to encrypt his and given the URL for a secure system  Jane was told to wait for an from Johnny  Johnny and Jane had to collaboratively set up and use secure  Brought in pairs of novice participants  Friends  Johnny ed Jane asking for help with his taxes  Johnny told to encrypt his and given the URL for a secure system  Jane was told to wait for an from Johnny  Johnny and Jane had to collaboratively set up and use secure

Pwm Integrates with Gmail Identity-based encryption

Tutanota depot PGP and password- based encryption

Virtru Integrated and depot Custom key escrow scheme

Results  Within subjects  25 participant pairs  50 total participants  Quantitative metrics  Qualitative feedback  Within subjects  25 participant pairs  50 total participants  Quantitative metrics  Qualitative feedback

System Usability Scale

PwmTutanotaVirtru

Task Completion Time

Mistakes  No mistakes with Pwm  One mistake with Virtru  Many mistakes with Tutanota  Two-third (68%, n=17) sent password through regular  Half (48%, n=12) selected easily guessed passwords  No mistakes with Pwm  One mistake with Virtru  Many mistakes with Tutanota  Two-third (68%, n=17) sent password through regular  Half (48%, n=12) selected easily guessed passwords

Favorite System

Paired-participants  Two novices  Struggled using Tutanota’s password-based encryption  Two perspectives  Jane strongly preferred Pwm  More natural behavior  Relaxed during the study  Willing to believe other side made mistakes  Two novices  Struggled using Tutanota’s password-based encryption  Two perspectives  Jane strongly preferred Pwm  More natural behavior  Relaxed during the study  Willing to believe other side made mistakes

Quotes “I thought it was good, I dunno, might’ve taken the pressure off too, where it’s like, ‘Okay, he’s figuring this out too’, so I can just, y’know, I don’t have to feel as ‘under-the-microscope’ in the study.”

Quotes “...I was more at ease probably than I would’ve been if it was someone random on the other end...It would’ve felt more mechanical, robotic, whereas I know [her] and I was calling my wife, ‘Hi wife! What’s the password?’ It felt a lot more personable for me I think....”

Quotes “It was good in that you saw the troubles, like the third system [Tutanota], I didn’t even know how it worked, so I ended up sending an to myself on Gmail so then I could see what was happening on her end, to know like how it works on the other end. So I think it’s good to have two people on each end that don’t know what’s going on, because if it weren’t I’d assume the person on the other side had done it before...”

Other Lessons Learned  Hiding security details leads to a lack of trust  Participants prefer integrated secure  Integrated tutorials are essential  Users are interested in secure  Hiding security details leads to a lack of trust  Participants prefer integrated secure  Integrated tutorials are essential  Users are interested in secure

Summary  Paired participant studies are helpful  Assesses the usability of grassroots adoption  Allows for interesting interactions between two novice users  Leads to more natural participant behavior  We are getting closer to usable, secure for the masses  Paired participant studies are helpful  Assesses the usability of grassroots adoption  Allows for interesting interactions between two novice users  Leads to more natural participant behavior  We are getting closer to usable, secure for the masses

Questions