1. Madison Russo & Mohammad Anwer
Auto Inquiry and Change & Insurance Card Refresh - User Experience Study March 2017
Madison Russo & Mohammad Anwer

2. AIC and AICR UX Study - Approach
Method
Moderated 1:1 Interview with task-based testing - presenting participants with a clickable prototype of the mobile and desktop app while asking them to perform specific actions
Link to Study Script
Used New England Focus Group for recruitment and Sapient for their facility
Participants
7 people who met the requirements of our PI Customer Screener
Link to Screener
Scope
MyTravelers policy summary screen, add a vehicle flow, and insurance docs and cards screen
Link to Screen Flow
Proprietary & Confidential

3. AIC and AICR UX Study - Mobile Study

4. AIC and AICR UX Study - Desktop Study

5. AIC and AICR UX Study – High-Level Findings
Validations:
Navigation to ‘Make changes’ link was intuitive (Auto policy icon > summary > ‘Make changes’ link).
Although some participants expressed concerns over having to search for this, I think this was because they were explicitly told to add a vehicle so they were trying to search for this text.
The linear flow of adding a vehicle is very straight forward and easy.
The majority of users understood how to download a PDF on their laptop and it to the driver.
Users understood how to use native phone features to cards.
Users were not familiar with Apple Wallet but recognized the value of it if they used it.
Proprietary & Confidential

6. AIC and AICR UX Study – High-Level Findings
Key Issues:
Users expected to see a dollar value associated with each coverage option to better compare their deductibles with the effect on their premium.
Users could not clearly distinguish the differences among our roadside assistance plans and new car replacement plans.
Discounts were not clearly visible; users did not know if they were eligible for discounts online.
The premium change amount was clear but users wanted to see their billing payment breakdown.
Proprietary & Confidential

7. AIC and AICR UX Study – Key Issue 1
Users expected to see a dollar value associated with each coverage option to better compare their deductibles with the effect on their premium.
“I would have a package at the bottom that would reflect what my choices are [so I could] see some options I might have.”
“I think this would have the same deductibles as my other cars but I would want to see what the total [premium] is if I wanted to change them.”
Proprietary & Confidential

8. AIC and AICR UX Study – Key Issue 2
Users could not clearly distinguish differences among our roadside assistance plans and new car replacement plans.
“I assume the premier new car replacement is better than this one (new car replacement) but I don’t know what is in this one... I would also want to know how much more expensive the premier is.”
Proprietary & Confidential

9. AIC and AICR UX Study – Key Issue 3
Discounts were not clearly visible; users did not know if they were eligible for discounts online.
“I would expect to see something about discounts.. I didn’t see any questions about that.”
Proprietary & Confidential

10. AIC and AICR UX Study – Key Issue 4
The premium change amount was clear but users wanted to see their billing payment breakdown.
Total bill amount for the next bill
Total premium amount for their remaining 6-month / 12-month contract (currently in detailed coverage modal)
“I pay my insurance bill [in lump sum] so I would want to see what the total amount is… yeah I would want this to be on this page, I don’t want to have to look for it”
“I am looking for what my next bill would be…is that in this link (How are my payments affected…)?
Proprietary & Confidential

11. AIC and AICR UX Study - SUS
System Usability Score (SUS)
Average = 90.4
(Industry average = 68)
Address Change Study - Nov 2016 SUS Score:
Next steps, next week:
Review full list of issues with Business and determine course of action
New designs to address issues, if needed
Proprietary & Confidential

12. AIC and AICR UX Study – Issue Severity List
Number of Issues Identified:
Critical Issues: 2
High Issues: 4
Medium Issues: 4
Low Issues: 5
Total: 15
Issue Severity List
Raw data and full list of issues
Link to Issue Severity List

13. AIC and AICR UX Study - Appendix
UX Issue Severity Model:
PI Usability Research established paradigm based on difficulty to overcome and participants’ frustration
Frustration: perceived change in participant’s body language or behavior
Difficulty: delta from expected time to complete baseline established before the study
Four severity levels: Low, Medium, High, Critical
Breakdown of Questions:
Was the issue DIFFICULT to overcome?
Did the user ask for help?
Did they express frustration through body language or other behavioral cues?
Did it take longer than reasonable for the participant to correct the issue?
Was it PERSISTENT?
Did the issue affect at least 50% of the participants?
Is there significant BUSINESS impact?
Could the issue result in calls to the call center?
Does it reduce the efficiency of a participant and cost us more person-hours?
Could it reduce conversion?
Would users speak poorly of us if they encounter this issue?
Do we miss opportunities to sell more?