1. Lessons Learned: Two-Step Verification
HUIT Senior Leadership Team | January 5, 2017
Smith Campus Center

2. Agenda What did we do? What were the goals? How did it go?
Ground rules
What did we learn about the project?
What would we do next time?
Other thoughts/suggestions

3. What did we do?
We invited all of HUIT and anyone in the Schools who was part of the two-step verification team to get together to talk about the lessons we could learn from the project
The session was held December 20, 2016 for 80 minutes at Piper Auditorium at Gund Hall
About 150 people responded affirmatively to the calendar invite
52 people participated
In addition to notes, we tried capturing video for those unable to attend
Christian Hamer (Project Sponsor) moderated the session and Dennis Ravenelle (PM) took notes

4. What were the goals?
Learn what we could from anyone with any kind of input about how we can make projects like this better
Be inclusive and involve anyone who was interested in either participating or just watching
Model that it’s OK (even good!) to make mistakes if we learn from them
Have a forum where it really was OK to say anything, as long as it was offered in a constructive spirit

5. How did it go? The reaction we’ve heard so far was positive
We got a lot of good feedback on the project and heard some perspectives we hadn’t heard or considered
To try to organize the conversation, we broke it into seven sections:
Planning
Internal communications
External communications
Rollout
Support
Project management/coordination
Coordination with the Schools
And tried to focus on three central questions:
What worked well?
What could we have done better?
What would we change if we had it to do all over again?

8. Ground rules Logistics: No one takes anything that is said personally
We explained that we were experimenting with video (so that people would know it was being recorded) and what that meant in terms of microphone and repeating questions
The space: because we planned for the 150 responses, the space was not ideal, but we reinforced the point that we wanted this to be interactive and collaborative
No one takes anything that is said personally
The project went very well, but we can always learn – so people who worked on the project should not take any feedback critically
NOTHING is off limits as long as it is offered in the spirit of learning

9. What did we learn about the project?
Did well
Could do better
Tool/technology is easy to use
Early testing and feedback, especially on the user interface
Strong support from University leadership and strong community acceptance
Good project coordination, information sharing and reporting, aided by Office 365
Good training for help desk and field support, with feedback from early adopters to inform FAQs
Focus on first-call resolution
Great documentation and website
Adapted rollout to meet audience needs
“Ear to the ground” from field support
Personal appearances
Better support for edge but important cases (international phone numbers, replaced phone)
Address UI/usability issues sooner
More clear communications (internally and externally) about what resources would be impacted
Provide less information in reports to Schools
Better internal communications when unforeseen issues cause schedule changes
Ensure that support teams have screenshots of all possible paths for all user types
Make sure key processes (list generation) are documented and not reliant on individuals
Create a central/shared glossary sooner
Consistent message from the start on hardware tokens
Tell the community “why” from the start
Earlier focus on how to use Duo while traveling

10. What would we do next time?
Different space:
We didn’t need a space that large and could have used Lamont, MWD or other room that could hold in a setup more suited to collaboration
While we didn’t want to turn people away, the space made it more challenging
Repeat the feedback:
We did this as a practical matter because we were recording and wanted as few mics as possible (one) – the feedback was that it really helped the audience feel like we understood what was being said
Don’t justify or respond to constructive criticism:
That was the most difficult thing for me to (not) do, but really important
Have some ”seed” feedback:
I had some notes of my own that I thought it was important to share (mainly feedback on things I would do differently) and I also had a nice list from someone who was unable to attend but sent feedback in advance
This seemed to help “prime the pump” for the audience and, because not all of it was mine, did so in an authentic way
Make it clear from the start that ANYTHING (constructive) is welcomed, and deliver on that by not justifying when feedback is offered

11. What would we do next time?
Video?
We did it, and we will publish it for those who couldn’t make it, but it meant that we had to use a mic (no speakers though), which made it seem “big” (or, perhaps more clearly, the opposite of “intimate”) and may or may not have limited feedback
Who moderates?
I thought it was really important for me to do this – but there may be a way to achieve the goals we set out using a moderator who was not at all involved with the project, but trained in conducting these sessions

12. Other thoughts/suggestions
Consider sending a survey in advance to help identify themes
Consider using clickers to help understand how strongly a theme is echoed/felt
Consider asking the same set of questions across projects so that we can do some evaluation across projects over time
THE REAL TEST IS GOING TO BE DOING THIS, AS GRACEFULLY AS WE CAN, FOR A PROJECT THAT FAILED!