1. Vanquishing the Measurement Dragon
McKinsey Learning | A better you, a better Firm October 2017
ABSTRACT
This document was developed to inform clients about best practices when building capability development vehicles with special attention on developing capability in high-consequence environments. First, the authors discuss design success factors for main questions to be addressed when designing and executing a corporate academy: objectives and purpose (clear role and strategy), governance and structure (business-driven approach), learning approaches (high-quality delivery), faculty (credible faculty), and measurement and impact (measuring learning impact and marketing the institute). They then present qualities that leading companies exhibit and main insights on culture building and development approaches. Finally, the authors describe common themes that are observed from capability development in high-consequence situations. Key slides include 1 (document summary), 2-3 (main questions: overview), 37 (insights), and 46 (themes: overview).
CONFIDENTIAL AND PROPRIETARY
Any use of this material without specific permission of McKinsey & Company is strictly prohibited

2. A vision for impact measurement and broader learning analytics
Harness distinctive Learning Analytics to shape our L&D agenda; help ensure we are delivering against the Firm’s dual mission; and maximize the impact we have on our colleagues
Aspiration
3 Themes
Leading edge impact measurement
Integration with People Analytics
Engine and fuel source for personalized learning
“EEI/C2” framework
‘Journey View’ that looks across individual programs/courses
“Smart assessment” strategy
Linking of learning behaviors with “People Mission” outcomes
Learning data as a not only an explainer but also a predictor
“Machine learning” recs that go beyond simple “course + demographic” algorithms
Adaptive “branching” in select courses and programs
Enablers & capabilities
People & processes
Robust communication and change management strategy
Integration into core processes (needs analysis, design, performance dialogues)
Dedicated internal capability
External networks, knowledge
Technology
Efficient, easy-to-use survey and assessment platforms
Modernized, powerful reporting and visualization (e.g., Tableau)
Integrated “People” data
Tools to capture micro, event-driven behaviors and data
SOURCE: Learning Analytics & Measurement team

3. “EEI/C2” as a governing framework for measurement
Do colleagues “own” their journey?
Do they utilize what’s available?
Do they attend on-time?
Are our offerings satisfying and enjoyable?
Are they high-quality (faculty, interfaces, content)?
Are we building relevant capabilities?
Are behaviors changing?
Are outcomes improving?
Engagement x
(Experience +
Impact) x
Coverage
Cost
Do we provide sufficient topic coverage for each role & journey?
Is the coverage accessible and timely?
How efficient are we from a cash cost perspective?
How efficient are we from an opportunity cost perspective?
SOURCE: Learning Impact & Measurement Team

4. Digital Learning measurement aspiration
Digital Learning will:
Be seamless
and embedded
“…at the time and place they need it…”
# of assets by archetype / sub-archetype
Average age of assets and/or churn rate
Breakdown of usage by channel
User experience rating
Engage
our learners
“…in a way that our learners find useful and engaging…”
Penetration
Usage:
Completion rate
# of logins/launches
Time spent in learning
Quality (star-rating, or other)
Create
impact
“…and in a way that drives impact for our learners and for the Firm.”
Learner effectiveness
Learner efficiency
Provide
access
“Digital learning gives learners access to the capabilities they need…”
# of training modules / assets developed
# of external content assets imported
Coverage against capabilities (taxonomy that includes capabilities for all roles and profiles)
Coverage against target audience
“At a reasonable cost” – the costs of digital learning need to be linked to these metrics

6. Breakouts for Deeper Discussion
Framework
Digital
Learning Journey
SOURCE: Learning Impact and Measurement Team