1. Rank ordering the "intangibles”
Stefan Wolejszo
Vancouver Hockey Analytics Conference
10 March 2017

2. “Intangibles” matter to teams

3. Teams usually “eyeball” them

4. Measuring “intangibles”
Measuring and collecting empirical data typically outperforms the eye test
Social scientists have measured “intangibles” for a long time
At some point a team may allow someone access to collect (limited) data
Question: If I was asked which variables to measure to get the most bang for the buck which would I pick?

5. The 5 criteria Five criteria will be used
1) Conceptual clarity: definitions are clear and consistent
2) Empirical research: evidence outside of hockey
3) Elite occupations: research is with elite occupation groups
4) Practicality: possible to collect meaningful data
5) Applicability: teams can use results to improve
Each will be coded as follows:
Strong evidence  (green) points
Some issues  (yellow) point
Problematic (red) 0 points

6. Psychosocial dynamics
There are many psychosocial factors
Focus on five of them:
a) Cognitive Fatigue
b) Resilience
c) Leadership
d) Motivation
e) Grit

7. Cognitive Fatigue
Cognitive fatigue (AKA mental fatigue) refers to being mentally taxed from concentrating on a task for a long period of time
Errors, taking poor risks, loss of concentration, impaired memory, less willing to exert self to reach rewards (intrinsic motivation)
Has been empirically researched since 1880s (students)
Doctors (surgeons), pilots, astronauts

8. Cognitive fatigue trajectories in WWII pilots (Davis 1946)

9. Cognitive Fatigue (9/10)
Surveys calibrated with physical testing (“time on task” and “cognitive load”)
Change in player and team routines to reduce errors
Identifying warning signs of cognitive fatigue
Improve performance while reducing injuries
Conceptual clarity 
Empirical Research
Elite Occupations
Practicality 
Applicability

10. Resilience
Resilience refers to having good outcomes in the face of adversity
Unclear whether it is a trait, a process, or both
Potential Traumatic Events (PTEs) can be chronic or isolated
Researched since 1970s (schizophrenia)
Mainly children and people with disabilities, but also doctors, military, high performance athletes

11. Prototypical outcome trajectories (Bonanno and Diminich 2013)

12. Resilience (8/10) Survey and interview methods
“Tough guy” not conducive to data collection (frame as “what pisses you off”)
Identify players who need help with building resilience (sports psychologist)
Conceptual clarity 
Empirical Research 
Elite Occupations
Practicality
Applicability

13. Leadership
Generally thought of as the ability to motivate people to willingly do the things you want them to do (dozens of types of leadership)
Not always conceptually distinct from “influence”
Is it a trait or a process? (i.e. “great man” versus multi-level network approaches)
Thousands of empirical articles of varying quality
Research in business has mixed results, but shown to be important in some organizations (e.g. military)

14. Leadership and process (Rupprecht et al, 2013)

15. Leadership (6/10) Surveys and interviews
Process of developing leadership is long, players with teams for a short time
Very limited ability to meaningfully act on results
Better used for evaluating rather than directing process
Conceptual clarity 
Empirical Research 
Elite Occupations
Practicality
Applicability

16. Motivation Initiates and guides goal-orientated behaviors
Divided into “intrinsic motivation” (internal) and “extrinsic motivation” (rewards and punishments)
Studies of motivation (willpower) have been around for a lot longer, but resurgence in the 1970s
Commonly in education and individual sports (tennis, fencing, skiing)
There is evidence of a “motivation funnel” where as you move through higher levels motivation goes up while SDs drop

17. Motivation clusters in professional tennis players (Gilbert et al 2013)

18. Motivation (6/10)
Survey instrument already exists (Sports Motivation Scale, Luc Pelletier at U Ottawa)
Can provide a very limited amount of new information about a player
Conceptual clarity 
Empirical Research
Elite Occupations 
Practicality
Applicability -

19. Grit
Recent concept defined in different ways, but common theme include perseverance, character strength, and passion
“Perseverance” is similar to resilience and/or motivation in many respects
Research most commonly in education
Minimal amount in elite groups: West Point and Beast Barracks, high level spelling bees

20. Grit scale –short form (Duckworth 2004)

21. Grit (5/10)
Duckworth’s grit scale would need to be re-worked in hockey context
Small potential to produce new information
Best use is to measure trade off between grit and skill-based measures
Conceptual clarity 
Empirical Research
Elite Occupations
Practicality
Applicability

22. Scorecard Psychosocial Factor Score Cognitive Fatigue 9/10 Resilience
8/10
Leadership
6/10
Motivation
Grit
5/10

23. Conclusion “Intangibles” can be measured
Key shift from “do intangibles matters” to “how much does a given intangible matter”
May have limited opportunity to do this type of research with professional hockey teams
Need to make our best bet about which metrics will matter most and help to improve the team
Cognitive fatigue and resilience are the best choices
Others need work before they can be of good use