Rank ordering the "intangibles” Stefan Wolejszo Vancouver Hockey Analytics Conference 10 March 2017

“Intangibles” matter to teams

Teams usually “eyeball” them

Measuring “intangibles” Measuring and collecting empirical data typically outperforms the eye test Social scientists have measured “intangibles” for a long time At some point a team may allow someone access to collect (limited) data Question: If I was asked which variables to measure to get the most bang for the buck which would I pick?

The 5 criteria Five criteria will be used 1) Conceptual clarity: definitions are clear and consistent 2) Empirical research: evidence outside of hockey 3) Elite occupations: research is with elite occupation groups 4) Practicality: possible to collect meaningful data 5) Applicability: teams can use results to improve Each will be coded as follows: Strong evidence  (green) 2 points Some issues  (yellow) 1 point Problematic - (red) 0 points

Psychosocial dynamics There are many psychosocial factors Focus on five of them: a) Cognitive Fatigue b) Resilience c) Leadership d) Motivation e) Grit

Cognitive Fatigue Cognitive fatigue (AKA mental fatigue) refers to being mentally taxed from concentrating on a task for a long period of time Errors, taking poor risks, loss of concentration, impaired memory, less willing to exert self to reach rewards (intrinsic motivation) Has been empirically researched since 1880s (students) Doctors (surgeons), pilots, astronauts

Cognitive fatigue trajectories in WWII pilots (Davis 1946)

Cognitive Fatigue (9/10) Surveys calibrated with physical testing (“time on task” and “cognitive load”) Change in player and team routines to reduce errors Identifying warning signs of cognitive fatigue Improve performance while reducing injuries Conceptual clarity  Empirical Research Elite Occupations Practicality  Applicability

Resilience Resilience refers to having good outcomes in the face of adversity Unclear whether it is a trait, a process, or both Potential Traumatic Events (PTEs) can be chronic or isolated Researched since 1970s (schizophrenia) Mainly children and people with disabilities, but also doctors, military, high performance athletes

Prototypical outcome trajectories (Bonanno and Diminich 2013)

Resilience (8/10) Survey and interview methods “Tough guy” not conducive to data collection (frame as “what pisses you off”) Identify players who need help with building resilience (sports psychologist) Conceptual clarity  Empirical Research  Elite Occupations Practicality Applicability

Leadership Generally thought of as the ability to motivate people to willingly do the things you want them to do (dozens of types of leadership) Not always conceptually distinct from “influence” Is it a trait or a process? (i.e. “great man” versus multi-level network approaches) Thousands of empirical articles of varying quality Research in business has mixed results, but shown to be important in some organizations (e.g. military)

Leadership and process (Rupprecht et al, 2013)

Leadership (6/10) Surveys and interviews Process of developing leadership is long, players with teams for a short time Very limited ability to meaningfully act on results Better used for evaluating rather than directing process Conceptual clarity  Empirical Research  Elite Occupations Practicality Applicability

Motivation Initiates and guides goal-orientated behaviors Divided into “intrinsic motivation” (internal) and “extrinsic motivation” (rewards and punishments) Studies of motivation (willpower) have been around for a lot longer, but resurgence in the 1970s Commonly in education and individual sports (tennis, fencing, skiing) There is evidence of a “motivation funnel” where as you move through higher levels motivation goes up while SDs drop

Motivation clusters in professional tennis players (Gilbert et al 2013)

Motivation (6/10) Survey instrument already exists (Sports Motivation Scale, Luc Pelletier at U Ottawa) Can provide a very limited amount of new information about a player Conceptual clarity  Empirical Research Elite Occupations  Practicality Applicability -

Grit Recent concept defined in different ways, but common theme include perseverance, character strength, and passion “Perseverance” is similar to resilience and/or motivation in many respects Research most commonly in education Minimal amount in elite groups: West Point and Beast Barracks, high level spelling bees

Grit scale –short form (Duckworth 2004)

Grit (5/10) Duckworth’s grit scale would need to be re-worked in hockey context Small potential to produce new information Best use is to measure trade off between grit and skill-based measures Conceptual clarity  Empirical Research Elite Occupations Practicality Applicability

Scorecard Psychosocial Factor Score Cognitive Fatigue 9/10 Resilience 8/10 Leadership 6/10 Motivation Grit 5/10

Conclusion “Intangibles” can be measured Key shift from “do intangibles matters” to “how much does a given intangible matter” May have limited opportunity to do this type of research with professional hockey teams Need to make our best bet about which metrics will matter most and help to improve the team Cognitive fatigue and resilience are the best choices Others need work before they can be of good use