School of Computing Science Simon Fraser University Vancouver, Canada What is the value of an action in ice hockey? Deep Learning for The National Hockey League Oliver Schulte Guiliang Liu If you use “insert slide number” under “Footer”, that text box only displays the slide number, not the total number of slides. So I use a new textbox for the slide number in the master. This is a version of “Equity”.

Overview Evaluate NHL players by the total aggregate value of their actions (decisions) Evaluate values of actions by how they impact a teams’ chance of scoring the next goal Use a neural net to estimate the chance that a team scores the next goal

Evaluating Actions by Expected Future Success

Expected vs. Actual Success Important insight: players should create chances to score but actual scoring success depends on other factors beyond their control (e.g. goalie, teammates, luck) for shots, look at the expected goals, not the actual goals So far has been done for the immediate effects of actions (e.g. chance of a goal occuring within 30 sec, THOR Shuckers and Curro 2013) What about actions with indirect and delayed effects (e.g. drawing penalties, strong passes)? Consider the chance of success in the future, down the line.

Examples Basketball Expected Point Value (Cervone, Bornn et al. 2014): The Spurs have the ball with 15 sec left on the shot clock. What is the chance that they will score 3 points? In-game win probabilities (Pettigrew 2015) big impact smaller impact

Our Version: Scoring The Next Goal Our model estimates the probability that a team scores the next goal (within any amount of time) big impact but no goal Q = chance of scoring the next goal Blue Jackets vs. Penguins Nov 17, 2015 - more fine-grained information than in-game win probabilities shouldn’t this be “blue jacket advantages”? And Penguins passed around the goal? the three numbers add up

Take-Aways If you can estimate the chance of future success (winning, scoring), you can assign values to actions Value of an action = impact on expected success The impact can be measured for all actions, offensive and defensive Action Type Impact Offensive (e.g. pass) increases chance that my team scores the next goal Defensive (e.g. block) decreases chance that the other team scores the next goal 

Estimating Expected Future Success Deep Learning for the NHL

Function Estimation Q = 0.2 Think of a function that maps a game state to the chance of scoring/winning This is called the value function or the Q-function in machine learning What is the mathematical form of this function (linear, polynomial,...)? Beats me!

Neural Networks To The Rescue No reason to think that the Q-function for hockey has a nice mathematical form Use a universal function approximator (non-parametric statistics) Neural networks are a well-developed type of universal function approximator

Neural Network Training Applies reinforcement learning Reinforcement learning builds models for estimating expected future success from actions Google Deepmind uses RL to build artificial intelligence SportLogiq dataset with >3M NHL events 2015-2016 We’ve also used nhl.com play-by-play data

From Action Values to Player Evaluation

Total Goal Impact Metric (GIM) Traditional Approach: Add up the impact values for each player to get a total score We are looking at every action can attribute to specific player rather than all players on the ice

Player Ranking Top-20 highest impact players Identify undervalued players Johnny Gaudreau and Mark Scheifele Later they received a $5M+ contract for the 2016-17 season.

Impact Metric Correlates Well With Standard Stats Looked at 14 standard stats, also auto-correlation Table shows correlation with season points only Performance Metric Correlation With Points +/- 0.237 Goals Above Replacement (GAR) 0.622 Wins Above Replacement (WAR) 0.612 Expected Goals 0.854 Scoring Impact 0.87 Goal Impact 0.93

Goal Impact Predicts Salary For worth of new contracts following the 2015-2016 season Correlation increases for contracts 2 seasons away

Visualization In the 2016-17 season (left), we find many underestimated players (with high GIM but low salary). But the percentage of players who are undervalued decreases in the next season. This suggests that the Goal Impact Metric provides an early signal of a player's value.

Conclusions If you can estimate the chance of future success (winning, scoring), you can assign values to all actions Value of an action = impact on expected success Estimating chance of future success can be done with neural nets (deep reinforcement learning) Total action value = goal impact metric for each player Powerful new performance metric: Correlates well with points, goals, games played, etc. predicts worth of new contracts