1. February 19, 2010 How to Catch a Tiger: Understanding Putting Performance on the PGA TOUR Jason Acimovic MIT Operations Research Center, acimovic@mit.edu Douglas Fearing MIT Operations Research Center, dfearing@mit.edu Professor Stephen Graves MIT Sloan School of Management, sgraves@mit.edu

2. February 19, 2010 Agenda 2 Introduction – Project Question – Applications – Approach and contribution Golf and data overview Putting model Off-green model Situational analysis

3. February 19, 2010 Project Question How well do people perform on tasks? – Tasks differ from each other – Not everyone performs every task – Even the same task can be different from person to person 3

4. February 19, 2010 Applications Evaluating employees in a distribution center – Pickers in a warehouse vary in skill (picks per hour) – Pick zones vary in difficulty (books vs. electronics) – Difficulty also varies by hour of day and day of week – Pickers shift around, but not enough to ensure perfect mixing – How do you compensate the best employees and identify underperformers? Golf putting – Different golfers play different tournaments – Greens vary in their difficulty – Different golfers start on the green from different distances – How do we identify the best putters? 4

5. February 19, 2010 Project approach and contribution Develop statistical models to predict strokes-to-go Correct for player skill and course difficulty Evaluate incremental value of each shot taken relative to the expectation for the field – Compare predicted strokes-to-go before and after shot Aggregate shot value across players, shot types, etc. to better understand player performance Compare our model to current metrics, namely, Putting Average Paper: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1538300 (or email us) 5

6. February 19, 2010 Agenda 6 Introduction Golf and data overview – Strokes-to-go example – ShotLink data Putting model Off-green model Situational analysis

7. February 19, 2010 Quick golf primer The goal is to get from the tee to the pin in the fewest number of strokes 18 holes in a round of golf Typically 4 rounds in a tournament Lowest total score wins 7 Tee Green Fairway

8. February 19, 2010 Strokes-to-go example Shot LocationStrokes-To-Go 14.4 31.8 23.0 Shot Gain 0.4 0.8 0.2 8 4.4 – 3.0 – 1 = 0.4

9. February 19, 2010 ShotLink Data 9 Every tournament, 250 volunteers gather data on every shot – Lasers pinpoint the ball location to within an inch – Field volunteers gather qualitative characteristics Data is used for both real time reporting as well as detailed analyses 5 Million shot data points 2 Million putt data points

10. February 19, 2010 Visual explanation of ShotLink TM dataset Course Year Round Number Hole Number Tee Location Ball Location Pin Location Player Shot Number Location Type Ball Lie Hole Par Stimp Reading Green Length X Coordinate Y Coordinate Z Coordinate 16 th Hole on Colonial 10 X Coordinate Y Coordinate Z Coordinate

11. February 19, 2010 Data for the 14 th hole at Quail Hollow – 1 day 11

12. February 19, 2010 Agenda 12 Introduction Golf and data overview Putting model – Empirical data – Two stage model Holing out submodel Distance-to-go submodel – Markov chain – Correct for hole difficulty and player skill – Putts-gained per round and results Off-green model Situational analysis

13. February 19, 2010 Empirical mean and std. dev. of putts-to-go MeanStd. Dev. 13

14. February 19, 2010 Two-stage model to predict putts-to-go First stage sub-model – From anywhere on the green, the first model predicts the probability of sinking the putt 14 Probability of 0.1 of making it in on this putt

15. February 19, 2010 Second stage sub-model – If the golfer misses the putt, the second model calculates the distribution of the distance-to-go for the green If I miss, I have a 0.0021 probability of being in this blue area. (calculate this for entire green) Second stage finds conditional distance-to-go 15

16. February 19, 2010 We can calculate the putts-to-go distribution from anywhere on the green Combine and … 16 Consider only distance in our model

17. February 19, 2010 Empirical probabilities of holing out 17 Empirical probability of holing out vs. distance

18. February 19, 2010 Normal regression is inappropriate With Ordinary Least Squares regression, “one” might predict the probability of making a putt based on starting distance…. But… – We want to predict a probability with a range between 0 and 1 – Errors are not normal 18

19. February 19, 2010 One-putt logistic regression model Y – putts-to-go d – initial distance to the pin Fitted model parameters: Probability: 19

20. February 19, 2010 Model holing out as a logistic regression 20 Model probability of holing out vs. distance

21. February 19, 2010 2 nd -stage problem, determining distance-to-go What happens if we miss the first putt? 21 z

22. February 19, 2010 Empirical mean and std. dev. of distance-to-go MeanStd. Dev. 22

23. February 19, 2010 Empirical distributions of distance-to-go From 10 ft.From 30 ft. 23

24. February 19, 2010 Distance-to-go gamma regression model d – initial distance to the pin z – distance-to-go (assuming a miss) Fitted model parameters: Mean: Density: 24

25. February 19, 2010 Distance-to-go model: mean and std. dev. MeanStd. Dev. 25 October 19, 2015

26. February 19, 2010 Distance-to-go model distributions From 10 ft.From 30 ft. 26

27. February 19, 2010 Putts-to-go as Markov chain 27 distanceH p = 1 p = [ 1 + exp(…) ] -1 g (z|d) = (1 - [ 1 + exp(…) ] -1 ) x f(z|d) z Where g(z|d): probability density of ending up at z conditioned on starting at d f(z|d)probability density of ending up at z conditioned on missing and starting at d (from the distance-to-go gamma regression model) d Probability of holing out in n putts is probability of reaching absorbing state in n transitions

28. February 19, 2010 Making it within n putts (model prediction) Over 90% of golfers 2-putt or better within 35 ft. Only a 1.6% chance of 4-putting or worse at 100 ft. 28 Two-Stage Model Within N Putts

29. February 19, 2010 Two-stage model mean and std. dev. MeanStd. Dev. 29

30. February 19, 2010 Comparing putt quality Greens vary in difficulty – Fast vs. slow greens – Type and length of grass Good putts on a hard green should be valued more than the same on an easy green Adjust parameters for each hole to the logistic and gamma regression models 30

31. February 19, 2010 Revised logistic and gamma regressions Every player p and hole h have their own dummy variables and specific holing-out probabilities * – I p is the indicatory variable, and is equal to 1 if observation i contains player p and is zero otherwise. – Instead of a regression with 6 parameters, we now have thousands of parameters E.g., there is a β 0h parameter for every hole 31 *The actual analysis accounts for the number of observations per player and per hole, so that the model is more complex for players about whom we know more. The gamma regression is adjusted similarly

32. February 19, 2010 Visualizing player skill level differences 32 Comparison of above average (Brent Geiberger), below average (John Huston), and field average putter for an average green

33. February 19, 2010 Visualizing green difficulty differences Comparison of an easy green (Bay Hill #9), difficult green (Sawgrass #1), and average green based on a field average golfer 33

34. February 19, 2010 Calculating putts gained per round Calculate the gain associated with each putt – Relative to the putts-to-go for each specific hole – Example: Golfer starts at 12 ft. and takes 2 putts to sink ball Expected putts-to-go: 1.71 Actual number of putts: 2 Relative gain: (- 0.29) Sum the relative gains for each player Divide by the number of rounds played 34 12 feet 1.71 putts to go

35. February 19, 2010 Top 10 putts gained per round 35 RankGolfer Putts Gained / Round Number of Rounds Putts Gained / Round Stdev 1 Tiger Woods0.692300.12 2 David Frost0.671130.16 3 Fredrik Jacobson0.562480.11 4 Nathan Green0.551970.12 5 Aaron Baddeley0.533030.10 6 Jesper Parnevik0.503150.10 7 Stewart Cink0.493750.09 8 Darren Clarke0.451070.17 9 Ben Crane0.442730.11 10 Willie Wood0.42720.20

36. February 19, 2010 Putting average is the most popular metric today Putting Average – Average number of putts per green * When a golfer reaches a green – Count the putts it takes to get it in the hole – Average this among all his green appearances – Regardless of how close he starts on the green 36 * Actually, a green in regulation, which means the green was reached in no more than (par – 2) strokes

37. February 19, 2010 Comparing with putting average 37 Golfer Putts Gained / Round PG/R RankPutting Average PA Rank Tiger Woods0.69 1 1.711 David Frost0.67 2 1.7760 Fredrik Jacobson0.56 3 1.744 Nathan Green0.55 4 1.745 Aaron Baddeley0.53 5 1.743 Jesper Parnevik0.50 6 1.7647 Stewart Cink0.49 7 1.7512 Darren Clarke0.45 8 1.7519 Ben Crane0.44 9 1.7517 Willie Wood0.42 10 1.7792

38. February 19, 2010 Understanding the discrepancies Insert first-putt distance histograms for most severe outlier. 38 PG/R PercentileGolfer Putts Gained / RoundPutting Average PA Percentile 9 th Stephen Leaney 0.261.79 59 th 88 th Ernie Els -0.631.75 5 th 54% for All Players 51% for Stephen Leaney 60% for Ernie Els Percentage of 1 st putts 20 ft. or closer On average he starts closer to the hole, so his putting average is inflated by his excellent approach shots

39. February 19, 2010 Agenda 39 Introduction Golf and data overview Putting model Off-green model Situational analysis

40. February 19, 2010 Evaluating off-green performance For each hole, calculate “field par” – Empirical average number of strokes corrected for player skill and hole difficulty Calculate total strokes gained per round for each player Calculate off-green strokes gained per round 40 (Off-green strokes gained = Total strokes gained – putts gained)

41. February 19, 2010 Top 10 golfers (on and off green performance) 41 RankGolfer Putts Gained / Round Off-Green Gain / RoundTotal 1 Tiger Woods0.692.533.22 2 Vijay Singh-0.362.652.29 3 Jim Furyk0.002.03 4 Phil Mickelson0.191.741.94 5 Ernie Els-0.632.481.85 6 Adam Scott0.081.691.77 7 Sergio Garcia-0.672.201.52 8 David Toms0.161.271.43 9 Retief Goosen-0.441.841.40 10 Stewart Cink0.490.891.39

42. February 19, 2010 Agenda 42 Introduction Golf and data overview Putting model Off-green model Situational analysis – Player specific putts – Fourth round pressure – Tiger woods’ fourth round performance

43. February 19, 2010 Situational putting performance Above, we used the general putting model to evaluate putting relative to the field of professionals We also have the capability to evaluate a golfer’s putting relative to his own expected performance For instance, even if Tiger Woods usually putts better than the field, we can also determine when he putts worse than himself – Does he putt better or worse after the cut? – Does he putt better or worse for birdie vs. for par? 43

44. February 19, 2010 Player-specific putts gained – example On the 10 th green at Quail Hollow, 9 feet from the pin: – Tiger Woods’ personal expected putts-to-go is 1.54 – Vijay Singh’s personal expected putt-to-go is 1.59 – If they each sink it, Tiger gains only 0.54 strokes whereas Vijay gains 0.59 strokes 44 Tiger: E[putts] = 1.54 Vijay: E[putts] = 1.59 9ft

45. February 19, 2010 Advantages of player-specific putts gained Easy to test various hypotheses – After calculating the shot value for every putt, we need only to filter and aggregate the results Describes the magnitude in terms of score impact Suggests areas for further investigation – Standard deviation of putts gained provides the relative significance of the effect 45

46. February 19, 2010 Fourth round pressure 46 Putting does not seem to be affected by the pressures of being in the fourth round

47. February 19, 2010 Tiger Woods’ fourth round performance A common perception is that Tiger has the ability to kick it up a notch during the final round Looking at his putts-gained suggests otherwise 47

48. February 19, 2010 Conclusion Developed a model for putting – Corrected for player skill and hole difficulty – Intuitive model that describes how putts occur Demonstrated the differences between our metric and current putting statistics Developed a “field par” which corrects for hole difficulty and quality of field Compared on- and off-green performance Examined situational putting performance 48