Modeling the Model Athlete Automatic Coaching of Rowing Technique Simon Fothergill, Fourth Year Ph.D. student, Digital Technology Group, Computer Laboratory, University of Cambridge DTG Monday Meeting, 10 th November 2008 Based on paper: Modelling the Model Athlete : Automatic Coaching of Rowing Technique; Simon Fothergill, Rob Harle, Sean Holden; S+SSPR08; Orlando, Florida, USA, December 2008

Supplementary Sports coaching Feedback is vital Rowing technique is complex, precise and easy to capture Good coaches aren’t enough Sensor signals need interpreting Biomechanical rules are complex and require specific sensors, if they exist at all

Pattern Recognition Statistical Arbitrary features that summerise the data in some way. E.g. RGB values, number of X Structural Consider constituent parts and how they are related. E.g. “contains”, “above”, “more red” Combination –Distance –Shape moments / smoothness

System overview Population of strokes Stroke quality classifier Good Bad Individual aspect of technique Motion capture system Lightweight markers Preprocessing of motion data Feature extraction Classification stroke

Motion capture Bat system Inertial sensors Optical motion capture –VICON –Nintendo Wii controllers

Preprocessing Compensate for occlusions Transform to the “erg co-ordinate system” defined by seat Segment performance into strokes using handle trajectory extremities

Feature extraction Art Modified various algorithms until found “a good one” for a set of strokes where each stroke is obviously different in over-all quality.

Abstract features Length Height Distance Shape moments ( λ 11, λ 12, λ 21, λ 02, λ 20 ) Speed moments : (μ 11, μ 12, μ 21, μ 02, μ 20 ) ψ(s)

Physical Performance features Wobble (lateral variance) Speed smoothness –μ-subtract, –LPF (3Hz) –dS/dt/dt, –∑ Shape smoothness –LPF (6Hz), –|dS/dt/dt|, –++ > threshold (0.4ms -2 )

Domain features Ratio (drive time : recovery time) Drive and recovery angles

System overview Population of strokes Stroke quality classifier Good Bad Individual aspect of technique Motion capture system Lightweight markers Preprocessing of motion data Feature extraction Classification stroke

Machine learning Normalisation and Negation –Each feature’s values are normalised to roughly between 0 and 1 –Highly negatively correlated features are negated –Good strokes are scored as 1 –Bad strokes are scored as 0

Machine learning Classification Feature 0 Feature N Weight 0 Weight N Linear combination Bias Bias weight Composite representation of motion Method 1: Moore-Penrose F w = s (F -1 = Moore-Penrose pseudo-inverse of feature matrix) Method 2: Gradient descent Error function: Sum of the square of the differences weights initialised to iterations learning rate

Machine learning Validation of models –Training repeated using populations formed by leaving out different sets of strokes –Unseen strokes are then classified –Each stroke left out exactly once –Multiple performers (each performer left out) Sensitivity analysis –Threshold computed to minimise misclassification –Features –Iterations

Empirical Validation Population –Six novice, male rowers in their mid-twenties –60kg and 90kg –Very little or no rowing experience. –Not initially fatigued, comfortable rate, uncontrived manner. Scoring –Single expert (coach) –Score whole performances (95% representative) –Bad = Expert considers a significant floor in technique –Good = Expert considers a noticeable improvement Experimental method –Basic explanation –Give performance (~30 strokes) –Repeat to fatigue Identify fault Teach correction Give performance (~30 strokes) whilst coach helps to maintain improved technique (for accumulating aspects)

Empirical Validation For an Individual and specific aspect –Training just that single aspects –Recognition of that single aspects with realistic combinations of different qualities for different aspects Rower Coached aspect (chronological order) Moore-Penrose trainingGradient Descent training Single aspectAll aspectsSingle aspectAll aspects 1 Separate arms/legs0000 Overreaching0001 2Separate arms/legs Overreaching Shins vertical0000 Early open back Leaning back0306 Quick hands0407 Rushing slide0206 Early open back Overreaching0000 Separate arms/legs0013 Quick hands0011

Empirical Validation Across Individuals RowersAspectMoore-Penrose trainingGradient Descent training 2Quick hands95 2Early open back3329 3Separate arms/legs21 4Overreaching12

Discussion and Conclusions Useful features λ 02, λ 20 μ 02 and μ 20 used in at least 90% of the final feature sets for both algorithms. Comparison of techniques –For single athletes, gradient descent not as fast –For multiple athletes, gradient descent more reliable Encouragingly low misclassification Suggets inter-variation from different athletes > athlete’s intra-variation

Further Work Characterisation of the process –Population –Domain –Algorithms Reversing the models to allow prediction of optimal individual aspects of technique that can be merged to an optimal technique for an individual

References Modelling the Model Athlete : Automatic Coaching of Rowing Technique; Simon Fothergill, Rob Harle, Sean Holden; S+SSPR08; Orlando, Florida, USA, December 2008 –Ilg, Mezger & Giese. Estimation of Skill Levels in Sports Based on Hierarchical Spatio-Temporal Correspondences. DAGM 2003, LNCS 2781, pp , –Murphy, Vignes, Yuh, Okamura. Automatic Motion Recognition and Skill Evaluation for Dynamic Tasks. EuroHaptics 2003, –Gordon. Automated Video Assessment of Human Performance. J. Greer (ed) Proceedings of AI-ED 95. pp , –Rosen, Solazzo, Hannaford & Sinanan. Objective Laparoscopic Skills Assessments of Surgical Residents Using Hidden Markov Models Based on Haptic Information and Tool/Tissue Interactions. The Ninth Conference on Medicine Meets Virtual Reality, Joint IAPR International Workshops on Structural and Syntactic Pattern Recognition and Statistical Techniques in Pattern Recognition (S+SSPR 2008) Orlando, Florida, USA, December 4-6, 2008 ( 19th International Conference of Pattern Recognition, ICPR 2008 ( Computer Laboratory, University of Cambridge (

Acknowledgements Professor Andy Hopper Dr Sean Holden Dr Rob Harle Dr Joseph Newman Brian Jones Dr Mbou Eyole-Monono The Digital Technology Group, Computer Laboratory The Rainbow Group, Computer Laboratory

Thank you! Questions?