1. Player Action Recognition in Broadcast Tennis Video with Applications to Semantic Analysis of Sport Game Guangyu Zhu, Changsheng Xu Qingming Huang, Wen Gao Liyuan Xing

2. Outline Introduction Framework Overview Player Action Recognition Video Analysis Experimental Results

3. Introduction Semantic gap –between user semantics and low-level feature –Object in sports video can consider as an effective mid-level representation Action Recognition –Far-view –Foreside-swing backside-Swing

4. Introduction Multimodal Framework –Action recognition method based on motion analysis –High-level analysis Video Indexing Highlight ranking Tactic analysis

5. Framework Overview Sports video database Low-level analysis Middle-level analysis Fusion scheme High-level analysis

6. Framework Overview

8. Low-level Analysis Dominant color-based algorithm in [16] was used to identify all the in-play shots

9. Player Action Recognition Related Work –Shah[8], Gavrila[9] recognition with close-up views –Motion representation Motion history/energy image [12] Spatial arrangement of moving points [13] Several Constraints –Efroes[11] Motion descriptor in a spatio-temporal volume NNC similarity measure –Miyamori[14][15] Base on silhouette transition Appearance feature is not preserved across videos

10. Player Action Recognition

11. Player Tracking and Stabilization Player Tracking –Initial position: detection algo. in [16] –SVR particle filter [24] Player region centroid

12. Optical Flow Computation Background subtraction

13. Optical Flow Computation Noise elimination

14. Local Motion Representation S-OFHs –slice based optical flow histogram The prob. of bin(u) The prob. of bin(u) in slice

15. Local Motion Representation Two slice of the figure is used Horizontal and vertical optical field is used

16. Action Classification Using SVM The concatenation of four S-OFHs is fed as feature vector Audio keywords –Silence, hitting ball, applause

17. Action Classification Action clip window is set to 25 frames Voting Strategy

18. Video Analysis Fusion of mid-level features Action Based Tennis Video Indexing Highlights Ranking and Browsing Tactics Analysis and Statistics

19. Video Indexing Based on action recognition and domain knowledge

20. Highlights Ranking Player action recognition Real-world trajectory computation

21. Highlights Ranking Affective Features(4 for this paper) Features on action –Swing Switching Rate

22. Highlights Ranking Features on trajectory –Speed of Player (SOP) –Maximum Covered Court The rectangle shaped with left most, rightmost, topmost, and bottommost points –Direction Switching Rate

23. Highlights Ranking The feature vector comprised of four affective features is fed into the ranking model Support vector regression User defined threshold

24. Tactics Analysis and Statictics

25. Experimental Results Action Recognition (6 seq, 194 clips)

27. Experimental Results Video Indexing

28. Experimental Results Highlights ranking

29. Experimental Results

32. Future Work More effective slice partition Involve more semantic action –Ex. Overhead-swing Action recognition apply to more applications such as 3-D scene reconstruction Include the ranking accuracy by combining audio features

33. Thank You