Trajectory-Based Ball Detection and Tracking with Aid of Homography in Broadcast Tennis Video Xinguo Yu, Nianjuan Jiang, Ee Luang Ang Present by komod Visual Communications and Image Processing 2007 Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6508

Introduction The ball is the most important object in tennis (and in many kind of sports) Very challenging problem –Camera motion –presence of many ball-like objects –small size and the high speed of the ball –Object-indistinguishable

Introduction Method –Trajectory-based the ball is the “most active” object in tennis video previous work ： A Trajectory-based ball detection and tracking algorithm in broadcast tennis video, Proc. of ICIP –Homography Goal –find projection locations of the ball on the ground –find landing positions

Introduction

Feature Point Extraction Court Segmentation –Find the court color range and paint all the pixels in this range with a single color –find the lines separating the audience from the playing field detecting the change pattern of color for each row and column of the image –paint the audience area in the court color.

Feature Point Extraction Straight Line Detection –gridding Hough transform Court Fitting –Detect the net and use it as reference –find the intersection of lines

Homography Acquisition Standard Frame –whose lookat is the cluster center of all lookats of all the frames in the considered clip –The lookat of frame is a point in the real world that corresponds to the center of the frame

Homography Acquisition Disparity Measure of Two Court Images –For i = 1 to 9 Measure Function –Let C std be the court in the standard frame and C trn denote the transformed court from the segmented court in frame F –For given H and F

Homography Acquisition Initial Matrix –transforms an image point X' (x 1 ', y 2 ', 1) to a point X (x 1, y 2, 1) in another image –X = HX‘ Tuning of Homography –The homograph matrix computed based on feature points –A small hough space enclosing it

Homography Acquisition Tuning procedure Frame transform

Ball Location In Hitting Frame Hitting frame detection –Find the sound emitted by the racket hitting M. Xu et al, Creating audio keywords for event detection in soccer video, In Proc. of ICME Hitting racket detection –Maybe player tracking

Ball Candidate Detection Object segmentation from standard frame Four sieve are used for non-ball object removal –Court Sieve Θ 1 filter out audience area filter out court lines –Ball Size Sieve Θ 2 filter out the objects out of the ball-size range homography from ground model to standard frame use a range of allowable ball sizes (estimate error) –Ball Color Sieve Θ 3 filter out the objects with too few ball color pixels –Shape Sieve Θ 4 filter out objects out of the range of width-to-height ratio 2.5 is suggested in previous paper

–Each sieve is a Boolean function on domain Ο(F) –The set of remaining objects is C(F) C(F) = {o : o ∈ O(F), Θ i (o)=1 for i = 1 to 4} Candidate Classification –Three features are use Size, color, and distance from other objects –The ball-candidates are classified into 3 Categories Ball Candidate Detection

Candidate Trajectory Generation No detail explanation in this paper –X. Yu et al, Trajectory-based ball detection and tracking of broadcast soccer video, IEEE Transactions on Multimedia, issue 6, Candidate Feature Plots (CFPs) –CFP-y –CFP-l

The algorithm is actually works on the CFP-l which are 3-D plots

Candidate Trajectory Generation

Trajectory Processing Trajectory Confidence Index –Let T be a candidate trajectory –and λ 1,λ 2,…,λ m, be all properties of trajectory T –confidence index Ω(T)

Trajectory Processing Trajectory Discrimination

Trajectory Processing Ball Projection Location – y = an 3 + bn 2 + cn + d. Ball Land Detection –form a ball position function against frame number i, y = f(i) –find the maximum of f '(i) between each pair of hittings

Experimental Results 5 clips extracted from mpeg2 704x576 average time for acquiring ball candidates ALG new for a frame is 86.15s on a P4/1.7Ghz PC with 512MB RAM ALG old is 19.21s

Experimental Results BPL

Experimental Results

average discrepancy of all detected balls from the groundtruth previous result

Experimental Results frames with inserted 3D projected virtual content Homography in home surveillance video

Conclusion and Future Works The previous algorithm mainly alleviated the challenges raised by causes besides camera motion The algorithm presented in this paper additionally counteracts the challenges brought to us by the camera motion The contributions of this paper are two-fold –it develops a procedure to robustly acquire an accurate homograph matrix of each frame –it forms an improved version of ball detection and tracking algorithm Two future works –evolve the algorithm into an end-to-end system for ball detection and tracking of broadcast tennis video –analyze the tactics of players and winning-patterns, and hence produce rich indexing of broadcast tennis video by making use of the ball position

Any Question?

Thank You

Experimental Results 7 segments, total 120 s, mpeg1 video, Men’s Final of FRENCH OPEN 2003 back