Semantic Scenes Detection and Classification in Sports Videos Soo-Chang Pei ( 貝蘇章 ) and Fan Chen ( 陳 凡 ) Conference on Computer Vision, Graphics and Image Processing (CVGIP 2003) 指導老師：吳宗憲 教授 組員 p 王志宏 p 趙郁婷 p 蔡佩珊 1

Outline × Introduction × Method ■ Scene change detection ■ Semantic scenes detection in baseball game videos × Experimental Result × Discussion 2

Introduction × Motivation 3

Introduction (Cont.) 4 Infield Outfield Player Pitching ？ scene

Introduction (Cont.) × Index and retrieval the baseball video × Semantic scene detection methods ■ Combining with domain-specific knowledge ■ Index keyframes by low-level features 5 Skeleton outline of our semantic scenes detection

Introduction (Cont.) × Video content analysis methods may be classified into the following three categories: ■ Syntactic structurization of video ■ Video classification ■ Extraction of semantics × Don’t use video object tracking in object level verification in order to reduce the complexity × Use object-location to verify which view type the keyframe belongs to 6

Method × Scene change detection × Semantic scenes detection in baseball game videos 7

Scene change detection × Execute the action of scene change detection and key frame extraction ■ IBM VideoAnnEx Annotation Tool annotate video sequences with MPEG-7 metadata 8

Semantic scenes detection × Analysis and structuring in baseball ■ A play usually starts with a pitching scene ■ After the play starts, if after a scene change the camera is shooting the field, then the current play should continue; otherwise, the current play ends when switched to a player scene 9 The model of baseball broadcasting videos the play continuous the play is end

Semantic scenes detection (Cont.) × Block diagram 10

Semantic scenes detection (Cont.) × Field color percentage ■ Detected field color distribution and percentage ■ Three situations Medium(20%~45%) ： pitching scene Large (>=45%) ： outfield scene or an infield scene Small (<20%) ： close-up scene or others 11 grass color range ： soil color range ： grass color range ： soil color range ： 0 100

Semantic scenes detection (Cont.) × Block diagram --- Field color percentage 12 20~45% >45% <20%

Semantic scenes detection (Cont.) × Pitching scene verification ■ First build a binary image by assigning field color to 1-pixel and non-field color to 0-pixel 13 (a) original pitching scene (b) the result of binarizing(c) histogram of horizontal projection T = 0.15 Horizontal projection histogram

Semantic scenes detection (Cont.) × Pitching scene verification 14 (a) original pitching scene (b) the result of binarizing(d) histogram of vertical projection Vertical projection histogram m = 15 M = 100

× Block diagram --- Pitching scene verification 15 Semantic scenes detection (Cont.)

(Cont.) Semantic scenes detection (Cont.) × Close-up scene detection ■ A close-up scene always target on one’s face Face detection is a key point ■ Skin color can be segmented out of an image Hues are between 0 and 50 degrees and saturation between 0.23 and

Semantic scenes detection (Cont.) × Close-up scene detection ■ Two step Step 1 ： Label skin color to 1-pixel, or label to 0-pixel Step 2 ： Find the largest region in our defined “red-block” 17 depicts the skin color distribution depicts the region considered as face region

Semantic scenes detection (Cont.) × Block diagram --- Close-up scene detection 18

(Cont.) Semantic scenes detection (Cont.) × Player scene detection ■ Player scene ： Lead role is a figure but background is composed of lots of field components 19 Can be taken as a scene between infield or outfield and close-up scene

(Cont.) Semantic scenes detection (Cont.) × Player scene detection ■ Field color percentage is large and there are some big concaves in the vertical projection diagram 20 K

Semantic scenes detection (Cont.) × Block diagram --- Player scene detection 21

(Cont.) Semantic scenes detection (Cont.) × Infield and outfield scene detection ■ Calculate the ratio of grass to soil Infield sceneOutfield scene 22 >5>5 ≦ 5

Experimental Result × Data : MPEG-1 × Frame size : 360*240 × Frame rate : 30Hz 23

Experimental Result (Cont.) × Result 24 Original Binarizing Infield_scene

Experimental Result (Cont.) × Result 25 Original Binarizing Outfield_scene

Discussion × pitching 畫面會因為土和草的比率而偵測有誤 × 當投手站於右方時，可能造成錯誤 26

27