Behavior Classification by Eigen-decomposition of Periodic Motions Michael Rudzsky Joint work with Roman Goldenberg, Ron Kimmel, Ehud Rivlin Computer Science Department Technion-Israel Institute of Technology Geometric Image Processing Lab

Dynamism of a Dog on a Leash Giacomo Balla, 1912

The Red Horseman Carlo Carra, 1914

Muybridge Horse Eadweard Muybridge, Animals in Motion, 1887

Horse - decomposition

Segmentation and Tracking qActive Contour qFast Geodesic Active Contours uAOS uLevel Sets uFast Marching Goldenberg, Kimmel, Rivlin, Rudzsky, IEEE T-IP 2001

Tracking in color movies Goldenberg, Kimmel, Rivlin, Rudzsky, IEEE T-IP 2001

Background qBackground subtraction Chan, Vese, Active Contours without Edges, IEEE T-IP 2001 Paragios, Deriche, Geodesic Active Regions for Motion Estimation and Tracking, ICCV-99

Tracking

Information extraction

Walking man - periodicity

Walking cat -periodicity

Periodicity Analysis tt+3t+6t+9  +3  +6  +9

Inter-frame correlation

Spatial Alignment 50x50

Temporal Scaling Original period - 11 frames Resampled period - 10 frames

Temporal Alignment

Parameterization qn - number of frames in the training set q50x50 - normalized images qM 2500 x n - training samples matrix  MM T = U  V T, the principle basis {U i, i=1..k}

Distinguishing by static appearance qImage I written as a vector v I qParameterized representation in basis B, p = B T v I qDTFS ||p - v I ||

Back-projection Original 11 frame one period subsequence Projection to the `dogs & cats’ basis and the DTFS values

Recognizing motions q{I f, f=1..T} - one period, temporally aligned set of normalized object images qp f, f=1..T - projection of the image I f onto the principal basis B of size k qOne-period subsequence representation Vector P of size kT - (p f, f=1..T) qIf k = 20 and normalized duration of one-period is T=10, then P is of size 200.

Classification -dogs & cats walkrungallopcat...

Classification -dogs & cats

Classification -people walkrunrun45

Classification -people

Learning curves Dogs and catsPeople

Parameterized modeling Ju, Black, Yacoob, Cardboard People, ICFG-96 Black, Yacoob, Parameterized Modeling and Recognition of Activities, ICCV-98

Optical Flow Polaba, Nelson, Recognizing Activities, ICPR-94

Motion History Images (MHI) James W. Davis, OSU Motion Recognition Lab

Star Skeleton Distance from the center of mass After low pass filter

Phase portrays Shavit, Jepson, Motion Understanding from Qualitative Visual Dynamics, 1993

Summary qSegmentation uactive contours qPeriodicity analysis uglobal contour characteristics qAlignment uspatial utemporal qParameterization uPrincipal basis projection qClassification uNearest neighbors