H UMAN A CTION R ECOGNITION USING L OCAL S PATIO -T EMPORAL D ISCRIMINANT E MBEDDING Kui Jia and Dit-Yan Yeung, IEEE Conference on Computer Vision and Pattern Recognition Instructor: Jenn-Jier Lien Reporter: Mei-Hsuan Chao

O UTLINE Introduction Related work Local spatio-temporal discriminant embedding Experiments Conclusion 2

I NTRODUCTION Recognizing human activities in videos has many important computer vision applications. A human silhouette contains both instant spatial information about the body pose and dynamic temporal motion information of the global body and local body parts. Human silhouettes can be considered as data points on nonlinear dynamic shape manifolds. The aim in this paper is to find a manifold embedding method which can optimally make use of the discriminative temporal shape variation information between different types of actions. 3

R ELATED WORK LPP(Locality preserving projections) LPP constructs a nearest neighbor graph. By using the Laplacian of the graph, LPP can find a mapping which optimally preserves the local neighborhood information. LSDA(locality sensitive discriminant analysis) LSDA first constructs one nearest neighbor graph, and then splits it into the within-class graph and the between-class graph. 4

L OCAL SPATIO - TEMPORAL DISCRIMINANT EMBEDDING Neighbor graph G Short video segment Si 5

L OCAL SPATIO - TEMPORAL DISCRIMINANT EMBEDDING Objective functions 6

L OCAL SPATIO - TEMPORAL DISCRIMINANT EMBEDDING Principal angles between Si and Sj 7

L OCAL SPATIO - TEMPORAL DISCRIMINANT EMBEDDING Optimal embedding The columns of an optimal A can be obtained as the generalized eigenvectors corresponding to the l largest eigenvalues. 8

L OCAL SPATIO - TEMPORAL DISCRIMINANT EMBEDDING Iterative learning 9

E XPERIMENTS Data setting Design of two-stage recognition scheme 10 Frame by frame basis Short segment basis Test silhouette frame Recognition result

E XPERIMENTS 11

C ONCLUSION Propose a novel local spatio-temporal discriminant embedding (LSTDE) method. Perform recognition on a frame-by-frame or short video segment basis. Experimental results demonstrate that the proposed method can accurately recognize human actions, and outperforms some representative manifold embedding methods. 12