1. Latent Boosting for Action Recognition Zhi Feng Huang et al. BMVC 2011 2014. 6. 12. Jeany Son

2. Background – learning with latent variables Multiple Instance Learning (MI-SVM, mi-SVM) (-) Single plain latent variable Latent SVM (+) Structured latent variable (-) Control parameters / Normalize different features MILboost (+) Not require to normalize different features (-) Single latent variable / Not structured hCRF Learning parameters and weights for features  Latent Boosting : structured latent variable, not require to normalize different features, feature selection

3. Boosting Combining many weak predictors to produce an ensemble predictor training examples with high error are weighted higher than those with lower error Difficult instances get more attention AdaBoost : “shortcoming” are identified by high-weight data points Gradient Boosting : “shortcomings” are identified by gradients

4. Gradient Boost Gradient Boosting = Gradient Descent + Boosting Analogous to line search in steepest descent Construct the new base-learners to be maximally correlated with the negative gradient of the loss function, associated with the whole ensemble. Arbitrary loss functions can be applied

5. Function estimate (parametric) Change the function optimization problem into the parameter estimation one Function estimation given

6. Steepest descent optimization “greedy stage-wise” approach of function incrementing with the base-learners The optimal step-size rho should be specified at each iteration The optimization rule is defined as:

7. Gradient Boost Solution to the parameter estimates can be difficult to obtain Choose new function h to be most correlated with –g(x) Classic least-squares minimization problem

8. : Line search by Newton’s method

9. Newton’s method

10. K-class Gradient Boost Goal : learn a set of scoring function by minimizing negative log-loss of the training data Probability of an example x being class k : Weak classifier

11. Solve the optimization problem Select h to the most parallel with the –g(X) by following minimization problem Scoring function is updated as

13. LatentBoost for Human Action Recognition l1l1 l2l2 l3l3 l4l4 l5l5 x1x1 x2x2 x3x3 x4x4 x5x5

14. Features Optical flow features (unary) Split into 4 scalar fields channels & motion magnitude Color histogram features (pairwise) difference between color histograms in rectangular sub-windows taken from adjacent frames

15. Positive optical flow features (a) Bend (b) Jack (c) Jump (d) pJump (e) run (f) side (g) walk (h) wave1 (i) wave2

16. Latent Boosting Assume that an example (x,y) is associated with a set of latent variables L={l 1, l 2, …, l T } These latent variables are constrained by an undirected graph structure G=(V,E) Scoring function of (x,L) pair for the k-th class where l1l1 l2l2 l3l3 l4l4 l5l5 x1x1 x2x2 x3x3 x4x4 x5x5

17. Weak learners of the unary & pairwise potential : gradient of loss function w.r.t. unary potential : gradient of loss function w.r.t. pairwise potential

18. Marginal distributions  These can be computed efficiently by using Belief Propagation l1l1 l2l2 l3l3 l4l4 l5l5 x1x1 x2x2 x3x3 x4x4 x5x5 y

19. F

20. Weizmann dataset (83 videos, 9 events)

21. Typical tracklets (29x60) from the Weizmann dataset Jacking Running Jumping Waving

22. TRECVID dataset (5 cameras, 10 videos, 7 events) Typical tracklets (29x60) from the TRECVID dataset

25. Limitations Not guaranteed to find the global optimum in a non-convex problem Performance of the final classifier is very sensitive to the initialization If the latent structure is not a tree, LatentBoost can perform inference with LBP : slow and not exact than BP Summation over all the possible latent variable may cause problems

26. Summary Novel boosting algorithm with latent variables Applying to the task of human action recognition New way to solve problems with a structure of latent variables