1. Data Driven Attributes for Action Detection
Week 2
Presented by Christina Peterson

2. Background
Liu et al. [1] propose a unified framework for action recognition where manually specified attributes are:
Selected discriminatively to account for intra-class variability
Integrated with data-driven attributes to make the attribute set more descriptive
Yu et al. [2] propose a framework for an attribute-based query by using a large pool of weak attributes composed of automatic classifier scores that are easily acquired with no human labor.
Query attributes are acquired by human labeling process
Weak attributes are generated automatically by machine
Query attributes are mapped to weak attributes

3. Background
Malisiewicz et al. [3] propose a method for object detection which combines a discriminative object detector with a nearest-neighbor approach.
A separate linear SVM classifier is trained for each exemplar in the dataset
Each exemplar is represented using a rigid HOG template
This results in a large collection of simple individual Exemplar-SVM detectors rather than a single complex category detector
Farhadi et al. [4] propose an attribute based approach to object detection.
Semantic and discriminative attributes
Feature selection method for learning attributes that can be generalized across categories
Base feature definition

4. Background
Tian et. al. [5] proposes a spatiotemporal deformable part model (SDPM) that stays true to the structure of the original deformable part model (DPM).
SDPM has volumetric parts that displace in both time and space
Root filter used to capture the overall information of the action cycle and is obtained by applying an SVM on the HOG3D features of the action cycle

5. Low Level Features STIP Color Texture
Histogram of Oriented Gradient (HOG)
72 element descriptor
Histogram of Optical Flow (HOF)
90 element descriptor
Color
Texture

6. Bag of Words Concatenate Low Level Features for each video
Cluster Features by Kmeans
128 for color, 256 for texture, 1000 for STIP
Each feature type will be clustered separately by Kmeans
3 x 3 x = 28 cells
Features collected for each cell
Create Histogram of cluster centers per feature for each cell in bounding box
( ) x 28
Normalize based on size of bounding box

7. Exemplar SVM
Train a separate linear SVM classifier for each exemplar in the dataset with a single positive example and many negative examples
This results in a large collection of simple individual Exemplar-SVM detectors rather than a single complex category detector
Example: The action Diving-side will have multiple linear SVM classifiers each based on a positive example within this action class
Test set will need to run all Exemplar-SVM detectors for the respective action class to calculate label prediction accuracy

8. Goals Implement the Exemplar SVM classifiers in matlab
Label Propagation
Finding relationship between labels and prediction results
Conditional Probability

9. References
[1] J. Liu, B. Kuipers, and S. Savarese. Recognizing Human Actions by Attributes. In CVPR, [2] F. Yu, R. Ji, M.-H. Tsai, G. Ye, and S.-F. Chang. Weak Attributes for Large-Scale Image Retrieval. In CVPR, [3] T. Malisiewicz, A. Gupta, and A. A. Efros. Ensemble of Exemplar SVMS for Object Detection and Beyond. In Proc. ICCV, [4] H. Farhadi, I. Endres, D. Hoiem, and D. Forsyth. Describing objects by their attributes. In CVPR, [5] Y. Tian, R. Sukthankar, and M. Shah. Spatiotemporal Deformable Part Models for Action Detection. In CVPR, [6] Y. Wang and G. Mori. Hidden Part Models for Human Action Recognition: Probabilistic vs. Max-Margin. In PAMI, 2011.