Chao-Yeh Chen and Kristen Grauman University of Texas at Austin Efficient Activity Detection with Max- Subgraph Search

Outline Introduction Approach Define weighted nodes Link nodes Search for the maximum-weight graph Experimental Result Conclusion

Introduction Existing methods tend to separate activity detection into two distinct stages: 1. generates space-time candidate regions of interest from the test video 2. scores each candidate according to how well it matches a given activity model (often a classifier).

How to detect human activity in continuous video? Status quo approaches:

Introduction We pose activity detection as a maximum-weight connected subgraph problem over a learned space-time graph constructed on the test sequence.

Approach

Classifier training for feature weights Learn a linear SVM from training data, the scoring function would have the form: let denote j-th bin count for histogram h(S), the j-th word is associated with a weight for j = 1,…,K, where K is the dimension of histogram h.

Classifier training for feature weights Thus the classifier response for subvolume S is: Candidate subvolume SVM weight for j-th word Num occurrences of j-th word SVM weight for i-th feature's word

Bag-of-feature(Bof)

Localized SpaceTime Features Low-level descriptors we use HoG and HoF computed in local space-time cubes [14, 10]. These descriptors capture the appearance and motion in the video. High-level descriptors

Define weighted nodes Divide space-time volume into frame-level or space-time nodes. Compute the weight of nodes from the features inside them.

Link nodes Two different link strategies: 1. Neighbors only for frame-level nodes(T-Subgraph) or space- time nodes(ST-Subgraph). 2. First two neighbors for frame-level nodes(T-Jump-Subgraph).

Search for the maximum-weight graph Transform max-weight subgraph problem into a prize- collecting Steiner tree problem. Solve efficiently with branch and cut method from [15]. [15]An algorithmic framework for the exact solution of the prize-collecting Steiner tree problem. Math. Prog., 2006.

Experimental Result Datasets

Baselines T-Sliding ST-Cube-Sliding ST-Cube-Subvolume[29] J. Yuan, Z. Liu, and Y. Wu. Discriminative subvolume search for efficient action detection. In CVPR, 2009.

UCF Sports data

Hollywood data

MSR dataset

Example of ST-Subgraph

Overview of all methods on the three datasets

High-level vs Low-level descriptors

Conclusion Compare to sliding window search,it significantly reduces computation time. Flexible node structure offers more robust detection in noisy backgrounds. High-level descriptor shows promise for complex activities by incorporating semantic relationships between humans and objects in video.