Discriminative Segment Annotation in Weakly Labeled Video Kevin Tang, Rahul Sukthankar Appeared in CVPR 2013 (Oral)

Research Problem Input: a weakly labeled video (eg., “dog”) Output: identify segments that correspond to the label to generate the semantic segmentation, i.e., classify each segment either as coming from concept “dog” (called concept segments), or not (called background segments). Pipeline – Perform unsupervised spatiotemporal segmentation. – Propose an algorithm to identify the meaningful segment.

Contributions Present a interpretation framework to analyze a broad class of existing weakly supervised learning algorithms about segment annotation problem. Propose a discriminative algorithm CRANE (Concept Ranking According to Negative Exemplars) for segment annotation.

Interpretation framework Pairwise distance matrix between segments Segment: spatiotemporal volume (3D), represented as a point in feature space (such as RGB histogram, local binary pattern histogram, or dense optical histogram). Positive segment  Concept segment  Background segment Negative segment Goal: classify the from in. Rank the elements in in decreasing order of a score, such that top-ranked Elements correspond to.

Interpretation framework Baseline algorithms about segment annotation. – Kernel density estimation for Negative segments. Intuition: the distribution of is similar to distribution of. Construct a probability density operated on block C. Rank the elements according to. – Negative Mining (MIN) Intuition: distance from to the nearest > distance from to nearest. Operated on block D.

CRANE Each negative segment in penalizes nearby segments in. Segments in should be those far from negatives. Penalty function

CRANE Advantages 1.Robust to noise. 2.Parallelizable.

Experimental Results YouTube Objects datasets

Experimental Results (a): Sucesses (b): Failures