Jifeng Dai 2011/09/27

 Introduction  Structural SVM  Kernel Design  Segmentation and parameter learning  Object Feature Descriptors  Experimental results  Conclusions and Future Work

CVPR 2011 Oral

 Things to do:

 Contributions: 1) Propose a kernelized structural support vector machine approach to learn discriminatively the mapping from image to a segmentation mask. 2) Combine high level object similarity information with multiple low level segmentation cues into a novel kernel. 3) Traditional segmentation regularizations are preserved in the framework and explicitly enforced during the learning process. This way smoothness of the solution does not need to be “re-learned” from training examples.

 Complex output The dog chased the cat x S VPNP DetNV NP DetN y2y2 S VP DetNV NP VN y1y1 S VP DetNV NP DetN ykyk …

 Training Examples:  Hypothesis Space: The dog chased the cat x S VPNP DetNV NP DetN y1y1 S VP DetNV NP VN y2y2 S VP DetNV NP DetN y 58 S VPNP DetNV NP DetN y 12 S VPNP DetNV NP DetN y 34 S VPNP DetNV NP DetN y4y4 Training: Find that solve Problems How to predict efficiently? How to learn efficiently? Manageable number of parameters?

 The idea behind Structured SVM is to discriminatively learn a scoring function over input/output pairs (i.e. over image/mask pairs).

 Loss function:  Two important choices: 1) Restrict the search to Ys, subset of Y composed by smooth segmentation masks.

 Two important choices: 1) Restrict the search to Ys, subset of Y composed by smooth segmentation masks. 2) using kernel functions so that we could work in the dual formulation.

HOG… Object Similarity Kernel Mask Similarity Kernel

1) Shape Kernel 2) Local Color Model Kernel 3) Global Color Model Kernel

 Graph cuts Mask smooth term In which So (6) and (7) take the form: Graph cuts!!!

 Parameters are optimized on a validation set

 HOG grid or detector response feature

 Datasets: 1) the Dresses dataset (600 images) 2) the Weizmann horses dataset (328 images) 3) the Oxford 17 category flower dataset (849 images)

 How to measure performance?

 Comparison with previous works:

Oxford Flower Dataset Previous work:

 Examples:

Contributions: 1) Propose a kernelized structural support vector machine approach to learn discriminatively the mapping from image to a segmentation mask. 2) Combine high level object similarity information with multiple low level segmentation cues into a novel kernel. 3) Traditional segmentation regularizations are preserved in the framework and explicitly enforced during the learning process. This way smoothness of the solution does not need to be “re-learned” from training examples.

Future Work: 1)Model the boundary curves (driven by low-level cues). 2) Instead of relying on a single global object similarity kernel, dividing the kernel into a parts-based representation. 3) Establish a theoretical connection between the complexity of the top-down models the algorithm can learn and the number of segmentations needed in the training set.