1. INTRODUCTION Heesoo Myeong and Kyoung Mu Lee Department of ECE, ASRI, Seoul National University, Seoul, Korea http://cv.snu.ac.kr Tensor-based High-order Semantic Relation Transfer for Semantic Scene Segmentation Goal PROPOSED METHOD Results on Jain et al. Dataset Quantitative Results on Standard Datasets   Jain et al. dataset (Jain et al., ECCV10): 250 training images, 100 test images, 19 labels   SIFT Flow dataset (Liu et al., CVPR09): 2,488 training images, 200 test images, 33 labels   Polo dataset (Zhang et al., CVPR11): 80 training images, 237 test images, 6 labels Table 1: Per-pixel classification rates and (average per-class rates) 1. For a test image, retrieve similar training images using global features 2. Apply semantic relation transfer algorithm to transfer third-order semantic relation from each training image to the test image 3. Integrate the high-order score into MRF optimization framework and obtain semantic scene segmentation EXPERIMENTS Inference   Use fully connected third-order Markov random field (MRF) model: Key idea Our Contributions   The use of high-order semantic relations for semantic segmentation   A novel tensor-based representation of high-order semantic relations   A quadratic objective function for learning the semantic tensor and an efficient approximate algorithm   Semantic scene segmentation: identifying and segmenting all objects in a scene   We have presented a novel approach to learn high-order semantic relations of regions in a nonparametric manner   We develop a novel semantic tensor representation of the high-order semantic relations   We cast the high-order semantic relation transfer problem as a quadratic objective function of semantic tensors and propose an efficient approximate algorithm   Exploiting high-order(mostly third-order) semantic relation sky tree building car road Test imageSemantic scene segmentation building car sky road sky person car sky tree High-order relations in the training dataset Previous works & Limitations   Conventional context models mainly focus on learning pairwise relationships between objects   Pairwise relations are not enough to represent high-level contextual knowledge within images sky tree building car road car sky person tree building Query imageIntegrated high-order relationSemantic segmentation Retrieved images … Groundtruth sky road building tree car sidewalk Annotations of retrieved images Predicted top scored high-order relation tree sky road building sky car tree person road Results on LMO Dataset Results on Polo Dataset Overview Semantic Relation Transfer Algorithm sky sky road Pairwise semantic relation car building car building grass grass grass grass grassgrass horse horse horse horse horse horse horse horse grass grass person person person person person person QueryGround truthProposed QueryGround truthProposed building building skyskyskyskyskybuilding building building sidewalk sidewalk mountain mountain window door window door building sky window car car QueryGround truthProposed QueryGround truthProposed skytree building car roadskyroad building tree car sidewalk building building tree car tree roadroad building building car road bison mountain grass grass bison tree QueryGround truthProposed QueryGround truthProposed CONCLUSION