Presentation is loading. Please wait.

Presentation is loading. Please wait.

Models for Scene Understanding – Global Energy models and a Style-Parameterized boosting algorithm (StyP-Boost) Jonathan Warrell, 1 Simon Prince, 2 Philip.

Published byWinfred Dalton Modified over 9 years ago

Similar presentations

Presentation on theme: "Models for Scene Understanding – Global Energy models and a Style-Parameterized boosting algorithm (StyP-Boost) Jonathan Warrell, 1 Simon Prince, 2 Philip."— Presentation transcript:

1 Models for Scene Understanding – Global Energy models and a Style-Parameterized boosting algorithm (StyP-Boost) Jonathan Warrell, 1 Simon Prince, 2 Philip Torr, 1 Lubor Ladicky, 1 Chris Russell 1 1 Oxford Brookes University, 2 University College London

2 Overview CRF-based semantic segmentation Recent models Detectors Stereo Co-occurence Hierarchical Energies Style parameterized boosting (StyP-Boost) Open questions / problems

3 CRF-based semantic segmentation Semantic segmentation = dense labeling using fixed object set

4 CRF-based semantic segmentation Conditional Random Field model (pairwise) Observed Variables Hidden Variables UnaryPairwise

5 Example:  -expansion Sky House Tree Ground Initialize with Tree Status: Expand GroundExpand HouseExpand Sky Courtesy: Pushmeet Kohli

6 Move Making Algorithms Search Neighbourhood Current Solution Optimal Move Solution Space Energy

7 Higher order CRF models Higher order models UnaryPairwiseHigher-order

8 Segment-based Potentials No. of pixels not taking l in c

9 Detector-based Potentials Strength of detector response Lubor Ladicky, Paul Sturgess, Karteek Alahari, Chris Russell, Philip H.S. Torr, What,Where & How Many? Combining Object Detectors and CRFs, ECCV 2010 What,Where & How Many? Combining Object Detectors and CRFs

10 Co-occurrence Potentials Lubor Ladicky, Chris Russell, Pushmeet Kohli, Philip H.S. Torr, Graph Cut based Inference with Co-occurrence Statistics, ECCV, 2010Graph Cut based Inference with Co-occurrence Statistics Global image label set

11 Joint Stereo + Segmentation Joint potentials Lubor Ladicky, Paul Sturgess, Chris Russell, Sunando Sengupta, Philip H.S. Torr, Joint Optimisation for Object Class Segmentation and Dense Stereo Reconstruction, BMVC 2010Joint Optimisation for Object Class Segmentation and Dense Stereo Reconstruction Object only potentials

12 Hierarchical Energies Lubor Ladicky, Chris Russell, Pushmeet Kohli, Philip H.S. Torr, Associative Hierarchical CRFs for Object Class Image Segmentation, ICCV, 2009.Associative Hierarchical CRFs for Object Class Image Segmentation Energy between levels 1 and 0

13 Style-based Potentials Jonathan Warrell, Simon Prince, Philip H.S. Torr, StyP-Boost: A Bilinear Boosting Algorithm for Learning Style-Parameterized Classifiers, BMVC, 2010StyP-Boost: A Bilinear Boosting Algorithm for Learning Style-Parameterized Classifiers Style-based unary potential Style 1:Style 2:

14 TextonBoost (Shotton et al ’09) Image first convolved with 17-d filter bank Vectors are clustered, and assigned to ~150 texton indices

15 TextonBoost (Shotton et al ’09) Texture-layout features derived from textons Boosted classifier predicts semantic class

16 DenseBoost (Ladicky et al ’09) DenseBoost extends TextonBoost to include HOG ColourHOG Structure / Motion features State of the art performance on MSRC (Ladicky et al ’09) CamVid (Sturgess et al ’09) Paul Sturgess, Karteek Alahari, Lubor Ladicky, Philip H.S. Torr, Combining Appearance and Structure from Motion Features for Road Scene Understanding, BMVC, 2009Combining Appearance and Structure from Motion Features for Road Scene Understanding Lubor Ladicky, Chris Russell, Pushmeet Kohli, Philip H.S. Torr, Associative Hierarchical CRFs for Object Class Image Segmentation, ICCV, 2009.Associative Hierarchical CRFs for Object Class Image Segmentation

17 StyP-Boost Framework (Training) Training Set Objective Classifier form Local featuresStyle ParametersTarget vectors

18 StyP-Boost Framework (Training) Training Set Objective Classifier form Loss for class k Strong learner for class k

19 StyP-Boost Framework (Training) Training Set Objective Classifier form Weak learner mStyle s

20 Corel: Styles through clustering Styles found in Corel through clustering 2-styles (98%) 3-styles(96%) 4-styles (89%)

21 Corel: Styles through clustering Cluster images based on label histograms during training (2-4 clusters) Train classifier to predict cluster from image Use smoothed classifier posteriors as style parameters (training and testing) cluster label

22 Corel: Qualitative results StyP-Boost reduces noise from classes which don’t co-occur

23 Corel: Qualitative results StyP-Boost provides better discrimination of co-occuring classes

24 Corel: Quantitative results Training set Test set

25 Open questions / Problems Learning from sparsely labeled data Lamp-post Sign

26 Open questions / Problems Incorporating 3D and Video Image CRF Ground-plan CRF Volumetric CRF

27 Open questions / Problems Using temporal information Extend detector potentials to include tracking Use global scene variables for times of day, seasons etc.

28 Further Questions Further Questions?

Download ppt "Models for Scene Understanding – Global Energy models and a Style-Parameterized boosting algorithm (StyP-Boost) Jonathan Warrell, 1 Simon Prince, 2 Philip."

Similar presentations

Using Strong Shape Priors for Multiview Reconstruction Yunda SunPushmeet Kohli Mathieu BrayPhilip HS Torr Department of Computing Oxford Brookes University.

Using Strong Shape Priors for Multiview Reconstruction Yunda SunPushmeet Kohli Mathieu BrayPhilip HS Torr Department of Computing Oxford Brookes University.
POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of.

POSE–CUT Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph Cuts Mathieu Bray Pushmeet Kohli Philip H.S. Torr Department of.
Mean-Field Theory and Its Applications In Computer Vision1 1.

Mean-Field Theory and Its Applications In Computer Vision1 1.
Bayesian Belief Propagation

Bayesian Belief Propagation
O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.
The Layout Consistent Random Field for detecting and segmenting occluded objects CVPR, June 2006 John Winn Jamie Shotton.

The Layout Consistent Random Field for detecting and segmenting occluded objects CVPR, June 2006 John Winn Jamie Shotton.
Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29.

Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29.
Pose Estimation and Segmentation of People in 3D Movies Karteek Alahari, Guillaume Seguin, Josef Sivic, Ivan Laptev Inria, Ecole Normale Superieure ICCV.

Pose Estimation and Segmentation of People in 3D Movies Karteek Alahari, Guillaume Seguin, Josef Sivic, Ivan Laptev Inria, Ecole Normale Superieure ICCV.
Ľubor Ladický1 Phil Torr2 Andrew Zisserman1

Ľubor Ladický1 Phil Torr2 Andrew Zisserman1
Joint Optimisation for Object Class Segmentation and Dense Stereo Reconstruction Ľubor Ladický, Paul Sturgess, Christopher Russell, Sunando Sengupta, Yalin.

Joint Optimisation for Object Class Segmentation and Dense Stereo Reconstruction Ľubor Ladický, Paul Sturgess, Christopher Russell, Sunando Sengupta, Yalin.
Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.

Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.
Scene Labeling Using Beam Search Under Mutex Constraints ID: O-2B-6 Anirban Roy and Sinisa Todorovic Oregon State University 1.

Scene Labeling Using Beam Search Under Mutex Constraints ID: O-2B-6 Anirban Roy and Sinisa Todorovic Oregon State University 1.
Carolina Galleguillos, Brian McFee, Serge Belongie, Gert Lanckriet Computer Science and Engineering Department Electrical and Computer Engineering Department.

Carolina Galleguillos, Brian McFee, Serge Belongie, Gert Lanckriet Computer Science and Engineering Department Electrical and Computer Engineering Department.
Semantic Texton Forests for Image Categorization and Segmentation We would like to thank Amnon Drory for this deck הבהרה : החומר המחייב הוא החומר הנלמד.

Semantic Texton Forests for Image Categorization and Segmentation We would like to thank Amnon Drory for this deck הבהרה : החומר המחייב הוא החומר הנלמד.
Shuai Zheng et al. Dense Semantic Image Segmentation with Objects and Attributes. IEEE CVPR 2014.01/06/20141/6 Dense semantic image segmentation with objects.

Shuai Zheng et al. Dense Semantic Image Segmentation with Objects and Attributes. IEEE CVPR /06/20141/6 Dense semantic image segmentation with objects.
Structured Hough Voting for Vision-based Highway Border Detection

Structured Hough Voting for Vision-based Highway Border Detection
INTRODUCTION Heesoo Myeong, Ju Yong Chang, and Kyoung Mu Lee Department of EECS, ASRI, Seoul National University, Seoul, Korea Learning.

INTRODUCTION Heesoo Myeong, Ju Yong Chang, and Kyoung Mu Lee Department of EECS, ASRI, Seoul National University, Seoul, Korea Learning.
Efficient Inference for Fully-Connected CRFs with Stationarity

Efficient Inference for Fully-Connected CRFs with Stationarity
LOCUS (Learning Object Classes with Unsupervised Segmentation) A variational approach to learning model- based segmentation. John Winn Microsoft Research.

LOCUS (Learning Object Classes with Unsupervised Segmentation) A variational approach to learning model- based segmentation. John Winn Microsoft Research.
Analyzing Semantic Segmentation Using Hybrid Human-Machine CRFs Roozbeh Mottaghi 1, Sanja Fidler 2, Jian Yao 2, Raquel Urtasun 2, Devi Parikh 3 1 UCLA.

Analyzing Semantic Segmentation Using Hybrid Human-Machine CRFs Roozbeh Mottaghi 1, Sanja Fidler 2, Jian Yao 2, Raquel Urtasun 2, Devi Parikh 3 1 UCLA.

Similar presentations

Ads by Google