Context-based vision system for place and object recognition Antonio Torralba Kevin Murphy Bill Freeman Mark Rubin Presented by David Lee Some slides borrowed.

Slides:

Advertisements

Similar presentations

1 Gesture recognition Using HMMs and size functions.

Advertisements

Pattern Finding and Pattern Discovery in Time Series

An Approach to ECG Delineation using Wavelet Analysis and Hidden Markov Models Maarten Vaessen (FdAW/Master Operations Research) Iwan de Jong (IDEE/MI)

Modeling Uncertainty over time Time series of snapshot of the world “state” we are interested represented as a set of random variables (RVs) – Observable.

An Overview of Machine Learning

Machine Learning Hidden Markov Model Darshana Pathak University of North Carolina at Chapel Hill Research Seminar – November 14, 2012.

Patch to the Future: Unsupervised Visual Prediction

Hidden Markov Models Reading: Russell and Norvig, Chapter 15, Sections

Generative learning methods for bags of features

December 2006 Cairo University Faculty of Computers and Information HMM Based Speech Synthesis Presented by Ossama Abdel-Hamid Mohamed.

Introduction to Hidden Markov Models

Hidden Markov Models Bonnie Dorr Christof Monz CMSC 723: Introduction to Computational Linguistics Lecture 5 October 6, 2004.

What is the temporal feature in video sequences?

Robust Moving Object Detection & Categorization using self- improving classifiers Omar Javed, Saad Ali & Mubarak Shah.

Hilbert Space Embeddings of Hidden Markov Models Le Song, Byron Boots, Sajid Siddiqi, Geoff Gordon and Alex Smola 1.

Probabilistic reasoning over time So far, we’ve mostly dealt with episodic environments –One exception: games with multiple moves In particular, the Bayesian.

… Hidden Markov Models Markov assumption: Transition model:

HMM-BASED PATTERN DETECTION. Outline  Markov Process  Hidden Markov Models Elements Basic Problems Evaluation Optimization Training Implementation 2-D.

Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

An opposition to Window- Scanning Approaches in Computer Vision Presented by Tomasz Malisiewicz March 6, 2006 Advanced The Robotics Institute.

Generative learning methods for bags of features

1 Segmentation with Scene and Sub-Scene Categories Joseph Djugash Input Image Scene/Sub-Scene Classification Segmentation.

Handwritten Character Recognition using Hidden Markov Models Quantifying the marginal benefit of exploiting correlations between adjacent characters and.

Crash Course on Machine Learning

Cognitive Computer Vision Kingsley Sage and Hilary Buxton Prepared under ECVision Specific Action 8-3

Gaussian Mixture Model and the EM algorithm in Speech Recognition

Alignment and classification of time series gene expression in clinical studies Tien-ho Lin, Naftali Kaminski and Ziv Bar-Joseph.

3D SLAM for Omni-directional Camera

CAP 5415 Computer Vision Fall 2004

A brief overview of Speech Recognition and Spoken Language Processing Advanced NLP Guest Lecture August 31 Andrew Rosenberg.

Dynamic 3D Scene Analysis from a Moving Vehicle Young Ki Baik (CV Lab.) (Wed)

Understanding the Semantics of Media Lecture Notes on Video Search & Mining, Spring 2012 Presented by Jun Hee Yoo Biointelligence Laboratory School of.

Reading Between The Lines: Object Localization Using Implicit Cues from Image Tags Sung Ju Hwang and Kristen Grauman University of Texas at Austin Jingnan.

Structure Discovery of Pop Music Using HHMM E6820 Project Jessie Hsu 03/09/05.

Using Inactivity to Detect Unusual behavior Presenter : Siang Wang Advisor : Dr. Yen - Ting Chen Date : Motion and video Computing, WMVC.

UIUC CS 498: Section EA Lecture #21 Reasoning in Artificial Intelligence Professor: Eyal Amir Fall Semester 2011 (Some slides from Kevin Murphy (UBC))

Putting Context into Vision Derek Hoiem September 15, 2004.

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: ML and Simple Regression Bias of the ML Estimate Variance of the ML Estimate.

Prostate Cancer CAD Michael Feldman, MD, PhD Assistant Professor Pathology University Pennsylvania.

Epitomic Location Recognition A generative approach for location recognition K. Ni, A. Kannan, A. Criminisi and J. Winn In proc. CVPR Anchorage,

CS Statistical Machine learning Lecture 24

1 CS 552/652 Speech Recognition with Hidden Markov Models Winter 2011 Oregon Health & Science University Center for Spoken Language Understanding John-Paul.

School of Computer Science 1 Information Extraction with HMM Structures Learned by Stochastic Optimization Dayne Freitag and Andrew McCallum Presented.

1 CSE 552/652 Hidden Markov Models for Speech Recognition Spring, 2006 Oregon Health & Science University OGI School of Science & Engineering John-Paul.

Pattern Recognition and Machine Learning-Chapter 13: Sequential Data

 Present by 陳群元.  Introduction  Previous work  Predicting motion patterns  Spatio-temporal transition distribution  Discerning pedestrians  Experimental.

A Dynamic Conditional Random Field Model for Object Segmentation in Image Sequences Duke University Machine Learning Group Presented by Qiuhua Liu March.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Elements of a Discrete Model Evaluation.

Contextual models for object detection using boosted random fields by Antonio Torralba, Kevin P. Murphy and William T. Freeman.

1.Learn appearance based models for concepts 2.Compute posterior probabilities or Semantic Multinomial (SMN) under appearance models. -But, suffers from.

CS Statistical Machine learning Lecture 25 Yuan (Alan) Qi Purdue CS Nov

Using the Forest to see the Trees: A computational model relating features, objects and scenes Antonio Torralba CSAIL-MIT Joint work with Aude Oliva, Kevin.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition Objectives: Reestimation Equations Continuous Distributions.

Max-Margin Training of Upstream Scene Understanding Models Jun Zhu Carnegie Mellon University Joint work with Li-Jia Li *, Li Fei-Fei *, and Eric P. Xing.

Biologically Inspired Vision-based Indoor Localization Zhihao Li, Ming Yang

EEL 6586: AUTOMATIC SPEECH PROCESSING Hidden Markov Model Lecture

Particle Filtering for Geometric Active Contours

Nonparametric Semantic Segmentation

Chapter 3: Maximum-Likelihood and Bayesian Parameter Estimation (part 2)

Context-based vision system for place and object recognition

An opposition to: Context-Based Vision System for Place and Object Recognition Contextual Models for Object Detection Using BRFs.

Cheng-Ming Huang, Wen-Hung Liao Department of Computer Science

Lecture 25: Introduction to Recognition

Action Recognition in Temporally Untrimmed Videos

Brief Review of Recognition + Context

Instructors: Fei Fang (This Lecture) and Dave Touretzky

Nonparametric Bayesian Texture Learning and Synthesis

Visual Recognition of American Sign Language Using Hidden Markov Models 문현구 문현구.

Chapter 3: Maximum-Likelihood and Bayesian Parameter Estimation (part 2)

Presentation transcript:

Context-based vision system for place and object recognition Antonio Torralba Kevin Murphy Bill Freeman Mark Rubin Presented by David Lee Some slides borrowed from Kevin Murphy

Object out of context

Object in context

Wearable test-bed

System diagram

Computing the features

24 filtered Images Downsample to 4x4 4x4x24 =384 dim 80 dim

Visualizing the filter bank output Images 80-dimensional representation

Place recognition system

Hidden Markov Model Hidden states = location (63 values) Observations = v G t ∈ R 80 Transition model encodes topology of environment Observation model is a mixture of Gaussians (100 views per place)

Hidden Markov Model Observation Likelihood Prediction Prior Transition Matrix Mixture of Gaussians MLE (counting)

Scene Categorization 17 Categories (Office, Corridor, Street, etc) Train a separate HMM on category labels

Place recognition demo

Specific location Location category Indoor/outdoor Ground truth System estimate Performance on known env.

Performance on new env.

Comparison of features Recognition Categorization

Effect of HMM on recognition With Without (But with temporal smoothing)

From place to object recognition

Object priming Predict object properties based on context (top-down signals):  Visual gist, v t G  Specific Location, Q t  Kind of location, C t

Object Priming Again… MLE Probability of object i Probability of object i in image v i given entire video sequence Probability of object i Given current observation & place Estimate of current place (Output of HMM) Mixture of Gaussians Observation Likelihood Prior probability of object i being in place q

Predicting object presence

ROC curves for object detection

Predicting object position and scale

Estimate of mask Probability of an object i being present and location being q (Output of previous system) Estimate of mask given current gist, place, and object delta Gaussian

Predicted segmentation

Conclusion Real world problem (and it works!) Uses only global feature (context) How much did {HMM / place prior} affect {place recognition / object detection}? Can we really say “context” did the job?