Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.

Slides:

Advertisements

Similar presentations

Shape Matching and Object Recognition using Low Distortion Correspondence Alexander C. Berg, Tamara L. Berg, Jitendra Malik U.C. Berkeley.

Advertisements

Three things everyone should know to improve object retrieval

Foreground Focus: Finding Meaningful Features in Unlabeled Images Yong Jae Lee and Kristen Grauman University of Texas at Austin.

Florian Schroff, Antonio Criminisi & Andrew Zisserman ICCV 2007 Harvesting Image Databases from the Web.

VisualRank: Applying PageRank to Large-Scale Image Search Yushi Jing, Member, IEEE, and Shumeet Baluja, Member, IEEE.

Discriminative Relevance Feedback With Virtual Textual Representation For Efficient Image Retrieval Suman Karthik and C.V.Jawahar.

MIT CSAIL Vision interfaces Towards efficient matching with random hashing methods… Kristen Grauman Gregory Shakhnarovich Trevor Darrell.

UCB Computer Vision Animals on the Web Tamara L. Berg CSE 595 Words & Pictures.

Part 1: Bag-of-words models by Li Fei-Fei (Princeton)

The Pyramid Match Kernel: Discriminative Classification with Sets of Image Features Kristen Grauman Trevor Darrell MIT.

Query Specific Fusion for Image Retrieval

1 Part 1: Classical Image Classification Methods Kai Yu Dept. of Media Analytics NEC Laboratories America Andrew Ng Computer Science Dept. Stanford University.

CS4670 / 5670: Computer Vision Bag-of-words models Noah Snavely Object

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Bag-of-features models Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

CVPR 2008 James Philbin Ondˇrej Chum Michael Isard Josef Sivic

Beyond bags of features: Part-based models Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

Large-scale matching CSE P 576 Larry Zitnick

Effective Image Database Search via Dimensionality Reduction Anders Bjorholm Dahl and Henrik Aanæs IEEE Computer Society Conference on Computer Vision.

Small Codes and Large Image Databases for Recognition CVPR 2008 Antonio Torralba, MIT Rob Fergus, NYU Yair Weiss, Hebrew University.

Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

Fast and Compact Retrieval Methods in Computer Vision Part II A. Torralba, R. Fergus and Y. Weiss. Small Codes and Large Image Databases for Recognition.

1 Image Recognition - I. Global appearance patterns Slides by K. Grauman, B. Leibe.

Lecture 28: Bag-of-words models

Agenda Introduction Bag-of-words model Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.

Automatic Image Annotation and Retrieval using Cross-Media Relevance Models J. Jeon, V. Lavrenko and R. Manmathat Computer Science Department University.

CS335 Principles of Multimedia Systems Content Based Media Retrieval Hao Jiang Computer Science Department Boston College Dec. 4, 2007.

Video Google: Text Retrieval Approach to Object Matching in Videos Authors: Josef Sivic and Andrew Zisserman ICCV 2003 Presented by: Indriyati Atmosukarto.

Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

CS294‐43: Visual Object and Activity Recognition Prof. Trevor Darrell Spring 2009 March 17 th, 2009.

Bag-of-features models

Unsupervised discovery of visual object class hierarchies Josef Sivic (INRIA / ENS), Bryan Russell (MIT), Andrew Zisserman (Oxford), Alyosha Efros (CMU)

Generative learning methods for bags of features

Agenda Introduction Bag-of-words model Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based.

Visual Object Recognition Rob Fergus Courant Institute, New York University

Multiple Object Class Detection with a Generative Model K. Mikolajczyk, B. Leibe and B. Schiele Carolina Galleguillos.

Discriminative and generative methods for bags of features

Large Scale Recognition and Retrieval. What does the world look like? High level image statistics Object Recognition for large-scale search Focus on scaling.

Machine learning & category recognition Cordelia Schmid Jakob Verbeek.

Efficient Image Search and Retrieval using Compact Binary Codes

Object Recognition and Augmented Reality

Review: Intro to recognition Recognition tasks Machine learning approach: training, testing, generalization Example classifiers Nearest neighbor Linear.

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Indexing Techniques Mei-Chen Yeh.

Wang, Z., et al. Presented by: Kayla Henneman October 27, 2014 WHO IS HERE: LOCATION AWARE FACE RECOGNITION.

Describing People: A Poselet-Based Approach to Attribute Classification Lubomir Bourdev 1,2 Subhransu Maji 1 Jitendra Malik 1 1 EECS U.C. Berkeley 2 Adobe.

Unsupervised Learning of Categories from Sets of Partially Matching Image Features Kristen Grauman and Trevor Darrel CVPR 2006 Presented By Sovan Biswas.

Keypoint-based Recognition Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 03/04/10.

MediaEval Workshop 2011 Pisa, Italy 1-2 September 2011.

Lecture #32 WWW Search. Review: Data Organization Kinds of things to organize –Menu items –Text –Images –Sound –Videos –Records (I.e. a person ’ s name,

Marcin Marszałek, Ivan Laptev, Cordelia Schmid Computer Vision and Pattern Recognition, CVPR Actions in Context.

1 Action Classification: An Integration of Randomization and Discrimination in A Dense Feature Representation Computer Science Department, Stanford University.

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Svetlana Lazebnik, Cordelia Schmid, Jean Ponce

Category Discovery from the Web slide credit Fei-Fei et. al.

Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao.

Fast Similarity Search for Learned Metrics Prateek Jain, Brian Kulis, and Kristen Grauman Department of Computer Sciences University of Texas at Austin.

Video Google: A Text Retrieval Approach to Object Matching in Videos Josef Sivic and Andrew Zisserman.

Minimal Loss Hashing for Compact Binary Codes

Beyond Sliding Windows: Object Localization by Efficient Subwindow Search The best paper prize at CVPR 2008.

Unsupervised Learning of Visual Sense Models for Polysemous Words Kate Saenko Trevor Darrell Deepak.

MSRI workshop, January 2005 Object Recognition Collected databases of objects on uniform background (no occlusions, no clutter) Mostly focus on viewpoint.

CVPR 2006 New York City Spatial Random Partition for Common Visual Pattern Discovery Junsong Yuan and Ying Wu EECS Dept. Northwestern Univ.

Efficient Image Search and Retrieval using Compact Binary Codes Rob Fergus (NYU) Jon Barron (NYU/UC Berkeley) Antonio Torralba (MIT) Yair Weiss (Hebrew.

Content-Based Image Retrieval

The topic discovery models

Digit Recognition using SVMS

Rob Fergus Computer Vision

Brief Review of Recognition + Context

Presented by Wanxue Dong

Presentation transcript:

Agenda Introduction Bag-of-words models Visual words with spatial location Part-based models Discriminative methods Segmentation and recognition Recognition-based image retrieval Datasets & Conclusions

Retrieval domains Internet image search Video search for people/objects Searching home photo collections

Learning from Internet Image Search Joint learning of text and images Large scale retrieval

Noisy labels

Improving Google’s Image Search Fergus, Fei-Fei, Perona, Zisserman, ICCV 2005 Variant of pLSA that includes spatial information

Topics in model Re-ranking result: Motorbike Automatically chosen topic

Animals on the Web Berg and Forsyth, CVPR 2006 Gather images using text search Use LDA to discover “good” images using features based on nearby text, shape, color

Boostrapping of Image Search 2 4 Images returned with PENGUIN query Removal of drawings and abstract images Naives Bayes ranking using noisy metadata Train SVM……. Schroff, Zisserman, Criminisi, Harvesting Image Databases from the Web, ICCV 2007 Final ranking using SVM

OPTIMOL Li, Wang, Fei-Fei CVPR 07

Learning from Internet Image Search Joint learning of text and images Large scale retrieval

Matching Words and Pictures Barnard, Duygulu, de Freitas, Forsyth, Blei, Jordan, JMLR 2003

Text to Images

Images to text Use Blobworld or nCuts to segments images into regions Need to deduce labels attached to each image

Images to text result

Names and Faces in the News Berg, Berg, Edwards, Maire, White, Teh, Learned-Miller, Forsyth. CVPR Find faces (standard face detector), rectify them to same pose. 2.Perform Kernel PCA and Linear Discriminant Analysis (LDA). 3.Extract names from text. 4.Cluster faces, with each name corresponding to a cluster. 5.Use language model to refine results Collected 500,000 images and text captions from Yahoo! News

Initial clusters

Clusters refined with language model

Learning from Internet Image Search Joint learning of text and images Large scale retrieval

Vocabulary tree Nistér & Stewénius CVPR KD-tree in descriptor space Inverse lookup of features Specific object recognition  Not category-level

Slide from D. Nister

Pyramid Match Hashing Grauman & Darell, CVPR 2007 Combines Pyramid Match Kernel (efficient computation of correspondences between two set of vectors) with Locality Sensitive Hashing (LSH) [Indyk & Motwani 98] Allows matching of the set of features in a query image to sets of features in other images in time that is sublinear in # images Theoretical guarantees

Salakhutdinov and Hinton, SIGIR 2007 Torralba, Fergus, Weiss, CVPR 2008 Map images to compact binary codes Hash codes for fast lookup Semantic Hashing