Overcoming Dataset Bias: An Unsupervised Domain Adaptation Approach Boqing Gong University of Southern California Joint work with Fei Sha and Kristen Grauman.

Slides:



Advertisements
Similar presentations
Learning visual representations for unfamiliar environments Kate Saenko, Brian Kulis, Trevor Darrell UC Berkeley EECS & ICSI.
Advertisements

Sharing Features Between Visual Tasks at Different Levels of Granularity Sung Ju Hwang 1, Fei Sha 2 and Kristen Grauman 1 1 University of Texas at Austin,
Improving the Fisher Kernel for Large-Scale Image Classification Florent Perronnin, Jorge Sanchez, and Thomas Mensink, ECCV 2010 VGG reading group, January.
A Survey on Transfer Learning Sinno Jialin Pan Department of Computer Science and Engineering The Hong Kong University of Science and Technology Joint.
Adding Unlabeled Samples to Categories by Learned Attributes Jonghyun Choi Mohammad Rastegari Ali Farhadi Larry S. Davis PPT Modified By Elliot Crowley.
Location Recognition Given: A query image A database of images with known locations Two types of approaches: Direct matching: directly match image features.
Foreground Focus: Finding Meaningful Features in Unlabeled Images Yong Jae Lee and Kristen Grauman University of Texas at Austin.
Rajat Raina Honglak Lee, Roger Grosse Alexis Battle, Chaitanya Ekanadham, Helen Kwong, Benjamin Packer, Narut Sereewattanawoot Andrew Y. Ng Stanford University.
Classification using intersection kernel SVMs is efficient Joint work with Subhransu Maji and Alex Berg Jitendra Malik UC Berkeley.
Patch to the Future: Unsupervised Visual Prediction
From Virtual to Reality: Fast Adaptation of Virtual Object Detectors to Real Domains Baochen Sun and Kate Saenko UMass Lowell.
Steerable Part Models Hamed Pirsiavash and Deva Ramanan
Learning on Probabilistic Labels Peng Peng, Raymond Chi-wing Wong, Philip S. Yu CSE, HKUST 1.
1 Building a Dictionary of Image Fragments Zicheng Liao Ali Farhadi Yang Wang Ian Endres David Forsyth Department of Computer Science, University of Illinois.
Mid-level Visual Element Discovery as Discriminative Mode Seeking Harley Montgomery 11/15/13.
Discriminative Segment Annotation in Weakly Labeled Video Kevin Tang, Rahul Sukthankar Appeared in CVPR 2013 (Oral)
Watching Unlabeled Video Helps Learn New Human Actions from Very Few Labeled Snapshots Chao-Yeh Chen and Kristen Grauman University of Texas at Austin.
Enhancing Exemplar SVMs using Part Level Transfer Regularization 1.
Jun Zhu Dept. of Comp. Sci. & Tech., Tsinghua University This work was done when I was a visiting researcher at CMU. Joint.
Beyond bags of features: Adding spatial information Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.
Statistical Recognition Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and Kristen Grauman.
Semi-Supervised Learning in Gigantic Image Collections Rob Fergus (NYU) Yair Weiss (Hebrew U.) Antonio Torralba (MIT) TexPoint fonts used in EMF. Read.
Object Recognition: Conceptual Issues Slides adapted from Fei-Fei Li, Rob Fergus, Antonio Torralba, and K. Grauman.
Spatial Pyramid Pooling in Deep Convolutional
Hierarchical Subquery Evaluation for Active Learning on a Graph Oisin Mac Aodha, Neill Campbell, Jan Kautz, Gabriel Brostow CVPR 2014 University College.
Learning from Multiple Outlooks Maayan Harel and Shie Mannor ICML 2011 Presented by Minhua Chen.
Relaxed Transfer of Different Classes via Spectral Partition Xiaoxiao Shi 1 Wei Fan 2 Qiang Yang 3 Jiangtao Ren 4 1 University of Illinois at Chicago 2.
Online Learning Algorithms
Introduction to domain adaptation
Training and future (test) data follow the same distribution, and are in same feature space.
Learning to Learn By Exploiting Prior Knowledge
Transfer Learning From Multiple Source Domains via Consensus Regularization Ping Luo, Fuzhen Zhuang, Hui Xiong, Yuhong Xiong, Qing He.
Unsupervised Learning of Categories from Sets of Partially Matching Image Features Kristen Grauman and Trevor Darrel CVPR 2006 Presented By Sovan Biswas.
Presented By Wanchen Lu 2/25/2013
Cao et al. ICML 2010 Presented by Danushka Bollegala.
Person-Specific Domain Adaptation with Applications to Heterogeneous Face Recognition (HFR) Presenter: Yao-Hung Tsai Dept. of Electrical Engineering, NTU.
Object Bank Presenter : Liu Changyu Advisor : Prof. Alex Hauptmann Interest : Multimedia Analysis April 4 th, 2013.
Thesis Proposal PrActive Learning: Practical Active Learning, Generalizing Active Learning for Real-World Deployments.
Computer Vision CS 776 Spring 2014 Recognition Machine Learning Prof. Alex Berg.
DeepFont: Large-Scale Real-World Font Recognition from Images
IIIT Hyderabad Synthesizing Classifiers for Novel Settings Viresh Ranjan CVIT,IIIT-H Adviser: Prof. C. V. Jawahar, IIIT-H Co-Adviser: Dr. Gaurav Harit,
IEEE TRANSSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
Transfer Learning Task. Problem Identification Dataset : A Year: 2000 Features: 48 Training Model ‘M’ Testing 98.6% Training Model ‘M’ Testing 97% Dataset.
Building high-level features using large-scale unsupervised learning Anh Nguyen, Bay-yuan Hsu CS290D – Data Mining (Spring 2014) University of California,
Semantic Embedding Space for Zero ­ Shot Action Recognition Xun XuTimothy HospedalesShaogang GongAuthors: Computer Vision Group Queen Mary University of.
A Two-level Pose Estimation Framework Using Majority Voting of Gabor Wavelets and Bunch Graph Analysis J. Wu, J. M. Pedersen, D. Putthividhya, D. Norgaard,
BAGGING ALGORITHM, ONLINE BOOSTING AND VISION Se – Hoon Park.
UNBIASED LOOK AT DATASET BIAS Antonio Torralba Massachusetts Institute of Technology Alexei A. Efros Carnegie Mellon University CVPR 2011.
Sharing Features Between Objects and Their Attributes Sung Ju Hwang 1, Fei Sha 2 and Kristen Grauman 1 1 University of Texas at Austin, 2 University of.
Extending the Multi- Instance Problem to Model Instance Collaboration Anjali Koppal Advanced Machine Learning December 11, 2007.
PSEUDO-RELEVANCE FEEDBACK FOR MULTIMEDIA RETRIEVAL Seo Seok Jun.
Geodesic Flow Kernel for Unsupervised Domain Adaptation Boqing Gong University of Southern California Joint work with Yuan Shi, Fei Sha, and Kristen Grauman.
Visual Categorization With Bags of Keypoints Original Authors: G. Csurka, C.R. Dance, L. Fan, J. Willamowski, C. Bray ECCV Workshop on Statistical Learning.
Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification John Blitzer, Mark Dredze and Fernando Pereira University.
Image Classification over Visual Tree Jianping Fan Dept of Computer Science UNC-Charlotte, NC
Iterative similarity based adaptation technique for Cross Domain text classification Under: Prof. Amitabha Mukherjee By: Narendra Roy Roll no: Group:
Context Neelima Chavali ECE /21/2013. Roadmap Introduction Paper1 – Motivation – Problem statement – Approach – Experiments & Results Paper 2 Experiments.
Finding Clusters within a Class to Improve Classification Accuracy Literature Survey Yong Jae Lee 3/6/08.
1 Bilinear Classifiers for Visual Recognition Computational Vision Lab. University of California Irvine To be presented in NIPS 2009 Hamed Pirsiavash Deva.
Guillaume-Alexandre Bilodeau
Data Mining, Neural Network and Genetic Programming
Learning Mid-Level Features For Recognition
Cross Domain Distribution Adaptation via Kernel Mapping
Can Computer Algorithms Guess Your Age and Gender?
Video Summarization via Determinantal Point Processes (DPP)
Finding Clusters within a Class to Improve Classification Accuracy
CS 1674: Intro to Computer Vision Scene Recognition
Bilinear Classifiers for Visual Recognition
Brief Review of Recognition + Context
[Figure taken from googleblog
Presentation transcript:

Overcoming Dataset Bias: An Unsupervised Domain Adaptation Approach Boqing Gong University of Southern California Joint work with Fei Sha and Kristen Grauman 1

Motivation Vision datasets Key to computer vision research Instrumental to benchmark different algorithms Recent trends Growing fast in scale: Gbytes to web-scale Providing better coverage: objects, scenes, activities, etc 2

Object recognition Many factors affect visual appearance Fundamentally, have to cover an exponentially large space Example 3 Limitation: datasets are biased Images from [Saenko et al.’10]. Amazon online catalogueOffice Monitor from two datasets/domains

Consequence: dataset-specific classifier 4 TRAINTEST Poor cross-dataset generalization Datasets have different underlying distributions Classifiers overfit to datasets’ idiosyncrasies Images from [Saenko et al.’10].

Mismatch is common Object recognition Video analysis Pedestrian detection 5 [Torralba & Efros’11, Perronnin et al.’10] [Duan et al.’09, 10] [Dollár et al.’09]

Our solution: unsupervised domain adaptation Setup Source domain (labeled dataset) Target domain (unlabeled dataset) Objective Train classification model to work well on the target 6 The two distributions are not the same! ?

Our algorithms Geodesic flow kernel (GFK): Learning domain-invariant representation Mitigating domain idiosyncrasy Landmarks Bridging two domains to have similar distributions Training discriminatively w.r.t. target without using labels 7 T S [Gong, Shi, Sha, Grauman, CVPR’12] [Gong, Sha, Grauman, ICML’13]

Correcting sample bias Ex. [Shimodaira’00, Huang et al.’06, Bickel et al.’07] Assumption: marginal distributions are the only difference Learning transductively Ex. [Bergamo & Torresani’10, Bruzzone & Marconcini’10] Assumption: shift classifiers via high-confident predictions Learning a shared representation Ex. [Daumé III’07, Pan et al.’09, Gopalan et al.’11] Assumption: a latent space exists s.t. Ps(X,Y)=Pt(X,Y) Related work 8

Our algorithms Geodesic flow kernel (GFK): learning domain-invariant representation mitigating domain idiosyncrasy Landmarks Bridging two domains to have similar distributions Training discriminatively w.r.t. target without using labels 9 T S [Gong, Shi, Sha, Grauman, CVPR’12] [Gong, Sha, Grauman, ICML’13]

Assume low-dimensional structure Ex. PCA, Partial Least Squares (source only) Modeling data with linear subspaces 10 Target Source

GFK: Domain-invariant features 11 TargetSource More similar to source.

GFK: Domain-invariant features 12 TargetSource More similar to target.

GFK: Domain-invariant features 13 TargetSource Blend the two.

We define the geodesic flow kernel (GFK): Advantages Analytically computable Encoded domain-invariant features Broadly applicable: can kernelize many classifiers GFK in closed form 14

Our algorithms Geodesic flow kernel (GFK): learning domain-invariant representation mitigating domain idiosyncrasy Landmarks Bridging two domains to have similar distributions Training discriminatively w.r.t. target without using labels 15 [Gong, Shi, Sha, Grauman, CVPR’12] [Gong, Sha, Grauman, ICML’13]

Landmarks Labeled source instances that are distributed as could be sampled from the target serve as conduit for discriminative loss in target domain 16 Landmarks Target Source

Key steps 17 Landmarks Target Source 1 Identify landmarks at multiple scales.

Key steps 18 2 Construct auxiliary domain adaptation tasks 3 MKL 4

Identifying landmarks Select landmarks such that 1)landmark distribution ≈ target distribution 2)#landmarks are balanced across different categories 19 Select landmarks Landmarks Target

Identifying landmarks 20 Landmarks Target Select landmarks 1)landmark distribution 2)#landmarks are bala Integer P  QP Distribution Category balance

Multi-scale analysis : GFK σ : multi-scale analysis  multiple sets of landmarks 21 Select landmarks such that 1)landmark distribution 2)#landmarks are bala QP

Exemplar images of landmarks 22 Target Source

Use of multiple sets of landmarks 1. Auxiliary task At each scale, 23 Target Source Landmarks Source’ = source - landmarks Target’ = target + landmarks

Use of multiple sets of landmarks 1. Auxiliary task At each scale, Provably easier to solve 24 Target Source Landmarks Source’ = source - landmarks Target’ = target + landmarks

Use of multiple sets of landmarks 2. Discriminative learning, w.r.t. target Train classifiers using labeled landmarks Discriminative loss is measured towards target via landmarks 25 Target Source Landmarks

Experimental study Setup Cross-dataset generalization for object recognition among four domains Re-examine dataset’s values in light of domain adaptation Evaluation Caltech-256AmazonDSLRWebcam [Griffin et al.’07, Saenko et al.’10]

Classification accuracy: baseline 27 Accuracy (%) Source  Target

Classification accuracy: existing methods 28 Accuracy (%) Source  Target

Classification accuracy: GFK (ours) 29 Accuracy (%) Source  Target

Classification accuracy: landmark (ours) 30 Accuracy (%) Source  Target

Cross-dataset generalization [Torralba & Efros’11] 31 Accuracy (%) Caltech-101 generalizes the worst. Performance drop of ImageNet is big. Analyzing datasets in light of domain adaptation Performance drop!

Cross-dataset generalization [Torralba & Efros’11] 32 Accuracy (%) Performance drop becomes smaller! Caltech-101 generalizes the worst (w/ or w/o adaptation). There is nearly no performance drop of ImageNet. Analyzing datasets in light of domain adaptation

Summary Datasets are biased Overcome by domain adaptation (DA) Our solutions: new learning methods for DA Geodesic flow kernel (GFK) Learn a shared representation of domains Landmarks Discriminative loss is measured towards target via landmarks Data collection guided by DA Informative data: those cannot be classified well by adapting from existing datasets 33

34 Target Source

35

Discussion Scalability GFK: eigen-decomposition, Landmarks: QP Overall, polynomial time 36 Target Source Landmarks

Which domain should be used as the source? 37 DSLR Caltech-256 Webcam Amazon

We introduce the Rank of Domains measure: Intuition –Geometrically, how subspaces disagree –Statistically, how distributions disagree 38 Automatically selecting the best

39 Possible sources Our ROD measure Caltech Amazon 0 DSLR 0.26 Webcam 0.05 Caltech-256 adapts the best to Amazon. Accuracy (%) Source  Target Automatically selecting the best