Group Norm for Learning Latent Structural SVMs

Slides:

Advertisements

Similar presentations

Self-Paced Learning for Semantic Segmentation

Advertisements

Presenter: Duan Tran (Part of slides are from Pedro’s)

Learning Specific-Class Segmentation from Diverse Data M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011 VGG reading group, 29.

Learning Shared Body Plans Ian Endres University of Illinois work with Derek Hoiem, Vivek Srikumar and Ming-Wei Chang.

Learning deformable models Yali Amit, University of Chicago Alain Trouvé, CMLA Cachan.

Curriculum Learning for Latent Structural SVM

Generalizing Backpropagation to Include Sparse Coding David M. Bradley and Drew Bagnell Robotics Institute Carnegie.

Lecture 31: Modern object recognition

Many slides based on P. FelzenszwalbP. Felzenszwalb General object detection with deformable part-based models.

Learning Structural SVMs with Latent Variables Xionghao Liu.

More sliding window detection: Discriminative part-based models Many slides based on P. FelzenszwalbP. Felzenszwalb.

Jun Zhu Dept. of Comp. Sci. & Tech., Tsinghua University This work was done when I was a visiting researcher at CMU. Joint.

Restrict learning to a model-dependent “easy” set of samples General form of objective: Introduce indicator of “easiness” v i : K determines threshold.

Support Vector Machines (SVMs) Chapter 5 (Duda et al.)

1 Transfer Learning Algorithms for Image Classification Ariadna Quattoni MIT, CSAIL Advisors: Michael Collins Trevor Darrell.

Learning of Pseudo-Metrics. Slide 1 Online and Batch Learning of Pseudo-Metrics Shai Shalev-Shwartz Hebrew University, Jerusalem Joint work with Yoram.

Introduction to Machine Learning course fall 2007 Lecturer: Amnon Shashua Teaching Assistant: Yevgeny Seldin School of Computer Science and Engineering.

Learning to Segment from Diverse Data M. Pawan Kumar Daphne KollerHaithem TurkiDan Preston.

Pattern Recognition. Introduction. Definitions.. Recognition process. Recognition process relates input signal to the stored concepts about the object.

Lecture 29: Recent work in recognition CS4670: Computer Vision Noah Snavely.

Group Norm for Learning Latent Structural SVMs Overview Daozheng Chen (UMD, College Park), Dhruv Batra (TTI Chicago), Bill Freeman (MIT), Micah K. Johnson.

Generic object detection with deformable part-based models

Ranking with High-Order and Missing Information M. Pawan Kumar Ecole Centrale Paris Aseem BehlPuneet DokaniaPritish MohapatraC. V. Jawahar.

Introduction to variable selection I Qi Yu. 2 Problems due to poor variable selection: Input dimension is too large; the curse of dimensionality problem.

Loss-based Learning with Weak Supervision M. Pawan Kumar.

Self-paced Learning for Latent Variable Models

Loss-based Learning with Latent Variables M. Pawan Kumar École Centrale Paris École des Ponts ParisTech INRIA Saclay, Île-de-France Joint work with Ben.

Ranking with High-Order and Missing Information M. Pawan Kumar Ecole Centrale Paris Aseem BehlPuneet KumarPritish MohapatraC. V. Jawahar.

计算机学院计算感知 Support Vector Machines. 2 University of Texas at Austin Machine Learning Group 计算感知计算机学院 Perceptron Revisited: Linear Separators Binary classification.

CSC Lecture 8a Learning Multiplicative Interactions Geoffrey Hinton.

Object Detection with Discriminatively Trained Part Based Models

Optimizing Average Precision using Weakly Supervised Data Aseem Behl IIIT Hyderabad Under supervision of: Dr. M. Pawan Kumar (INRIA Paris), Prof. C.V.

Lecture 31: Modern recognition CS4670 / 5670: Computer Vision Noah Snavely.

BING: Binarized Normed Gradients for Objectness Estimation at 300fps

Deformable Part Models (DPM) Felzenswalb, Girshick, McAllester & Ramanan (2010) Slides drawn from a tutorial By R. Girshick AP 12% 27% 36% 45% 49% 2005.

Max-Margin Classification of Data with Absent Features Presented by Chunping Wang Machine Learning Group, Duke University July 3, 2008 by Chechik, Heitz,

Fast Maximum Margin Matrix Factorization for Collaborative Prediction Jason Rennie MIT Nati Srebro Univ. of Toronto.

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Unsupervised Learning: Kmeans, GMM, EM Readings: Barber

Tell Me What You See and I will Show You Where It Is Jia Xu 1 Alexander G. Schwing 2 Raquel Urtasun 2,3 1 University of Wisconsin-Madison 2 University.

Guest lecture: Feature Selection Alan Qi Dec 2, 2004.

Ariadna Quattoni Xavier Carreras An Efficient Projection for l 1,∞ Regularization Michael Collins Trevor Darrell MIT CSAIL.

Final Exam Review CS479/679 Pattern Recognition Dr. George Bebis 1.

Learning from Big Data Lecture 5

Feature Selction for SVMs J. Weston et al., NIPS 2000 오장민 (2000/01/04) Second reference : Mark A. Holl, Correlation-based Feature Selection for Machine.

Convolutional Restricted Boltzmann Machines for Feature Learning Mohammad Norouzi Advisor: Dr. Greg Mori Simon Fraser University 27 Nov

Loss-based Learning with Weak Supervision M. Pawan Kumar.

Spectral Algorithms for Learning HMMs and Tree HMMs for Epigenetics Data Kevin C. Chen Rutgers University joint work with Jimin Song (Rutgers/Palentir),

1 Bilinear Classifiers for Visual Recognition Computational Vision Lab. University of California Irvine To be presented in NIPS 2009 Hamed Pirsiavash Deva.

Discriminative Machine Learning Topic 4: Weak Supervision M. Pawan Kumar Slides available online

Strong Supervision from Weak Annotation: Interactive Training of Deformable Part Models S. Branson, P. Perona, S. Belongie.

1 C.A.L. Bailer-Jones. Machine Learning. Data exploration and dimensionality reduction Machine learning, pattern recognition and statistical data modelling.

Combining Models Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya.

Learning Deep Generative Models by Ruslan Salakhutdinov

Object detection with deformable part-based models

Lecture 07: Soft-margin SVM

Action Recognition in the Presence of One

Classification of Hand-Written Digits Using Scattering Convolutional Network Dongmian Zou Advisor: Professor Radu Balan.

Object Localization Goal: detect the location of an object within an image Fully supervised: Training data labeled with object category and ground truth.

Using Transductive SVMs for Object Classification in Images

Janardhan Rao (Jana) Doppa, Alan Fern, and Prasad Tadepalli

Pattern Recognition CS479/679 Pattern Recognition Dr. George Bebis

Bilinear Classifiers for Visual Recognition

Probabilistic Models with Latent Variables

Lecture 07: Soft-margin SVM

Lecture 08: Soft-margin SVM

Dimension reduction : PCA and Clustering

Deep Learning for Non-Linear Control

Neural networks (3) Regularization Autoencoder

Primal Sparse Max-Margin Markov Networks

An Efficient Projection for L1-∞ Regularization

Presentation transcript:

Group Norm for Learning Latent Structural SVMs Daozheng Chen (UMD, College Park), Dhruv Batra (TTI-Chicago), Bill Freeman (MIT), Micah K. Johnson (GelSight, Inc.) Overview Induce Group Norm Data with complete annotation is rarely ever available. Latent variable models capture interaction between observed data (e.g. gradient histogram image features) latent or hidden variables not observed in the training data (e.g. location of object parts). Parameter estimation involve a difficult non-convex optimization problem (EM, CCCP, self-paced learning) Our goal Estimate model parameters Learn the complexity of latent variable space. Our approach norm for regularization to estimate the parameters of a latent-variable model. Key Contribution Inducing Group Norm w partitioned into P groups; each group corresponds to the parameters of a latent variable state Felzenszwalb et al. car model on the PASCAL VOC 2007 data. Each row is a component of the model. Root filters Part filters Part displacement Component #1 Component #2 norm for regularization Digit Recognition Rotation (Latent Var.) Feature Vector Images Latent Structural SVM Label space Latent Space Joint feature vector Prediction Rule: Learning objective: At group level, the norm behave like norm and induces group sparsity. Within each group, the norm behave like norm and does not promote sparsity. Alternating Coordinate and Subgradient Descent Experiment Digit recognition experiment (following the setup of Kumar et al. NIPS ‘10) MNIST data: binary classification on four difficult digit pairs (1,7), (2,7), (3,8), (8,9) Training data 5,851 - 6,742, and testing data 974 - 1,135 Rotate digit images with angles from -60o to 60o PCA to form 10 dimensional feature vector Rewrite Learning Objective nonconvex convex -60o -48o -36o -24o -12o 0o 12o 24o 36o 48o 60o convex Minimize Upper bound of convex if {hi} is fixed l2 norm of the parameter vectors for different angles over the 4 digit pairs. Select only a few angles, much fewer than 22 angles Angles Not Selected -60o -48o -12o 0o -36o Subgradient Significantly higher accuracy than random sampling. 66% faster than full model with no loss in accuracy!