Yu-Feng Li 1, James T. Kwok2, Ivor W. Tsang3 and Zhi-Hua Zhou1 A Convex Method for Locating Regions of Interest with Multi-Instance Learning Yu-Feng Li 1, James T. Kwok2, Ivor W. Tsang3 and Zhi-Hua Zhou1 1 LAMDA Group, Nanjing University, China 2 Hong Kong University of Science & Technology, Hong Kong 3 Nanyang Technological University, Singapore {liyf,zhouzh}@lamda.nju.edu.cn, jamesk@cse.ust.hk, IvorTsang@ntu.edu.sg

Multi-Instance Learning MIL [Dietterich et al., AIJ97] attempts to learn from a training set consists of bags each containing many instances - A bag is positive if it contains at least one positive instances; otherwise negative. - The labels of training bags are known, however, the labels of instances in the bags are unknown. Multi-instance learning Traditional supervised learning In MIL, identifying positive instances is an important problem understanding the relation between the bag and input patterns. Figure reprinted from [Zhi-Hua Zhou et al., icml09]

Example: ROI In relevant feedback of CBIR, usually user is only interested in some regions, i.e., regions of interest (ROI), in the images. Locating ROIs could be used in other area, i.e., image screening…

The Problem How to develop an efficient and theoretical supported method for locating the ROIs? …This presentation

Outline Introduction Our Methods Experiments Conclusion

Multi-Instance Learning Introduction Multi-Instance Learning Originated from the research on drug activity prediction [Dietterich et al. AIJ97] Drugs are small molecules working by binding to the target area For molecules qualified to make the drug, one of its shapes could tightly bind to the target area A molecule may have many alternative shapes The difficulty: Biochemists know that whether a molecule is qualified or not, but do not know which shape responses for the qualification Figure reprinted from [Dietterich et al., AIJ97]

Multi-Instance Learning (con’t) Introduction Multi-Instance Learning (con’t) Each shape can be represented by a feature vector, i.e., an instance … [a1, a2, …, am]T [b1, b2, …, bm]T [u1, u2, …, um]T one bag one molecule Thus, a molecule is a bag of instances A bag is positive if it contains at least one positive instance; otherwise it is negative The labels of the training bags are known The labels of the instances in the training bags are unknown Figure reprinted from [Zhi-Hua Zhou et al., icml09]

MIL for Image Analysis Various applications Introduction MIL for Image Analysis Various applications Image categorization [Maron & Ratan, ICML’98; Chen & Wang, JMLR04; Chen et al., PAMI06] Image retrieval [Zhang et al., ICDE’02; Zhou et al., AJCAI’05] Face detection [Viola et al., NIPS’05; Zhang & Viola, NIPS’07] Computer-aided medical diagnosis [Fung et al., NIPS’06] … …

Related works for locating ROIs Introduction Related works for locating ROIs Diverse Density and its variants [Maron & Lozano-Pérez, NIPS98] [Maron & Ratan, ICML98] [Zhang & Goldman, NIPS02] [Ray & Craven, icml05] Effective in locating the ROIs Huge time cost, suffers from local minima CKNN-ROI [zhou et al., AJCAI05] A variation of Citation-kNN [wang & zucker,icml00] Efficient, but based on heuristic SVM based methods Very few focus on locating the ROIs, except MI-SVM [Andrew et al., nips03] Effect and efficient, but still suffers from local minima

Multi-Instance SVM [Andrew et al, nips03] Introduction Multi-Instance SVM [Andrew et al, nips03] Main Idea : Large Margin Principle Positive bag Negative bag Key instance, i.e., roi Optimal Hyperplane

A set of training bags: , where and Introduction Problem Setup A set of training bags: , where and The decision function: The goal is to learn f minimizes the structural risk  is strictly monotonically increasing function, l() is the loss function, C is the regularization parameter

consider and square hinge loss Introduction Problem Setup (cont.) consider and square hinge loss So, (1) becomes where This, however, is a non-convex problem because of the max operator and it may get stuck in local solution. In this paper, we propose a convex-relaxation method to solve such an non-convex problem.

Outline Introduction Our Methods Experiments Conclusion

Instance-level Key Instance SVM Our Methods Instance-level Key Instance SVM By introducing the indicator variables d and s(i,j) λ balances the slack variables from positive and negative bags

Bag-level Key Instance SVM Our Methods Bag-level Key Instance SVM Ins-KI-SVM may involves large number of constraints in optimization. Simply represent the negative bag by mean of its instances [Gartner et al., icml02] [Xu & Frank, pakdd04].

More General Formulation Our Methods More General Formulation It involves Ins-KI-SVM and Bag-KI-SVM as special cases when r = N and , , it reduces to Ins-KI-SVM when r = m and , it reduces to Bag-KI-SVM In this paper, we aim at solving above optimization problem instead!

Generate the most informative label Our Methods Convex relaxation – Intuitional View Our main observation efficient hard ? SVM 1 -1 SVM Note that Each label assignment corresponds to a kind of label-kernel in SVM dual. Learn weights for different label assignments reduces to multiple kernel learning which is convex and efficient in general. Our simple strategy 1 -1 SVM … 1 -1 ? Generate the most informative label

Convex relaxation approach Our Methods Convex relaxation approach Consider the dual Consider the minimax relaxation [Li et al., AISTATS’09] Multiple Kernel Learning

Convex relaxation approach (cont.) Our Methods Convex relaxation approach (cont.) Exponential number of base kernels…. Too expensive Cutting plane algorithm Adaptive SimpleMKL How?

Find the most violated d Our Methods Find the most violated d To find the most violated d, we need to solve the following maximization problem Rewritten as It is a concave QP, and could not be solved efficiently… However, the cutting plane method only requires to add a violated constraint at each iteration Approximated by Inf-norm Hence, we propose a simple and efficient method for finding a good approximation of the most violated d Linear Problem, can be solved by sorting

Outline Introduction Our Methods Experiments Conclusion

Experiments Two Kinds of Tasks CBIR image data Benchmark data sets

CBIR image data [zhou et al., AJCAI05] Experiments CBIR image data [zhou et al., AJCAI05] Some statistics Image size: 160 × 160 Feature representation by using SBN [Maron & Ratan ICML98] Evaluation: success rate, i.e., the ratio of the number of successes divided by the total number of relevant images

Experiments CBIR image data (cont.) One-vs-rest, 50 train/ 450 test (10/90 are relevant, respectively), 30 runs

Experiments Some examples

Different number of confident bags Experiments Different number of confident bags Ins-KI-SVM is consistently better than all the other SVM-based methods across all the settings.

Average cpu time at ordinary PC Experiments Speed Average cpu time at ordinary PC Performance per time cost = (1/rank)/time Bag-KI-SVM achieves highest performance per time cost among all svm-type methods.

10 fold Cross-validation Experiments Benchmark datasets 10 fold Cross-validation The performance of KI-SVMs are competitive with all these state-of-the-art methods

Thanks! Conclusion Main contribution: Future work: A convex method for locating ROIs with multi-instance learning Future work: Locating multiple rois or roi groups Thanks!