1. August 16, 2015EECS, OSU1 Learning with Ambiguously Labeled Training Data Kshitij Judah Ph.D. student Advisor: Prof. Alan Fern Qualifier Oral Presentation

2. August 16, 2015EECS, OSU2 Outline Introduction and Motivation Problem Definition Learning with Partial Labels Semi-Supervised Learning Multiple-Instance Learning Conclusion

3. August 16, 2015EECS, OSU3 Introduction and Motivation Training Data (D) Learning Algorithm f Loss Function Test Point  Supervised Learning (From CS534 Machine Learning slides)

4. August 16, 2015EECS, OSU4 Introduction and Motivation Obtaining labeled training data is difficult Reasons: Require substantial human effort (Information Retrieval) Require expensive tests (Medical Diagnosis, Remote Sensing) Disagreement among experts (Information Retrieval, Remote Sensing)

5. August 16, 2015EECS, OSU5 Introduction and Motivation Reasons: Specifying a single correct class is not possible but pointing incorrect classes is possible (Medical Diagnosis) Labeling is not possible at instance level (Drug Activity Prediction) Objective: Present the space of learning problems and algorithms that deal with ambiguously labeled training data

6. August 16, 2015EECS, OSU6 Space of Learning Problems Learning Problems Supervised Learning Semi-Supervised Learning Direct Supervision (Labels) Indirect Supervision (No Labels) Reinforcement Learning Probabilistic Labels Deterministic Labels Learning with Partial Labels Session-based Learning Unsupervised Learning Multiple-instance Learning

7. August 16, 2015EECS, OSU7 Problem Definition Let denote an instance space, being domain of the i th feature  Instance is a vector  denotes a discrete valued class variable with domain the set of all classes  Training data is a set of examples and is a vector of length m such that k th entry is 1 if is a possible class of, otherwise 0  In case of unambiguously labeled, exactly one entry in is 1

8. August 16, 2015EECS, OSU8 Problem Definition  In case of ambiguous labeling, more than one entries can be 1  The learning task is to select using D an optimal hypothesis from a hypothesis space  is used to classify a new instance  Optimality refers to expected classification accuracy

9. August 16, 2015EECS, OSU9 Learning with Partial Labels

10. August 16, 2015EECS, OSU10 Problem Definition Each training instance has a set of candidate class labels associated with it is assumed to contain true label of Different from multiple-label learning because each instance belongs to exactly one class Applications: Medical diagnosis, Bioinformatics, information retrieval.

11. August 16, 2015EECS, OSU11 Taxonomy of Algorithms for Learning with Partial Labels Probabilistic Generative Approaches Mixture Models + EM Probabilistic Discriminative Approaches Logistic Regression for Partial Labels

12. August 16, 2015EECS, OSU12 Probabilistic Generative Approaches Learn joint probability distribution can be used to either model data or perform classification learned using and Classification of is performed using Bayes rule: and assigning to class that maximizes

13. August 16, 2015EECS, OSU13 Probabilistic Generative Approaches Some examples of generative models are Gaussian Distribution Mixture Models e.g. Gaussian mixture model Directed Graphical Models e.g. Naïve Bayes, HMMs Undirected Graphical Models e.g. Markov Networks Applications: Robotics, speech recognition, computer vision, forecasting, time series prediction etc.

14. August 16, 2015EECS, OSU14 Mixture Models In a mixture model, the probability density function of is given by: Model parameters are often learned using maximum likelihood estimation (MLE)

15. August 16, 2015EECS, OSU15 Mixture Models In MLE approach, assuming instances in D are independently and identically distributed (i.i.d), the joint density of D is given by The goal is to find parameters that maximizes the log of above quantity

16. August 16, 2015EECS, OSU16 Expectation-Maximization Algorithm (EM) Difficult to optimize - log of sums Incomplete-data likelihood function – Generating component unknown Convert to complete-data likelihood by introducing a random variable which tells, for each, which component generated it If we observe, ( 6 )

17. August 16, 2015EECS, OSU17 Expectation-Maximization Algorithm (EM) We do not however observe directly, it is a hidden variable Use EM to handle hidden variables EM is an iterative algorithm with two basic steps: E-Step: Given current estimate of model parameters and data D, the expected value is calculated where expectation is to the marginal distribution of (i.e. each ):

18. August 16, 2015EECS, OSU18 Expectation-Maximization Algorithm (EM) M-Step: In this, we maximize (8) computed in E-step: Repeat until convergence.

19. August 16, 2015EECS, OSU19 Mixture Models + EM for Partial Labels - set of possible mixture components that could have generated the sample EM is modified so that is restricted to take values from Modified E-Step if

20. August 16, 2015EECS, OSU20 Pros and Cons of Generative Models Pros Clear probabilistic framework Effective when model assumptions are correct Cons Complex, lot of parameters Difficult to specify and verify their correctness Unlabeled data can hurt performance if model is incorrect Parameter estimation is difficult EM gets stuck in local maxima, computationally expensive No guarantee of better classification

21. August 16, 2015EECS, OSU21 Discriminative Approaches Discriminative approaches attempt to directly learn a mapping required for classification Simpler than generative models More focused towards classification or regression tasks Two types: Probabilistic: They learn posterior class probability Non-probabilistic: They learn a classifier Applications - text classification, information retrieval, machine perception, time series prediction, bioinformatics etc.

22. August 16, 2015EECS, OSU22 Logistic Regression (From CS534 Machine Learning slides) Learns the conditional class distribution For two class case {0, 1}, the probabilities are given as: For 0-1 loss function LR is a linear classifier For multiple class case, we learn weight parameters for each class

23. August 16, 2015EECS, OSU23 Maximum Entropy Logistic Regression Learns such that it has maximum entropy while being consistent with the partial labels in the D is close to uniform distribution over classes in candidate set and zero everywhere else Presence of unlabeled data results in with low discrimination capacity Disadvantages: Does not use unlabeled data to enhance learning Instead uses unlabeled data in unfavorable way that decreases discrimination capacity of

24. August 16, 2015EECS, OSU24 Minimum Commitment Logistic Regression Learns such that classes belonging to candidate set are predicted with high probability No preference is made among distributions that satisfy above property Presence of unlabeled data has no effect on discrimination capacity of Disadvantages : Does not use unlabeled data to enhance learning

25. August 16, 2015EECS, OSU25 Self-Consistent Logistic Regression Learns such that the entropy of the distribution is minimized Learning is driven by the predictions that are produced by the learner itself Advantage: Can make use of unlabeled data to enhance learning Disadvantage: If initial estimate is incorrect, the algorithm will go down the wrong path

26. August 16, 2015EECS, OSU26 Semi-Supervised Learning

27. August 16, 2015EECS, OSU27 Problem Definition Semi-supervised learning (SSL) is a special case of learning with partial labels, and For each in, = true class label of For each in, = Usually because unlabeled data is easily available as compared to labeled data e.g. information retrieval SSL approaches makes use of in conjunction with to enhance learning SSL has been applied to many domains like text classification, image classification, audio classification etc.

28. August 16, 2015EECS, OSU28 Taxonomy of Algorithms for Semi-Supervised Learning Probabilistic Generative Approaches Mixture Models + EM Discriminative Approaches Semi-supervised support vector machines (S3VMs) Self Training Co-Training

29. August 16, 2015EECS, OSU29 Mixture Models + EM for SSL Nigam et al. proposed a probabilistic generative framework for document classification Assumptions: (1) Documents are generated by a mixture model (2) One-to-one correspondence between mixture components and classes Probability of a document is given as: was modeled using Naïve Bayes

30. August 16, 2015EECS, OSU30 Mixture Models + EM for SSL is estimated using MLE by maximizing log likelihood of : where EM is used to maximize Applications: Text classification, remote sensing, face orientation discrimination, pose estimation, audio classification

31. August 16, 2015EECS, OSU31 Mixture Models + EM for SSL Problem: Unlabeled data decreased performance Reason: Incorrect modeling assumption One mixture component per class: Class includes multiple sub- topics Solutions: Decrease effect of unlabeled data Correct modeling assumptions: Multiple mixture components per class

32. August 16, 2015EECS, OSU32 SVMs Class -1 Class +1 margin

33. August 16, 2015EECS, OSU33 SVMs Idea: Do following things: A. Maximize margin B. Classify points correctly C. Keep data points outside margin SVM optimization problem:

34. August 16, 2015EECS, OSU34 SVMs A. is achieved by B. and C. are achieved using hinge loss C controls trade off between training error and generalization error From SSL tutorial by Xiaojin Zhu during ICML 2007

35. August 16, 2015EECS, OSU35 S3VMs From SSL tutorial by Xiaojin Zhu during ICML 2007  Also known as Transductive SVMs

36. August 16, 2015EECS, OSU36 S3VMs Idea: Do following things: A. Maximize margin B. correctly classify labeled data C. Keep data points (labeled and unlabeled) outside margin S3VM optimization problem:

37. August 16, 2015EECS, OSU37 S3VMs A. is achieved by B. is achieved using hinge loss C. is achieved for labeled data using hinge loss and for unlabeled data using hat loss From SSL tutorial by Xiaojin Zhu during ICML 2007

38. August 16, 2015EECS, OSU38 S3VMs Pros Same mathematical foundations as SVMs and hence applicable to tasks where SVMs are applicable Cons Optimization is difficult, finding exact solution is NP-Hard Applications: text classification, Bioinformatics, biological entity recognition, image retrieval etc.

39. August 16, 2015EECS, OSU39 Self Training A leaner uses its own predictions to teach itself Self training algorithm: 1.Train a classifier using 2.Use h to predict labels of 3.Add most confidently labeled instances in to 4.Repeat until a stopping criterion is met

40. August 16, 2015EECS, OSU40 Self Training Pros One of the simplest SSL algorithms Can be used with any classifier Cons Errors in initial predictions can build up and affect subsequent learning. A possible solution: identify and remove mislabeled examples from self- labeled training set during each iteration. E.g. SETRED algorithm [Ming Li and Zhi-Hua Zhou] Applications: object detection in images, word sense disambiguation, semantic role labeling, named entity recognition

41. August 16, 2015EECS, OSU41 Co-Training Idea: Two classifiers and collaborate with each other during learning and use disjoint subsets and of feature set and each is assumed to be: Sufficient for the classification task at hand Conditionally independent of the other given class

42. August 16, 2015EECS, OSU42 Co-Training Example: Web page classification can be text present on the web page can be anchor text on pages that link to this page The two feature sets are sufficient for classification and conditionally independent given class of web page because each is generated by two different persons that know the class

43. August 16, 2015EECS, OSU43 Co-Training Co-Training algorithm: 1.Use and to train ; and to train 2.Use to predict labels of 3.Use to predict labels of 4.Add h 1 ‘s most confidently labeled instances to 5.Add h 2 ‘s most confidently labeled instances to 6.Repeat until a stopping criterion is met

44. August 16, 2015EECS, OSU44 Co-Training Pros Effective when required feature split exists Can be used with any classifier Cons What if feature split does not exist? Answer: (1) Use random feature split (2) Use full feature set How to select confidently labeled instances? Answer: Use multiple classifiers instead of two and do majority voting. E.g. tri-training uses three classifiers [Zhou and Li], democratic co-learning uses multiple classifiers [Zhou and Goldman] Applications: Web page classification, named entity recognition, semantic role labeling

45. August 16, 2015EECS, OSU45 Multiple-Instance Learning

46. August 16, 2015EECS, OSU46 Problem Definition Set of objects Each object has multiple variants called instances, is called a bag Bag is positive if at least one instance is positive else negative Goal: learn

47. August 16, 2015EECS, OSU47 Why MIL is ambiguous label problem? To learn, learn maps an instance to a class does not contain labels at instance level

48. August 16, 2015EECS, OSU48 Taxonomy of Algorithms for Multiple-Instance Learning Probabilistic Generative Approaches Diverse Density Algorithm (DD) EM Diverse Density (EM-DD) Probabilistic Discriminative Approaches Multiple-instance logistic regression Non-probabilistic Discriminative Approaches Axis-Parallel Rectangles (APR) Multiple-instance SVM (MISVM)

49. August 16, 2015EECS, OSU49 Diverse Density Algorithm (DD) 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 Point B Point A x 1 x 2 2 3 1 2

50. August 16, 2015EECS, OSU50 Diverse Density Algorithm (DD) Diverse density is a probabilistic quantity: Using gradient ascent, maximum diverse density point can be located Cons: Computationally expensive Applications: person identification, stock selection, natural scene classification

51. August 16, 2015EECS, OSU51 EM Diverse Density (EM-DD) Views knowledge of which instance is responsible for the bag label as a missing data problem EM to maximize EM-DD algorithm: E-Step: Given current estimate of target location, use to find the most likely instance responsible for bag label M-Step: Find

52. August 16, 2015EECS, OSU52 EM Diverse Density (EM-DD) Pros Computationally less expensive than DD Avoids local maxima Shown to outperform DD and other MIL algorithms on musk data and artificial real-valued data

53. August 16, 2015EECS, OSU53 Multiple-Instance Logistic Regression Learn a logistic regression classifier for instances Use softmax function to combine the output probabilities to get bag probabilities MIL property satisfied: A bag has high probability of being positive if an instance has high probability of being positive

54. August 16, 2015EECS, OSU54 Axis-Parallel Rectangles (APR) Proposed by Dietterich et al. for drug activity prediction Three algorithms GFS elim-count APR GFS elim-kde APR Iterated discrimination APR

55. August 16, 2015EECS, OSU55 Axis-Parallel Rectangles (APR) x 1 x 2

56. August 16, 2015EECS, OSU56 Iterated Discrimination APR Inside-out algorithm: Start with single positive instance and grow APR to include additional positive instances Three basic steps: Grow: Construct smallest APR that covers at least one instance from each positive bag Discriminate: Select most discriminating features using APR Expand: Expand final APR to improve generalization Works in two phases

57. August 16, 2015EECS, OSU57 Iterated Discrimination APR Grow x 1 x 2

58. August 16, 2015EECS, OSU58 Iterated Discrimination APR Discriminate Greedily select feature with highest discrimination power Discrimination power depends on: How many negative instances are outside How far they are outside Expand Expand the final APR to improve generalization using kernel density estimation

59. August 16, 2015EECS, OSU59 Multiple-instance SVM (MISVM) Based on the idea similar to S3VMs S3VMs: No constraint on how unlabeled data disambiguates MISVM: Maintain MIL constraint All instances from negative bag as negative At least one instance from each positive bag as positive Find maximum margin classifier with MIL constraint satisfied

60. August 16, 2015EECS, OSU60 Multiple-instance SVM (MISVM) MISVM optimization problem:

61. August 16, 2015EECS, OSU61 Multiple-instance SVM (MISVM) Two notions of margin: instance level and bag level Figure from [Andrews et al., NIPS 2002]

62. August 16, 2015EECS, OSU62 Conclusion Motivated problem of learning from ambiguously labeled data Studied space of such learning problems Presented a taxonomy of proposed algorithms for each problem

63. August 16, 2015EECS, OSU63 Thank you