Combining Labeled and Unlabeled Data with Co-Training

Slides:

Advertisements

Similar presentations

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Advertisements

Semi-Supervised Learning Avrim Blum Carnegie Mellon University [USC CS Distinguished Lecture Series, 2008]

Co Training Presented by: Shankar B S DMML Lab

Distributed Machine Learning: Communication, Efficiency, and Privacy Avrim Blum [RaviKannan60] Joint work with Maria-Florina Balcan, Shai Fine, and Yishay.

Semi-Supervised Learning and Learning via Similarity Functions: Two key settings for Data- Dependent Concept Spaces Avrim Blum [NIPS 2008 Workshop on Data.

Semi-Supervised Learning

Multi-View Learning in the Presence of View Disagreement C. Mario Christoudias, Raquel Urtasun, Trevor Darrell UC Berkeley EECS & ICSI MIT CSAIL.

Maria-Florina Balcan Modern Topics in Learning Theory Maria-Florina Balcan 04/19/2006.

ALADDIN Workshop on Graph Partitioning in Vision and Machine Learning Jan 9-11, 2003 Welcome! [Organizers: Avrim Blum, Jon Kleinberg, John Lafferty, Jianbo.

Co-Training and Expansion: Towards Bridging Theory and Practice Maria-Florina Balcan, Avrim Blum, Ke Yang Carnegie Mellon University, Computer Science.

Text Learning Tom M. Mitchell Aladdin Workshop Carnegie Mellon University January 2003.

Improving the Graph Mincut Approach to Learning from Labeled and Unlabeled Examples Avrim Blum, John Lafferty, Raja Reddy, Mugizi Rwebangira Carnegie Mellon.

Unsupervised Models for Named Entity Classification Michael Collins and Yoram Singer Yimeng Zhang March 1 st, 2007.

GRAPHS OF MEANS How is a Graph of Means Constructed? What are Error Bars? How Can a Graph Indicate Statistical Significance?

Semi Supervised Learning Qiang Yang –Adapted from… Thanks –Zhi-Hua Zhou – ople/zhouzh/ –LAMDA.

Maria-Florina Balcan Carnegie Mellon University Margin-Based Active Learning Joint with Andrei Broder & Tong Zhang Yahoo! Research.

Combining Labeled and Unlabeled Data for Multiclass Text Categorization Rayid Ghani Accenture Technology Labs.

Semi-Supervised Learning Using Randomized Mincuts Avrim Blum, John Lafferty, Raja Reddy, Mugizi Rwebangira.

Co-training LING 572 Fei Xia 02/21/06. Overview Proposed by Blum and Mitchell (1998) Important work: –(Nigam and Ghani, 2000) –(Goldman and Zhou, 2000)

Graph-Based Semi-Supervised Learning with a Generative Model Speaker: Jingrui He Advisor: Jaime Carbonell Machine Learning Department

CS Ensembles and Bayes1 Semi-Supervised Learning Can we improve the quality of our learning by combining labeled and unlabeled data Usually a lot.

Maria-Florina Balcan A Theoretical Model for Learning from Labeled and Unlabeled Data Maria-Florina Balcan & Avrim Blum Carnegie Mellon University, Computer.

Improving the Graph Mincut Approach to Learning from Labeled and Unlabeled Examples Avrim Blum, John Lafferty, Raja Reddy, Mugizi Rwebangira.

Semi-supervised Learning Rong Jin. Semi-supervised learning  Label propagation  Transductive learning  Co-training  Active learing.

Information Extraction with Unlabeled Data Rayid Ghani Joint work with: Rosie Jones (CMU) Tom Mitchell (CMU & WhizBang! Labs) Ellen Riloff (University.

Incorporating Unlabeled Data in the Learning Process

Machine Learning Overview Tamara Berg Language and Vision.

Text Classification using SVM- light DSSI 2008 Jing Jiang.

Active Learning An example From Xu et al., “Training SpamAssassin with Active Semi- Supervised Learning”

EMNLP’01 19/11/2001 ML: Classical methods from AI –Decision-Tree induction –Exemplar-based Learning –Rule Induction –TBEDL ML: Classical methods from AI.

1 COMP3503 Semi-Supervised Learning COMP3503 Semi-Supervised Learning Daniel L. Silver.

Ensembles. Ensemble Methods l Construct a set of classifiers from training data l Predict class label of previously unseen records by aggregating predictions.

ISQS 6347, Data & Text Mining1 Ensemble Methods. ISQS 6347, Data & Text Mining 2 Ensemble Methods Construct a set of classifiers from the training data.

MLE’s, Bayesian Classifiers and Naïve Bayes Machine Learning Tom M. Mitchell Machine Learning Department Carnegie Mellon University January 30,

1 A Vital Web-based Resource for ESL/EFL Teachers and Students Presenter: Professor Lyra Riabov Southern New Hampshire University

Artificial Intelligence 8. Supervised and unsupervised learning Japan Advanced Institute of Science and Technology (JAIST) Yoshimasa Tsuruoka.

Downtime Reduction Ideas for a Symposium Presentation and Training

Combining labeled and unlabeled data for text categorization with a large number of categories Rayid Ghani KDD Lab Project.

Advisor : Prof. Sing Ling Lee Student : Chao Chih Wang Date :

Advisor : Prof. Sing Ling Lee Student : Chao Chih Wang Date :

Machine Learning Overview Tamara Berg CS 560 Artificial Intelligence Many slides throughout the course adapted from Svetlana Lazebnik, Dan Klein, Stuart.

Active learning Haidong Shi, Nanyi Zeng Nov,12,2008.

CoCQA : Co-Training Over Questions and Answers with an Application to Predicting Question Subjectivity Orientation Baoli Li, Yandong Liu, and Eugene Agichtein.

COP5992 – DATA MINING TERM PROJECT RANDOM SUBSPACE METHOD + CO-TRAINING by SELIM KALAYCI.

Learning from Labeled and Unlabeled Data Tom Mitchell Statistical Approaches to Learning and Discovery, and March 31, 2003.

Classification using Co-Training

My Wonderful World of Stuff This is a sample slide upload.

Return to Home! Go To Next Slide! Return to Home! Go To Next Slide!

Unsupervised Learning Part 2. Topics How to determine the K in K-means? Hierarchical clustering Soft clustering with Gaussian mixture models Expectation-Maximization.

Data Mining Practical Machine Learning Tools and Techniques

Advanced data mining with TagHelper and Weka

Tim Peake Project Space Ambassador Training

Sample Presentation. Slide 1 Info Slide 2 Info.

app today and share with all your clients!

Classification and Prediction

Semi-Supervised Learning

Computational Learning Theory

Classification and Prediction

Unsupervised Learning II: Soft Clustering with Gaussian Mixture Models

Computational Learning Theory

Subject selection for

Home loan for $185,000 for 30 years at 8.5%

Data Science in Industry

How To Extend the Training Data

Basics of ML Rohan Suri.

Unsupervised Learning: Clustering

Semi-Supervised Learning and Active Learning

Semi-Supervised Learning

One Set of Styles Connected to As Many Pages as You Want!!!

There is only a small amount of space for people to live because there are a lot of mountains and very little flat space to build homes on. The cities.

Presentation transcript:

Combining Labeled and Unlabeled Data with Co-Training Avrim Blum Tom Mitchell Madison, 1998

Semi-Supervised Learning Most of ML assumes labeled training data But labels can be expensive to maintain Semi-supervised learning: Make use of unlabeled data in combination of small amount of labeled data

Some Co-Training Slides from a Tutorial by Avrim Blum

Co-training Many problems have two different sources of info you can use to determine label. E.g., classifying webpages: can use words on page or words on links pointing to the page. My Advisor Prof. Avrim Blum x2- Text info x1- Link info x - Link info & Text info

Co-training Idea: Use small labeled sample to learn initial rules. E.g., “my advisor” pointing to a page is a good indicator it is a faculty home page. E.g., “I am teaching” on a page is a good indicator it is a faculty home page. my advisor

Co-training Idea: Use small labeled sample to learn initial rules. E.g., “my advisor” pointing to a page is a good indicator it is a faculty home page. E.g., “I am teaching” on a page is a good indicator it is a faculty home page. Then look for unlabeled examples where one rule is confident and the other is not. Have it label the example for the other. Training 2 classifiers, one on each type of info. Using each to help train the other. hx1,x2i

Results: webpages 12 labeled examples, 1000 unlabeled

Sample Run

Co-Training Theorems [BM98] if x1, x2 are independent given the label, and if have alg that is robust to noise, then can learn from an initial “weakly-useful” rule plus unlabeled data. Faculty home pages Faculty with advisees “My advisor”