Heterogeneous Consensus Learning via Decision Propagation and Negotiation Jing Gao† Wei Fan‡ Yizhou Sun†Jiawei Han† †University of Illinois at Urbana-Champaign.

Slides:



Advertisements
Similar presentations
Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao Wei Fan Jing JiangJiawei Han University of Illinois at Urbana-Champaign IBM T. J.
Advertisements

Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao, Wei Fan, Jing Jiang, Jiawei Han l Motivate Solution Framework Data Sets Synthetic.
A General Framework for Mining Concept-Drifting Data Streams with Skewed Distributions Jing Gao Wei Fan Jiawei Han Philip S. Yu University of Illinois.
When Efficient Model Averaging Out-Perform Bagging and Boosting Ian Davidson, SUNY Albany Wei Fan, IBM T.J.Watson.
Pseudo-Relevance Feedback For Multimedia Retrieval By Rong Yan, Alexander G. and Rong Jin Mwangi S. Kariuki
Random Forest Predrag Radenković 3237/10
Multi-label Classification without Multi-label Cost - Multi-label Random Decision Tree Classifier 1.IBM Research – China 2.IBM T.J.Watson Research Center.
Data Mining Classification: Alternative Techniques
Semi-supervised Learning Rong Jin. Semi-supervised learning  Label propagation  Transductive learning  Co-training  Active learning.
Large Scale Manifold Transduction Michael Karlen Jason Weston Ayse Erkan Ronan Collobert ICML 2008.
Christine Preisach, Steffen Rendle and Lars Schmidt- Thieme Information Systems and Machine Learning Lab (ISMLL) University of Hildesheim Germany Relational.
Unsupervised Transfer Classification Application to Text Categorization Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong Michigan State University.
Multiple Criteria for Evaluating Land Cover Classification Algorithms Summary of a paper by R.S. DeFries and Jonathan Cheung-Wai Chan April, 2000 Remote.
Bayesian Learning Rong Jin. Outline MAP learning vs. ML learning Minimum description length principle Bayes optimal classifier Bagging.
Sparse vs. Ensemble Approaches to Supervised Learning
On Community Outliers and their Efficient Detection in Information Networks Jing Gao 1, Feng Liang 1, Wei Fan 2, Chi Wang 1, Yizhou Sun 1, Jiawei Han 1.
On Appropriate Assumptions to Mine Data Streams: Analyses and Solutions Jing Gao† Wei Fan‡ Jiawei Han† †University of Illinois at Urbana-Champaign ‡IBM.
Ensemble Learning: An Introduction
Combining Labeled and Unlabeled Data for Multiclass Text Categorization Rayid Ghani Accenture Technology Labs.
Heterogeneous Consensus Learning via Decision Propagation and Negotiation Jing Gao † Wei Fan ‡ Yizhou Sun † Jiawei Han † †University of Illinois at Urbana-Champaign.
Three kinds of learning
Semi-Supervised Clustering Jieping Ye Department of Computer Science and Engineering Arizona State University
Semi-Supervised Learning D. Zhou, O Bousquet, T. Navin Lan, J. Weston, B. Schokopf J. Weston, B. Schokopf Presents: Tal Babaioff.
Cross Validation Framework to Choose Amongst Models and Datasets for Transfer Learning Erheng Zhong ¶, Wei Fan ‡, Qiang Yang ¶, Olivier Verscheure ‡, Jiangtao.
Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao† Wei Fan‡ Jing Jiang†Jiawei Han† †University of Illinois at Urbana-Champaign ‡IBM.
Sparse vs. Ensemble Approaches to Supervised Learning
Distributed Representations of Sentences and Documents
CS Ensembles and Bayes1 Semi-Supervised Learning Can we improve the quality of our learning by combining labeled and unlabeled data Usually a lot.
Semi-supervised Learning Rong Jin. Semi-supervised learning  Label propagation  Transductive learning  Co-training  Active learing.
EVENT IDENTIFICATION IN SOCIAL MEDIA Hila Becker, Luis Gravano Mor Naaman Columbia University Rutgers University.
Ensemble Learning (2), Tree and Forest
Relaxed Transfer of Different Classes via Spectral Partition Xiaoxiao Shi 1 Wei Fan 2 Qiang Yang 3 Jiangtao Ren 4 1 University of Illinois at Chicago 2.
Evaluating Performance for Data Mining Techniques
Transfer Learning From Multiple Source Domains via Consensus Regularization Ping Luo, Fuzhen Zhuang, Hui Xiong, Yuhong Xiong, Qing He.
Machine Learning1 Machine Learning: Summary Greg Grudic CSCI-4830.
Graph-based Consensus Maximization among Multiple Supervised and Unsupervised Models Jing Gao 1, Feng Liang 2, Wei Fan 3, Yizhou Sun 1, Jiawei Han 1 1.
LOGO Ensemble Learning Lecturer: Dr. Bo Yuan
Xiaoxiao Shi, Qi Liu, Wei Fan, Philip S. Yu, and Ruixin Zhu
Mining Social Network for Personalized Prioritization Language Techonology Institute School of Computer Science Carnegie Mellon University Shinjae.
On Node Classification in Dynamic Content-based Networks.
Combining multiple learners Usman Roshan. Bagging Randomly sample training data Determine classifier C i on sampled data Goto step 1 and repeat m times.
Today Ensemble Methods. Recap of the course. Classifier Fusion
Exploiting Context Analysis for Combining Multiple Entity Resolution Systems -Ramu Bandaru Zhaoqi Chen Dmitri V.kalashnikov Sharad Mehrotra.
Graph-based Text Classification: Learn from Your Neighbors Ralitsa Angelova , Gerhard Weikum : Max Planck Institute for Informatics Stuhlsatzenhausweg.
PSEUDO-RELEVANCE FEEDBACK FOR MULTIMEDIA RETRIEVAL Seo Seok Jun.
Finding Top-k Shortest Path Distance Changes in an Evolutionary Network SSTD th August 2011 Manish Gupta UIUC Charu Aggarwal IBM Jiawei Han UIUC.
1 LinkClus: Efficient Clustering via Heterogeneous Semantic Links Xiaoxin Yin, Jiawei Han Univ. of Illinois at Urbana-Champaign Philip S. Yu IBM T.J. Watson.
Advisor : Prof. Sing Ling Lee Student : Chao Chih Wang Date :
Consensus Extraction from Heterogeneous Detectors to Improve Performance over Network Traffic Anomaly Detection Jing Gao 1, Wei Fan 2, Deepak Turaga 2,
Classification Ensemble Methods 1
KAIST TS & IS Lab. CS710 Know your Neighbors: Web Spam Detection using the Web Topology SIGIR 2007, Carlos Castillo et al., Yahoo! 이 승 민.
Classification using Co-Training
Enhanced hypertext categorization using hyperlinks Soumen Chakrabarti (IBM Almaden) Byron Dom (IBM Almaden) Piotr Indyk (Stanford)
Tree and Forest Classification and Regression Tree Bagging of trees Boosting trees Random Forest.
Machine learning & object recognition Cordelia Schmid Jakob Verbeek.
Combining Models Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya.
Exploring Social Tagging Graph for Web Object Classification
Semi-Supervised Clustering
Sofus A. Macskassy Fetch Technologies
Constrained Clustering -Semi Supervised Clustering-
Cross Domain Distribution Adaptation via Kernel Mapping
A Consensus-Based Clustering Method
RankClus: Integrating Clustering with Ranking for Heterogeneous Information Network Analysis Yizhou Sun, Jiawei Han, Peixiang Zhao, Zhijun Yin, Hong Cheng,
Data Mining Practical Machine Learning Tools and Techniques
Learning with information of features
Jianping Fan Dept of Computer Science UNC-Charlotte
Design of Hierarchical Classifiers for Efficient and Accurate Pattern Classification M N S S K Pavan Kumar Advisor : Dr. C. V. Jawahar.
Knowledge Transfer via Multiple Model Local Structure Mapping
Semi-Supervised Learning
Advisor: Dr.vahidipour Zahra salimian Shaghayegh jalali Dec 2017
Presentation transcript:

Heterogeneous Consensus Learning via Decision Propagation and Negotiation Jing Gao† Wei Fan‡ Yizhou Sun†Jiawei Han† †University of Illinois at Urbana-Champaign ‡IBM T. J. Watson Research Center KDD’09 Paris, France

2/24 Information Explosion Fan Site Descriptions Pictures Videos Not only at scale, but also at available sources! Blogs descriptions reviews

3/24 Multiple Source Classification Image CategorizationLike? Dislike?Research Area images, descriptions, notes, comments, albums, tags……. movie genres, cast, director, plots……. users viewing history, movie ratings… publication and co- authorship network, published papers, …….

4/24 Model Combination helps! Some areas share similar keywords People may publish in relevant but different areas There may be cross- discipline co-operations supervised unsupervised Supervised or unsupervised

5/24 Motivation Multiple sources provide complementary information –We may want to use all of them to derive better classification solution Concatenation of information sources is impossible –Information sources have different formats –May only have access to classification or clustering results due to privacy issues Ensemble of supervised and unsupervised models –Combine their outputs on the same set of objects –Derive a consolidated solution –Reduce errors made by individual models –More robust and stable

6/24 Consensus Learning

7/24 Related Work Ensemble of Classification Models –Bagging, boosting, …… –Focus on how to construct and combine weak classifiers Ensemble of Clustering Models –Derive a consolidated clustering solution Semi-supervised (transductive) learning Link-based classification –Use link or manifold structure to help classification –One unlabeled source Multi-view learning –Construct a classifier from multiple sources

8/24 Problem Formulation Principles –Consensus: maximize agreement among supervised and unsupervised models –Constraints: Label predictions should be close to the outputs of the supervised models Objective function ConsensusConstraints NP-hard!

9/24 Methodology Step 1: Group-level predictions Step 2: Combine multiple models using local weights How to propagate and negotiate? How to compute local model weights?

10/24 Group-level Predictions (1) Groups: –similarity: percentage of common members –initial labeling: category information from supervised models

11/24 Group-level Predictions (2) Principles –Conditional probability estimates smooth over the graph –Not deviate too much from the initial labeling [ ] [ ] Labeled nodes Unlabeled nodes

12/24 Local Weighting Scheme (1) Principles –If M makes more accurate prediction on x, M’s weight on x should be higher Difficulties –“unsupervised” model combination—cannot use cross-validation

13/24 Local Weighting Scheme (2) Method –Consensus To compute M i ’s weight on x, use M 1,…, M i-1, M i+1, …,M r as the true model, and compute the average accuracy Use consistency in x’s neighbors’ label predictions between two models to approximate accuracy –Random Assign equal weights to all the models consensusrandom

14/24 Algorithm and Time Complexity Compute similarity and local consistency for each pairs of groups for each group iterate f steps Compute probability estimates based on the weighted average of neighbors Compute local weights for each example for each model Combine models’ predictions using local weights O(s 2 ) O(fcs 2 ) O(rn) linear in the number of examples!

15/24 Experiments-Data Sets 20 Newsgroup –newsgroup messages categorization –only text information available Cora –research paper area categorization –paper abstracts and citation information available DBLP –researchers area prediction –publication and co-authorship network, and publication content –conferences’ areas are known Yahoo! Movie –user viewing interest analysis (favored movie types) –movie ratings and synopses –movie genres are known

16/24 Experiments-Baseline Methods Single models –20 Newsgroup: logistic regression, SVM, K-means, min-cut –Cora abstracts, citations (with or without a labeled set) –DBLP publication titles, links (with or without labels from conferences) –Yahoo! Movies Movie ratings and synopses (with or without labels from movies) Ensemble approaches –majority-voting classification ensemble –majority-voting clustering ensemble –clustering ensemble on all of the four models

17/24 Experiments-Evaluation Measures Classification Accuracy –Clustering algorithms: map each cluster to the best possible class label (should get the best accuracy the algorithm can achieve) Clustering quality –Normalized mutual information –Get a “true” model from the groudtruth labels –Compute the shared information between the “true” model and each algorithm

18/24 Empirical Results -Accuracy

19/24 Empirical Results-NMI

20/24 Empirical Results- DBLP data

21/24 Empirical Results-Yahoo! Movies

22/24 Empirical Results-Scalability

23/24 Conclusions Summary –We propose to integrate multiple information sources for better classification –We study the problem of consolidating outputs from multiple supervised and unsupervised models –The proposed two-step algorithm solve the problem by propagating and negotiating among multiple models –The algorithm runs in linear time. –Results on various data sets show the improvements Follow-up Work –Algorithm and theory –Applications

24/24 Thanks! Any questions? Office: 2119B