Xiaoxiao Shi, Qi Liu, Wei Fan, Philip S. Yu, and Ruixin Zhu

Slides:

Advertisements

Similar presentations

Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao Wei Fan Jing JiangJiawei Han University of Illinois at Urbana-Champaign IBM T. J.

Advertisements

Actively Transfer Domain Knowledge Xiaoxiao Shi Wei Fan Jiangtao Ren Sun Yat-sen University IBM T. J. Watson Research Center Transfer when you can, otherwise.

Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao, Wei Fan, Jing Jiang, Jiawei Han l Motivate Solution Framework Data Sets Synthetic.

Type Independent Correction of Sample Selection Bias via Structural Discovery and Re-balancing Jiangtao Ren 1 Xiaoxiao Shi 1 Wei Fan 2 Philip S. Yu 2 1.

Feature Selection as Relevant Information Encoding Naftali Tishby School of Computer Science and Engineering The Hebrew University, Jerusalem, Israel NIPS.

Foreground Focus: Finding Meaningful Features in Unlabeled Images Yong Jae Lee and Kristen Grauman University of Texas at Austin.

Universal Learning over Related Distributions and Adaptive Graph Transduction Erheng Zhong †, Wei Fan ‡, Jing Peng*, Olivier Verscheure ‡, and Jiangtao.

Robust Object Tracking via Sparsity-based Collaborative Model

Unsupervised Transfer Classification Application to Text Categorization Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong Michigan State University.

IJCAI Wei Zhang, 1 Xiangyang Xue, 2 Jianping Fan, 1 Xiaojing Huang, 1 Bin Wu, 1 Mingjie Liu 1 Fudan University, China; 2 UNCC, USA {weizh,

Self Taught Learning : Transfer learning from unlabeled data Presented by: Shankar B S DMML Lab Rajat Raina et al, CS, Stanford ICML 2007.

Cross Domain Distribution Adaptation via Kernel Mapping Erheng Zhong † Wei Fan ‡ Jing Peng* Kun Zhang # Jiangtao Ren † Deepak Turaga ‡ Olivier Verscheure.

MSRC Summer School - 30/06/2009 Cambridge – UK Hybrids of generative and discriminative methods for machine learning.

Heterogeneous Consensus Learning via Decision Propagation and Negotiation Jing Gao† Wei Fan‡ Yizhou Sun†Jiawei Han† †University of Illinois at Urbana-Champaign.

Supervised Distance Metric Learning Presented at CMU’s Computer Vision Misc-Read Reading Group May 9, 2007 by Tomasz Malisiewicz.

OCFS: Optimal Orthogonal Centroid Feature Selection for Text Categorization Jun Yan, Ning Liu, Benyu Zhang, Shuicheng Yan, Zheng Chen, and Weiguo Fan et.

Semi-Supervised Clustering Jieping Ye Department of Computer Science and Engineering Arizona State University

Dept. of Computer Science & Engineering, CUHK Pseudo Relevance Feedback with Biased Support Vector Machine in Multimedia Retrieval Steven C.H. Hoi 14-Oct,

Cross Validation Framework to Choose Amongst Models and Datasets for Transfer Learning Erheng Zhong ¶, Wei Fan ‡, Qiang Yang ¶, Olivier Verscheure ‡, Jiangtao.

Feature Selection and Its Application in Genomic Data Analysis March 9, 2004 Lei Yu Arizona State University.

Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao† Wei Fan‡ Jing Jiang†Jiawei Han† †University of Illinois at Urbana-Champaign ‡IBM.

Bing LiuCS Department, UIC1 Learning from Positive and Unlabeled Examples Bing Liu Department of Computer Science University of Illinois at Chicago Joint.

Review Rong Jin. Comparison of Different Classification Models  The goal of all classifiers Predicating class label y for an input x Estimate p(y|x)

Relaxed Transfer of Different Classes via Spectral Partition Xiaoxiao Shi 1 Wei Fan 2 Qiang Yang 3 Jiangtao Ren 4 1 University of Illinois at Chicago 2.

Jinhui Tang †, Shuicheng Yan †, Richang Hong †, Guo-Jun Qi ‡, Tat-Seng Chua † † National University of Singapore ‡ University of Illinois at Urbana-Champaign.

(ACM KDD 09’) Prem Melville, Wojciech Gryc, Richard D. Lawrence

Transfer Learning From Multiple Source Domains via Consensus Regularization Ping Luo, Fuzhen Zhuang, Hui Xiong, Yuhong Xiong, Qing He.

Learning with Positive and Unlabeled Examples using Weighted Logistic Regression Wee Sun Lee National University of Singapore Bing Liu University of Illinois,

Watch, Listen and Learn Sonal Gupta, Joohyun Kim, Kristen Grauman and Raymond Mooney -Pratiksha Shah.

Predictive Modeling with Heterogeneous Sources Xiaoxiao Shi 1 Qi Liu 2 Wei Fan 3 Qiang Yang 4 Philip S. Yu 1 1 University of Illinois at Chicago 2 Tongji.

Wei Zhang Akshat Surve Xiaoli Fern Thomas Dietterich.

The Necessity of Combining Adaptation Methods Cognitive Computation Group, University of Illinois Experimental Results Title Ming-Wei Chang, Michael Connor.

1 SUPPORT VECTOR MACHINES İsmail GÜNEŞ. 2 What is SVM? A new generation learning system. A new generation learning system. Based on recent advances in.

Machine Learning Using Support Vector Machines (Paper Review) Presented to: Prof. Dr. Mohamed Batouche Prepared By: Asma B. Al-Saleh Amani A. Al-Ajlan.

Transfer Learning Task. Problem Identification Dataset : A Year: 2000 Features: 48 Training Model ‘M’ Testing 98.6% Training Model ‘M’ Testing 97% Dataset.

Transfer Learning with Applications to Text Classification Jing Peng Computer Science Department.

Kernel Methods A B M Shawkat Ali 1 2 Data Mining ¤ DM or KDD (Knowledge Discovery in Databases) Extracting previously unknown, valid, and actionable.

Partially Supervised Classification of Text Documents by Bing Liu, Philip Yu, and Xiaoli Li Presented by: Rick Knowles 7 April 2005.

Modern Topics in Multivariate Methods for Data Analysis.

Transfer Learning Motivation and Types Functional Transfer Learning Representational Transfer Learning References.

SVM Support Vector Machines Presented by: Anas Assiri Supervisor Prof. Dr. Mohamed Batouche.

Xiangnan Kong,Philip S. Yu Department of Computer Science University of Illinois at Chicago KDD 2010.

A Novel Local Patch Framework for Fixing Supervised Learning Models Yilei Wang 1, Bingzheng Wei 2, Jun Yan 2, Yang Hu 2, Zhi-Hong Deng 1, Zheng Chen 2.

PSEUDO-RELEVANCE FEEDBACK FOR MULTIMEDIA RETRIEVAL Seo Seok Jun.

Geodesic Flow Kernel for Unsupervised Domain Adaptation Boqing Gong University of Southern California Joint work with Yuan Shi, Fei Sha, and Kristen Grauman.

Visual Categorization With Bags of Keypoints Original Authors: G. Csurka, C.R. Dance, L. Fan, J. Willamowski, C. Bray ECCV Workshop on Statistical Learning.

Dual Transfer Learning Mingsheng Long 1,2, Jianmin Wang 2, Guiguang Ding 2 Wei Cheng, Xiang Zhang, and Wei Wang 1 Department of Computer Science and Technology.

Bing LiuCS Department, UIC1 Chapter 8: Semi-supervised learning.

HAITHAM BOU AMMAR MAASTRICHT UNIVERSITY Transfer for Supervised Learning Tasks.

Neural Text Categorizer for Exclusive Text Categorization Journal of Information Processing Systems, Vol.4, No.2, June 2008 Taeho Jo* 報告者 : 林昱志.

COP5992 – DATA MINING TERM PROJECT RANDOM SUBSPACE METHOD + CO-TRAINING by SELIM KALAYCI.

Multiple Instance Learning for Sparse Positive Bags Razvan C. Bunescu Machine Learning Group Department of Computer Sciences University of Texas at Austin.

Guest lecture: Feature Selection Alan Qi Dec 2, 2004.

Image Classification over Visual Tree Jianping Fan Dept of Computer Science UNC-Charlotte, NC

Iterative similarity based adaptation technique for Cross Domain text classification Under: Prof. Amitabha Mukherjee By: Narendra Roy Roll no: Group:

Efficient Semi-supervised Spectral Co-clustering with Constraints

Learning Photographic Global Tonal Adjustment with a Database of Input / Output Image Pairs.

Self-taught Clustering – an instance of Transfer Unsupervised Learning † Wenyuan Dai joint work with ‡ Qiang Yang, † Gui-Rong Xue, and † Yong Yu † Shanghai.

 Effective Multi-Label Active Learning for Text Classification Bishan yang, Juan-Tao Sun, Tengjiao Wang, Zheng Chen KDD’ 09 Supervisor: Koh Jia-Ling Presenter:

SUPERVISED AND UNSUPERVISED LEARNING Presentation by Ege Saygıner CENG 784.

UCSpv: Principled Voting in UCS Rule Populations Gavin Brown, Tim Kovacs, James Marshall.

Semi-Supervised Clustering

Data Driven Attributes for Action Detection

Cross Domain Distribution Adaptation via Kernel Mapping

Introductory Seminar on Research: Fall 2017

Image Segmentation Techniques

Semi-supervised Learning

Knowledge Transfer via Multiple Model Local Structure Mapping

Concave Minimization for Support Vector Machine Classifiers

Semi-Supervised Learning

Presentation transcript:

Xiaoxiao Shi, Qi Liu, Wei Fan, Philip S. Yu, and Ruixin Zhu Transfer Learning on Heterogeneous Feature Spaces via Spectral Transformation Xiaoxiao Shi, Qi Liu, Wei Fan, Philip S. Yu, and Ruixin Zhu

Motivation Standard Supervised Learning 85.5% Training documents (labeled) Test documents (unlabeled) Classifier 85.5%

Huge set of unlabeled documents Motivation How to improve the performance? In Reality… Training (labeled) Huge set of unlabeled documents Labeled data are insufficient! 47.3%

Learning Formulations

Learning from heterogeneous sources Labeled data from other sources Target domain test (unlabeled) ??? Heterogeneous datasets: Different data distributions: P(xtrain) and P(xtest) are different Different outputs: ytrain and ytest are different Different feature spaces: xtrain and xtest are different 3/18

Some Applications of Transfer Learning WiFi-based localization tracking [Pan et al'08] Collaborative Filtering [Pan et al'10] Activity Recognition [Zheng et al'09] Text Classification [Dai et al'07] Sentiment Classification [Blitzer et al‘07] Image Categorization [Shi et al’10] … …

Issues Different data distributions: P(xtrain) and P(xtest) are different focuses more on Chicago local news focuses more on global news focuses more on scientific/objective documents

Issues Different outputs: ytrain and ytest are different Wikipedia ODP Yahoo!

Issues Different feature spaces (the focus on the paper) Drug efficacy tests: Physical properties Topological properties Image Classification Wavelet features Color histogram

Unify different feature spaces Different number of features; different meanings of the features, no common feature, no overlap. Projection-based approach HeMap Find a projected space where (1) the source and target data are similar in distribution; (2) the original structure (separation) of each of the dataset is preserved.

Unify different feature spaces via HeMap Optimization objective of HeMap: The linear projection error The linear projection error The difference between the projected data

Unify different feature spaces via HeMap With some derivations, the objective can be reformulated as (more details can be found in the paper):

Algorithm flow of HeMap

Generalized HeMap to handle heterogeneous data (different distributions, outputs and feature spaces)

Unify different distributions and outputs Clustering based sample selection [Shi etc al,09] Unify different outputs Bayesian like schema

Generalization bound and are domain-specific parameters; is model complexity Principle I: minimize the difference between target and source datasets Principle II: minimize the combined expected error by maintaining the original structure (minimize projection error)

Experiments Drug efficacy prediction The dataset is collected by the College of Life Science and Biotechnology of Tongji University, China. It is to predict the efficacy of drug compounds against certain cell lines. The data are generated in two different feature spaces general descriptors: refer to physical properties of compounds drug-like index: refer to simple topological indices of compounds.

Experiments

Experiments Image classification Homer Simpson & Cactus Cartman & Bonsai Superman & CD Homer Simpson & Coin

Experiments

Conclusions Extends the applicability of supervised learning, semi-supervised learning and transfer learning by using heterogeneous data: Different data distributions Different outputs Different feature spaces Unify different feature spaces via linear projection with two principles Maintain the original structure of the data Maximize the similarity of the two data in the projected space