Transfer Learning in Astronomy: A New Machine Learning Paradigm

Slides:



Advertisements
Similar presentations
Applications of one-class classification
Advertisements

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California
Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao, Wei Fan, Jing Jiang, Jiawei Han l Motivate Solution Framework Data Sets Synthetic.
1 Semi-supervised learning for protein classification Brian R. King Chittibabu Guda, Ph.D. Department of Computer Science University at Albany, SUNY Gen*NY*sis.
Machine learning continued Image source:
Correlation Aware Feature Selection Annalisa Barla Cesare Furlanello Giuseppe Jurman Stefano Merler Silvano Paoli Berlin – 8/10/2005.
Learning on Probabilistic Labels Peng Peng, Raymond Chi-wing Wong, Philip S. Yu CSE, HKUST 1.
Relational Learning with Gaussian Processes By Wei Chu, Vikas Sindhwani, Zoubin Ghahramani, S.Sathiya Keerthi (Columbia, Chicago, Cambridge, Yahoo!) Presented.
Nov 4, Detection, Classification and Tracking of Targets in Distributed Sensor Networks Presented by: Prabal Dutta Dan Li, Kerry Wong,
Predictive Automatic Relevance Determination by Expectation Propagation Yuan (Alan) Qi Thomas P. Minka Rosalind W. Picard Zoubin Ghahramani.
1 Jun Wang, 2 Sanjiv Kumar, and 1 Shih-Fu Chang 1 Columbia University, New York, USA 2 Google Research, New York, USA Sequential Projection Learning for.
A Maximum Likelihood Method for Identifying the Components of Eclipsing Binary Stars Jonathan Devor and David Charbonneau Harvard-Smithsonian Center for.
Selective Sampling on Probabilistic Labels Peng Peng, Raymond Chi-Wing Wong CSE, HKUST 1.
Training and future (test) data follow the same distribution, and are in same feature space.
Learning to Learn By Exploiting Prior Knowledge
Transfer Learning From Multiple Source Domains via Consensus Regularization Ping Luo, Fuzhen Zhuang, Hui Xiong, Yuhong Xiong, Qing He.
1 What is learning? “Learning denotes changes in a system that... enable a system to do the same task more efficiently the next time.” –Herbert Simon “Learning.
Active Learning for Class Imbalance Problem
Processing of large document collections Part 2 (Text categorization) Helena Ahonen-Myka Spring 2006.
NAOKI YASUDA, MAMORU DOI (UTOKYO), AND TOMOKI MOROKUMA (NAOJ) SN Survey with HSC.
Query on Cosmic Dark Energy by Error Analyses of Type Ia Supernova Qiu-he Peng (Depart. of Astronomy, Nanjing University)
Boris Babenko Department of Computer Science and Engineering University of California, San Diego Semi-supervised and Unsupervised Feature Scaling.
Machine Learning CSE 681 CH2 - Supervised Learning.
Fuzzy Entropy based feature selection for classification of hyperspectral data Mahesh Pal Department of Civil Engineering National Institute of Technology.
Introduction to machine learning and data mining 1 iCSC2014, Juan López González, University of Oviedo Introduction to machine learning Juan López González.
GA-Based Feature Selection and Parameter Optimization for Support Vector Machine Cheng-Lung Huang, Chieh-Jen Wang Expert Systems with Applications, Volume.
EVALUATING TRANSFER LEARNING APPROACHES FOR IMAGE INFORMATION MINING APPLICATIONS Surya Durbha*, Roger King, Nicolas Younan, *Indian Institute of Technology(IIT),
Transfer Learning Task. Problem Identification Dataset : A Year: 2000 Features: 48 Training Model ‘M’ Testing 98.6% Training Model ‘M’ Testing 97% Dataset.
Modern Topics in Multivariate Methods for Data Analysis.
Transfer Learning Motivation and Types Functional Transfer Learning Representational Transfer Learning References.
Transfer Learning for Image Classification Group No.: 15 Group member : Feng Cai Sauptik Dhar Sauptik.
A Novel Local Patch Framework for Fixing Supervised Learning Models Yilei Wang 1, Bingzheng Wei 2, Jun Yan 2, Yang Hu 2, Zhi-Hong Deng 1, Zheng Chen 2.
Geodesic Flow Kernel for Unsupervised Domain Adaptation Boqing Gong University of Southern California Joint work with Yuan Shi, Fei Sha, and Kristen Grauman.
HAITHAM BOU AMMAR MAASTRICHT UNIVERSITY Transfer for Supervised Learning Tasks.
LSST and VOEvent VOEvent Workshop Pasadena, CA April 13-14, 2005 Tim Axelrod University of Arizona.
Guest lecture: Feature Selection Alan Qi Dec 2, 2004.
Searching High-z Supernovae with HSC and WFMOS
Speech Lab, ECE, State University of New York at Binghamton  Classification accuracies of neural network (left) and MXL (right) classifiers with various.
Iterative similarity based adaptation technique for Cross Domain text classification Under: Prof. Amitabha Mukherjee By: Narendra Roy Roll no: Group:
Dimensionality Reduction in Unsupervised Learning of Conditional Gaussian Networks Authors: Pegna, J.M., Lozano, J.A., Larragnaga, P., and Inza, I. In.
1 Baryon Acoustic Oscillations Prospects of Measuring Dark Energy Equation of State with LAMOST Xuelei Chen ( 陳學雷 ) National Astronomical Observatory of.
Machine Learning ICS 178 Instructor: Max Welling Supervised Learning.
Virtual Examples for Text Classification with Support Vector Machines Manabu Sassano Proceedings of the 2003 Conference on Emprical Methods in Natural.
Learning Photographic Global Tonal Adjustment with a Database of Input / Output Image Pairs.
 Effective Multi-Label Active Learning for Text Classification Bishan yang, Juan-Tao Sun, Tengjiao Wang, Zheng Chen KDD’ 09 Supervisor: Koh Jia-Ling Presenter:
Web-Mining Agents: Transfer Learning TrAdaBoost R. Möller Institute of Information Systems University of Lübeck.
Bayesian Template-Based Approach to Classifying SDSS-II Supernovae from 3-Year Survey Brian Connolly Photometric Supernova ID Workshop 3/16/12.
Overview G. Jogesh Babu. R Programming environment Introduction to R programming language R is an integrated suite of software facilities for data manipulation,
1 C.A.L. Bailer-Jones. Machine Learning. Data exploration and dimensionality reduction Machine learning, pattern recognition and statistical data modelling.
New Trends In Machine Learning and Data Science Ricardo Vilalta Dept
Learning Deep Generative Models by Ruslan Salakhutdinov
CEE 6410 Water Resources Systems Analysis
Data Mining, Neural Network and Genetic Programming
A photometric method to classify high-z supernovae found with HSC
Cross Domain Distribution Adaptation via Kernel Mapping
CSE 4705 Artificial Intelligence
Erasmus University Rotterdam
Alan Qi Thomas P. Minka Rosalind W. Picard Zoubin Ghahramani
Introductory Seminar on Research: Fall 2017
Machine Learning Basics
Using Transductive SVMs for Object Classification in Images
Classification of GAIA data
INF 5860 Machine learning for image classification
Learning Probabilistic Graphical Models Overview Learning Problems.
Multivariate Methods Berlin Chen
Multivariate Methods Berlin Chen, 2005 References:
Topological Signatures For Fast Mobility Analysis
Jia-Bin Huang Virginia Tech
MAS 622J Course Project Classification of Affective States - GP Semi-Supervised Learning, SVM and kNN Hyungil Ahn
Machine Learning: Lecture 5
Presentation transcript:

Transfer Learning in Astronomy: A New Machine Learning Paradigm What is Transfer Learning? What is Domain Adaptation? A Case Study in Supernova Ia Classification

MOTIVATION TRANSFER LEARNING Motivation for transfer learning The goal is to transfer knowledge gathered from previous experience. Also called Inductive Transfer or Learning to Learn. Once a predictive model is built, there are reasons to believe the model will cease to be valid at some point in time. Source Domain Target Domain

Traditional Approach to Classification DB1 DB2 DBn Learning System Learning System Learning System

Transfer Learning DB1 DB2 Source domain DB new Target domain Learning System Learning System Learning System Knowledge

Transfer Learning Scenarios: Labeling in a new domain is costly. DB1 (labeled) DB2 (unlabeled) Classification of Cepheids Classification of Long Period Variable Stars LPV

Transfer Learning Scenarios: 2. Data is outdated. Model created with one survey but a new survey is now available. Survey 1 Survey 2 ? Learning System

MULTITASK LEARNING: Train in Parallel with Combined Architecture Figure obtained from Brazdil, et. Al. Metalearning: Applications to Data Mining, Chapter 7, Springer, 2009.

Knowledge of Parameters Assume prior distribution of parameters Source domain Learn parameters and adjust prior distribution Target domain Learn parameters using the source prior distribution.

Knowledge of Parameters Find coefficients ws using SVMs Find coefficients wT using SVMs initializing the search with ws

Feature Transfer Identify common Features to all tasks

Transfer Learning in Astronomy: A New Machine Learning Paradigm What is Transfer Learning? What is Domain Adaptation? A Case Study in Supernova Ia Classification

Domain Adaptation Source Class 1 Source Class 2 Target

Domain Adaptation Target Class 1 Target Class 2

Domain Adaptation Labeled Source Data Train Source Domain algorithm Target Domain Unlabeled Target Data Test Classifier Predicted Labels

Covariate Shift Covariate Shift: PS(X) ≠ PT(X) , PS(Y|X) = PT(Y|X) Source Class 1 Source Class 2 Target Class 1 Target Class 2

Instance Based Methods Source Class 1 Source Class 2 Target

Instance Based Methods Source Class 1 Source Class 2 Target

Instance Based Methods Source Class 1 Data Source Class 2 Data Target Data Source Model Target Model

Instance Based Methods Source Class 1 Source Class 2 Target Class 1 Target Class 2

Instance Based Methods Target Class 1 Target Class 2

Feature-Based Methods When source instances cannot represent the target distribution at all in the parameter space, we can project source and target datasets to common feature space (i.e., we can align both datasets).

Assumptions Domain Adaptation NO O.K. Source Class 1 Source Class 2 Target

Expert labels (few) examples Active Learning DB new Target Domain (unlabeled data) Active Learning What examples need labeling to guarantee a hypothesis with small predictive error? Expert labels (few) examples Learning System

Active Learning Unlabeled Target Point Queried Data Target Class 1

Active Learning Unlabeled Target Point Queried Data Target Class 1

Active Learning Unlabeled Target Point Queried Data Target Class 1

Active Learning Unlabeled Target Point Queried Data Target Class 1

Active Learning Unlabeled Target Point Queried Data Target Class 1

Domain Adaptation + Active Learning Predicted Labels Labeled Source Data Source Domain Domain Adaptation Labeled Target Data Target Domain Unlabeled Target Data Active Learning Train Test Classifier

Transfer Learning in Astronomy: A New Machine Learning Paradigm What is Transfer Learning? What is Domain Adaptation? A Case Study in Supernova Ia Classification

Source and Target using supernova datasets Spectroscopy Enable parameter inference from astronomical data determine the presence of individual chemical elements (spectral lines) infer the distance (redshift) to extragalactic sources Expensive and time-consuming process Unfeasible to obtain measurements for all cataloged objects Photometry: Low resolution counterpart Summarize the intensity of electromagnetic radiation in a handful of broad wavelength windows (filters). Information on individual spectral lines is not accessible.

Overview Automatic SN Classification Using Machine Learning Training data labeled through spectroscopy Test data obtained through photometry Domain Adaptation Active Learning

Brighter nearby objects Sample Selection Bias Training Set Test Set Brighter nearby objects (spectroscopy) Extensive Deeper Surveys (photometry) What is machine learning? Main assumption in supervised learning is not warranted

Automated Classification of Supernova Ia How can we take advantage of existing supernova datasets already classified using spectroscopy to develop an automated classification method using new photometric surveys?

Experimental Setup Data Acquisition: Simulated data stemming from the Supernova Photometric Classification Challenge, traditionally called post-SNPCC Mimic the characteristics of Dark Energy Survey (DES) data Select only objects having at least 3 observed epochs per filter, with at least 1 of them being before -3 days and at least 1 after +24 days since maximum brightness. In each filter, light curve fitting is performed using Gaussian process regression, and the resulting function is sampled with a window of 1 day. No quality cuts are imposed (SNR>0). Datasets: Source Dataset - Labeled (Photometric) Test Dataset – Unlabeled (Spectroscopic) Classes: Supernova Ia Vs Supernova Ib and Supernova Ic

Experimental Setup Dimensionality Reduction : Original : 108 Features For this research : 20 Features, reduced by KPCA Active Learning : 50% Pool, 50% Test 10 Pairs 10 Runs

Results: Accuracy and Purity Accuracy and Precision on Target Data using Domain Adaptation and Active Learning.

Thank you!