Teaching Machines to Learn by Metaphors Omer Levy & Shaul Markovitch Technion – Israel Institute of Technology.

Slides:

Advertisements

Similar presentations

Explanation-Based Learning (borrowed from mooney et al)

Advertisements

Knowledge Transfer via Multiple Model Local Structure Mapping Jing Gao, Wei Fan, Jing Jiang, Jiawei Han l Motivate Solution Framework Data Sets Synthetic.

Learning visual representations for unfamiliar environments Kate Saenko, Brian Kulis, Trevor Darrell UC Berkeley EECS & ICSI.

Stream flow rate prediction x = weather data f(x) = flow rate.

A Survey on Transfer Learning Sinno Jialin Pan Department of Computer Science and Engineering The Hong Kong University of Science and Technology Joint.

1 Machine Learning: Lecture 3 Decision Tree Learning (Based on Chapter 3 of Mitchell T.., Machine Learning, 1997)

Image classification Given the bag-of-features representations of images from different classes, how do we learn a model for distinguishing them?

CHAPTER 2: Supervised Learning. Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 Learning a Class from Examples.

Combining Inductive and Analytical Learning Ch 12. in Machine Learning Tom M. Mitchell 고려대학교 자연어처리 연구실 한 경 수

EE462 MLCV Lecture 5-6 Object Detection – Boosting Tae-Kyun Kim.

Frustratingly Easy Domain Adaptation

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Classification Dr Eamonn Keogh Computer Science & Engineering Department University of California - Riverside Riverside,CA Who.

Machine Learning: Symbol-Based

The Implicit Mapping into Feature Space. In order to learn non-linear relations with a linear machine, we need to select a set of non- linear features.

What is Learning All about ?  Get knowledge of by study, experience, or being taught  Become aware by information or from observation  Commit to memory.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Learning from Multiple Outlooks Maayan Harel and Shie Mannor ICML 2011 Presented by Minhua Chen.

Introduction to domain adaptation

CS Machine Learning. What is Machine Learning? Adapt to / learn from data  To optimize a performance function Can be used to:  Extract knowledge.

Training and future (test) data follow the same distribution, and are in same feature space.

Learning to Learn By Exploiting Prior Knowledge

Transfer Learning From Multiple Source Domains via Consensus Regularization Ping Luo, Fuzhen Zhuang, Hui Xiong, Yuhong Xiong, Qing He.

Overcoming Dataset Bias: An Unsupervised Domain Adaptation Approach Boqing Gong University of Southern California Joint work with Fei Sha and Kristen Grauman.

Machine Learning Chapter 11. Analytical Learning

Kernel Classifiers from a Machine Learning Perspective (sec ) Jin-San Yang Biointelligence Laboratory School of Computer Science and Engineering.

Machine Learning CSE 681 CH2 - Supervised Learning.

Sampletalk Technology Presentation Andrew Gleibman

Universit at Dortmund, LS VIII

A Word at a Time: Computing Word Relatedness using Temporal Semantic Analysis Kira Radinsky (Technion) Eugene Agichtein (Emory) Evgeniy Gabrilovich (Yahoo!

Transfer Learning Task. Problem Identification Dataset : A Year: 2000 Features: 48 Training Model ‘M’ Testing 98.6% Training Model ‘M’ Testing 97% Dataset.

Kernel Methods A B M Shawkat Ali 1 2 Data Mining ¤ DM or KDD (Knowledge Discovery in Databases) Extracting previously unknown, valid, and actionable.

Modern Topics in Multivariate Methods for Data Analysis.

Transfer Learning Motivation and Types Functional Transfer Learning Representational Transfer Learning References.

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

Introduction to Machine Learning Supervised Learning 姓名 : 李政軒.

Machine Learning Chapter 2. Concept Learning and The General-to-specific Ordering Tom M. Mitchell.

George F Luger ARTIFICIAL INTELLIGENCE 6th edition Structures and Strategies for Complex Problem Solving Machine Learning: Symbol-Based Luger: Artificial.

Machine Learning Chapter 5. Artificial IntelligenceChapter 52 Learning 1. Rote learning rote( โรท ) n. วิถีทาง, ทางเดิน, วิธีการตามปกติ, (by rote จากความทรงจำ.

Overview Concept Learning Representation Inductive Learning Hypothesis

Geodesic Flow Kernel for Unsupervised Domain Adaptation Boqing Gong University of Southern California Joint work with Yuan Shi, Fei Sha, and Kristen Grauman.

Concept learning, Regression Adapted from slides from Alpaydin’s book and slides by Professor Doina Precup, Mcgill University.

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture.

HAITHAM BOU AMMAR MAASTRICHT UNIVERSITY Transfer for Supervised Learning Tasks.

Text Categorization With Support Vector Machines: Learning With Many Relevant Features By Thornsten Joachims Presented By Meghneel Gore.

Iterative similarity based adaptation technique for Cross Domain text classification Under: Prof. Amitabha Mukherjee By: Narendra Roy Roll no: Group:

1 Minimum Error Rate Training in Statistical Machine Translation Franz Josef Och Information Sciences Institute University of Southern California ACL 2003.

Anne Watson Hong Kong  grasp formal structure  think logically in spatial, numerical and symbolic relationships  generalise rapidly and broadly.

MACHINE LEARNING 3. Supervised Learning. Learning a Class from Examples Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

Web-Mining Agents: Transfer Learning TrAdaBoost R. Möller Institute of Information Systems University of Lübeck.

Transfer and Multitask Learning Steve Clanton. Multiple Tasks and Generalization “The ability of a system to recognize and apply knowledge and skills.

Multiple-goal Search Algorithms and their Application to Web Crawling Dmitry Davidov and Shaul Markovitch Computer Science Department Technion, Haifa 32000,

1 Bilinear Classifiers for Visual Recognition Computational Vision Lab. University of California Irvine To be presented in NIPS 2009 Hamed Pirsiavash Deva.

Combining Models Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya.

Support Vector Machine

Machine Learning: Symbol-Based

Machine Learning: Symbol-Based

CS 9633 Machine Learning Concept Learning

Transfer Learning in Astronomy: A New Machine Learning Paradigm

Kernels Usman Roshan.

Non-linear hypotheses

Semi-supervised Learning

Overview of Machine Learning

Course Introduction CSC 576: Data Mining.

Support Vector Machines and Kernels

Usman Roshan CS 675 Machine Learning

IES 511 Machine Learning Dr. Türker İnce (Lecture notes by Prof. T. M

Machine Learning Chapter 2

Machine Learning Chapter 2

Presentation transcript:

Teaching Machines to Learn by Metaphors Omer Levy & Shaul Markovitch Technion – Israel Institute of Technology

Concept Learning by Induction

Few Examples

Transfer Learning Target (New) Source (Original)

Define: Related Concept

Transfer Learning Approaches Common Inductive Bias Common Instances Common Features

Different Feature Space

Example

Example

Example

Common Inductive Bias

Common Inductive Bias

Common Instances

Common Features

New Approach to Transfer Learning

Our Solution: Metaphors

Metaphors Target (New) Source (Original)

Concept Learner Metaphor Learner Source Target +/-

Theorem

The Metaphor Theorem

Redefine Transfer Learning

Metaphor Learning Framework

Concept Learning Framework Search Algorithm Hypothesis Space Evaluation Function Data

Source Target Metaphor Learning Framework Search Algorithm Metaphor Space Evaluation Function

Metaphor Evaluation

Metaphor Spaces

General Few Degrees of Freedom Representation-Specific Bias

Geometric Transformations ЯR

Dictionary-Based Metaphors cheesequeso

Linear Transformations

Which metaphor space should I use?

Automatic Selection of Metaphor Spaces Which metaphor space should I use?

Occam’s Razor Automatic Selection of Metaphor Spaces Which metaphor space should I use?

Structural Risk Minimization Occam’s Razor Automatic Selection of Metaphor Spaces Which metaphor space should I use?

Automatic Selection of Metaphor Spaces

Empirical Evaluation

Reference Methods Baseline Target Only Identity Metaphor Merge State-of-the-Art Frustratingly Easy Domain Adaptation – Daumé, 2007 MultiTask Learning – Caruana, 1997; Silver et al, 2010 TrAdaBoost – Dai et al, 2007

Digits: Negative Image

Digits: Higher Resolution

Wine

Qualitative Results Transfer Learning Task Target Instance Target Sample Size Digits: Negative Image Digits: Higher Resolution

Discussion

Recap Problem: Concept learning with few examples Solution: Metaphors

Recap

What if the concepts are not related?

Metaphors are not a measure of relatedness

Metaphors are not a measure of relatedness Metaphors explain how concepts are related

Vision

Explaining how concepts are related since M E T A P H O R S

Concept Learning by Induction

Few Examples

Approaches Explanation-Based Learning Semi-Supervised Learning Transfer Learning

Explanation-Based Learning Axioms Data Logical Deduction

Semi-Supervised Learning

Transfer Learning

Target (New) Source (Original)

Transfer Learning

Target (New) Source (Original)

Define: Related Concept

Transfer Learning Approaches Common Inductive Bias Common Instances Common Features

Common Inductive Bias

Common Instances

Common Features 1.Perform feature selection on source 2.Use that selection on target

Which definition is better?

Different Feature Space

Example

Example

Example

Common Inductive Bias

Common Inductive Bias

Common Instances

Common Features

Our Solution: Metaphors

Performance with Automatic Selection of Metaphor Spaces Digits: Negative Image

Performance with Automatic Selection of Metaphor Spaces Digits: Negative Image Geometric Transformations Feature Reordering Orthogonal Linear Transformations Orthogonal Quadratic Transformations

Performance with Automatic Selection of Metaphor Spaces Digits: Negative Image Geometric Transformations Feature Reordering Orthogonal Linear Transformations Orthogonal Quadratic Transformations

What if I have more than one source?

Multiple Source Datasets B Я HRZ

B Я H R Z

Я R

Performance with Multiple Source Datasets Latin & Cyrillic

Performance with Multiple Source Datasets Latin & Cyrillic ABCDEFG HIJKLMN OPQRSTU VWXYZ

Performance with Multiple Source Datasets Latin & Cyrillic ABCDEFG HIJKLMN OPQRSTU VWXYZ

Performance with Multiple Source Datasets