Logistic Regression (Classification Algorithm)

Slides:

Advertisements

Similar presentations

CPSC 502, Lecture 15Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 15 Nov, 1, 2011 Slide credit: C. Conati, S.

Advertisements

Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation Lecture Notes for Chapter 4 Part I Introduction to Data Mining by Tan,

Classification: Definition Given a collection of records (training set ) –Each record contains a set of attributes, one of the attributes is the class.

Single Neuron Hung-yi Lee. Single Neuron … bias Activation function.

Lecture 13 – Perceptrons Machine Learning March 16, 2010.

Logistic Regression Classification Machine Learning.

Middle Term Exam 03/01 (Thursday), take home, turn in at noon time of 03/02 (Friday)

Lecture Notes for Chapter 4 Introduction to Data Mining

x – independent variable (input)

Greg GrudicIntro AI1 Introduction to Artificial Intelligence CSCI 3202 Fall 2007 Introduction to Classification Greg Grudic.

Lecture 5 (Classification with Decision Trees)

Greg GrudicIntro AI1 Introduction to Artificial Intelligence CSCI 3202: The Perceptron Algorithm Greg Grudic.

Logistic Regression 10701/15781 Recitation February 5, 2008 Parts of the slides are from previous years’ recitation and lecture notes, and from Prof. Andrew.

Machine learning Image source:

Machine learning Image source:

Outline Classification Linear classifiers Perceptron Multi-class classification Generative approach Naïve Bayes classifier 2.

Logistic Regression L1, L2 Norm Summary and addition to Andrew Ng’s lectures on machine learning.

Matlab Matlab Sigmoid Sigmoid Perceptron Perceptron Linear Linear Training Training Small, Round Blue-Cell Tumor Classification Example Small, Round Blue-Cell.

ISCG8025 Machine Learning for Intelligent Data and Information Processing Week 2B *Courtesy of Associate Professor Andrew Ng’s Notes, Stanford University.

Classification / Regression Neural Networks 2

Machine Learning Introduction Study on the Coursera All Right Reserved : Andrew Ng Lecturer:Much Database Lab of Xiamen University Aug 12,2014.

Model representation Linear regression with one variable

Andrew Ng Linear regression with one variable Model representation Machine Learning.

Logistic Regression Debapriyo Majumdar Data Mining – Fall 2014 Indian Statistical Institute Kolkata September 1, 2014.

CSE 446 Perceptron Learning Winter 2012 Dan Weld Some slides from Carlos Guestrin, Luke Zettlemoyer.

Nov 23rd, 2001Copyright © 2001, 2003, Andrew W. Moore Linear Document Classifier.

CSC 4510 – Machine Learning Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website:

M Machine Learning F# and Accord.net. Alena Dzenisenka Software architect at Luxoft Poland Member of F# Software Foundation Board of Trustees Researcher.

CS 782 – Machine Learning Lecture 4 Linear Models for Classification  Probabilistic generative models  Probabilistic discriminative models.

Logistic Regression Week 3 – Soft Computing By Yosi Kristian.

Machine Learning CUNY Graduate Center Lecture 4: Logistic Regression.

Week 1 - An Introduction to Machine Learning & Soft Computing

The problem of overfitting

Regularization (Additional)

Linear Models for Classification

Logistic Regression. Linear Regression Purchases vs. Income.

Linear Discriminant Analysis and Logistic Regression.

DECISION TREE Ge Song. Introduction ■ Decision Tree: is a supervised learning algorithm used for classification or regression. ■ Decision Tree Graph:

Regress-itation Feb. 5, Outline Linear regression – Regression: predicting a continuous value Logistic regression – Classification: predicting a.

8.3 Empirical Models. Definition An empirical model is based only on data and is used to predict, not explain, a system. An empirical model consists of.

1 Introduction to Machine Learning Chapter 1. cont.

1 Illustration of the Classification Task: Learning Algorithm Model.

CSE 446 Logistic Regression Perceptron Learning Winter 2012 Dan Weld Some slides from Carlos Guestrin, Luke Zettlemoyer.

Machine Learning in CSC 196K

CS 2750: Machine Learning Linear Models for Classification Prof. Adriana Kovashka University of Pittsburgh February 15, 2016.

LECTURE 05: CLASSIFICATION PT. 1 February 8, 2016 SDS 293 Machine Learning.

Transfer and Multitask Learning Steve Clanton. Multiple Tasks and Generalization “The ability of a system to recognize and apply knowledge and skills.

Support Vector Machines Optimization objective Machine Learning.

Learning by Loss Minimization. Machine learning: Learn a Function from Examples Function: Examples: – Supervised: – Unsupervised: – Semisuprvised:

Introduction to Azure Machine Learning and Data Mining algorithms Oleksandr Krakovetskyi CEO, DevRain Solutions PhD, Microsoft Regional

First Lecture of Machine Learning Hung-yi Lee. Learning to say “yes/no” Binary Classification.

Machine Learning & Deep Learning

Lecture 3: Linear Regression (with One Variable)

Classification / Regression Neural Networks 2

Logistic Regression Classification Machine Learning.

Logistic Regression Classification Machine Learning.

Lecture 8 Generalized Linear Models &

Logistic Regression Classification Machine Learning.

The Science of Predicting Outcome

Sigmoid and logistic regression

Classification Boundaries

How do we find the best linear regression line?

Machine Learning Algorithms – An Overview

Logistic Regression Chapter 7.

Logistic Regression Classification Machine Learning.

Logistic Regression Classification Machine Learning.

Linear regression with one variable

Logistic Regression Classification Machine Learning.

Logistic Regression Geoff Hulten.

Presentation transcript:

Logistic Regression (Classification Algorithm)

Classification Problem Email: Spam/Not Spam? Online Transactions: Fraudulent (Yes/No)? Tumor: Malignant/Benign

Classification Problem Email: Spam/Not Spam? Online Transactions: Fraudulent (Yes/No)? Tumor: Malignant/Benign Prediction Task:

Classification Problem Email: Spam/Not Spam? Online Transactions: Fraudulent (Yes/No)? Tumor: Malignant/Benign Prediction Task: This is an example of Binary Classification task. A generalized case of classification task in Multi-class Classification

Applying Linear Regression

Applying Linear Regression hθ(x)

Applying Linear Regression hθ(x)

Problem in Applying Linear Regression Yes(1) No(0) Tumor Size

Problem in Applying Linear Regression Yes(1) No(0) Tumor Size

Problem in Applying Linear Regression Yes(1) No(0) Tumor Size

Problem in Applying Linear Regression Yes(1) No(0) Tumor Size Benign if tumor size lies in this range Malignantif tumor size lies in this range

Problem in Applying Linear Regression

Hypothesis Representation hθ(x) = θTx

Hypothesis Representation g(z) Sigmoid function Logistic function z

Interpretation of Hypothesis Output hθ(x) = P(y=1|x;θ)

Interpretation of Hypothesis Output

Interpretation of Hypothesis Output g(z) z

Decision Boundary

Decision Boundary

Decision Boundary

Decision Boundary

Decision Boundary

Decision Boundary

Learning Task

Cost Function

Cost Function

Logistic Regression Cost Function

Logistic Regression Cost Function

Logistic Regression Cost Function

Logistic Regression Cost Function

Logistic Regression Cost Function

Logistic Regression Cost Function

Logistic Regression Cost Function

Logistic Regression Cost Function

Gradient Descent

Gradient Descent

Gradient Descent Algorithm looks identical to linear regression

Multi-class Classification: One-vs-All Algorithm

Binary vs Multi-class Classification Problem

One-vs-All (one-vs-rest)

One-vs-All (one-vs-rest)

One-vs-All (one-vs-rest)

One-vs-All (one-vs-rest)

One-vs-All Algorithm