Logistic Regression Week 3 – Soft Computing By Yosi Kristian.

Slides:

Advertisements

Similar presentations

Single Neuron Hung-yi Lee. Single Neuron … bias Activation function.

Advertisements

CSCI 347 / CS 4206: Data Mining Module 07: Implementations Topic 03: Linear Models.

Machine Learning Week 2 Lecture 1.

Machine Learning Week 1, Lecture 2. Recap Supervised Learning Data Set Learning Algorithm Hypothesis h h(x) ≈ f(x) Unknown Target f Hypothesis Set 5 0.

Logistic Regression Classification Machine Learning.

Artificial Neural Networks

Classification and Prediction: Regression Via Gradient Descent Optimization Bamshad Mobasher DePaul University.

Artificial Intelligence Lecture 2 Dr. Bo Yuan, Professor Department of Computer Science and Engineering Shanghai Jiaotong University

Machine Learning Neural Networks

Artificial Neural Networks

x – independent variable (input)

Greg GrudicIntro AI1 Introduction to Artificial Intelligence CSCI 3202 Fall 2007 Introduction to Classification Greg Grudic.

Greg GrudicIntro AI1 Introduction to Artificial Intelligence CSCI 3202: The Perceptron Algorithm Greg Grudic.

Logistic Regression 10701/15781 Recitation February 5, 2008 Parts of the slides are from previous years’ recitation and lecture notes, and from Prof. Andrew.

CSCI 347 / CS 4206: Data Mining Module 04: Algorithms Topic 06: Regression.

Machine learning Image source:

Neural Networks Lecture 8: Two simple learning algorithms

Data Mining Dr. Chang Liu. What is Data Mining Data mining has been known by many different terms Data mining has been known by many different terms Knowledge.

Where We’re At Three learning rules  Hebbian learning regression  LMS (delta rule) regression  Perceptron classification.

Classification Part 3: Artificial Neural Networks

Logistic Regression L1, L2 Norm Summary and addition to Andrew Ng’s lectures on machine learning.

1 Logistic Regression Adapted from: Tom Mitchell’s Machine Learning Book Evan Wei Xiang and Qiang Yang.

Matlab Matlab Sigmoid Sigmoid Perceptron Perceptron Linear Linear Training Training Small, Round Blue-Cell Tumor Classification Example Small, Round Blue-Cell.

CSC 4510 – Machine Learning Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website:

ISCG8025 Machine Learning for Intelligent Data and Information Processing Week 2B *Courtesy of Associate Professor Andrew Ng’s Notes, Stanford University.

CSC 4510 – Machine Learning Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website:

Machine Learning Introduction Study on the Coursera All Right Reserved : Andrew Ng Lecturer:Much Database Lab of Xiamen University Aug 12,2014.

Model representation Linear regression with one variable

Andrew Ng Linear regression with one variable Model representation Machine Learning.

Learning Theory Reza Shadmehr logistic regression, iterative re-weighted least squares.

CSC 4510 – Machine Learning Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website:

M Machine Learning F# and Accord.net. Alena Dzenisenka Software architect at Luxoft Poland Member of F# Software Foundation Board of Trustees Researcher.

Machine learning system design Prioritizing what to work on

CS 782 – Machine Learning Lecture 4 Linear Models for Classification  Probabilistic generative models  Probabilistic discriminative models.

Linear Discrimination Reading: Chapter 2 of textbook.

Week 1 - An Introduction to Machine Learning & Soft Computing

Linear Classification with Perceptrons

The problem of overfitting

Regularization (Additional)

Logistic Regression (Classification Algorithm)

M Machine Learning F# and Accord.net.

Regress-itation Feb. 5, Outline Linear regression – Regression: predicting a continuous value Logistic regression – Classification: predicting a.

Additive Models ， Trees ， and Related Models Prof. Liqing Zhang Dept. Computer Science & Engineering, Shanghai Jiaotong University.

Machine Learning in CSC 196K

WEEK 2 SOFT COMPUTING & MACHINE LEARNING YOSI KRISTIAN Gradient Descent for Linear Regression.

Data Mining: Concepts and Techniques1 Prediction Prediction vs. classification Classification predicts categorical class label Prediction predicts continuous-valued.

Machine Learning & Deep Learning

Lecture 3: Linear Regression (with One Variable)

A Simple Artificial Neuron

Announcements HW4 due today (11:59pm) HW5 out today (due 11/17 11:59pm)

Classification with Perceptrons Reading:

Logistic Regression Classification Machine Learning.

Machine Learning Week 1.

Logistic Regression Classification Machine Learning.

Lecture 8 Generalized Linear Models &

Logistic Regression Classification Machine Learning.

Overfitting and Underfitting

Classification Boundaries

Support Vector Machine I

Machine Learning Algorithms – An Overview

Logistic Regression Chapter 7.

Logistic Regression Classification Machine Learning.

Statistics 350 Lecture 12.

Multiple features Linear Regression with multiple variables

Multiple features Linear Regression with multiple variables

Logistic Regression Classification Machine Learning.

Linear regression with one variable

Logistic Regression Classification Machine Learning.

Logistic Regression Geoff Hulten.

Presentation transcript:

Logistic Regression Week 3 – Soft Computing By Yosi Kristian.

Classification o Spam / Not Spam? o Online Transactions: Fraudulent (Yes / No)? o Tumor: Malignant / Benign ? 0: “Negative Class” (e.g., benign tumor) 1: “Positive Class” (e.g., malignant tumor)

Using Linear Regression ( fail ) Tumor Size Threshold classifier output at 0.5: If, predict “y = 1” If, predict “y = 0” Tumor Size Malignant ? (Yes) 1 (No) 0 0,5

Other Reason Classification: y = 0 or 1 Linear regression can be > 1 or < 0 Logistic Regression: Although it has the term regression, Logistic Regression is actually a Classification Algorithm, the name was giver only for historical reasons.

HYPOTHESIS REPRESENTATION

Logistic Regression Model Sigmoid function / Logistic function Want

Interpretation of Hypothesis Output = estimated probability that y = 1 on input x Tell patient that 70% chance of tumor being malignant Example: If “probability that y = 1, given x, parameterized by ”

DECISION BOUNDARY

Boundaries Logistic regression Suppose predict “ “ if predict “ “ if z 1

Decision Boundary x1x1 x2x Predict “ “ if

Non-linear decision boundaries x1x1 x2x2 Predict “ “ if x1x1 x2x2 1 1

COST FUNCTION

Linear Regression Cost function (fail) Linear regression: “non-convex”“convex”

Logistic regression cost function If y = 1 1 0

Logistic regression cost function If y = 0 1 0

SIMPLIFIED COST FUNCTION AND GRADIENT DESCENT

Logistic regression cost function

Output To fit parameters : To make a prediction given new :

Gradient Descent Want : Repeat (simultaneously update all )

Gradient Descent Want : (simultaneously update all ) Repeat Algorithm looks identical to linear regression!