1. Logistic Regression Week 3 – Soft Computing By Yosi Kristian.

2. Classification o Email: Spam / Not Spam? o Online Transactions: Fraudulent (Yes / No)? o Tumor: Malignant / Benign ? 0: “Negative Class” (e.g., benign tumor) 1: “Positive Class” (e.g., malignant tumor)

3. Using Linear Regression ( fail ) Tumor Size Threshold classifier output at 0.5: If, predict “y = 1” If, predict “y = 0” Tumor Size Malignant ? (Yes) 1 (No) 0 0,5

4. Other Reason Classification: y = 0 or 1 Linear regression can be > 1 or < 0 Logistic Regression: Although it has the term regression, Logistic Regression is actually a Classification Algorithm, the name was giver only for historical reasons.

5. HYPOTHESIS REPRESENTATION

6. Logistic Regression Model Sigmoid function / Logistic function Want 1 0.5 0

7. Interpretation of Hypothesis Output = estimated probability that y = 1 on input x Tell patient that 70% chance of tumor being malignant Example: If “probability that y = 1, given x, parameterized by ”

8. DECISION BOUNDARY

9. Boundaries Logistic regression Suppose predict “ “ if predict “ “ if z 1

10. Decision Boundary x1x1 x2x2 1 23 1 2 3 Predict “ “ if

11. Non-linear decision boundaries x1x1 x2x2 Predict “ “ if x1x1 x2x2 1 1

12. COST FUNCTION

13. Linear Regression Cost function (fail) Linear regression: “non-convex”“convex”

14. Logistic regression cost function If y = 1 1 0

15. Logistic regression cost function If y = 0 1 0

16. SIMPLIFIED COST FUNCTION AND GRADIENT DESCENT

17. Logistic regression cost function

18. Output To fit parameters : To make a prediction given new :

19. Gradient Descent Want : Repeat (simultaneously update all )

20. Gradient Descent Want : (simultaneously update all ) Repeat Algorithm looks identical to linear regression!