1. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 1 Support Vector Machines l Find a linear hyperplane (decision boundary) that will separate the data

2. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 2 Support Vector Machines l One Possible Solution

3. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 3 Support Vector Machines l Another possible solution

4. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 4 Support Vector Machines l Other possible solutions

5. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 5 Support Vector Machines l Which one is better? B1 or B2? l How do you define better?

6. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 6 Support Vector Machines l Find hyperplane maximizes the margin => B1 is better than B2

7. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 7 X1 X2 此條線為由原點沿 方向走 與 垂直的線

8. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 8 Support Vector Machines

9. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 9 w d x1 x2 x1-x2 wx+b = -1 wx+b=1 WX +b =0 wx+b=0 0 此線在沿法向量走 -b/|w| 的距離，與法向量垂直

10. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 10 開始推導 l wx+b=0 這是一條怎樣的線呢？ x a, x b 為線上兩點  wx a +b=0, wx b +b=0 兩式相減 w (x a -x b )=0 線上兩點所形成的 向量與 w 垂直 所以 w 是法向量 l w x s +b=k ， x s 在上面的線 k > 0 l w x c +b=k’ ， x c 在下面的線 k’ < 0 l 方塊分類為 y=1 ， 圓圈分類為 y=-1 l 分類公式 z 是 y=1 類或 y=-1 類依據：

11. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 11 l 調整 w 與 b 得到 l w(x1-x2)=2 l |w|*d=2 l ∴

12. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 12 Support Vector Machines l We want to maximize: –Which is equivalent to minimizing: –But subjected to the following constraints:  This is a constrained optimization problem –Numerical approaches to solve it (e.g., quadratic programming)

13. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 13 l Min f(w)= l Subject to l Change to l Min f(w)= l Subject to

14. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 14

15. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 15 Nonlinear Support Vector Machines l What if decision boundary is not linear?

16. © Tan,Steinbach, Kumar Introduction to Data Mining 4/18/2004 16 Nonlinear Support Vector Machines l Transform data into higher dimensional space