1. Presenter : Lin, Shu-Han Authors : Jeen-Shing Wang, Jen-Chieh Chiang
A cluster validity measure with a hybrid parameter search method for the support vector clustering algorithm
為SVC演算法開發的一個以混合參數搜尋方法為基礎的分群有效性評估模式
Presenter : Lin, Shu-Han
Authors : Jeen-Shing Wang, Jen-Chieh Chiang
PR (2008)

2. Outline Introduction of SVC Motivation Objective Methodology
Experiments
Conclusion
Comments

3. SVC SVC is from SVMs SVMs is supervised clustering technique
Fast convergence
Good generalization performance
Robustness for noise
SVC is unsupervised approach
Data points map to HD feature space using a Gaussian kernel.
Look for smallest sphere enclose data.
Map sphere back to data space to form set of contours.
Contours are treated as the cluster boundaries. 3

4. SVC - Sphere Analysis
To find the minimal enclose sphere with soft margin:
To solve this problem, the Lagrangian function: a
where ||・|| is the Euclidean norm, a is the center of the sphere,
ξj are slack variables that loosen the constraints to allow some
data points lying outside the sphere, C is a constant, and CΣξj
is a penalty term. To solve the optimization problem as in (1),
it is convenient to introduce the Lagrangian function
The first and last terms are the objective function to be minimized.
The second term represents the inequality constraints associated with the slack variables.
The third term is the result of the non-negativity requirements on the values of ξj’s.
第一項與最後一項表示要最小化的目標函數，第二項表示加入差額變數的不等式限制式，第三項是在ξj為非負數的情形下所得到的結果。 4

5. SVC - Sphere Analysis 為了要使lagragian最小化，我們必須將R、a、ξj分別微分，並且設定為零。
Lagragian multiplier unknown 5

6. SVC - Sphere Analysis Karush-Kuhn-Tucker complementarity:
Based on (3)–(7), we can classify each data point into the
following: 1) an internal point; 2) an external point, and 3) a
boundary point in the feature space. Point xj is classified as an
internal point if βj = 0. When 0 < βj < C, the data point xj
is denoted as an SV. SVs lying on the surface of the featurespace
sphere are the so-called boundary points. These SVs can
be used to describe the cluster contour in the input space. When
βj = C, the data points located outside the feature space are
defined as the external points or BSVs. Note that if C ≥ 1, no
external points will exist. Hence, the value of C can be used to
control the existence of external points, namely, outliers, during
the clustering process.
所谓径向基函数 (Radial Basis Function 简称 RBF), 就是某种沿径向对称的标量函数。 通常定义为空间中任一点x到某一中心xc之间欧氏距离的单调函数 , 可记作 k(||x-xc||), 其作用往往是局部的 , 即当x远离xc时函数取值很小。 6

7. Wolfe dual optimization problem
SVC -Sphere Analysis
To find the minimal enclose sphere with soft margin:
C : existence of outliers allowed
Wolfe dual optimization problem
C越大，outlier越少，C等於（或大於）0的時候，就不容許有任何outlier產生了。 a
Bound SV; Outlier 7

8. SVC -Sphere Analysis Mercer kernel Kernel: Gaussian a
The distance (similarity) between x and a:
q : |clusters| & the smoothness/tightness of the cluster boundaries.
Mercer kernel
Kernel: Gaussian
By using the above conditions, (1) can be turned into a Wolfe
dual optimization problem with only variablesβj
所謂徑向基函數(Radial Basis Function簡稱RBF),就是某種沿徑向對稱的標量函數。 通常定義為空間中任一點x到某一中心xc之間歐氏距離的單調函數,可記作k(||x-xc||),其作用往往是局部的,即當x遠離xc時函數取值很小。
q : 如果不是使用高斯來當作kernel函數的話，就不會有q這個值了；但是無論你選什麼當kernel，都會面臨到選擇C這個數值。 Q是變異數的倒數；因為變異數越大的話，代表這個R越大，也代表這個分出來的群會比較寬鬆；相反的那麼也就是說，q越大的話，群就會越緊實。 a
Gaussian function: 8

9. Motivation Drawbacks of Cluster validation Compactness
Different densities or size
As the # of clusters increases, it will monotonic decrease
Separation
Irregular cluster structures 9 9

10. Motivation Their previous study Can handle Different sizes
Different densities
Arbitrary shape
But… 10 10

11. Objectives – A cluster validity method and a parameter search algorithm for SVC
Auto determine the two parameter:
Increasing q lead to increasing # of clusters
C regulates the existence of outliers and overlapping clusters
To Identify the optimal structure
C這個參數代表著對outlier 跟overlap的容忍度
Q是變異數的倒數；因為變異數越大的話，代表這個R越大，也代表這個分出來的群會比較寬鬆；相反的那麼也就是說，q越大的話，群就會越緊實。 11

12. Methodology - Idea N=64, max # of cluster = , 8
q is related to the densities of the clusters
Each cluster structure corresponds to an interval of q
Identify the optimal structure is equivalent to finding the largest interval
N=64, max # of cluster = , 8 12

13. Methodology - Problem How to locate overall search range of q
How to detect outliers/noises
How to identify the largest interval 13

14. Methodology – Locate range of q
Lower bound
Upper bound: Employ K-Means to get clusters, and get variance of each clusters vi
Ascending order: cluster size
意思是如果資料集的range很大的話， q min就會比較小，意思是就要從更小的地方開始搜尋
然後意思是如果群裡面的變異數很大的話，q max就會比較小，意思是如果群都很大的話，那就不用搜尋到這麼多q了
n =3, the biggest 3 clusters’ variance 14

15. Methodology – Outlier Detection
singleton
Set q = qmax , the tightest of q
And we get Copt, remove these
outlier 15

16. Methodology – the largest interval
qopt 16

17. Methodology – the largest interval
Fibonacci search: locate the interval where the cluster structure is the same
Bisection search
n: iteration 17

18. Methodology – Overview
Locate range of q
the largest interval
Outlier Detection 18

19. Experiments - Benchmark and Artificial Examples19

20. Experiments - Outlier
Copt 20

21. Experiments ？ 21

22. Conclusions q C A new measure:
Inspired from the observations of q
Determine the optimal cluster structure with its corresponding range of q and C q C

23. Comments Advantage Drawback Application
Inspired from observation of parameter
Drawback …
Application
SVC
DBSCAN: MinPts / Eps