1. A Study of the Relationship between SVM and Gabriel Graph ZHANG Wan and Irwin King, Multimedia Information Processing Laboratory, Department of Computer Science & Engineering, The Chinese University of Hong Kong

2. Outline  Discussion  Introduction  Related Background  Experiments  Support Vector Machine(SVM)  Gabriel Graph  Relative Neighborhood Graph  Other Concepts

3. Data Classification  Given training data in different classes(labels known) Predict test data (labels unknown)  Examples  Methods  Decision tree  Face recognition  Speech recognition  Handwritten digits recognition  Neural network  Nearest neighbor

4.  Gabriel graph, Relative neighborhood graph ---- from Computational Geometry  SVM(Support Vector Machine) ----- from Statistical learning theory  introduced by Vapnik in 1990’s  become more and more popular

5. Simple case of SVM Maximize distance between two parallel separating planes a vector determines the orientation of a discriminant plane Distance =

6. SVM and Gabriel graph SVM Convex Hull Gabriel GraphDelaunay triangulation Dual problem(Bennett,2000) Sub problem (Brown,1979) Sub graph(Howe,1978) Relative Neighborhood Graph Sub graph (Kirkpatrick,1985 ) ? -skeleton Special case (Kirkpatrick,1985)

7. Gabriel graph  Definition:  Decision boundary can be constructed from those Gabriel neighbors (p and q) such that p and q are of different classes.

8. Relative neighborhood graph Definition Let :Denotes an open sphere centered at x with radius r, i.e.

9. Summary  -Skeleton(Kirkpatrick,1985) --- a parameterized family of neighborhood graphs The neighborhood is defined,for any fixed,( ) as the intersection of two spheres: And GG(V)=G 1 (V), RNG(V)=G 2 (V).

10. Summary-2 -Skeleton of V, is neighborhood graph with the set of edges defined as follows: a useful feature of this family: its monotonicity with respect to,i.e

11. Gabriel editing Algorithm  Compute the Gabriel graph for the training set.  Visit each node, marking it if all its Gabriel neighbors are of the same class as the current node.  Delete all marked nodes, exiting with the remaining ones as the edited training set.

12. Algorithm for SVM  parameter C, the kernel function and any kernel parameters.  Solve Dual Quadratic problem using an appropriate quadratic programming.  Recover the primal threshold variable b using the support vectors  Obtain the decision function

13. Comparison of time Complexity Where n –- No. of dataset, d –- No. of dimension, -- obtained through an normalization of objective function,which depends on n. RNG GG uncertain SVM Worst caseAverage caseBest CaseMethods Neighbor graph --- more dimension-sensitive SVM --- data-sensitive

14. Experiments & Observations  Libsvm for SVM classification -skeleton algorithm implemented with C++.  Datasets include : Iris dataset, Wine Cultivar dataset, Glass identification data set.  The following is the parameters’ selected for SVM method to obtain an optimal solution. Parameter Iris Data Wine Data Glass Data Kernel Function RBF Error Penalty(C) 2 12 2 7 2 11 Gamma for RBF 2 -9 2 -10 2 -2

15. Experiments & Observations(2) RNG(V) -skeleton( =1.4) GG(V) where =1.4

16. Experiments & Observations(3) SV GG(V) and SV RNG SV~ -skeleton( (1,2) )? where =1.4

17. Conclusion  According to the observations we could improve SVM with Gabriel graph algorithm as follows:  Using the SVM's optimization steps to obtain the solution to the quadratic problem and find the separating plane.  Use the Gabriel graph algorithm to reduce the size of the training data.  Map the data to some other higher, possibly infinite, dimension space and fit an optimal linear classifier in that space.

18. Q&A