A Study of the Relationship between SVM and Gabriel Graph ZHANG Wan and Irwin King, Multimedia Information Processing Laboratory, Department of Computer Science & Engineering, The Chinese University of Hong Kong

Outline  Discussion  Introduction  Related Background  Experiments  Support Vector Machine(SVM)  Gabriel Graph  Relative Neighborhood Graph  Other Concepts

Data Classification  Given training data in different classes(labels known) Predict test data (labels unknown)  Examples  Methods  Decision tree  Face recognition  Speech recognition  Handwritten digits recognition  Neural network  Nearest neighbor

 Gabriel graph, Relative neighborhood graph ---- from Computational Geometry  SVM(Support Vector Machine) from Statistical learning theory  introduced by Vapnik in 1990’s  become more and more popular

Simple case of SVM Maximize distance between two parallel separating planes a vector determines the orientation of a discriminant plane Distance =

SVM and Gabriel graph SVM Convex Hull Gabriel GraphDelaunay triangulation Dual problem(Bennett,2000) Sub problem (Brown,1979) Sub graph(Howe,1978) Relative Neighborhood Graph Sub graph (Kirkpatrick,1985 ) ? -skeleton Special case (Kirkpatrick,1985)

Gabriel graph  Definition:  Decision boundary can be constructed from those Gabriel neighbors (p and q) such that p and q are of different classes.

Relative neighborhood graph Definition Let :Denotes an open sphere centered at x with radius r, i.e.

Summary  -Skeleton(Kirkpatrick,1985) --- a parameterized family of neighborhood graphs The neighborhood is defined,for any fixed,( ) as the intersection of two spheres: And GG(V)=G 1 (V), RNG(V)=G 2 (V).

Summary-2 -Skeleton of V, is neighborhood graph with the set of edges defined as follows: a useful feature of this family: its monotonicity with respect to,i.e

Gabriel editing Algorithm  Compute the Gabriel graph for the training set.  Visit each node, marking it if all its Gabriel neighbors are of the same class as the current node.  Delete all marked nodes, exiting with the remaining ones as the edited training set.

Algorithm for SVM  parameter C, the kernel function and any kernel parameters.  Solve Dual Quadratic problem using an appropriate quadratic programming.  Recover the primal threshold variable b using the support vectors  Obtain the decision function

Comparison of time Complexity Where n –- No. of dataset, d –- No. of dimension, -- obtained through an normalization of objective function,which depends on n. RNG GG uncertain SVM Worst caseAverage caseBest CaseMethods Neighbor graph --- more dimension-sensitive SVM --- data-sensitive

Experiments & Observations  Libsvm for SVM classification -skeleton algorithm implemented with C++.  Datasets include : Iris dataset, Wine Cultivar dataset, Glass identification data set.  The following is the parameters’ selected for SVM method to obtain an optimal solution. Parameter Iris Data Wine Data Glass Data Kernel Function RBF Error Penalty(C) Gamma for RBF

Experiments & Observations(2) RNG(V) -skeleton( =1.4) GG(V) where =1.4

Experiments & Observations(3) SV GG(V) and SV RNG SV~ -skeleton( (1,2) )? where =1.4

Conclusion  According to the observations we could improve SVM with Gabriel graph algorithm as follows:  Using the SVM's optimization steps to obtain the solution to the quadratic problem and find the separating plane.  Use the Gabriel graph algorithm to reduce the size of the training data.  Map the data to some other higher, possibly infinite, dimension space and fit an optimal linear classifier in that space.

Q&A