Learning piecewise linear classifiers via Dirichlet Process Ya Xue and Xuejun Liao 22 April 2005

Piecewise linear classifier A piecewise linear classifier is a classifier that generates a piecewise linear decision boundary. Consider the binary problem where the labeled data We assume that each Dj contains at least one positive sample and one negative sample. Associated with each Dj there is a linear classifier parameterized by wj Assume Dj is primitive and the samples in it are always linearly separable

Piecewise linear classifier (cont’d) It turns out that many of wj will become identical to each other, yielding a small number of distinct representative w’s. Each distinct w represents a linear boundary that separate a subset of the samples in D and the ensemble of w gives us the piecewise linear boundary. We use Dirichlet Process to learn wij , and automatically discover the piecewise linear boundary from wij

Gibbs Sampling of the Posterior Dirichlet Process

Results I 1000 burn-ins and 1000 samples.

Results II

Results M-Logit-DP 200 primary data and 200 auxiliary data. 5 labeled primary data. MCMC: 3000 burn-ins and 1000 samples.