1. Kernel Stick-Breaking Process
D. B. Dunson and J. Park
Discussion led by Qi An
Jan 19th, 2007

2. Outline Motivation Model formulation and properties Prediction rules
Posterior Computation
Examples
Conclusions

3. Motivation
Consider a problem of estimating the conditional density of a response variable using a mixture model, , where Gx is an unknown probability measure indexed by x.
The problem of defining priors for random probability measures on Gx has received increasing attention in recent year. For example, DP, DDP.

4. One model
In DDP, the atoms can vary with x according to a stochastic process while the weights are fixed
Dunson et al propose a model to allow the weights to vary with predictors
while this model lacks reasonable marginalization and updating properties.

5. Model formulation
Introduce a countable sequence of mutually independent random components
The kernel stick-breaking process (KSBP) can be defined as follows:

6. About the model
The model for Gx is a predictor-dependent mixture over an infinite sequence of basis probability measures, Gh* located at Γh.
Bases located close to x and having a smaller index, h, tend to receive higher probability weight.
KSBP accommodates dependency between Gx and Gx’

7. Special cases
If K(x,Γ)=1 for all and Gh*~DP(αG0), it is a stick-breaking mixture of DP.
If K(x,Γ)=1, and , we obtain Gx≡G, with G having a stick-breaking prior.
If and , we obtain a Pitman-Yor process.

8. Properties Let , we can obtain The correlation between measures
First moment
No dependency on V and Γ
Second moment
It can be proven and the value 1 in the limit as x x’
where

9. Alternative representation
The KSBP has an alternative representation
The moments and correlation coefficient has the form

10. Truncation
For stick-breaking Gibbs sampler, we need to make truncation approximation
Author proves that the residual weights decrease exponentially fast in N and an accurate approximation may be obtained for moderate N
The approximated model can be expressed as

11. Prediction rules Consider a special case in which
The model can be equivalently expressed as:

12. Prediction rules
Define and is a subset of the integers between 1 and n
It can be proven that the probability that subjects i and j belong to the same cluster is
The predictive distribution is obtained by marginalization
where and denote the set of possible r-
dimensional subsets of {1,…,s} that include i

13. Posterior Computation
From the prior, we can obtain
1, sample Si
2, sample CSi when Si=0 (assign subject I to a new atom at an occupied location)
3, sample θh

14. 4, sample Vh
5, sample Γh using a Metropolis-Hastings step or Gibbs step if H is a set of discrete potential locations
First sample
and
then, alternate between
(i) Sampling (Aih,Bih) from their conditional distribution
(ii) Updating Vh by sampling from conditional posterior

15. Simulated examples

18. Conclusions
This stick-breaking process is useful in setting in which there is uncertainty in an uncountable collection of probability measures
The process can be applied in predictor dependent clustering, dynamic modeling and spatial data analysis, besides the density regression.
The KSBP formulation can be applied to many tools developed for exchangeable stick-breaking processes with minimal modification.
A predicator dependent urn scheme is obtained, which generalizes the Polya urn scheme