Download presentation
Presentation is loading. Please wait.
1
Reversing Label Switching:
An Interactive Talk Earl Duncan 20 July 2017
2
Introduction Given observed data ๐= ๐ฆ 1 ,โฆ, ๐ฆ ๐ , the ๐พ-component mixture model is expressed as ๐ ~ ๐ ๐ ๐,๐ = ๐=1 ๐ ๐=1 ๐พ ๐ค ๐ ๐ ๐ ๐ฆ ๐ ๐ ๐ where ๐ ๐ denotes unknown component-specific parameter(s), and ๐ ๐ (โ) is the ๐ th component density with corresponding mixture weight ๐ค ๐ subject to: ๐=1 ๐พ ๐ค ๐ =1 and ๐ค ๐ โฅ0 for ๐=1,โฆ,๐พ. Marin, J-M., K. Mengersen, and C. P. Robert โBayesian modelling and inference on mixtures of distributionsโ In Handbook of Statistics edited C. Rao and D. Dey. New York: Springer-Verlag. Earl Duncan BRAG 20 July 2017: Reversing Label Switching 1/12
3
Introduction A latent allocation variable ๐ ๐ is used to identify which component ๐ ๐ belongs to. ๐ ๐ ๐ง ๐ ,๐ ~ ๐ ๐ง ๐ ๐ฆ ๐ ๐ ๐ง ๐ ๐ ๐ |๐ ~ Cat ๐ค 1 , โฆ, ๐ค ๐พ The likelihood is exchangeable meaning that it is invariant to permutations of the labels identifying the mixture components ๐ ๐ ๐ฝ =๐ ๐ ๐ ๐ฝ E.g ๐ ๐ ๐ 1 , ๐ 2 =๐ ๐ ๐ 2 , ๐ for some permutation ๐. If the posterior distribution is invariant to permutations of the labels, this is known as label switching (LS). Earl Duncan BRAG 20 July 2017: Reversing Label Switching 2/12
4
Introduction Consider the conditions:
the prior is (at least partly) exchangeable the sampler is efficient at exploring the posterior hypersurface If condition 1 holds, the posterior will have (up to) ๐พ! symmetric modes. If condition 1 and 2 hold, LS will occur (i.e. the symmetric modes will be observed). No label switching LS between all 3 groups LS between groups 1 and 2 Earl Duncan BRAG 20 July 2017: Reversing Label Switching 3/12
5
Introduction If label switching occurs, the marginal posterior distributions are identical for each component. Therefore, it is impossible to make inferences! K = 3 K = 4 Earl Duncan BRAG 20 July 2017: Reversing Label Switching 4/12
6
Introduction To make sensible inferences, one must first reverse the label switching using a relabelling algorithm. If/when LS occurs, determine the permutations ๐ (1) ,โฆ, ๐ (๐) to undo the label switching. Apply the permutations to ๐, ๐, and inverse permutations to ๐. The function ๐(โ) can be regarded as a generic permutation function which either permutes or relabels. Let ๐=( ๐ 1 , โฆ, ๐ ๐พ ) be a permutation of the index set 1,โฆ,๐พ , let ๐=( ๐ฃ 1 ,โฆ, ๐ฃ ๐พ ) be an arbitrary ๐พ-length vector, and let ๐=( ๐ 1 , ๐ 2 , ๐ 3 ,โฆ) be an arbitrary length vector (or possibly scalar) containing only the values 1,โฆ,๐พ . Then: Permute: ๐ ๐ฃ 1 ,โฆ, ๐ฃ ๐พ = ๐ฃ ๐ 1 ,โฆ, ๐ฃ ๐ ๐พ Relabel: ๐ ๐ 1 , ๐ 2 , ๐ 3 ,โฆ = ๐ ๐ 1 , ๐ ๐ 2 , ๐ ๐ 3 ,โฆ Earl Duncan BRAG 20 July 2017: Reversing Label Switching 5/12
7
Example Example: determining ๐
๐ (๐) can be determined from the posterior estimates ๐ (๐) and a reference allocation vector ๐ โ = ๐ง 1 ,โฆ, ๐ง ๐ ( ๐ โ ) . Earl Duncan BRAG 20 July 2017: Reversing Label Switching 6/12
8
Exercises Consider the following cross-tabulation of reference allocation vector ๐ โ = ๐ ( ๐ โ ) and ๐ (7) (here ๐=200). ๐ โ ๐ (7) Question 1: What should the permutation ๐ (7) be to reverse the labels of a component-specific parameter, ๐ฝ (7) ? Hint: (3, 1, 4, 2) or (2, 4, 1, 3) Answer: ๐ (7) =(3, 1, 4, 2) Earl Duncan BRAG 20 July 2017: Reversing Label Switching 7/12
9
Exercises The second step requires this permutation to be applied to the component-specific parameters and the labels. Question 2: If ๐ (7) =(0.5, 0.1, 0.3, 0.2) and ๐ (7) =(3, 4, 2, 2, 3, โฆ), what are the resulting estimates after relabelling? Recall ๐ (7) =(3, 1, 4, 2). Hint: Permuting: ๐ ๐ฃ 1 ,โฆ, ๐ฃ ๐พ = ๐ฃ ๐ 1 ,โฆ, ๐ฃ ๐ ๐พ Relabelling: ๐ ๐ 1 , ๐ 2 , ๐ 3 ,โฆ = ๐ ๐ 1 , ๐ ๐ 2 , ๐ ๐ 3 ,โฆ Answer: ๐ (7) := ๐ , 0.1, 0.3, 0.2 = 0.3, 0.5, 0.2, 0.1 ๐ (7) := ๐ โ๐ (3, 4, 2, 2, 3, โฆ) = ( ๐ ๐ 1 โ1 , ๐ ๐ 2 โ1 , ๐ ๐ 3 โ1 , ๐ ๐ 4 โ1 , ๐ ๐ 5 โ1 ,โฆ) = ( ๐ 3 โ1 , ๐ 4 โ1 , ๐ 2 โ1 , ๐ 2 โ1 , ๐ 3 โ1 ,โฆ) = (1, 3, 4, 4, 1,โฆ) Earl Duncan BRAG 20 July 2017: Reversing Label Switching 8/12
10
Exercises Question 3: Why is the inverse permutation used to relabel ๐? Hint: Consider drawing values from 3 component densities. Introduce LS, and note how the new values of ๐ฝ and ๐ are recorded. ๐ฝ ? ? ? ? ? ? โฎ โฎ โฎ w/o LS w/ LS Answer: Draw values without LS, then with LS: ๐ฝ โฎ โฎ โฎ โ ๐ LS =(2, 3, 1) โ๐= ๐ LS โ1 =(3, 1, 2) But how are the values of ๐ recorded? Earl Duncan BRAG 20 July 2017: Reversing Label Switching 9/12
11
Exercises Answer continued: ๐ LS =(2, 3, 1) ๐=(3, 1, 2)
๐ฝ โฎ โฎ โฎ โ Draw from Middle, but label it โ1โ Draw from Right, but label it โ2โ Draw from Left, but label it โ3โ 2โ 1 3โ 2 1โ 3 ๐ โฏ โฏ โฎ โฎ โฎ โฎ โฎ โฑ โ โ ๐ โ1 ( ๐ 2 )=( ๐ ๐ 1 โ1 , ๐ ๐ 2 โ1 , ๐ ๐ 3 โ1 , ๐ ๐ 4 โ1 , ๐ ๐ 5 โ1 ,โฆ) =( ๐ 2 โ1 , ๐ 2 โ1 , ๐ 3 โ1 , ๐ 1 โ1 , ๐ 3 โ1 ,โฆ) =(3, 3, 1, 2, 1,โฆ) ๐ โฏ ? ? ? ? ? โฏ โฎ โฎ โฎ โฎ โฎ โฑ Earl Duncan BRAG 20 July 2017: Reversing Label Switching 10/12
12
Comparison of Relabelling Algorithms
Earl Duncan BRAG 20 July 2017: Reversing Label Switching 11/12
13
Questions? Any questions?
Earl Duncan BRAG 20 July 2017: Reversing Label Switching 12/12
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.