On Hierarchical Type Covering Ertem Tuncel 1, Jayanth Nayak 2, and Kenneth Rose 2 1 University of California, Riverside 2 University of California, Santa Barbara

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Types and Typical Sequences Type class : Collection of data points with empirical distribution P, i.e., Conditional type class (a.k.a. V-shell): Collection of data points with empirical conditional distribution V(x|y) w.r.t., i.e.,

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Type Covering Covering of using “distortion spheres” about codevectors, i = 1, 2,…, M, i.e., where Type covering lemma (Csiszár and Körner) states that there exists a covering strategy with

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Uses of Type Covering Source coding: Proof of achievability of the rate-distortion curve Proof of universal achievability of Marton’s error exponent in lossy source coding Results on rate redundancy Guessing subject to distortion: Alice presents Bob fixed guesses until Minimum guessing exponent is achieved by the concatenation of type-covering codebooks in a certain order.

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING 2-Stage Type Covering Weak covering: Implicit in Tuncel and Rose’s work on error exponents in scalable source coding, is the building of a tree of spheres and such that for each, at least one parent-child pair (i, j) simultaneously covers. Two possibilities: Strong covering: Introduced by Kanlis and Narayan, is the covering of using spheres, followed by a covering of each using spheres,.

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Illustration of the Difference STRONG COVERINGWEAK COVERING Every x  is covered by at least one parent-child pair

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Need for Strong vs. Weak The difference was previously overlooked. Even though strong covering was introduced in the context of scalable source coding, weak covering suffices for that purpose. Strong covering, on the other hand, is useful in the generalization of the guessing game, introduced by Merhav, Arikan, and Roth: Alice presents guesses until and then guesses until Minimum guessing exponent

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Main Results We derive necessary and sufficient conditions for achievability of covering rates and for both strategies. The claimed rate region in Kanlis and Narayan’s work for strong covering is in fact the rate region for weak covering. All the type covering based results regarding error exponents remain valid, as weak covering is sufficient for those analyses. Strong covering rates can be strictly higher than weak covering rates (shown by example). The minimum achievable hierarchical guessing exponent result must be re-examined, as it relies on strong covering.

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Let R P (D 1,D 2 ) denote the region of all (R 1,R 2 ) satisfying the following:  Q 1 (y 1 ), Q 2|1 (y 2 |y 1 ), V(x|y 1 ), and W(x| y 1, y 2 ) such that V  V (P,Q 1,D 1 ), i.e., V and Q 1 are consistent with P and induce distortion  D 1, W  W (V,Q 2|1,Q 1,D 2 ), i.e., W and Q 2|1 are consistent with V, and (together with Q 1 ) induce distortion  D 2, Weak Covering Rates

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Weak Covering Rates If (R 1,R 2 )  R P (D 1,D 2 ), then there exists a weak covering with Conversely, if there exists a weak covering with codebook sizes M 1 and M 2, then

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Strong Covering Rates Let T P (D 1,D 2 ) denote the region of all (R 1,R 2 ) with R 2  R 1 satisfying the following:  Q 1 (y 1 ) such that, For all V  V (P,Q 1,D 1 ),  Q 2|1 (y 2 |y 1 ) such that Note that T P (D 1,D 2 )  R P (D 1,D 2 )

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Strong Covering Rates If (R 1,R 2 )  T P (D 1,D 2 ), then there exists a strong covering with Conversely, if there exists a strong covering with codebook sizes M 1 and M 2, then

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Example of T P (D 1,D 2 )  R P (D 1,D 2 ) When R 1 = R P (D 1 ), there exists a unique pair (Q 1,V) such that V  V (P,Q 1,D 1 ) and I(Q 1,V)  R 1 Then it suffices to find V  V (P,Q 1,D 1 ) such that We found such V

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Consequences on Guessing Merhav, Arikan, and Roth proved the following lower bound for the minimum guessing exponent where Using strong type covering, they also proved achievability of this lower bound when Alice knows the type of x in advance.

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING Consequences on Guessing Proof assumes every rate pair in R P (D 1,D 2 ) is sufficient for covering. What is achievable, instead, is where For the example yielding T P (D 1,D 2 )  R P (D 1,D 2 ), we also show that L(D 1,D 2,P) > K(D 1,D 2,P).

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING To sum up… Contrasted two possibilities for hierarchical type covering. Presented rate regions for both cases. Rate regions are not identical. Investigated the effect of this distinction on the guessing problem.