A Unified Achievability Theorem

1

A Unified Achievability Theorem Si-Hyeon Lee and Sae-Young Chung Department of EE, KAIST, Daejeon, Korea

arXiv:1401.6023v2 [cs.IT] 28 Jan 2014

Email: [email protected], [email protected]

Abstract We prove a single achievability theorem that recovers many achievability results in network information theory as simple corollaries. Our result is enabled by many novel elements such as a generalized network model that includes multiple sources and channels interacting with each other, a unified typicality constraint that generalizes cost and distortion constraints, a unified coding strategy that generalizes many coding schemes in the literature, and novel proof techniques. Our unified coding scheme performs at least as good as many previous schemes. Our theorem does not only recover many classical results such as Marton coding, Gelfand-Pinsker coding, and WynerZiv coding, but also many recent results such as coding for channels with action-dependent states by Weissman, interference decoding by Bandemer and El Gamal, hybrid coding by Minero, Lim, and Kim, noisy network coding by Lim, Kim, El Gamal, and Chung, and N -node decode-and-forward by Kramer, Gastpar, and Gupta. Using our theorem, obtaining many new achievability results in network information theory can now be done without any tedious proof steps.

I. I NTRODUCTION In network information theory, deriving achievability results has been done by setting up models and by tweaking coding schemes case by case, which makes proving the results difficult and tedious. Furthermore, it becomes worse as the network size and the complexity of the problem grow. In this paper, we prove a very general achievability theorem that unifies many separate approaches in the network information theory literature into a single framework. To this end, we first consider a fully general network model that includes multiple sources and channels interacting with each other. We then focus our attention to memoryless sources and channels and assume a unified typicality constraint that covers many objectives and constraints studied in the literature including cost and distortion constraints. Under such assumptions, we prove our main theorem of unified achievability, which includes

January 29, 2014

DRAFT

2

many achievability results in the literature as simple corollaries. Our achievability is based on a unified coding scheme that generalizes many coding schemes in the literature. Examples of known results recovered by our theorem include Marton’s inner bound for the broadcast channel [1], Han-Kobayashi inner bound for the interference channel [2], [3], Gelfand-Pinsker coding [4], coding for channels with action-dependent states by Weissman [5], Wyner-Ziv coding [6], coding for computing by Yamamoto [7], Berger-Tung inner bound for distributed lossy compression [8], [9], ZhangBerger inner bound for multiple description coding [10], hybrid coding by Minero, Lim, and Kim [11], interference decoding for a 3-user interference channel by Bandemer and El Gamal [12], [13], and many more. We can also show [14] that our theorem recovers many achievability results for discrete memoryless networks by unfolding the network. Examples include a combination of partial decode-and-forward and compress-and-forward for the relay channel by Cover and El Gamal [15], Slepian-Wolf coding over cooperative relay networks by Yassaee and Aref [16], noisy network coding by Lim, Kim, El Gamal, and Chung [17], short-message noisy network coding by Hou and Kramer [18], decode-and-forward for the N -node relay channel by Kramer, Gastpar, and Gupta [19], offset encoding for the multiple access relay channel by Sankar, Kramer, and Mandayam [20], Cover-Leung inner bound for the multiple access channel with feedback [21], and many more. Based on our theorem, we also establish a duality theorem that relates the sufficient conditions for achievability for both the original network and its dual network. Examples of such a duality include a duality between Gelfand-Pinsker coding [4] and Wyner-Ziv coding [6], a duality between optimal coding for the multiple access channel [22] and optimal coding for multiple description without combined reconstruction [23], and a duality between Marton’s inner bound [1] and Berger-Tung inner bound [8], [9]. This paper is organized as follows. In Section II, we describe our network model. In Section III, we present the main theorem of unified achievability and its proof. Examples of known results recovered by the main theorem are illustrated in Section IV. In Section V, a duality theorem and several examples are shown. In Section VI, we apply the main theorem to the Gaussian case. The following notation is used throughout the paper. A. Notation For two integers i and j , [i : j] denotes the set {i, i + 1, . . . , j}. For a set S of real numbers, S[i] denotes the i-th smallest element in S and S[i] denotes {j : j ∈ S, j < i}. For constants u1 , . . . , uk and January 29, 2014

DRAFT

3

S ⊆ [1 : k], uS denotes the vector (uS[1] , . . . , uS[|S|] ) and uji denotes u[i:j] where the subscript is omitted

when i = 1, i.e., uj = u[1:j] . For random variables U1 , . . . , Uk and S ⊆ [1 : k], US and Uij are defined S similarly. For sets T1 , . . . , Tk and S ⊆ [1 : k], TS denotes j∈S Tj and Tij denotes T[i:j] where the

subscript is omitted when i = 1. For constants u1 , . . . , uk , v1 , . . . , vk and S ⊆ [1 : k], we say that uS is

smaller than vS and write uS < vS if there exists k such that uS[j] = vS[j] for all j < k and uS[k] < vS[k] . 1 denotes an all-one vector and I denotes an identity matrix. When U is a Gaussian random vector with

mean µ and covariance matrix ΛU , we write U ∼ N (µ, ΛU ).

1u=v is the indicator function, i.e., it is 1

if u = v and 0 otherwise. δ(ǫ) > 0 denotes a function of ǫ that tends to zero as ǫ tends to zero. We follow the notion of typicality in [24], [25]. Let πxn (x) denote the number of occurrences of x ∈ X in the sequence xn . Then, xn is said to be ǫ-typical (or just typical) for ǫ > 0 if for every x ∈ X , |πxn (x)/n − p(x)| ≤ ǫp(x). (n)

The set of all ǫ-typical xn is denoted as Tǫ set (or just a typical set) such as

(n) Tǫ (X, Y

is naturally defined from the definition of

(n)

(X), which is shortly denoted as Tǫ

. A jointly typical (n)

) for multiple variables, which will also be denoted as Tǫ

,

(n) Tǫ (X).

II. N ETWORK M ODEL Most network information theory problems can be transformed to the following canonical form. QN k−1 , xk−1 ), where x ∈ X and y ∈ Y , k ∈ [1 : N ] • Source and channel: k k k k k=1 p(yk |y •

Node processing functions: Xk = fk (Yk ), k ∈ [1 : N ]

•

Objective: (X1 , . . . , XN , Y1 , . . . , YN ) ∈ Θ

where N is the number of nodes in the network, Xk and Yk , k ∈ [1 : N ] are finite sets1 , and Θ ⊆ X1 × · · · × XN × Y1 × · · · × YN . Note that this model includes noisy channels from X[1:N −1] to Y[2:N ] .

Furthermore, Y[1:N ] can play a role of distributed sources possibly affected by X[1:N −1] . Note that this is general enough to include most network and channel models and coding strategies in the literature. In this paper, we focus on memoryless sources and channels, i.e., Xk and Yk , k ∈ [1 : N ] are replaced with their n-fold super-symbols Xkn and Ykn that follow N Y n Y

p(yk,i |y1,i , . . . , yk−1,i , x1,i , . . . , xk−1,i ),

k=1 i=1

where xk,i ∈ Xk and yk,i ∈ Yk , k ∈ [1 : N ], i ∈ [1 : n]. We call it a generalized memoryless network. In this case, the node processing functions become Xkn = fk (Ykn ), k ∈ [1 : N ]. Note that this model 1

They can also be Rn when we apply our results to Gaussian sources and channels.

January 29, 2014

DRAFT

4

p(y1 )

Y1n

f1

X1n

p(y2 |x1 , y1 )

Node 1

Fig. 1.

Y2n

X2n

f2

p(y3 |y 2 , x2 )

Y3n

Node 2

X3n

f3 Node 3

A three-node generalized memoryless network.

is general enough to include many discrete memoryless network models in the literature with strictly causal, causal, and non-causal node processing functions.2 A three-node generalized memoryless network is depicted in Fig. 1. Furthermore, we assume cost and distortion constraints specified using a jointly typical set, i.e., (n)

n , Y n, . . . , Y n) ∈ T (X1n , . . . , XN ǫ 1 N

(n)

, where the typical set Tǫ

is assumed to be with respect to a target

joint distribution p∗ (x[1:N ] , y[1:N ] ).3 (n)

(n)

n n The probability of error is defined as Pe (ǫ) = P ((X[1:N / Tǫ ] , Y[1:N ] ) ∈

). We say the target distribu(n)

tion p∗ is achievable if there exists a sequence of node processing functions such that limn→∞ Pe (ǫ) = 0 for any ǫ > 0. III. M AIN R ESULTS

G ENERALIZED M EMORYLESS N ETWORKS Q k−1 , xk−1 ), p∗ ) In this section, we present our main result. Let Ω(X1 , . . . , XN , Y1 , . . . , YN , N k=1 p(yk |y FOR

denote the set of ω = (µ, ν, Uj , Wk , Dk , Bk , Γj , Aj , p(uWk |uDk , yk ), xk (uWk , uDk , yk ) for k ∈ [1 : N ] and j ∈ [1 : ν]) that satisfy the following properties: 1) positive integers µ and ν 2) alphabets Uj , j ∈ [1 : ν]

3) sets Wk ⊆ [1 : ν] \ W k−1 , Dk ⊆ W k−1, Bk ⊆ W k−1 \ Dk , Γj ⊆ [1 : µ], and Aj ⊆ [1 : ν] for k ∈ [1 : N ] and j ∈ [1 : ν] that satisfy

A-1 ΓWk \ ΓDk ’s are disjoint. A-2 ΓAj ⊆ Γj and j ′ < j if j ′ ∈ Aj . A-3 AWk ⊆ Wk ∪ Dk , ABk ⊆ Dk ∪ Bk , and ADk ⊆ Dk . 2

Assuming block-wise processing at each node and allowing unfolding of the network if necessary.

3

Since we only consider sufficient conditions for achievability in this paper, it is enough to consider a single target distribution

that results in typical cost and typical distortion instead of a set of distributions.

January 29, 2014

DRAFT

5

W1

W2

D2 1 2

D3 Node 1

Fig. 2.

B3 3

Node 2

An example of ω ∈ Ω for a three-node generalized memoryless network is illustrated, where W1 = {1, 2}, W2 =

{3}, W3 = D1 = ∅, D2 = {1}, D3 = {1, 2}, B1 = B2 = ∅, B3 = {3}, A1 = ∅, and A2 = A3 = {1}. An arrow represents a superposition relationship for the codebooks of U ’s. Since W1 = {1, 2}, node 1 covers Y1n by U1n and U2n . Since D2 = {1} and W2 = {3}, node 2 decodes U1n and covers Y3n by U3n . Since D3 = {1, 2} and B3 = {3}, node 3, which is not drawn, decodes U1n and U2n with the help of U3n , i.e., it decodes U3n non-uniquely.

4) a set of conditional pmfs p(uWk |uDk , yk ) and functions xk (uWk , uDk , yk ) for k ∈ [1 : N ] such that p(x[1:N ] , y[1:N ] ) induced by N Y

p(yk |y k−1 , xk−1 )p(uWk |uDk , yk )1xk =xk (uWk ,uDk ,yk )

(1)

k=1

is the same as the target distribution p∗ (x[1:N ] , y[1:N ] ). If it is clear from the context, Ω will be used instead of Ω(X1 , . . . , XN , Y1 , . . . , YN , p∗ ).

QN

k=1 p(yk |y

k−1 , xk−1 ),

In the above, Uj is the alphabet set for Uj , which is the j -th auxiliary variable used for covering, Wk is the set of indices of U ’s used for covering at node k, Dk and Bk are sets of indices of U ’s decoded uniquely and non-uniquely at node k, respectively, and Aj is the set of indices of U ’s over which Uj codebook is constructed using a superposition, for j ∈ [1 : ν] and k ∈ [1 : N ]. An example of ω ∈ Ω is illustrated in Fig. 2. We also define the following notations ¯ k , ΓDk , W ¯ k , ΓW k \ D ¯ k, B ¯k , ΓDk ∪Bk \ D ¯k D ¯k, W ¯ k , and B ¯k will be clearer as we give our for k ∈ [1 : N ]. Meaning of the notations such as Γj , D

proof. The following is our main result. Theorem 1: For an N -node generalized memoryless network, p∗ is achievable if there exist ω ∈ Ω

January 29, 2014

DRAFT

6

and µ-tuple (r1 , . . . , rµ ) such that for 1 ≤ k ≤ N X

rj


j∈S¯k

X

I(Uj ; USk [j]∪Skc , Yk |UAj )

X

I(Uj ; UTk [j]∪Dk , Yk |UAj )

j∈Sk

j∈T¯k

j∈Tk

¯k ∪ B ¯k such that S¯k ∩ D ¯ k 6= ∅ and for all T¯k ⊆ W ¯ k such that T¯k = for all S¯k ⊆ D 6 ∅, where Sk , {j : j ∈ Dk ∪ Bk , Γj ∩ S¯k 6= ∅},

(2)

¯ k )c = ∅} Tk , {j : j ∈ Wk , Γj ∩ (T¯k ∪ D

(3)

and Skc = (Dk ∪ Bk ) \ Sk . Proof: Consider ω ∈ Ω and µ-tuple (r1 , . . . , rµ ). Let 0 < ǫk < ǫ′k < ǫ′′k for all k ∈ [1 : N ] such that ǫ′′k−1 < ǫk and ǫ′′N < ǫ. In the following, lj ∈ [1 : 2nrj ] for j ∈ [1 : µ]. nrΓ

1) Codebook generation: For each j ∈ [1 : ν] and lΓj ∈ [1 : 2nrΓj,[1] ] × · · · × [1 : 2 j,[|Γj |] ], generate Q unj (lΓj ) conditionally independently according to ni=1 p(uj,i |uAj ,i (lΓAj )), where unS (lΓS ) for S ⊆ [1 : ν]

denote {uni (lΓi ) : i ∈ S}.

2) Operation at node k ∈ [1 : N ]: After receiving Ykn , node k finds the smallest ˆlD¯ k ,k such that (unDk ∪Bk (ˆlD¯ k ,k , lB¯k ), ykn ) ∈ Tǫ(n) k

(4)

for some lB¯k . If there is no such index vector, let ˆlD¯ k ,k = 1. Next, node k finds the smallest lW¯ k such that4 (n) n (unDk (ˆlD¯ k ,k ), unWk (ˆlD¯ k ,k , lW ¯ k ), yk ) ∈ Tǫ′ . k

(5)

If there is no such index vector, let lW lD¯ k ,k ), uWk ,i (ˆlD¯ k ,k , lW ¯ k ), yk,i ) for ¯ k = 1. Send xk,i = xk (uDk ,i (ˆ i ∈ [1 : n]. ˆ¯ 3) Error analysis: For k ∈ [1 : N ], let L ¯ k denote the chosen index vectors at node k . Dk ,k and LW

Let us define the error event as follows: E=

N [

(Ek,1 ∪ Ek,2 ∪ Ek,3 ∪ Ek,4 )

k=1

4

In (5), (ˆlΓW

k

¯ ,k , lW ¯ ) \W k k

n ˆ¯ ,k , lW suffices to specify the index set of un ¯ ) as the index set of uWk for Wk , but we write (lD k k

notational convenience.

January 29, 2014

DRAFT

7

where n n Ek,1 = {(UW / Tǫ(n) } k−1 (LW ¯ k−1 ), Y[1:k] ) ∈ k n for some lD¯ k 6= LD¯ k , lB¯k } (lD¯ k ∪B¯k ), Ykn ) ∈ Tǫ(n) Ek,2 = {(UD k k ∪Bk (n) n ˆ n ˆ ¯ , L ¯ ), Y n ) ∈ / Tǫ′ } Ek,3 = {(UD (LD¯ k ,k ), UW (L k Wk Dk ,k k k k

Ek,4 =

n n {(UW k (LW ¯ k ), Y[1:k] )

∈ /

(n) Tǫ′′ }. k

ˆ ¯ = L ¯ for all k ∈ [1 : N ] and (U n N (L ¯ N ), Y n ) ∈ Tǫ(n) , which means Note that E c implies L W Dk Dk ,k W [1:N ] (n)

n n (X[1:N ] , Y[1:N ] ) ∈ Tǫ

(n)

. Hence, Pe (ǫ) ≤ P (E).

The probability of the error event can be upper-bounded as follows: P (E) ≤

N X

(P (Ek,1 ∩

k−1 \

c

(Ej,1 ∪ Ej,2 ) ∩

c ) Ek−1,4

+ P (Ek,2 ∩

(Ej,1 ∪ Ej,2 ∪ Ej,3 )c )

j=1

j=1

k=1

k−1 \

c c c c )). ∩ Ek,2 ) + P (Ek,4 ∩ Ek,1 ∩ Ek,2 + P (Ek,3 ∩ Ek,1

(6)

ˆ ¯ = L¯ . Note that (Ek,1 ∪ Ek,2 )c implies L Dk Dk ,k

Let us bound each term in the summation in (6) for given k ∈ [1 : N ]. First, we have P (Ek,1 ∩

k−1 \

c (Ej,1 ∪ Ej,2 )c ∩ Ek−1,4 )

j=1

(n)

n n n n n ≤ P ((UW / Tǫ(n) , k−1 (LW ¯ k−1 ), Y[1:k−1] ) ∈ Tǫ′′ , (UW k−1 (LW ¯ k−1 ), Y[1:k−1] , Yk ) ∈ k k−1

ˆ ¯ = L ¯ for all j ∈ [1 : k − 1]), L Dj Dj ,j

which tends to zero as n tends to infinity from the conditional typicality lemma [25]. Next, we show in Appendix A that the second term in the summation in (6) tends to zero as n tends to infinity if X

rj


X

I(Uj ; UTk [j]∪Dk , Yk |UAj ) + 4(1 + ν)δ(ǫ′k )

(7)

j∈Tk

¯ k such that T¯k 6= ∅, where Tk is defined in (3). for all T¯k ⊆ W

January 29, 2014

DRAFT

8

Finally, the fourth term in the summation in (6) is proved in Appendix C to tend to zero as n tends ¯ k such to infinity for sufficiently small ǫk and ǫ′k under the aforementioned condition (7) for all T¯k ⊆ W

that T¯k = 6 ∅. (n)

Therefore, P (E) and thus Pe (ǫ) tend to zero as n tends to infinity if rate tuple (r1 , . . . , rµ ) satisfies for 1 ≤ k ≤ N , X

rj


j∈S¯k

X

I(Uj ; USk [j]∪Skc , Yk |UAj )

X

I(Uj ; UTk [j]∪Dk , Yk |UAj )

j∈Sk

j∈T¯k

j∈Tk

¯k ∪ B ¯k such that S¯k ∩ D ¯ k 6= ∅ and for all T¯k ⊆ W ¯ k such that T¯k = for all S¯k ⊆ D 6 ∅. This completes the

proof. Remark 1: Theorem 1 can be improved using coded time sharing [2]. By considering a subset Ω′ of Ω which is the set of ω ′ ∈ Ω such that ν = µ and Γj = {j} ∪ Aj for j ∈ [1 : ν], we obtain the following corollary.

Corollary 1: For an N -node generalized memoryless network, p∗ is achievable if there exist ω ′ ∈ Ω′ and µ-tuple (r1 , . . . , rµ ) such that for 1 ≤ k ≤ N X

rj


X

I(Uj ; USk [j]∪Skc , Yk |UAj )

X

I(Uj ; UTk [j]∪Dk , Yk |UAj )

j∈Sk

j∈Sk

j∈Tk

j∈Tk

for all Sk ⊆ Dk ∪ Bk such that Sk ∩ Dk 6= ∅ and if j ∈ Skc , then Aj ⊆ Skc and for all Tk ⊆ Wk such that Tk 6= ∅ and if j ∈ Tk , then Aj ∩ Wk ⊆ Tk . ¯ k = Wk , D ¯ k = Dk , B ¯k = Bk , S¯k ⊆ Sk , and Tk ⊆ T¯k for all Proof: For ω ′ ∈ Ω′ , we have W ¯k ∪ B ¯k such that S¯k ∩ D ¯ k 6= ∅ and for all T¯k ⊆ W ¯ k such that T¯k = S¯k ⊆ D 6 ∅. Hence, it is enough to

consider S¯k and T¯k such that S¯k = Sk and T¯k = Tk . IV. S PECIAL C ASES In this section, we show that many previous results can be obtained as simple corollaries of our theorem. In the following, unspecified components Wk , Dk , Bk , and Aj of ω ∈ Ω in Theorem 1 or ω ′ ∈ Ω′ in Corollary 1 are assumed to be empty5 . 5

For ω ′ ∈ Ω′ , we do not explicitly specify Γj since Γj = {j} ∪ Aj .

January 29, 2014

DRAFT

9

Node 1 Y1n

Fig. 3.

U1 = (Y1 , X1 )

Node 2 X1n

p(y2 |x1 )

Y2n

ˆ1 = (Yˆ1 , X ˆ1) U

X2n = Yˆ1n

Point-to-point channel coding

1) Point-to-point channel coding [26]: •

We assume N = 2, Y1 is the source message such that H(Y1 ) = R, and p(y2 |y1 , x1 ) = p(y2 |x1 ) is the noisy channel from X1 to Y2 .

•

Since we want to recover the source message at node 2, we assume target distribution p∗ such that X2 = Y1 .

•

For ω ′ ∈ Ω′ in Corollary 1, we choose the following (illustrated in Fig. 3): – µ = 1, U1 = (Y1 , X1 ), W1 = {1}, D2 = {1}. – p(x1 |y1 ) = p(x1 ), x2 (u1 , y2 ) = y1 .

•

From Corollary 1, we conclude – T1 = {1}: r1 > R – S2 = {1}: r1 < I(X1 ; Y2 ) – By the Fourier-Motzkin (F-M) elimination, we get R < I(X1 ; Y2 ).

•

(n)

Note that if p∗ is achievable, P (X2n 6= Y1n ) → 0 as n → ∞ since (Y1n , X2n ) ∈ Tǫ

implies

X2n = Y1n . By considering all p∗ such that X2 = Y1 , we conclude that all R < maxp(x1 ) I(X1 ; Y2 )

is achievable in the usual sense. •

It is possible to include an input cost constraint given as

1 n

Pn

i=1 b(Xi )

≤ b, where b(x) ≥ 0 is a

cost function. To do so, the target distribution needs to satisfy E[b(X1 )] ≤ b/(1 + ǫ) in addition to X2 = Y1 , where ǫ > 0. By taking ǫ → 0 and by considering all p∗ satisfying X2 = Y1 and E[b(X1 )] ≤ b/(1 + ǫ), we can see all rates less than maxp(x1 ):E[b(X1 )]≤b I(X1 ; Y2 ) is achievable.

2) Point-to-point lossless compression [26]: •

Source and channel: N = 2, log |X1 | = R, Y2 = X1 .

•

Target distribution: p∗ such that X2 = Y1 .

•

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 1, U1 = (Y1 , X1 ), W1 = {1}, D2 = {1}. – p(x1 |y1 ) = 1/|X1 |, x2 (u1 , y2 ) = y1 .

•

From Corollary 1,

January 29, 2014

DRAFT

10

– T1 = {1}: r1 > H(Y1 ) – S2 = {1}: r1 < R – F-M elimination: R > H(Y1 ) 3) Point-to-point lossy compression [27]: •

Source and channel: Same as that of point-to-point lossless compression.

•

Target distribution: p∗ such that E[d(Y1 , X2 )] ≤

d 1+ǫ ,

where d(·, ·) ≥ 0 is a distortion measure

between two arguments and ǫ > 0. •

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 1, U1 = (X2 , X1 ), W1 = {1}, D2 = {1}. – p(u1 |y1 ) = p(x2 |y1 )/|X1 |.

•

From Corollary 1, – T1 = {1}: r1 > I(Y1 ; X2 ) – S2 = {1}: r1 < R – F-M elimination: R > I(Y1 ; X2 )

•

By taking ǫ → 0 and by considering all p∗ such that E[d(Y1 , X2 )] ≤

d 1+ǫ ,

we conclude R >

minp(x2 |y1 ):E[d(Y1 ,X2 )]≤d I(Y1 ; X2 ) is achievable.

4) Hybrid coding [11]: •

Source and channel: N = 2, p(y2 |y1 , x1 ) = p(y2 |x1 ) (point-to-point channel, see below for other channels.).

•

Target distribution: p∗ such that E[d(Y1 , X2 )] ≤

•

Choice of ω ′ ∈ Ω′ in Corollary 1:

d 1+ǫ .

– µ = 1, W1 = {1}, D2 = {1}. •

From Corollary 1, – T1 = {1}: r1 > I(Y1 ; U1 ) – S2 = {1}: r1 < (U1 ; Y2 ) – F-M elimination: I(Y1 ; U1 ) < I(U1 ; Y2 )

•

By taking ǫ → 0, hybrid coding [11] for point-to-point channel is recovered. Similarly, hybrid coding for the other network models in [11] can also be recovered from Corollary 1.

5) Gelfand-Pinsker coding [4]: •

Source and channel: N = 2, Y1 = (M, S), H(M ) = R, p(y1 ) = p(m)p(s), p(y2 |y1 , x1 ) = p(y2 |x1 , s).

January 29, 2014

DRAFT

11

Node 1 Y1n = (M n , S n )

Fig. 4.

U1 = (M, U ) X1 = x1 (U, S)

Node 2 X1n

p(y2 |x1 , s)

Y2n

ˆ 1 = (M ˆ,U ˆ) U

ˆn X2n = M

Gelfand-Pinsker coding.

•

Target distribution: p∗ such that X2 = M .

•

Choice of ω ′ ∈ Ω′ in Corollary 1 (illustrated in Fig. 4): – µ = 1, U1 = (M, U ), W1 = {1}, D2 = {1}. – p(u|y1 ) = p(u|s), x1 (u1 , y1 ) = x1 (u, s), x2 (u1 , y2 ) = m.

•

From Corollary 1, – T1 = {1}: r1 > R + I(U ; S) – S2 = {1}: r1 < I(U ; Y2 ) – F-M elimination: R < I(U ; Y2 ) − I(U ; S)

6) Channels with action-dependent states [5]: •

Source and channel: N = 3, H(Y1 ) = R, Y2 = (X1 , Y1 , S), p(s|x1 , y1 ) = p(s|x1 ), p(y3 |y[1:2] , x[1:2] ) = p(y3 |x2 , s).

•

Target distribution: p∗ such that X3 = Y1 .

•

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 2, U1 = (Y1 , X1 ), W1 = {1}, W2 = {2}, D2 = {1}, D3 = {1}, B3 = {2}, A2 = {1}. – p(x1 |y1 ) = p(x1 ), p(u2 |y2 , u1 ) = p(u2 |s, x1 ), x2 (y2 , u1 , u2 ) = x2 (s, u2 ), x3 (y3 , u1 ) = y1 .

•

From Corollary 1, – T1 = {1}: r1 > R – S2 = {1}: r1 < R + H(X1 ) – T2 = {2}: r2 > I(U2 ; S|X1 ) – S3 = {1, 2}: r1 + r2 < I(X1 , U2 ; Y3 ) – F-M elimination: R < I(X1 , U2 ; Y3 ) − I(U2 ; S|X1 )

7) Wyner-Ziv coding [6] and coding for computing [7]: •

Source and channel: N = 2, log |X1 | = R, Y2 = (X1 , S), p(s|y1 , x1 ) = p(s|y1 ).

•

Target distribution: p∗W Z for Wyner-Ziv coding such that E[d(Y1 , X2 )] ≤ for computing such that E[d(f (Y1 , S), X2 )] ≤

January 29, 2014

d 1+ǫ ,

d 1+ǫ

and p∗CC for coding

where f (y1 , s) is a function of y1 and s and DRAFT

12

Node 1 Y1n

Fig. 5.

U1 = (U, X1 )

Node 2 X1n

p(s|y1 )

Y2n

=

(X1n , S n )

ˆ 1 = (U ˆ, X ˆ1) U ˆ , S) X 2 = x2 (U

X2n

Wyner-Ziv coding and coding for computing.

ǫ > 0. •

Choice of ω ′ ∈ Ω′ in Corollary 1 (illustrated in Fig. 5): – µ = 1, U1 = (U, X1 ), W1 = {1}, D2 = {1}. – p(u1 |y1 ) =

1 |X1 | p(u|y1 ),

x2 (u1 , y2 ) = x2 (u, s) such that p = p∗W Z for Wyner-Ziv coding and

p = p∗CC for coding for computing. •

From Corollary 1, – T1 = {1}: r1 > I(U ; Y1 ) – S2 = {1}: r1 < R + I(U ; S) – F-M elimination: R > I(U ; Y1 ) − I(U ; S) = I(U ; Y1 |S)

8) Marton’s inner bound with common message [28]: •

Source and channel: N = 3, Y1 = (M1 , M2 , M3 ), H(Mk ) = Rk for k ∈ [1 : 3], p(y1 ) = p(m1 )p(m2 )p(m3 ), p(y2 |y1 , x1 ) = p(y2 |x1 ), p(y3 |y[1:2] , x[1:2] ) = p(y3 |x1 , y2 ).

•

Target distribution: p∗ such that X2 = (M1 , M2 ), X3 = (M1 , M3 ).

•

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 3, Uj = (Mj , Vj ) for j ∈ [1 : 3], W1 = {1, 2, 3}, D2 = {1, 2}, D3 = {1, 3}, A2 = {1}, A3 = {1}.

– p(v[1:3] |y1 ) = p(v[1:3] ), x1 (u[1:3] , y1 ) = x1 (v[1:3] ), x2 (u[1:2] , y2 ) = (m1 , m2 ), x3 (u{1,3} , y3 ) = (m1 , m3 ). •

From Corollary 1, – T1 = {1}: r1 > R1 – T1 = {1, 2}: r1 + r2 > R1 + R2 – T1 = {1, 3}: r1 + r3 > R1 + R3 – T1 = {1, 2, 3}: r1 + r2 + r3 > R1 + R2 + R3 + I(V2 ; V3 |V1 ) – S2 = {1, 2}: r1 + r2 < I(V1 , V2 ; Y2 ) – S2 = {2}: r2 < I(V2 ; Y2 |V1 )

January 29, 2014

DRAFT

13

– S3 = {1, 3}: r1 + r3 < I(V1 , V3 ; Y3 ) – S3 = {3}: r3 < I(V3 ; Y3 |V1 ) – F-M elimination: R1 + R2 < I(V1 , V2 ; Y2 ) R1 + R3 < I(V1 , V3 ; Y3 ) R1 + R2 + R3 < I(V1 , V2 ; Y2 ) + I(V3 ; Y3 |V1 ) − I(V2 ; V3 |V1 ) R1 + R2 + R3 < I(V1 , V3 ; Y3 ) + I(V2 ; Y3 |V1 ) − I(V2 ; V3 |V1 ) 2R1 + R2 + R3 < I(V1 , V2 ; Y2 ) + I(V1 , V3 ; Y3 ) − I(V2 ; V3 |V1 ),

which recovers the Marton’s inner bound with common message. 9) Three-receiver multilevel broadcast channel [29]: •

Source and channel: N = 4, Y1 = (M0 , M10 , M11 ), H(M0 ) = R0 , H(M10 ) = R10 , H(M11 ) = R11 , R1 = R10 + R11 , p(y1 ) = p(m0 )p(m10 )p(m11 ), p(y2 |y1 , x1 ) = p(y2 |x1 ), p(y3 |y[1:2] , x[1:2] ) = p(y3 |y2 ), p(y4 |y[1:3] , x[1:3] ) = p(y4 |y2 , x1 ).

•

Target distribution: p∗ such that X2 = (M0 , M10 , M11 ), X3 = M0 , X4 = M0 .

•

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 3, U1 = (M0 , V1 ), U2 = (M10 , V2 ), U3 = (M11 , X1 ), W1 = {1, 2, 3}, D2 = {1, 2, 3}, D3 = {1}, D4 = {1}, B4 = {2}, A2 = {1}, A3 = {1, 2}.

– p(v[1:2] , x1 |y1 ) = p(v[1:2] )p(x1 |v2 ). •

From Corollary 1, – T1 = {1}: r1 > R0 – T1 = {1, 2}: r1 + r2 > R0 + R10 – T1 = {1, 2, 3}: r1 + r2 + r3 > R0 + R10 + R11 – S2 = {1, 2, 3}: r1 + r2 + r3 < I(X1 ; Y2 ) – S2 = {2, 3}: r2 + r3 < I(X1 ; Y2 |V1 ) – S2 = {3}: r3 < I(X1 ; Y2 |V2 ) – S3 = {1}: r3 < I(V1 ; Y3 ) – S4 = {1, 2}: r1 + r2 < I(V2 ; Y4 ) – By the F-M elimination, the achievable rate region of three-receiver multilevel broadcast channel is recovered.

January 29, 2014

DRAFT

14

10) Han-Kobayashi inner bound [2]: •

Source and channel: N = 4, Yk = (Mk0 , Mkk ), H(Mk0 ) = Rk0 , H(Mkk ) = Rkk , Rk = Rk0 + Rkk for k ∈ [1 : 2], p(y1 ) = p(m10 )p(m11 ), p(y2 |y1 , x1 ) = p(m20 )p(m22 ), p(y3 |y[1:2] , x[1:2] ) = p(y3 |x[1:2] ), p(y4 |y[1:3] , x[1:3] ) = p(y4 |y3 , x[1:2] ).

•

Target distribution: p∗ such that X3 = Y1 , X4 = Y2 .

•

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 4, U1 = (M10 , V1 ), U2 = (M11 , X1 ), U3 = (M20 , V2 ), U4 = (M22 , X2 ), W1 = {1, 2}, W2 = {3, 4}, D3 = {1, 2}, D4 = {3, 4}, B3 = {3}, B4 = {1}, A2 = {1}, A4 = {3}.

– p(vk , xk |yk ) = p(vk , xk ) for k ∈ [1 : 2]. •

From Corollary 1, – T1 = {1}: r1 > R10 – T1 = {1, 2}: r1 + r2 > R10 + R11 – T2 = {3}: r2 > R20 – T2 = {3, 4}: r3 + r4 > R20 + R22 – S3 = {1, 2}: r1 + r2 < I(X1 ; Y3 |V2 ) – S3 = {1, 2, 3}: r1 + r2 + r3 < I(X1 , V2 ; Y3 ) – S3 = {2}: r2 < I(X1 ; Y3 |V1 , V2 ) – S3 = {2, 3}: r2 + r3 < I(X1 , V2 ; Y3 |V1 ) – S4 = {3, 4}: r3 + r4 < I(X2 ; Y4 |V1 ) – S4 = {1, 3, 4}: r1 + r3 + r4 < I(X2 , V1 ; Y4 ) – S4 = {4}: r4 < I(X2 ; Y4 |V1 , V2 ) – S4 = {1, 4}: r1 + r4 < I(X2 , V1 ; Y4 |V2 ) – The F-M elimination gives the Han-Kobayashi inner bound.

11) Interference decoding for a 3-user deterministic interference channel [12]: •

Source and channel: N = 15, H(Yk ) = Rk for k ∈ [1 : 3], p(y2 |x1 , y1 ) = p(y2 ), p(y3 |x[1:2] , y[1:2] ) =

January 29, 2014

DRAFT

15

p(y3 ), Y[4:15] is a function of X1 , X2 , and X3 as follows: Y4 = X2,1 , Y5 = X3,1 , Y6 = V1 Y7 = X1,2 , Y8 = X3,2 , Y9 = V2 Y10 = X1,3 , Y11 = X2,3 , Y12 = V3 Y13 = f1 (X1,1 , V1 ) Y14 = f2 (X2,2 , V2 ) Y15 = f3 (X3,3 , V3 )

where Xk,j = gk,j (Xk ), V1 = h1 (X2,1 , X3,1 ), V2 = h2 (X1,2 , X3,2 ), and V3 = h3 (X1,3 , X2,3 ), where hk and fk are one-to-one when either one of their arguments is fixed. By letting X[4:12] = ∅, this

network model is equivalent to that of [12]. •

Target distribution: p∗ such that X13 = Y1 , X14 = Y2 , X15 = Y3 .

•

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 12, Uj = (Yj , Xj ) for j ∈ [1 : 3], Uj = Yj for j ∈ [4 : 12], Wk = {k} for k ∈ [1 : 12], D13 = {1}, D14 = {2}, D15 = {3}.

– p(xk |yk ) = p(xk ) for k ∈ [1 : 3]. – Let m1 denote min(R2 + R3 , R2 + H(X3,1 ), H(X2,1 ) + R3 , H(V1 )). Then,    {6} if m1 = H(V1 )       {2, 3} if R2 < H(X2,1 ), R3 < H(X3,1 ), m1 < H(V1 ) B13 =    {2, 5} if R2 < H(X2,1 ), R3 ≥ H(X3,1 ), m1 < H(V1 )      {3, 4} if R ≥ H(X ), R < H(X ), m < H(V ) 2 2,1 3 3,1 1 1 Similarly, we set B14 and B15 .

•

By Corollary 1 followed by the F-M elimination, an inner bound is obtained that is at least as good as that in [12] and has a simpler form.

•

The interference decoding inner bound in [12] was improved by using rate splitting, Marton coding, and superposition coding in [13]. By introducing some virtual nodes whose input alphabets are empty and setting Bk for destination node k appropriately, an inner bound can be obtained from Corollary 1, which includes that in [13] and has a simpler form.

12) Berger-Tung inner bound for distributed lossy compression [8], [9]:

January 29, 2014

DRAFT

16

•

Source and channel: N = 3, log |X1 | = R1 , p(y2 |y1 , x1 ) = p(y2 |y1 ), log |X2 | = R2 , Y3 = (X1 , X2 ), X3 = X3,1 × X3,2 .

•

Target distribution: p∗ such that E[dk (Yk , X3,k )] ≤

dk 1+ǫ

for k ∈ [1 : 2], where dk (·, ·) ≥ 0 is a

distortion measure between two arguments and ǫ > 0. •

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 2, Uk = (Xk , Vk ), Wk = {k} for k ∈ [1 : 2], D3 = {1, 2}. – p(uk |yk ) = p(vk |yk )/|Xk | for k ∈ [1 : 2], x3 (u1 , u2 , y3 ) = (x3,1 (v1 , v2 ), x3,2 (v1 , v2 )).

•

By Corollary 1 followed by the F-M elimination, Berger-Tung inner bound is recovered.

13) Zhang-Berger inner bound for multiple description coding [10], [30], [31]: •

Source and channel: N = 4, X1 = M0 × M11 × M22 × M01 × M02 , log |M0 | = R0 , log |Mkk | = Rkk , log |M0k | = R0k , Rk = R0 + Rkk + R0k for k ∈ [1 : 2], Y2 = (M0 , M11 , M01 ), Y3 = (M0 , M22 , M02 ), Y4 = (Y2 , Y3 ).

•

Target distribution: p∗ such that E[dk (Y1 , Xk )] ≤

•

Choice of ω ′ ∈ Ω′ in Corollary 1:

dk 1+ǫ

for k ∈ [2 : 4], where ǫ > 0.

– µ = 4, U1 = (V, M0 ), U2 = (X2 , M11 ), U3 = (X3 , M22 ), U4 = (X4 , M01 , M02 ), W1 = {1, 2, 3, 4}, D2 = {1, 2}, D3 = {1, 3}, D4 = {1, 2, 3, 4}, A2 = {1}, A3 = {1}, A4 = {1, 2, 3}.

– p(u[1:4] |y1 ) = p(v, x[2:4] |y1 ) · •

1 |M0 |·|M11 |·|M22 |·|M01 |·|M02 | ,

X1 = (M0 , M11 , M22 , M01 , M02 ).

By Corollary 1 followed by the F-M elimination, Zhang-Berger inner bound is recovered.

14) Lossless communication of a 2-DMS over a multiple access channel [32]: •

Source and channel: N = 4, let V2 and V3 be two discrete memoryless sources and let V1 denote the common part of V2 and V3 , where the common part of two discrete memoryless sources is defined in [33], [34]. Y1 = V1 , Y2 = (V2 , X1 ), Y3 = (V3 , X1 ), p(y4 |y[1:3] , x[1:3] ) = p(y4 |x[2:3] ). Note that this model is equivalent to the 2-DMS over multiple access channel.

•

Target distribution: p∗ such that X4 = (V2 , V3 ).

•

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 3, U1 = (V1 , U, X1 ), U2 = (V2 , X2 ), U3 = (V3 , X3 ), Wk = {k} for k ∈ [1 : 3], D2 = {1}, D3 = {1}, D4 = {1, 2, 3}, A2 = {1}, A3 = {1}.

– p(u, x1 |y1 ) = p(u)/|X1 |, p(x2 |y2 , u1 ) = p(x2 |v2 , u), p(x3 |y3 , u1 ) = p(x3 |v3 , u). •

By Corollary 1 followed by the F-M elimination, the sufficient condition for lossless communication of a 2-DMS over a multiple access channel [32] is recovered.

January 29, 2014

DRAFT

17

15) Wiretap channel [35]: •

Source and channel: N = 3, Y1 = (M, M1 , M2 ), H(M ) = R, H(M1 ) = R1 , H(M2 ) = R2 , p(y1 ) = p(m)p(m1 )p(m2 ), p(y2 |y1 , x1 ) = p(y2 |x1 ), p(y3 |y[1:2] , x[1:2] ) = p(y3 |x1 , y2 ).

•

Target distribution: p∗ such that X2 = M .

•

Choice of ω ′ ∈ Ω′ in Corollary 1: – µ = 2, U1 = (M, M1 , U ), U2 = (M2 , X1 ), W1 = {1, 2}, D2 = {1}, A2 = {1}. – p(u, x1 |y1 ) = p(u, x1 ).

•

From Corollary 1, – T1 = {1}: r1 > R + R1 – T1 = {1, 2} r1 + r2 > R + R1 + R2 – S1 = {1}: r1 < I(U ; Y2 )

•

If R1 > I(U ; Y3 ) and R2 > I(X1 ; Y3 |U ) in addition to above conditions, it is shown to satisfy the leakage constraint at node 3 [35]. By the F-M elimination, the secrecy capacity CS = maxp(u,x1 ) I(U ; Y2 ) − I(U ; Y3 ) is recovered.

16) Discrete memoryless network: Discrete memoryless network (with strictly causal node processing functions) can be converted to a generalized memoryless network by unfolding the network. Many previous results for the network are shown to be recovered from Theorem 1 in [14]. Examples include a combination of decode-and-forward and compress-and-forward for the three-node relay channel [15], Slepian-Wolf coding over cooperative relay networks [16], noisy network coding [17], N -node decodeand-forward [19], [36], offset encoding for the multiple access relay channel [20], and Cover-Leung inner bound [21] for the multiple access channel with feedback. Remark 2: Though all the above examples are recovered from Corollary 1, there are some achievability results where Theorem 1 is helpful [14]. V. D UALITY Consider an N -node generalized memoryless network (X1 , . . . , XN , Y1 , . . . , YN ,

QN

k=1 p(yk |y

k−1 , xk−1 ))

with target joint distribution p∗ (x[1:N ] , y[1:N ] ). Its dual network is an N -node generalized memoryless N , xN )) with target joint distribution p∗ (x network (X1 , . . . , XN , Y1 , . . . , YN , pd (yk |yk+1 k+1 d [1:N ] , y[1:N ] ). We

note that the input and output alphabets of the dual network are the same as those of the original network while the order of nodes is reversed and that p∗d (x[1:N ] , y[1:N ] ) is not necessarily the same as p∗ (x[1:N ] , y[1:N ] ).

January 29, 2014

DRAFT

18

Let Ωd denote the set of ωd = (µ, Uj , Wk , Dk , p(uWk |uDk , yk ), pd (uDk |uWk , yk ), xk (uWk , uDk , yk ), xk,d (uWk , uDk , yk ) for k ∈ [1 : N ] and j ∈ [1 : µ]) such that QN k−1 , xk−1 ), p∗ ), where ω ′ (ω ) = (µ, U , W , D , • ω ′ (ωd ) ∈ Ω′ (X1 , . . . , XN , Y1 , . . . , YN , j d k k k=1 p(yk |y •

Bk , Aj , p(uWk |uDk , yk ), xk (uWk , uDk , yk ) for k ∈ [1 : N ] and j ∈ [1 : µ]) with Bk = ∅ and Aj = ∅, Q k−1 , xk−1 ), p∗ ), where ω ′ (ω ) = (µ(d) , U (d) , ωd′ (ωd ) ∈ Ω′ (X1 , . . . , XN , Y1 , . . . , YN , N d d j d k=1 pd (yk |y

(d) (d) (d) (d) (d) , y ), xk (u (d) , u (d) , yk ) for k ∈ [1 : |u Dk Wk Wk(d) Dk(d) k (d) (d) (d) (d) (d) with µ(d) = µ, Uj = Uj , Wk = Dk , Dk = Wk , Bk = ∅, Aj = ∅, (d) (d) (d) pd (uDk |uWk , yk ), and xk (u (d) , u (d) , yk ) = xk,d (uWk , uDk , yk ). Dk Wk (d)

(d)

(d)

(d)

Wk , Dk , Bk , Aj , pd (u

N ] and j ∈ [1 : µ(d) ]) (d) (d) (d) |u (d) , yk ) Dk Wk

pd (u

=

For given ωd ∈ Ωd , note that we have two different joint distributions, i.e., for the original network N Y

p(uWk |uDk , yk )1xk =xk (uWk ,uDk ,yk ) p(yk |y k−1 , xk−1 )

(8)

k=1

that is marginalized to

p∗ (x N Y

[1:N ] , y[1:N ] )

and for the dual network

N pd (uDk |uWk , yk )1xk =xk,d (uWk ,uDk ,yk ) pd (yk |yk+1 , xN k+1 )

(9)

k=1

that is marginalized to p∗d (x[1:N ] , y[1:N ] ). The following duality theorem is directly obtained from Corollary 1. Theorem 2: Consider ωd ∈ Ωd . For the original network, p∗ is achievable if there exists µ-tuple (r1 , . . . , rµ ) such that for 1 ≤ k ≤ N

X

rj


j∈Sk

X

I(Uj ; USk [j]∪Skc , Yk )

(10)

X

I(Uj ; UTk [j]∪Dk , Yk )

(11)

j∈Sk

j∈Tk

j∈Tk

for all Sk ⊆ Dk such that Sk 6= ∅ and for all Tk ⊆ Wk such that Tk 6= ∅. For the dual network, p∗d is achievable if there exists µ-tuple (r1 , . . . , rµ ) such that for 1 ≤ k ≤ N X

rj


j∈Tk

j∈Sk

X

Id (Uj ; UTk [j]∪Tkc , Yk )

(12)

X

Id (Uj ; USk [j]∪Wk , Yk )

(13)

j∈Tk

j∈Sk

for all Sk ⊆ Dk such that Sk 6= ∅ and for all Tk ⊆ Wk such that Tk 6= ∅. We note that the mutual information terms in (10) and (11) are with respect to the joint distribution (8) and the mutual information terms in (12) and (13) are with respect to the joint distribution (9). In the following, several examples of duality are illustrated. January 29, 2014

DRAFT

19

Example 1 (Gelfand-Pinsker coding [4] and Wyner-Ziv coding [6]): Consider Y1 = M × S , X1 , Y2 , and X2 = M, where log |M| = R. For the original network, assume p(y1 ) = p(m)p(s), p(y2 |y1 , x1 ) = p(y2 |x1 , s), and target distribution p∗ (x[1:2] , y[1:2] ) such that X2 = M . For the dual network, in which

the order of nodes is reversed, assume pd (y2 ), Y1 = (X2 , S) where pd (s|x2 , y2 ) = pd (s|y2 ), and target distribution p∗d (x[1:2] , y[1:2] ). Choose ωd ∈ Ωd as follows: µ = 1, U1 = (M, U ), W1 = {1}, D2 = {1}, p(u|y1 ) = p(u|s), pd (u1 |y2 ) =

1 |M| pd (u|y2 ),

x1 (u1 , y1 ) = x1 (u, s), x2 (u1 , y2 ) = x2,d (u1 , y2 ) = m, and

x1,d (u1 , y1 ) = x1,d (u, s). Note that the resultant choices of ω ′ (ωd ) and ωd′ (ωd ) correspond to Gelfand-

Pinsker coding and Wyner-Ziv coding, respectively. From Theorem 2, p∗ is achievable for the original network if there exists r1 such that r1 > I(U1 ; Y1 ) = R + I(U ; S) r1 < I(U1 ; Y2 ) = I(U ; Y2 ),

or equivalently if R < I(U ; Y2 ) − I(U ; S), and p∗d is achievable for the dual network if there exists r1 such that r1 < Id (U1 ; Y1 ) = R + Id (U ; S) r1 > Id (U1 ; Y2 ) = Id (U ; Y2 ),

or equivalently if R > Id (U ; Y2 ) − Id (U ; S). Example 2 (Multiple access channel [22] and multiple description [23]): Consider Y1 = M1 , X1 , Y2 = M2 , X2 , Y3 , and X3 = M1 × M2 , where log |M1 | = R1 and log |M2 | = R2 . For the original

network, assume p(y1 ) = p(m1 ), p(y2 |y1 , x1 ) = p(m2 ), p(y3 |y[1:2] , x[1:2] ) = p(y3 |x[1:2] ), and target distribution p∗ (x[1:3] , y[1:3] ) such that X3 = (M1 , M2 ). For the dual network, in which the order of nodes is reversed, assume pd (y3 ), Y2 = M2 , Y1 = M1 , and target distribution p∗d (x[1:3] , y[1:3] ). Choose ωd ∈ Ωd as follows: µ = 2, U1 = (M1 , X1 ), U2 = (M2 , X2 ), W1 = {1}, W2 = {2}, D3 = {1, 2}, p(x1 |y1 ) = p(x1 ), p(x2 |y2 ) = p(x2 ), pd (u[1:2] |y3 ) =

1 |M1 |·|M2 | pd (x1 |y3 )pd (x2 |y3 ),

x1 (u1 , y1 ) = x1,d (u1 , y1 ) =

x1 , x2 (u2 , y2 ) = x2,d (u2 , y2 ) = x2 , and x3 (u[1:2] , y3 ) = x3,d (u[1:2] , y3 ) = (m1 , m2 ). Note that the

resultant choices of ω ′ (ωd ) and ωd′ (ωd ) correspond to coding for a multiple access channel and coding for multiple description without combined reconstruction, respectively. From Theorem 2, p∗ is achievable

January 29, 2014

DRAFT

20

for the original network if there exist r1 and r2 such that r1 > I(U1 ; Y1 ) = R1 r2 > I(U2 ; Y2 ) = R2 r1 < I(U1 ; U2 , Y3 ) = I(X1 ; X2 , Y3 ) r2 < I(U2 ; U1 , Y3 ) = I(X2 ; X1 , Y3 ) r1 + r2 < I(U1 , U2 ; Y3 ) + I(U1 ; U2 ) = I(X1 , X2 ; Y3 ) + I(X1 ; X2 )

and p∗d is achievable for the dual network if there exist r1 and r2 such that r1 < Id (U1 ; Y1 ) = R1 r2 < Id (U2 ; Y2 ) = R2 (a)

r1 > Id (U1 ; Y3 ) = Id (X1 ; Y3 ) = Id (X1 ; X2 , Y3 ) (b)

r2 > Id (U2 ; Y3 ) = Id (X2 ; Y3 ) = Id (X2 ; X1 , Y3 ) r1 + r2 > Id (U1 , U2 ; Y3 ) + Id (U1 ; U2 ) = Id (X1 , X2 ; Y3 ) + Id (X1 ; X2 ),

where (a) and (b) are because pd (x1 , x2 |y3 ) = pd (x1 |y3 )pd (x2 |y3 ). Example 3 (Marton’s inner bound [1] and Berger-Tung inner bound [8], [9]): Consider Y1 = M1 × M2 , X1 , Y2 , X2 = M1 , Y3 , and X3 = M2 , where log |M1 | = R1 and log |M2 | = R2 . For the original

network, assume p(y1 ) = p(m1 )p(m2 ), p(y2 |y1 , x1 ) = p(y2 |x1 ), p(y3 |y[1:2] , x[1:2] ) = p(y3 |y2 , x1 ), and target distribution p∗ (x[1:3] , y[1:3] ) such that X2 = M1 and X3 = M2 . For the dual network, in which the order of nodes is reversed, assume pd (y3 ), pd (y2 |y3 , x3 ) = pd (y2 |y3 ), Y1 = (X2 , X3 ), and target distribution p∗d (x[1:3] , y[1:3] ). Choose ωd ∈ Ωd as follows: µ = 2, U1 = (M1 , V1 ), U2 = (M2 , V2 ), W1 = {1, 2}, D2 = {1}, D3 = {2}, p(v1 , v2 |y1 ) = p(v1 , v2 ), pd (u2 |y3 ) = pd (u1 |y2 ) =

1 |M1 | pd (v1 |y2 ),

1 |M2 | pd (v2 |y3 ),

x1 (u1 , u2 , y1 ) = x1 (v1 , v2 ), x2 (u1 , y2 ) = x2,d (u1 , y2 ) = m1 , x3 (u2 , y3 ) =

x3,d (u2 , y3 ) = m2 , and x1,d (u1 , u2 , y1 ) = x1,d (v1 , v2 ). Note that the resultant choices of ω ′ (ωd ) and ωd′ (ωd ) correspond to Marton’s inner bound and Berger-Tung inner bound, respectively. From Theorem

January 29, 2014

DRAFT

21

2, p∗ is achievable for the original network if there exist r1 and r2 such that r1 > I(U1 ; Y1 ) = R1 r2 > I(U2 ; Y1 ) = R2 r1 + r2 > I(U1 , U2 ; Y1 ) + I(U1 ; U2 ) = R1 + R2 + I(V1 ; V2 ) r1 < I(U1 ; Y2 ) = I(V1 ; Y2 ) r2 < I(U2 ; Y3 ) = I(V2 ; Y3 )

and p∗d is achievable for the dual network if there exist r1 and r2 such that r1 < Id (U1 ; U2 , Y1 ) = R1 + Id (V1 ; V2 ) r2 < Id (U2 ; U1 , Y1 ) = R2 + Id (V1 ; V2 ) r1 + r2 < Id (U1 , U2 ; Y1 ) + Id (U1 ; U2 ) = R1 + R2 + Id (V1 ; V2 ) r1 > Id (U1 ; Y2 ) = Id (V1 ; Y2 ) r2 > Id (U2 ; Y3 ) = Id (V2 ; Y3 ).

Remark 3: Note that the first and second examples show a strong duality in a sense that the mutual information terms in the original and dual networks have the same form while inequality directions are reversed. On the other hand, the third example shows a weak duality, in which the mutual information terms in the original and dual networks are similar, yet slightly different. VI. G AUSSIAN N ETWORK In this section, we consider an N -node Gaussian generalized memoryless network, in which the channel output Yk and channel input Xk at node k are rk -dimensional and tk -dimensional vectors, respectively, and the channel from nodes 1, . . . , k − 1 to node k is given as X X ′ Yj + Yk′ , Yk = Hkj Xj + Hkj j∈[1:k−1]

where Hkj is an rk × tj matrix,

′ Hkj

(14)

j∈[1:k−1]

is an rk × rj matrix, and Yk′ ∼ N (0, ΛYk′ ) is independent from

X k−1 and Y k−1 . We assume target distribution f ∗ (x[1:N ] , y[1:N ] ) that satisfies covariance contraint Σk

on Xk for k ∈ [1 : N ], i.e., ΛXk  Σk . Consider a subset Ωg of Ω, where Uj is an aj -dimensional vector and UWk and Xk have the form of ′ UWk = Gk UDk + G′k Yk + UW k

(15)

Xk = Fk UDk ∪Wk + Fk′ Yk , January 29, 2014

DRAFT

22

P P ′ which induce the target distribution. In the above, Gk is a j∈Dk aj matrix, Gk is a j∈Wk aj × P P P ′ j∈Dk ∪Wk aj matrix, and j∈Wk aj -dimensional vector, Fk is a tk × j∈Wk aj × rk matrix, UWk is a ′ ′ ) is independent from UDk and Yk . ∼ N (0, ΛUW Fk′ is a tk × rk matrix, where UW k k

Note that UWk and Xk can be rewritten as follows: X

UW k =

′ Gkj UWj + G′k Yk + UW k

(16)

j∈[1:k−1]

X

Xk =

Fkj UWj + Fk′ Yk

(17)

j∈[1:k]

where the columns of Gkj and Fkj corresponding to UDk and UDk ∪Wk are from Gk and Fk , respectively, and the other columns of Gkj and Fkj are zero vectors. The following lemma gives UWk and Yk . Lemma 1: For k ∈ [1 : N ], we have 

UWk



Yk



X

=

Φkj Ψj ,

(18)

j∈[1:k]

where Φkj ,

 P  

S:{j,k}⊆S⊆[j:k]

Q

i∈[1:|S|−1] ΥS[i+1] S[i]



Ψj , 

Υk ′ j ′

,

(19)

if j = k

 I



if j < k

Gk′ j ′ + G′k′  , P

P

′ G′j Yj′ + UW j

Yj′

i∈[j ′ :k ′ −1] Hk ′ i Fij ′

i∈[j ′ :k ′ −1] Hk ′ i Fij ′



,

(20)

G′k′ (Hk′ j ′ Fj′′ + Hk′ ′ j ′ ) Hk′ j ′ Fj′′ + Hk′ ′ j ′



.

(21)

From Lemma 1, for any S ⊆ W N , we can construct a matrix ΦUS such that US = ΦUS Ψk(S) , where k(S) = max({k : S ∩ Wk 6= ∅}). Also, for any k ∈ [1 : N ] and S ⊆ W k , we can construct matrices ΦUS ,Yk such that [USt Ykt ]t = ΦUS ,Yk Ψk .

Now we are ready to present a sufficient condition for achieving f ∗ for an N -node Gaussian generalized memoryless network.

January 29, 2014

DRAFT

23

Theorem 3: For an N -node Gaussian generalized memoryless network, f ∗ is achievable if there exist ωg ∈ Ωg and µ-tuple (r1 , . . . , rµ ) such that for 1 ≤ k ≤ N Q |ΦUSc ,Yk ΛΨk ΦtUSc ,Yk | · j∈Sk |ΦUj ,UAj ΛΨk({j}∪Aj ) ΦtUj ,UA | X 1 k j k Q rj < log t t k |Φ Φ | · 2 |Φ Λ k(A ) Φ Λ UAj Ψ j UDk ∪Bk ,Yk Ψ j∈Sk UDk ∪Bk ,Yk UAj | j∈S¯k Q t X j∈Tk |ΦUj ,UAj ΛΨk({j}∪Aj ) ΦUj ,UAj | 1 Q rj > log 2 |ΛUT′ | · j∈Tk |ΦUAj ΛΨk(Aj ) ΦtUA | ¯ j∈Tk

k

j

¯ k ∪B ¯k such that S¯k ∩ D ¯ k 6= ∅ and for all T¯k ⊆ W ¯ k such that T¯k 6= ∅, where ΛΨk , k ∈ [1 : N ] for all S¯k ⊆ D

is a block diagonal matrix with diagonal blocks   ′ Λ ′ G′t G′ Λ ′ ′ ΛUW + G j Yj j j Yj  j ΛΨj =  , j ∈ [1 : k]. ′t ΛYj′ Gj ΛYj′ ¯k ∪ B ¯k such that S¯k ∩ D ¯ k 6= ∅, and T¯k ⊆ W ¯k Proof: We apply Theorem 1. For k ∈ [1 : N ], S¯k ⊆ D

such that T¯k = 6 ∅, we have X

rj


j∈T¯k

X

I(Uj ; UTk [j]∪Dk , Yk |UAj )

X

h(Uj |UAj ) − h(UTk |UDk , Yk )

j∈Tk

=

j∈Tk

=

X

j∈Tk


h(Uj , UAj ) − h(UAj ) − h(UT′ k )

Q t j∈Tk |ΦUj ,UAj ΛΨk({j}∪Aj ) ΦUj ,UAj | 1 Q = log , 2 |ΛUT′ | · j∈Tk |ΦUAj ΛΨk(Aj ) ΦtUA | k

j

which proves Theorem 3.

January 29, 2014

DRAFT

24

Remark 4: If Uj , j ∈ [1 : K] has the form of G′′j UAj + Uj′′ as a special case of (15) for some matrix G′′j , where Uj′′ is independent of UAj , then we have |ΦUj ,UAj ΛΨk({j}∪Aj ) ΦtUj ,UA | j

|ΦUAj ΛΨk(Aj ) ΦtUA | j

= |ΛUj′′ |.

Proof of Lemma 1: By substituting (17) into (14), Yk is written as follows:   X X X ′ Fij UWj + Fi′ Yi  + Yk = Hki  Hkj Yj + Yk′ j∈[1:i]

i∈[1:k−1]

=

X

X

j∈[1:k−1]

Hki Fij UWj +

i∈[1:k−1] j∈[1:i] (a)

=

X

X

X

′ (Hkj Fj′ + Hkj )Yj + Yk′

j∈[1:k−1]

Hki Fij UWj +

j∈[1:k−1] i∈[j:k−1]

X

′ (Hkj Fj′ + Hkj )Yj + Yk′ ,

(22)

j∈[1:k−1]

where (a) is by changing the summation order. Next, by substituting (22) into (16), UWk is given as follows:   X X X X ′ ′ )Yj + Yk′  + UW UWk = Hki Fij UWj + Gkj UWj + G′k  (Hkj Fj′ + Hkj k j∈[1:k−1] i∈[j:k−1]

j∈[1:k−1]

=

X

(Gkj + G′k

j∈[1:k−1]

X

j∈[1:k−1]

X

Hki Fij )UWj +

i∈[j:k−1]

′ ′ . )Yj + G′k Yk′ + UW G′k (Hkj Fj′ + Hkj k

j∈[1:k−1]

Hence, we have  

UW k Yk



=

X



j∈[1:k−1]

Υkj 

UW j Yj



 + Ψk

(23)

where Υkj and Ψk are defined in (21) and (20), respectively. We prove Lemma 1 by solving the recursive formula in (23) using strong induction. For k = 1, t t Yjt ]t = Y1t ]t = Ψ1 from (23), and hence Lemma 1 holds trivially. For k > 1, assume that [UW [UW j 1 P i∈[1:j] Φji Ψi for all j < k . Then, (a)

t [UW Ykt ]t = k

X

t Υkj [UW Yjt ]t + Ψk j

j∈[1:k−1] (b)

=

X

X

Υkj Φji Ψi + Ψk

j∈[1:k−1] i∈[1:j] (c)

=

X

X

Υkj Φji Ψi + Ψk ,

(24)

i∈[1:k−1] j∈[i:k−1]

January 29, 2014

DRAFT

25

where (a) is from (23), (b) is from the induction assumption, and (c) is by chaning the summation order. Now, we have X

Υkj Φji = Υki +

j∈[i:k−1]

X

Υkj Φji

X

Υkj

j∈[i+1:k−1]

= Υki + =

Y

ΥS[i′ +1] S[i′ ]

S:{i,j}⊆S⊆[i:j] i′ ∈[1:|S|−1]

j∈[i+1:k−1]

X

X

Y

ΥS[i′ +1] S[i′ ]

S:{i,k}⊆S⊆[i:k] i′ ∈[1:|S|−1]

= Φki . t Ykt ]t = Hence, we have [UW k

P

j∈[1:k] Φkj Ψj ,

and Lemma 1 is proved by strong induction. A PPENDIX

A. Bounding the second term in the summation in (6) For given k ∈ [1 : N ], we have P (Ek,2 ∩

k−1 \

(Ej,1 ∪ Ej,2 ∪ Ej,3 )c )

j=1

′ ′ (n) n ′ n ˆ ¯ = L¯ , (lD ≤ P ((UD ¯k, L ¯ k 6= LD ¯ k ∪B ¯k s.t. lD ¯ k ∪B ¯k ), Yk ) ∈ Tǫk for some lD Dj Dj ,j k ∪Bk (n)

n n n n (UD (LD¯ j , LB¯j ), Yjn ) ∈ Tǫ(n) , (UD (LD¯ j ), UW for all j ∈ [1 : k − 1]) (LD¯ j , LW ¯ j ), Yj ) ∈ Tǫ′ j ∪Bj j j j j X n ˆ ¯ = L¯ , P ((UD for some lS′¯k s.t. li′ 6= Li for all i ∈ S¯k , L (lS′¯k , LS¯kc ), Ykn ) ∈ Tǫ(n) ≤ Dj Dj ,j k k ∪Bk ¯ ⊆D ¯ ∪B ¯ S k k k ¯ ∩D ¯ 6=∅ S k k

(n)

n n n n (UD (LD¯ j , LB¯j ), Yjn ) ∈ Tǫ(n) , (UD (LD¯ j ), UW (LD¯ j , LW ¯ j ), Yj ) ∈ Tǫ′ j ∪Bj j j j

j

for all j ∈ [1 : k − 1]).

¯k ∪ B ¯k such that S¯k ∩ D ¯ k 6= ∅, we have For S¯k ⊆ D n ˆ ¯ = L¯ , P ((UD for some lS′¯k s.t. li′ 6= Li for all i ∈ S¯k , L (lS′¯k , LS¯kc ), Ykn ) ∈ Tǫ(n) Dj Dj ,j k k ∪Bk (n)

n n n n (LD¯ j , LW (LD¯ j ), UW , (UD (LD¯ j , LB¯j ), Yjn ) ∈ Tǫ(n) (UD ¯ j ), Yj ) ∈ Tǫ′ j j j j ∪Bj j

for all j ∈ [1 : k − 1])

n ˆ ¯ = l¯ , =P ((UD (lS′¯k , lS¯kc ), Ykn ) ∈ Tǫ(n) for some lS′¯k s.t. li′ 6= li for all i ∈ S¯k , L Dj Dj ,j k ∪Bk k (n)

n n n n (lD¯ j , lW (lD¯ j ), UW , (UD (lD¯ j , lB¯j ), Yjn ) ∈ Tǫ(n) (UD ¯ j ), Yj ) ∈ Tǫ′ j j j j ∪Bj j

for all j ∈ [1 : k − 1],

LW ¯ k−1 ) ¯ k−1 for some lW ¯ k−1 = lW n n =P ((UD for some lS′¯k s.t. li′ 6= li for all i ∈ S¯k , UD (·) ∈ Aj (Yjn , lD¯ j ∪B¯j ), (lS′¯k , lS¯kc ), Ykn ) ∈ Tǫ(n) k j ∪Bj k ∪Bk n n (UD (lD¯ j ), UW (lD¯ j , ·)) ∈ Bj (Yjn , lD¯ j ∪W ¯ j ) for all j ∈ [1 : k − 1] for some lW ¯ k−1 ) j j

January 29, 2014

(25) DRAFT

26

where Aj (yjn , lD¯ j ∪B¯j ) is the ensemble of codebooks unDj ∪Bj (·) such that node j with received channel output yjn decodes ˆlD¯ j ,j as lD¯ j and the joint typicality (4) is satisfied for given lD¯ j ∪B¯j and Bj (yjn , lD¯ j ∪W ¯j) is the ensemble of codebooks (unDj (lD¯ j ), unWj (lD¯ j , ·)) such that node j with received channel output yjn and decoded index vector lD¯ j chooses lW¯ j and the joint typicality (5) is satisfied for given lD¯ j ∪W¯ j . More precisely, we define Aj (yjn , lD¯ j ∪B¯j ) , {unDj ∪Bj (·) : (unDj ∪Bj (˜lD¯ j ∪B¯j ), yjn ) ∈ / Tǫj for all ˜lD¯ j < lD¯ j , ˜lB¯j , (unDj ∪Bj (lD¯ j ∪B¯j ), yjn ) ∈ Tǫj } n n n n n lW / Tǫ′j for all ˜lW Bj (yjn , lD¯ j ∪W ¯j , ˜ ¯ j ), yj ) ∈ ¯ j < lWj , ¯ j ), uWj (lD ¯ j , ·)) : (uDj (lD ¯ j ), uWj (lD ¯ j ) , {(uDj (lD n (unDj (lD¯ j ), unWj (lD¯ j ∪W ¯ j ), yj ) ∈ Tǫ′j }.

Continuing with the bound in (25), we have n n (·) ∈ Aj (Yjn , lD¯ j ∪B¯j ), for some lS′¯k s.t. li′ 6= li for all i ∈ S¯k , UD (lS′¯k , lS¯kc ), Ykn ) ∈ Tǫ(n) P ((UD j ∪Bj k k ∪Bk n n (UD (lD¯ j ), UW (lD¯ j , ·)) ∈ Bj (Yjn , lD¯ j ∪W ¯ j ) for all j ∈ [1 : k − 1] for some lW ¯ k−1 ) j j n n n n (lD¯ j , ·)) ∈ Bj (Yjn , lD¯ j ∪W (lD¯ j ), UW (·) ∈ Aj (Yjn , lD¯ j ∪B¯j ), (UD , UD (lS′¯k , lS¯kc ), Ykn ) ∈ Tǫ(n) = P ((UD ¯j) j j j ∪Bj k k ∪Bk

for all j ∈ [1 : k − 1] for some lW¯ k−1 s.t. li 6= li′ for all i ∈ S¯k for some lS′¯k ) X n n ≤ P ((UD , UD (lS′¯k , lS¯kc ), Ykn ) ∈ Tǫ(n) (·) ∈ Aj (Yjn , lD¯ j ∪B¯j ), k k ∪Bk j ∪Bj ′ lS ¯

k

n n ′ ¯k ) (UD (lD¯ j ), UW (lD¯ j , ·)) ∈ Bj (Yjn , lD¯ j ∪W ¯ j ) for all j ∈ [1 : k − 1] for some lW ¯ k−1 s.t. li 6= li for all i ∈ S j j (a) X n n = (·) ∈ Aj (Yjn , lD¯ j ∪B¯j ), , UD (lS′¯k , L′′S¯c ), Y˜kn ) ∈ Tǫ(n) P ((UD j ∪Bj k k ∪Bk ′ lS ¯

k

k

′ n n ¯k ) (lD¯ j , ·)) ∈ Bj (Yjn , lD¯ j ∪W (lD¯ j ), UW (UD ¯ j ) for all j ∈ [1 : k − 1] for some lW ¯ k−1 s.t. li 6= li for all i ∈ S j j

(26) ′′ ˜¯ ¯ ) ˆ′′¯ (y n , l′¯ , un where Y˜kn is the channl output at node k assuming that decoding index ˆlD ¯ j ,j = lD j Sk Dj ∪Bj (lD j ∪Bj j ,j

¯j ∪ B ¯j )) and covering index l′′¯ = l′′¯ (y n , l′¯ , ˆl′′¯ , for all ˜lD¯ j ∪B¯j such that ˜li = 6 li′ for all i ∈ S¯k ∩ (D Wj j Sk Dj ,j Wj n ˆ′′ ′′ ˜ ¯ ) for all ˜l ¯ such that ˜li = ¯ j ) at node j ∈ [1 : k − 1] are 6 li′ for all i ∈ S¯k ∩ W unDj (ˆlD ¯ j ,j , lW ¯ j ,j ), uWj (lD Wj j

chosen according to the following rule: •

′′ ˜¯ : ˜ ¯ j } such that li = 6 li′ for all i ∈ S¯k ∩ D Find the smallest ˆlD ¯ j ,j ∈ {lD j

˜˜′′ ), y n ) ∈ T (n) ′′ (unDj ∪Bj (ˆlD ¯ j ,j , lB ¯j j ǫj

January 29, 2014

DRAFT

27 ′′ ∈ {˜ ¯j }. If there is no such index vector, let ˆl′′¯ be the lB¯j : ˜li = 6 li′ for all i ∈ S¯k ∩ B for some ˜˜lB ¯j Dj ,j

¯ j }. smallest one in {˜lD¯ j : ˜li = 6 li′ for all i ∈ S¯k ∩ D •

′′ ∈ {˜ ¯ j } such that lW li = 6 li′ for all i ∈ S¯k ∩ W Find the smallest lW ¯j : ˜ ¯j (n)

′′ n ˆ′′ n ′′ (unDj (ˆlD ¯ j ,j ), uWj (lD ¯ j ,j , lW ¯ j ), yj ) ∈ Tǫ′ . j

′′ be the smallest one in {˜ ¯ j }. lW li = 6 li′ for all i ∈ S¯k ∩ W If there is no such index vector, let lW ¯j : ˜ ¯j n (l ), U n (l , ·)) ∈ B (Y n , l n (·) ∈ Aj (Yjn , lD¯ j ∪B¯j ), (UD Note that (a) follows because if UD ¯j) ¯j ¯ j ∪W ¯j j j ¯j D D Wj D j ∪Bj

ˆ ′′¯ = l ¯ , L′′¯ = l ¯ for for all j ∈ [1 : k − 1] for some lW¯ k−1 such that li 6= li′ for all i ∈ S¯k , then L Wj Dj Dj ,j Wj

all j ∈ [1 : k − 1], and Y˜kn = Ykn . By discarding some constraints, (26) is upper-bounded by X n ). (lS′¯k , L′′S¯c ), Y˜kn ) ∈ Tǫ(n) P ((UD k k ∪Bk k

′ lS ¯

k

n (lS′¯k , L′′S¯c ), Y˜kn ) is given as follows: Now, we can show that the joint distribution of (UD k ∪Bk k

n (lS′¯k , L′′S¯c ) = unDk ∪Bk , Y˜kn = ykn ) = p(unSkc , ykn ) P (UD k ∪Bk k

n YY

p(uj,i |uAj ,i ),

(27)

j∈Sk i=1

where Sk is defined in (2). Now, we can obtain the following upper bound: X n ) (lS′¯k , L′′S¯c ), Y˜kn ) ∈ Tǫ(n) P ((UD k k ∪Bk k

′ lS ¯

k

= 2n

P

¯ j∈S k

X

rj

,ykn )∈Tǫ(n) (un k Sc k

≤ 2n

P

¯ j∈S k

X

rj

X

n YY

p(unSkc , ykn )

p(uj,i |uAj ,i )

j∈Sk i=1

(n) n n un Sk ∈Tǫk (USk |uS c ,yk ) k

p(unSkc , ykn ) · 2n(H(USk |USk ,Yk )+δ(ǫk )) · c

Y

2−n(H(Uj |UAj )−δ(ǫk ))

j∈Sk

(n)

n (un S c ,yk )∈Tǫk k

≤ 2n = 2n = 2n

P P P

¯ j∈S k

¯ j∈S k

¯ j∈S k

P

rj

· 2−n(

rj

· 2−n(

rj

· 2−n(

P

P

j∈Sk

H(Uj |UAj )−H(USk |USkc ,Yk )−(1+ν)δ(ǫk ))

c j∈Sk (H(Uj |UAj )−H(Uj |UAj ,USk [j] ,USk ,Yk ))−(1+ν)δ(ǫk ))

j∈Sk

I(Uj ;USk [j]∪Skc ,Yk |UAj )−(1+ν)δ(ǫk ))

,

which tends to zero as n tends to infinity if X X rj < I(Uj ; USk [j]∪Skc , Yk |UAj ) − (1 + ν)δ(ǫk ). j∈S¯k

Therefore, P (Ek,2 ∩

Tk−1

(28)

j∈Sk

c j=1 (Ej,1 ∪ Ej,2 ∪ Ej,3 ) )

tends to zero as n tends to infinity when (28) is satisfied

¯k ∪ B ¯k such that S¯k ∩ D ¯ k 6= ∅. for all S¯k ⊆ D January 29, 2014

DRAFT

28

B. Bounding the third term in the summation in (6) The proof follows similar steps to the mutual covering lemma in [37]. For given k ∈ [1 : N ], we have c c ) ∩ Ek,2 P (Ek,3 ∩ Ek,1 (n)

n n n n ≤ P ((UD (LD¯ k ), Ykn ) ∈ Tǫ(n) , (UD (LD¯ k ), UW (LD¯ k , lW / Tǫ′ for all lW ¯ k ), Yk ) ∈ ¯ k) k k k k k X n n (lD¯ k ) = unDk ) = (lD¯ k ) = unDk , Ykn = ykn )P (|Lk (lD¯ k , unDk , ykn )| = 0|UD P (LD¯ k = lD¯ k , UD k k (n)

n lD¯ k ,(un Dk ,yk )∈Tǫk

where (n)

n n n Lk (lD¯ k , unDk , ykn ) , {lW ¯ k : (uDk , UWk (lD ¯ k ∪W ¯ k ), yk ) ∈ Tǫ′ }. k

(n)

Consider lD¯ k and (unDk , ykn ) ∈ Tǫk . From the Chebyshev lemma, we have n (lD¯ k ) = unDk ) ≤ P (|Lk (lD¯ k , unDk , ykn )| = 0|UD k

n (l ) = un ) Var(|Lk (lD¯ k , unDk , ykn )||UD ¯k D Dk k . n n n n (E[|Lk (lD¯ k , uDk , yk )||UDk (lD¯ k ) = uDk ])2

Now, define the indicator function I(lW ¯k) =

  1

(n)

n n (l if (unDk , UW ¯ k ), yk ) ∈ Tǫ′ ¯ k ∪W D k

k

 0

otherwise P n n I(lW for each lW ¯ k ). ¯ k . Note that |Lk (lD ¯ k , uDk , yk )| = lW ¯ k

′ , and T ′ ,˜ ¯k ⊆ W ¯ k such lW Due to the symmetry of the codebook generation, for any lW ¯k, ˜ ¯ k , lW ¯k ¯ k lW

′ )|U n (l ) = 6 ˜li′ for all i ∈ / T¯k , we have E[I(lW that lT¯k = lT′¯k , ˜lT¯k = ˜lT′¯k and li 6= li′ , ˜li = ¯k ¯ k )I(lW ¯k Dk D ′ ′ )|l ′ )|U n (l ) = un ]. Let p ¯ ¯ unDk ] = E[I(˜lW¯ k )I(˜lW ¯ k )I(lW ¯k ¯ k T¯k = lT¯k , li 6= ¯k T¯k for Tk ⊆ Wk denote E[I(lW Dk Dk D n (l ) = un ]. li′ for all i ∈ / T¯k , UD ¯k D Dk k

Then, we have n E[|Lk (lD¯ k , unDk , ykn )||UD (lD¯ k ) = unDk ] = k

X

n E[I(lW¯ k )|UD (lD¯ k ) = unDk ] k

=

X

n n E[I 2 (lW ¯ k ) = uD k ] ¯ k )|UDk (lD

lW ¯ k

lW ¯ k

= 2n

January 29, 2014

P

¯ i∈W k

ri

pW ¯k

(29)

DRAFT

29

and n E[|Lk (lD¯ k , unDk , ykn )|2 |UD (lD¯ k ) = unDk ] = k

X X

X

′ n n E[I(lW ¯ k )I(lT¯k , lT¯c )|UDk (lD ¯ k ) = uD k ]

X

n n ′ E[I(lW ¯ k ) = uD k ] ¯ k )I(lT¯k , lT¯c )|UDk (lD

k

¯ ¯ ′¯ c :li′ 6=li ,∀i∈T¯c lW ¯ k k Tk ⊆Wk lT k

=

X X

k

¯ k lWk l′¯ c :li′ 6=li ,∀i∈T¯c T¯k ⊆W k T k

≤

P P n( i∈T¯ ri +2 i∈T¯ c ri )

X

2

k

k

pT¯k .

¯k T¯k ⊆W

Since p∅ = p2W ¯ k , we have n Var(|Lk (lD¯ k , unDk , ykn )||UD (lD¯ k ) = unDk ) k n n = E[|Lk (lD¯ k , unDk , ykn )|2 |UD (lD¯ k ) = unDk ] − E2 [|Lk (lD¯ k , unDk , ykn )||UD (lD¯ k ) = unDk ] k k P P X n( i∈T¯ ri +2 i∈T¯ c ri ) k k ≤ pT¯k . 2

(30)

¯ k ,T¯k 6=∅ T¯k ⊆W

¯ k such that T¯k = Now, for T¯k ⊆ W 6 ∅, we have (n)

(n)

k

k

′ n ′ n n n n (lD¯ k ∪W pT¯k = P ((unDk , UW ¯ k , lW ¯ k ), yk ) ∈ Tǫ′ , (uDk , UWk (lD ¯ k ), yk ) ∈ Tǫ′ |lT¯k = lT¯k , k n li 6= li′ for all i ∈ / T¯k , UD (lD¯ k ) = unDk ) k P P −n( j∈T I(Uj ;UTk [j]∪Dk ,Yk |UAj )+2 j∈T c I(Uj ;UTk ∪Tkc [j]∪Dk ,Yk |UAj )−2(1+ν)δ(ǫ′k ))

≤2

k

k

¯ k such that T¯k = by the joint typicality lemma [25], where Tk is defined in (3). Similarly, for T¯k ⊆ W 6 ∅,

we have P P −n( j∈T I(Uj ;UTk [j]∪Dk ,Yk |UAj )+ j∈T c I(Uj ;UTk ∪Tkc [j]∪Dk ,Yk |UAj )+(1+ν)δ(ǫ′k ))

pW ¯k ≥ 2

k

k

.

By substituting the above bounds into (29) and (30), we obtain n (l ) = un ) Var(|Lk (lD¯ k , unDk , ykn )||UD ¯k D Dk k n (l ) = un ])2 ≤ (E[|Lk (lD¯ k , unDk , ykn )||UD ¯k D Dk k

X

P

2−n(

¯ j∈T k

rj −

P

j∈Tk

I(Uj ;UTk [j]∪Dk ,Yk |UAj )−4(1+ν)δ(ǫ′k ))

¯ k ,T¯k 6=∅ T¯k ⊆W

c c n (l ) = un ) and thus P (E Therefore, P (|Lk (lD¯ k , unDk , ykn )| = 0|UD ¯k k,3 ∩ Ek,1 ∩ Ek,2 ) tend to zero as n D Dk k

tends to infinity if X

j∈T¯k

rj >

X

I(Uj ; UTk [j]∪Dk , Yk |UAj ) + 4(1 + ν)δ(ǫ′k )

(31)

j∈Tk

¯ k such that T¯k = for all T¯k ⊆ W 6 ∅.

January 29, 2014

DRAFT

.

30

C. Bounding the fourth term in the summation in (6) For given k ∈ [1 : N ], we have c c ) ∩ Ek,2 P (Ek,4 ∩ Ek,1 (n)

n n n (n) n ˆ ¯ = L¯ ) ≤ P ((UW / Tǫ′′ , L k−1 (LW ¯ k−1 ), Y[1:k] ) ∈ Tǫk , (UW k (LW ¯ k ), Y[1:k] ) ∈ Dk Dk ,k k

n ˆ ¯ , L ¯ k−1 ¯ ), Y n ) ∈ Tǫ(n) , ≤ P ((UW k−1 (LD W \Dk [1:k] k k ,k

=

(n) n n ˆ ¯ , L ¯ )) ∈ ˆ ¯ , L ¯ k−1 ¯ ), Y n , UW / Tǫ′′ ) (L (UW k−1 (LD Wk Dk ,k W \Dk [1:k] k k ,k k X n ˆ ¯ = l ¯ , U k−1 (l ¯ , L ¯ k−1 ¯ ) = un k−1 , Y n = y n ) P (L Dk W \Dk Dk Dk ,k W W [1:k] [1:k]

n lD¯ k ,(un ,y[1:k] )∈Tǫ(n) k W k−1

(n)

n n ˆ ¯ = l ¯ , U n k−1 (l ¯ , L ¯ k−1 ¯ ) = un k−1 , Y n = y n ) (lD¯ k , LW / Tǫ′′ |L , UW · P ((unW k−1 , y[1:k] ¯ k )) ∈ Dk W \Dk Dk Dk ,k W W [1:k] [1:k] k k

We use the following modified Markov lemma to bound the above, which can be proved from the proof of the Markov lemma in [9], [25] with some minor modification. Lemma 2: Consider random variables X, Y, Z, A such that X → Y → Z form a Markov chain. Let (n)

(xn , y n ) ∈ Tǫ

and a ∈ A. Suppose that P (Z n = z n |X n = xn , Y n = y n , A = a) = P (Z n = z n |Y n =

y n , A = a), where P (Z n = z n |Y n = y n , A = a) satisfies the following conditions for ǫ′ > ǫ: (n)

1) limn→∞ P ((y n , Z n ) ∈ Tǫ′ |Y n = y n , A = a) = 1. (n)

2) For every z n ∈ Tǫ′ (Z|y n ) and n sufficiently large, ′

P (Z n = z n |Y n = y n , A = a) ≤ 2−n(H(Z|Y )−δ(ǫ )) .

Then, for sufficiently small ǫ and ǫ′ such that ǫ < ǫ′ < ǫ′′ , (n)

lim P ((xn , y n , Z n ) ∈ Tǫ′′ |X n = xn , Y n = y n , A = a) = 1.

n→∞

(n)

n )∈T Fix lD¯ k and (unW k−1 , y[1:k] ǫk . We use Lemma 2 to show (n)

n n ˆ ¯ = l¯ , (lD¯ k , LW , UW lim P ((unW k−1 , y[1:k] ¯ k )) ∈ Tǫ′′ |L Dk Dk ,k k

n→∞

k

n n n n UW k−1 (lD ¯ k , LW ¯ k−1 \D ¯ k ) = uW k−1 , Y[1:k] = y[1:k] ) = 1.

(32)

Note that (UW k−1 \Dk , Y[1:k−1] ) − (UDk , Yk ) − UWk form a Markov chain and n n ˆ ¯ = l ¯ , U n k−1 (l ¯ , L ¯ k−1 ¯ ) = un k−1 , Y n = y n ) P (UW (lD¯ k , LW ¯ k ) = uWk |L W \Dk Dk Dk Dk ,k W W [1:k] [1:k] k n n n ˆ ¯ = l ¯ , UD (lD¯ k ) = unDk , Ykn = ykn ). (lD¯ k , LW = P (UW ¯ k ) = uWk |L Dk Dk ,k k k

January 29, 2014

DRAFT

31

¯ k such that T¯k = The first condition in Lemma 2 is safisfied if (31) is satisfied for all T¯k ⊆ W 6 ∅ since (n)

n ˆ ¯ = l ¯ , U n (l ¯ ) = un , Y n = y n ) P ((unDk , ykn , UW (lD¯ k , LW ¯ k )) ∈ Tǫ′ |L Dk Dk Dk Dk Dk ,k k k k k

(n)

n (lD¯ k , lW / Tǫ′ = 1 − P ((unDk , ykn , UW ¯ k )) ∈ k

k

(n)

n = 1 − P ((unDk , ykn , UW (lD¯ k , lW / Tǫ′ ¯ k )) ∈ k

k

n ˆ ¯ = l ¯ , UD (lD¯ k ) = unDk , Ykn = ykn ) for all lW ¯ k |L Dk Dk ,k k n n for all lW ¯ k |UDk (lD ¯ k ) = uDk )

and we have showed in the analysis of the third error event (n)

n (lD¯ k , lW / Tǫ′ lim P ((unDk , ykn , UW ¯ k )) ∈ k

n→∞

k

n (lD¯ k ) = unDk ) = 0 for all lW¯ k |UD k

under the aforementioned condition. (n)

Now, let us show that the second condition in Lemma 2 is satisfied. For every unWk ∈ Tǫ′ (UWk |unDk , ykn ), k

n n ˆ ¯ = l ¯ , U n (l ¯ ) = un , Y n = y n ) P (UW (lD¯ k , LW ¯ k ) = uWk |L Dk Dk Dk k k Dk Dk ,k k (n) n n n n n n ˆ n n n = P (UW (lD¯ k , LW ¯ k ) = uWk , UWk (lD ¯ k , LW ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )|L ¯ k , UDk (lD ¯ k ,k = lD ¯ k ) = uD k , Y k = y k ) D k k

(n) n n n n n n ˆ ¯ = l ¯ , UD (lD¯ k , LW (lD¯ k ) = unDk , Ykn = ykn , UW (lD¯ k , LW ≤ P (UW ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )) ¯ k ) = uWk |L Dk Dk ,k k k k k X n n ˆ ¯ = l ¯ , U n (l ¯ ) = un , Y n = y n , U n (l ¯ , L ¯ ) ∈ T (n) = P (LW ¯ k = lW ¯ k |L Dk Dk Dk k k W k Dk Dk Dk ,k Wk ǫ′ (UWk |uDk , yk )) k

lW ¯ k

n n n ˆ ¯ = l ¯ , L ¯ = l ¯ , UD (lD¯ k ) = unDk , Ykn = ykn , (lD¯ k ∪W · P (UW ¯ k ) = uWk |L Dk Wk Wk Dk ,k k k (n)

n n n (lD¯ k ∪W UW ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )). k k

For given lW ¯ k , we have n n n ˆ ¯ = l ¯ , L ¯ = l ¯ , UD (lD¯ k ) = unDk , Ykn = ykn , (lD¯ k ∪W P (UW ¯ k ) = uWk |L Dk Wk Wk Dk ,k k k (n)

n n n (lD¯ k ∪W UW ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )) k k

(a)

(n)

n n n n n n n (lD¯ k ∪W = P (UW ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )) ¯ k ) = uWk |UDk (lD ¯ k ∪W ¯ k ) = uDk , UWk (lD k k

=

n n n n (l P (UW ¯ k ) = uWk |UDk (lD ¯ k ) = uDk ) ¯ k ∪W D k (n) n (l n n n P (UW ¯ k ∪W ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )|UDk (lD ¯k ) = D k k

unDk )

′

≤ 2−n(H(UWk |UDk ,Yk )−δ(ǫk )) ,

January 29, 2014

DRAFT

32

where (a) is because n n ˆ ¯ = l ¯ , L ¯ = l ¯ , U n (l ¯ ) = un , Y n = y n , P (UW (lD¯ k ∪W ¯ k ) = uW k , L Dk Dk Dk Dk Wk Wk Dk ,k k k k (n)

n n n (lD¯ k ∪W UW ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )) k k

ˆ ¯ = l ¯ , U n (l ¯ ) = un , Y n = y n ) = P (L Dk Dk Dk k k Dk Dk ,k (n)

n n n n n n n (lD¯ k ∪W × P (UW ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )|UDk (lD ¯ k ) = uWk , UWk (lD ¯ k ) = uD k ) ¯ k ∪W k k

(n) n n n n ˆ ¯ = l ¯ , UD (lD¯ k ∪W (lD¯ k ) = unDk , Ykn = ykn , UW × P (LW ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )) ¯ k = lW ¯ k |L Dk Dk ,k k k k

ˆ ¯ = l ¯ , U n (l ¯ ) = un , Y n = y n , U n (l ¯ ¯ ) ∈ = P (L Dk Dk Dk k k Wk Dk ∪Wk Dk Dk ,k

(n) Tǫ′ (UWk |unDk , ykn ), LW ¯k k

= lW ¯k)

(n)

n n n n n n n (lD¯ k ∪W × P (UW ¯ k ) ∈ Tǫ′ (UWk |uDk , yk )). ¯ k ) = uWk |UDk (lD ¯ k ∪W ¯ k ) = uDk , UWk (lD k k

Hence, the second condition in Lemma 2 is satisfied. c ∩ E c ) tends to zero as n tends to infinity Now, from Lemma 2, (32) holds and hence P (Ek,4 ∩ Ek,1 k,2

¯ k such that T¯k = 6 ∅. for sufficiently small ǫk and ǫ′k if (31) is satisfied for all T¯k ⊆ W

R EFERENCES [1] K. Marton, “A coding theorem for the discrete memoryless broadcast channel,” IEEE Trans. Inf. Theory, vol. 25, pp. 306–311, May 1979. [2] T. S. Han and K. Kobayashi, “A new achievable rate region for the interference channel,” IEEE Trans. Inf. Theory, vol. 27, pp. 49–60, Jan. 1981. [3] H.-F. Chong, M. Motani, H. K. Garg, and H. El Gamal, “On the Han-Kobayashi region for the interference channel,” IEEE Trans. Inf. Theory, vol. 54, pp. 3188–3195, Jul. 2008. [4] S. I. Gelfand and M. S. Pinsker, “Coding for channel with random parameters,” Probl. Control Inf. Theory, vol. 9, pp. 19–31, 1980. [5] T. Weissman, “Capacity of channels with action-dependent states,” IEEE Trans. Inf. Theory, vol. 56, pp. 5396–5411, Nov. 2010. [6] A. D. Wyner and J. Ziv, “The rate-distortion function for source coding with side information at the decoder,” IEEE Trans. Inf. Theory, vol. 22, pp. 1–10, Jan. 1976. [7] H. Yamamoto, “Wyner-Ziv theory for a general function of the correlated sources,” IEEE Trans. Inf. Theory, vol. 28, pp. 803–807, Sep. 1982. [8] T. Berger, “Multiterminal source coding,” The Information Theory Approach to Communications, New York: SpringerVerlag, 1977. [9] S.-Y. Tung, “Multiterminal source coding,” Ph.D. dissertation, Cornell University, Ithaca, NY, 1978. [10] Z. Zhang and T. Berger, “New results in binary multiple descriptions,” IEEE Trans. Inf. Theory, vol. 33, pp. 502–521, Jul. 1987. [11] P. Minero, S. H. Lim, and Y.-H. Kim, “Hybrid coding: An interface for joint source-channel coding and network communication,” IEEE Trans. Inf. Theory, submitted for publication. [Online]. Available: http://arxiv.org/abs/1306.0530.

January 29, 2014

DRAFT

33

[12] B. Bandemer and A. El Gamal, “Interference decoding for deterministic channels,” IEEE Trans. Inf. Theory, vol. 57, pp. 2966–2975, May 2011. [13] ——, “An achievable rate region for the 3-user-pair deterministic interference channel,” in Proc. 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton House, UIUC, Illinois, USA, Sep. 2011, pp. 38–44. [14] S.-H. Lee and S.-Y. Chung, “A unified achievability theorem: More results,” in preparation. [15] T. M. Cover and A. El Gamal, “Capacity theorems for the relay channel,” IEEE Trans. Inf. Theory, vol. 25, pp. 572–584, Sep. 1979. [16] M. H. Yassaee and M. R. Aref, “Slepian-Wolf coding over cooperative relay networks,” IEEE Trans. Inf. Theory, vol. 57, pp. 3462–3482, Jun. 2011. [17] S. H. Lim, Y.-H. Kim, A. El Gamal, and S.-Y. Chung, “Noisy network coding,” IEEE Trans. Inf. Theory, vol. 57, pp. 3132–3152, May 2011. [18] J. Hou and G. Kramer, “Short message noisy network coding with a decode-forward option,” IEEE Trans. Inf. Theory, submitted for publication. [Online]. Available: http://arxiv.org/abs/1304.1692v2. [19] G. Kramer, M. Gastpar, and P. Gupta, “Cooperative strategies and capacity theorems for relay networks,” IEEE Trans. Inf. Theory, vol. 51, pp. 3037–3063, Sep. 2005. [20] L. Sankar, G. Kramer, and N. B. Mandayam, “Offset encoding for multiple-access relay channels,” IEEE Trans. Inf. Theory, vol. 53, pp. 3814–3821, Oct. 2007. [21] T. M. Cover and C. Leung, “An achievable rate region for the multiple-access channel with feedback,” IEEE Trans. Inf. Theory, vol. 27, pp. 292–298, May 1981. [22] D. Slepian and J. K. Wolf, “A coding theorem for multiple access channels with correlated sources,” Bell Syst. Tech. J., vol. 52, pp. 1037–1076, 1973. [23] T. M. Cover and A. El Gamal, “Achievable rates for multiple descriptions,” IEEE Trans. Inf. Theory, vol. 28, pp. 851–857, Nov. 1982. [24] A. Orlitsky and J. R. Roche, “Coding for computing,” IEEE Trans. Inf. Theory, vol. 47, pp. 903–917, Mar. 2001. [25] A. El Gamal and Y.-H. Kim, Network information theory.

Cambridge, U.K.: Cambridge Univ. Press, 2011.

[26] C. E. Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J., vol. 27, pp. 379–423 and 623–656, 1948. [27] ——, “Coding theorems for a discrete source with a fidelity criterion,” IRE Nat. Conv. Rec.,, pp. 142–163, Mar. 1959. [28] Y. Liang, “Multiuser communications with relaying and user cooperation,” Ph.D. dissertation, Univ. Illinois at UrbanaChampaign, Urbana, IL, 2005. [29] C. Nair and A. El Gamal, “The capacity region of a class of three-receiver broadcast channels with degraded message sets,” IEEE Trans. Inf. Theory, vol. 55, pp. 4479–4493, Oct. 2009. [30] R. Venkataramani, G. Kramer, and V. K. Goyal, “Multiple description coding with many channels,” IEEE Trans. Inf. Theory, vol. 49, pp. 2106–2114, Sep. 2003. [31] J. Wang, J. Chen, L. Zhao, P. Cuff, and H. Permuter, “On the role of the refinement layer in multiple description coding and scalable coding,” IEEE Trans. Inf. Theory, vol. 57, pp. 1443–1456, Mar. 2011. [32] T. Cover, A. El Gamal, and M. Salehi, “Multiple access channels with arbitrarily correlated sources,” IEEE Trans. Inf. Theory, vol. 26, pp. 648–657, Nov. 1980. [33] P. G. J. K¨orner, “Common information is far less than mutual information,” Probl. Control Inf. Theory, vol. 2, pp. 149–162, 1973.

January 29, 2014

DRAFT

34

[34] W. Witsenhausen, “On sequences of pairs of dependent random variables,” SIAM J. Appl. Math, vol. 28, pp. 100–113, 1975. [35] I. Csisz´ar and J. K¨orner, “Broadcast channels with confidential messages,” IEEE Trans. Inf. Theory, vol. 24, pp. 339–348, May 1978. [36] L.-L. Xie and P. R. Kumar, “An achievable rate for the multiple-level relay channel,” IEEE Trans. Inf. Theory, vol. 51, pp. 1348–1358, Apr. 2005. [37] A. El Gamal and E. C. van der Meulen, “A proof of Marton’s coding theorem for the discrete memoryless broadcast channel,” IEEE Trans. Inf. Theory, vol. 27, pp. 120–122, Jan. 1981.

January 29, 2014

DRAFT