IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.10 OCTOBER 2005
2808
PAPER
Special Section on Information Theory and Its Applications
Information-Spectrum Characterization of Broadcast Channel with General Source Ken-ichi IWATA†a) and Yasutada OOHAMA††b) , Members
SUMMARY This paper clarifies a necessary condition and a sufficient condition for transmissibility for a given set of general sources and a given general broadcast channel. The approach is based on the informationspectrum methods introduced by Han and Verd´u. Moreover, we consider the capacity region of the general broadcast channel with arbitrarily fixed error probabilities if we send independent private and common messages over the channel. Furthermore, we treat the capacity region for mixed broadcast channel. key words: general sources, general broadcast channels, information spectrum, joint source-channel coding, transmissibility, capacity region, mixed channel
1.
Introduction
The study of multi-terminal channels was started from Shannon’s paper in 1961 [1]. The broadcast channel is one of the typical multi-user channels, which was introduced by Cover [2] in 1972. For the surveys of broadcast channels, we can refer to [3] and its references. Han and Costa [4] have considered the matching problem for the transmission of sources and a broadcast channel. These problems are still far from being completely understood. In this paper, we confine our discussion to a class of block codes, and consider matching problem for the transmission of a set of general sources and a general broadcast channel. These results are based on a generalization of Feinstein’s lemma [5] and a generalization of Verd´u–Han’s lemma [6], [7], and are related to the extension of some results obtained by [8]. Theorem 1 in Sect. 3 clarifies a sufficient condition for transmissibility for a given set of general sources and a general broadcast channel, and Theorem 2 clarifies a necessary condition of the transmissibility for the same one. One of the fundamental problems for the channel coding is to find the amount of information that we can transmit reliably over the channel. However, the capacity region of the broadcast channel has not been clarified yet even in general. We derive upper and lower bounds for the region of Manuscript received January 24, 2005. Manuscript revised May 16, 2005. Final manuscript received June 24, 2005. † The author is with the Graduate School of Information Systems, the University of Electro-Communications, Chofu-shi, 1828585 Japan. †† The author is with the Department of Computer Science and Communication Engineering, Kyushu University, Fukuokashi, 812-8581 Japan. a) E-mail:
[email protected] b) E-mail:
[email protected] DOI: 10.1093/ietfec/e88–a.10.2808
the broadcast channels in Sect. 4.1 as a special case of Theorem 1 and Theorem 2 if arbitrarily fixed error probabilities are permitted. We treat the capacity region for mixed channel of two general broadcast channels in Sect. 4.2. Proofs of the main results are described in Sect. 5 based on the information-spectrum methods devised by Han [7]. 2.
General Source and General Broadcast Channel
Let Sk , k ∈ K be source alphabet which may be countable infinite set, where K denotes a set of all integers from 1 to its cardinality |K|, 1 ≤ |K| < ∞. We use calligraphic letters to indicate a set and use |K| to denote the cardinality of K. Let us denote the n-th Cartesian product of an alphabet Sk by Snk for each k ∈ K and n ∈ N, where N denotes a set of all positive integers, N = {1, 2, . . . }. For k ∈ K, let the kth general source defined by Sk = {S kn }n∈N , where S kn denotes a random variable vector (S 1(n) , S 2(n) , . . . , S n(n) )k ∈ Snk . The general source Sk generates outputs sk ∈ Snk of length n according to the distribution of random variables (S 1(n) , . . . , S n(n) )k for each k ∈ K and n ∈ N. Suppose we have a set of general correlated sources SK ≡ (Sk )k∈K ≡ ({S kn }n∈N )k∈K which generates outputs (sk )k∈K ∈ Sn1 ×Sn2 ×· · ·×Sn|K| . Let us use sK as an abbreviation for the sets of outputs (sk )k∈K ∈ Sn1 × Sn2 × · · · × Sn|K| for arbitrary fixed n ∈ N. Throughout the paper, the same abbreviation should be used for rann ≡ (S kn )k∈K , dom variables, alphabets and so on, that is S K n n n n SK ≡ S1 ×S2 ×· · ·×S|K| . We assume that the probability disn of outputs from general sources tribution PS Kn ≡ PS 1n S 2n ...S |K| SK is arbitrary for all n ∈ N, and we consider quite a wide class of sources. To encode the outputs from general sources SK by a block-length to block-length code, we consider a mapping from SnK to set of codewords of length n, and each codeword is used as input for the broadcast channel. Let us denote the set of codewords of block length n by Cn . In this case, there is only one encoder, and the encoder can observe output sK from SK . Let us denote a sequence of encoders by {ϕn }n∈N , ϕn : SnK → Cn ⊂ Xn , where the channel input alphabet Xn may be countable infinite set. The channel input X ≡ {X n = (X1(n) , X2(n) , · · · , Xn(n) )}n∈N depends on source outputs and {ϕn }n∈N , and X n is defined by n ). X n = ϕn (S K
c 2005 The Institute of Electronics, Information and Communication Engineers Copyright
IWATA and OOHAMA: INFORMATION-SPECTRUM CHARACTERIZATION OF BROADCAST CHANNEL WITH SOURCE
2809
We suppose that there is a general channel in order to transmit the information of source output to receivers. The channel has one input terminal and |J| output terminals, where J is index set of all integers from 1 to |J|, |J| denotes the cardinality of the set, and 1 ≤ |J| < ∞. If there are two or more receivers, |J| ≥ 2, then this channel is called the broadcast channel. Let Y j denote an alphabet on the jth output terminal of the channel, and Y j may be countable infinite set for each j ∈ J. In order to define a channel, we arbitrarily fix a collection of conditional probabilities Wn ≡ {W n (yJ |x)}x∈Xn ,yJ ∈YJn n n for each n ∈ N, where YJ ≡ Y1n × · · · × Y|J| . We call n W the transition probabilities from one input terminal to |J| output terminals of a channel as far as it satisfies W n (yJ |x) = 1 n yJ ∈YJ
for all x ∈ Xn . Let us call the sequence W ≡ {Wn }n∈N a general channel with one input terminal and |J| output terminals. Let us denote W nj (y j |x), the conditional probability of the jth output terminal for given input of the channel x ∈ Xn by W n (yJ |x), W nj (y j |x) ≡ n (yi )i∈J ∈Y1n ×···×Yin ×···×Y|J| , i j
and we define Wnj ≡ {W nj (y j |x)}x∈Xn ,y j ∈Ynj . Hence, this system satisfies the following property: Property a) PS Kn X n Y nj (sK , x, y j ) = PS Kn (sK )PX n |S Kn (x|sK )W nj (y j |x) (1)
n ≡ (S n ) Fig. 1 Sources S K , K ≡ {1, 2, . . . , |K|} and broadcast chank k∈K nel Wn of 1 input terminal and |J| output terminals, and the jth output n ≡ (S n ) , K j ⊂ K for ∀ j ∈ J ≡ {1, 2, . . . , |J|}. terminal decodes S K k k∈K j j
∅ denotes empty set. If we want to show these relations in figure then we obtain a sketch of Fig. 1. For output information sK ∈ SnK from SK , we define the error probability εn( j) (sK ) on the jth output terminal by εn( j) (sK ) = W nj (Dn( j) (sK j )c |ϕn (sK )), where Dn( j) (sK j )c denotes the complement set of Dn( j) (sK j ) with respect to Ynj , and W nj (Dn( j) (sK j )c |x) = y∈D(nj) (sK )c j
W nj (y|x). We define the error probability εn( j) on the jth terminal by PS Kn (sK )εn( j) (sK ) (2) εn( j) ≡ sK ∈SnK
for each j ∈ J and n ∈ N. Hereafter, we call a set (ϕn , (ψn( j) ) j∈J ) of one encoder and |J| decoders with a set of error probabilities (εn( j) ) j∈J a (ϕn , (ψn( j) ) j∈J , (εn( j) ) j∈J )-code. Definition 1: Set SK of sources is (ε j ) j∈J -transmissible def
⇐⇒ There exists a sequence of for channel W . (ϕn , (ψn( j) ) j∈J , (εn( j) ) j∈J )-codes satisfying
for all j ∈ J, n ∈ N, and (sK , x, y j ) ∈ SnK × Xn × Ynj . On the jth output terminal of the channel for each j ∈ J, the receiver can observe the output y j ∈ Ynj via the channel Wn , and can not see any one of the other terminals. He or she tries to reproduce the full or partial information of the set SK of sources that he or she wants to know. We denote the reproduced information on the jth output terminal by sK j ≡ (sk )k∈K j , sk ∈ Snk , K j ⊂ K, |K j | ≥ 1 corresponding to the encoded information sK ≡ (sk )k∈K for each j ∈ J. Thus K j denotes the index set of information that each receiver wants to know among K. We denote the decoder at the jth output terminal by ψn( j) : Ynj → SnK j . We assume that every decoder ψn( j) is deterministic mapping, and satisfies the relation Dn( j) (sK j ) ∩ Dn( j) (s¯ K j ) = ∅ for sK j s¯ K j , j ∈ J, where Dn( j) (sK j ) ≡ {y j ∈ Ynj |ψn( j) (y j ) = sK j }, and
lim sup εn( j) ≤ ε j n→∞
for ∀ j ∈ J.
If ε j = 0 for all j ∈ J on the above definition then we say that a set SK of sources is transmissible over a channel W , and there exists a sequence of (ϕn , (ψn( j) ) j∈J , (εn( j) ) j∈J )codes satisfies limn→∞ εn( j) = 0 for ∀ j ∈ J. We consider the conditions of transmissibility for a set of general sources and a general channel in the next section. 3.
Some Conditions for Joint Source-Channel Coding Problem
We describe some conditions for the joint source-channel coding problem such that a set SK of general sources is
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.10 OCTOBER 2005
2810
(ε j ) j∈J -transmissible over a general channel W . For a given set SK of general sources, we use the following abbreviation. We define collections of random varin n n ∀ ≡ (S kn )k∈A j , and S A A j ⊂ K j, ables S A c ≡ (S )k∈Ac for k j j j c A j ∅, where A j denotes the complement set of A j with respect to K j ⊂ K. For any subset A j ⊂ K j , A j ∅, and each j ∈ J, we define (conditional) mutual information density rate, and (conditional) entropy density rate as n n |S A i(Y nj ; S A c) ≡ j j
n PY nj |S Kn (Y nj |S K ) j j
1 log n , n PY nj |S An c (Y nj |S A c) j
j
1 n n log h(S A |S A c) ≡ j j n PS An
j
j
where Y nj denotes the random variables on the jth output n and the encoder terminal of the channel induced by the S K n n ϕn , and Y j can take values on Y j . For a set SK of sources and a channel W = {Wn }n∈N , we obtain the following theorems and corollaries. We prove Theorem 1 and 2 in Sects. 5.1 and 5.2, respectively. Theorem 1 (Direct theorem): For a set SK of sources and a channel W , let X be a channel input sequence, and let YJ be output sequence from the channel W corresponding to X, where (SK , X, YJ ) satisfies property a). If it holds that εj ≥ lim sup Pr |J| n→∞ A ⊂K , j
∀
j
A j ∅
n n |S A i(Y nj ; S A c) j j
n n ≤ h(S A |S A c) + γ j
j
(3)
∀
for γ > 0, j ∈ J, then the set SK of sources is (ε j ) j∈J transmissible over the channel W , where A j ⊂K j , BA j deA j ∅
notes the disjunction of statements BA j , A j ⊂ K j , A j ∅. Theorem 2 (Converse theorem): If a set SK of sources is (ε j ) j∈J -transmissible for a channel W , then it holds that ε j ≥ lim sup Pr n→∞
A j ⊂K j , A j ∅ n n n n |S A i(Y nj ; S A c ) ≤ h(S A |S Ac ) − γ j j j
j
A j ⊂K j , A j ∅
(4)
for ∀ γ > 0 and ∀ j ∈ J, where YJ denotes output sequences from the channel W corresponding to a channel input se quence X, and (SK , X, YJ ) satisfies property a). We also obtain the following corollaries for transmissible from the above theorems. Corollary 1: For a set SK of sources and a channel W , let X be a channel input sequence, and let YJ be output sequence from the channel W corresponding to X, where (SK , X, YJ ) satisfies property a). If it holds that
n n n n |S (5) lim Pr i(Y nj ; S A c ) ≤ h(S A |S c) + γ =0 A A j j
n→∞
j
j
SK of sources is transfor ∀ γ > 0, ∀ j ∈ J, then the set missible for the channel W , where A j ⊂K j , BA j denotes the A j ∅
conjunction of statements BA j , A j ⊂ K j , A j ∅.
Corollary 2: If a set SK of sources is transmissible for a channel W , then it holds that
n n n n |S (6) lim Pr i(Y nj ; S A c ) ≤ h(S A |S Ac ) − γ = 0 A j j A j ⊂K j , A j ∅
1 n n , n |S Ac (S A j |S Ac ) j
n→∞
j
j
for ∀ γ > 0 and ∀ j ∈ J, where YJ denotes output sequences from the channel W corresponding to a channel input se quence X, and (SK , X, YJ ) satisfies property a). Remark 1: The difference between (5) on Corollary 1 and (6) on Corollary 2 only appears in the signs of γ. If we neglect this difference of the signs, we may say that the combination of the above two corollaries essentially provides a necessary and sufficient condition for a set SK of sources is transmissible over a channel W . Remark 2: If we consider the special case of Theorem 1 with J = K = K1 = A1 = {1}, then (3) is rewritten as n n ) ≤ h(S K ) + γ ≤ ε1 , lim sup Pr i(Y1n ; S K n→∞
or, by using the notation of the channel input X, n n ) ≤ h(S K )+γ lim sup Pr i(Y1n ; X n ) − i(Y1n ; X n |S K n→∞
≤ ε1 .
(7)
where (SK , X, Y1 ) satisfies property a), and n i(Y1n ; X n |S K )≡
n PY n |X n S n (Y1n |X n , S K ) 1 log 1 K n n . n PY nj |S Kn (Y j |S K )
Because of the nonnegativity of the spectral conditional infmutual information rate n I(Y1 ; X|SK ) ≡ sup{α| lim Pr{i(Y1n ; X n |S K ) < α} = 0} n→∞
with the notation of [7], we have n lim sup Pr i(Y1n ; X n ) ≤ h(S K ) + γ ≤ ε1 n→∞
from (7). Thus, by replacing γ with a sequence {γn }n∈N , which satisfies γn > γn+1 > 0, lim γn → 0 and lim nγn → ∞, n→∞
n→∞
(8)
Theorem 1 can be reduced to Theorem 3.8.5 in [7]. Through the same argument, Theorem 2 is also reduced to Theorem 3.8.6 in [7] by using the relation (4) with J = K = K1 = A1 = {1}. Hence these results are generalizations of joint source-channel coding problem with one input terminal and one output terminal.
IWATA and OOHAMA: INFORMATION-SPECTRUM CHARACTERIZATION OF BROADCAST CHANNEL WITH SOURCE
2811
4.
4.1 (ε1 , ε2 )-Capacity Region of the Broadcast Channels
Capacity Region of Broadcast Channels
The results in the foregoing sections increase the understanding of transmission of information of a set of general sources through a general broadcast channel by using information-spectrum methods. One of the fundamental problems with the broadcast channels is to find a characterization of the capacity region defined by the set of all achievable rates. In this section, we now consider the capacity region of broadcast channels with two output terminals as a special case of the general channel W . We now assume that two private and one common independent messages are transmitted over this channel, and the each receiver decodes both the private and common messages. Hence, by using previous notations J, K, K j and etc., we define J = {1, 2}, K = {1, 2, 3}, K1 = {1, 3}, and K2 = {2, 3}, and three sources S kn , k = 1, 2, 3 in accordance with the uniformly and inde-
In order to describe the (ε1 , ε2 )-capacity region BW (ε1 , ε2 ) of the broadcast channel W , we define J j (R j , R3 |U j , U3 , X) for j = 1, 2 by J j (R j , R3 |U j , U3 , X) ≡ lim sup Pr{[i(Y nj ; U nj |U 3n ) ≤ R j ] ∨ [i(Y nj ; U3n ) ≤ R3 ]}, n→∞
Y nj
where denotes the random variables on the jth output terminal of the channel Wn induced by the X n for n ∈ N. Moreover, we define the following sets RW (ε1 , ε2 |U1 , U2 , U3 , X) = Cl{(R1 , R2 , R3 )|J j (R j , R3 |U j , U3 , X) ≤ ε j , for j = 1, 2}, RW (ε1 , ε2 ) = RW (ε1 , ε2 |U1 , U2 , U3 , X),
(k)
(k) Mn pendent distributed on the message sets M(k) n = {mak }ak =1 , that is 3 (1) (2) (3) (k) PS 1n S 2n S 3n (sa1 , sa2 , sa3 ) = k=1 P Mn(k) (mak ) 1 P M (k) (m(k) ak ) = |M(k) | n n
for ak ∈ {1, 2, · · · , |M(k) n |}, k = 1, 2, 3 and n ∈ N. Thus the message set M(3) n is a common message set for both the (2) first and the second receivers, and M(1) n and Mn is a private message set for the first receiver and the second receiver, respectively. The codebook Cn is defined by , Cn = ϕn (m(1) , m(2) , m(3) ) (k) (k)
(U1 ,U2 ,U3 ,X)∈P∗
where Cl() denotes the closure operation, and P∗ denotes the set of all (U1 , U2 , U3 , X) satisfying property b) below: Property b) (U1n , U2n , U3n ) − X n has Markov chain property for n ∈ N, and Ukn1 and Ukn2 is independent random variable for n ∈ N, k1 k2 , 1 ≤ k1 , k2 ≤ 3, i.e., PU1n U2n U3n X n (u1 , u2 , u3 , x) = PU1n (u1 )PU2n (u2 )PU3n (u3 )PX n |U1n U2n U3n (x|u1 , u2 , u3 ) (9) holds for all n ∈ N and all (u1 , u2 , u3 , x) ∈ U1n × U2n × U3n × Xn .
m ∈Mn ,k=1,2,3
and we call Cn with the error probability εn( j) of the jth output ( j) terminal for j = 1, 2 the (n, (|M(k) n |)k=1,2,3 , (εn ) j=1,2 )-code, ( j) where εn is defended by (2). Let (Rk )k=1,2,3 be any nonnegative real numbers. We say that the set of rates (R1 , R2 , R3 ) is (ε1 , ε2 )-achievable for ( j) W if there exists a sequence of (n, (|M(k) n |)k=1,2,3 , (εn ) j=1,2 )codes satisfying 1 lim inf log |M(k) n | ≥ Rk , n→∞ n lim sup εn( j) ≤ ε j , n→∞
for k = 1, 2, 3, for j = 1, 2.
We have following theorem for (ε1 , ε2 )-capacity region BW (ε1 , ε2 ). Theorem 3: The (ε1 , ε2 )-capacity region BW (ε1 , ε2 ) of broadcast channel W is bounded by ε ε 1 2 , RW ⊂ BW (ε1 , ε2 ) ⊂ RW (ε1 , ε2 ) . 2 2 In cast that (ε1 , ε2 ) = (0, 0), BW can be rewritten as BW = RW (U1 , U2 , U3 , X), (U1 ,U2 ,U3 ,X)∈P∗
where RW (U1 , U2 , U3 , X) is equal to
If ε j = 0 for j = 1, 2 on the above definition then we say that the set of rates (R1 , R2 , R3 ) is achievable for W . The set of all (ε1 , ε2 )-achievable rate is called the (ε1 , ε2 )-capacity region for W , and denoted it by BW (ε1 , ε2 ) = {(Rk )k=1,2,3 |Rk ≥ 0 for k = 1, 2, 3 and (Rk )k=1,2,3 is (ε1 , ε2 )-achievable for given W }. We use BW as an abbreviation for BW (0, 0), and BW is called the capacity region for W .
RW (U1 , U2 , U3 , X) = {(R1 , R2 , R3 )|0 ≤ R j ≤ I(Y j ; U j |U3 ), 0 ≤ R3 ≤ I(Y j ; U3 ) for j = 1, 2}, and
(10)
I(Y j ; U j |U3 ) ≡ sup α| lim Pr{i(Y nj ; U nj |U3n ) < α} = 0 , n→∞ I(Y j ; U3 ) ≡ sup α| lim Pr{i(Y nj ; U3n ) < α} = 0 n→∞
for j = 1, 2.
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.10 OCTOBER 2005
2812
The proof of Theorem 3 is given in Sect. 5.3. Remark 3: We obtain Theorem 3, but, to bound the range of the auxiliary random variables U j , j = 1, 2, 3 are still unsolved problem for general broadcast channel. Theorem 2 and its comment of Han and Costa [4] give the partial result to bound the range of the auxiliary random variables for the discrete memoryless stationary broadcast channel. Remark 4: On the above theorem, the inner bound and the outer bound of BW (ε1 , ε2 ) are not identical for ε1 > 0 and/or ε2 > 0. A stricter result for BW (ε1 , ε2 ) is still an unsolved problem. If W is discrete memoryless case and one of the private message rate is equal to 0 then K¨orner and Marton [9] obtain BW (ε, ε) for all 0 < ε < 1. But, for general W , the inner bound and the outer bound of BW (ε1 , ε2 ) may not be identical.
In this subsection, let us turn our attention to finding the capacity region of mixed broadcast channels as an application example of Theorem 3 based on the approach devised by Han(cf. [7]). Thus let us consider a special case that a broadcast channel W is the mixed channel of the component channels W [α1 ] and W [α2 ] until the end of this subsection. We now define the channel W = {{W n (y1 , y2 |x)}(x,y1 ,y2 )∈Xn ×Y1n ×Y2n }n∈N by 2
n
αi W [αi ] (y1 , y2 |x),
(11)
i=1
for all n ∈ N, x ∈ Xn , y1 ∈ Y1n , y2 ∈ Y2n , where α1 > 0, α2 > 0 are constant such that α1 + α2 = 1. In order to establish a capacity formula for the mixed channel, let U1 = {U 1n }n∈N , U2 = {U2n }n∈N , U3 = {U3n }n∈N , and X = {X n }n∈N , be arbitrary auxiliary and input processes, respectively, and n i] }n∈N the corresponding let Y j = {Y nj }n∈N and Y j[αi ] = {Y [α j the output processes be defined by (9) and PU1n U2n U3n X n Y1n Y2n (u1 , u2 , u3 , x, y1[αi ] , y2[αi ] ) = PU1n (u1 )PU2n (u2 )PU3n (u3 ) n
· PX n |U1n U2n U3n (x|u1 , u2 , u3 )W [αi ] (y1[αi ] , y2[αi ] |x) for all n ∈ N, i = 1, 2, j = 1, 2, u1 ∈ U1n , u2 ∈ U2n , u3 ∈ U3n , x ∈ Xn , y1 ∈ Y1n , and y2 ∈ Y2n . For these relations, we have following lemma. Lemma 1: I(Y j ; U j |U3 ) = min{I(Y j[α1 ] ; U j |U3 ), I(Y j[α2 ] ; U j |U3 )}, I(Y j ; U3 ) = min{I(Y j[α1 ] ; U3 ), I(Y j[α2 ] ; U3 )} for j = 1, 2.
(U1 ,U2 ,U3 ,X)∈P∗
[m] where RW (U1 , U2 , U3 , X) is the set of all triple (R1 , R2 , R3 ) such that
0 ≤ R j ≤ min{I(Y j[α1 ] ; U j |U3 ), I(Y j[α2 ] ; U j |U3 )}, 0 ≤ R3 ≤ min{I(Y j[α1 ] ; U3 ), I(Y j[α2 ] ; U3 )} for j = 1, 2. 5.
Proofs of Theorems
5.1 Proof of Theorem 1 Theorem 1 is obtained from the following Lemma 2.
4.2 Capacity Region of Mixed Broadcast Channel
W n (y1 , y2 |x) =
Theorem 4: The capacity region of the mixed broadcast channels W , which is defined by (11), is given by [m] RW (U1 , U2 , U3 , X), BW =
The proof of this lemma follows from Lemma 7.9.1 and Lemma 3.3.1 in [7]. From Theorem 3 and Lemma 1, we obtain the following capacity formula for the mixed broadcast channel.
Lemma 2: For a set SK of sources and a channel W , let X be a channel input sequence, and let YJ be output sequence from the channel W corresponding to X, such that (SK , X, YJ ) satisfies property a). Then there exists a (ϕn , (ψn( j) ) j∈J , (εn( j) ) j∈J )-code satisfying εn( j) n n n n n ≤ Pr i(Y j ; S A j |S Ac ) ≤ h(S A j |S Ac ) + γ j j |J| A ⊂K , j
j
A j ∅
+ (2|K j | − 1)e−nγ
(12)
n for S K , Wn , n ∈ N, j ∈ J, where γ > 0 is an arbitrary constant.
[Proof of Lemma 2]: It suffices to prove the existence of a sequence of block codes which satisfies (12) by using the random coding methods. Generation of Codebook : For each sK ∈ SnK , we independently generate ϕn (sK ) ∈ Xn randomly according to the conditionally probability PX n |S Kn (·|sK ), and define ϕn (sK ) as a codeword for sK . Both the encoder and every decoder are also assumed to know the codebook. Thus, we define the encoder ϕCn : SnK → Cn ⊂ Xn , where Cn = {ϕn (sK ) ∈ Xn |sK ∈ SnK }. Encoding ϕCn : SnK → Cn . The encoder ϕCn can see sK from the correlated sources SK , and send ϕCn (sK ) as the input of the broadcast channel. Decoding ψC( j)n : Ynj → SnK j for ∀ j ∈ J. The jth decoder receives the jth output of the broadcast channel Wn . In order to define the jth decoder ψC( j)n : Ynj → SnK j for ∀ j ∈ J, we set (K ) (A ) Tn j ≡ Tn j , A j ⊂K j , A j ∅
(A j )
Tn
≡ (sK j , y j ) ∈ SnK j × Ynj | i(y j ; sA j |sAcj ) > h(sA j |sAcj ) + γ
IWATA and OOHAMA: INFORMATION-SPECTRUM CHARACTERIZATION OF BROADCAST CHANNEL WITH SOURCE
2813
for ∀ A j ⊂ K j , A j ∅. For each j ∈ J, suppose that a decoder ψC( j)n receives an output y j ∈ Ynj from the channel Wn . If there exists a unique sK j ∈ SnK j
n sK ∈SnKx∈X , y j ∈Ynj , s.t. (K ) (sK j ,y j )Tn j
(K )
satisfying (sK j , y j ) ∈ Tn j then we declare ψC( j)n (y j ) = sK j . If there exist no such sK j or exist more than one such sK j then let us fix sˆ K j ∈ SnK j and declare ψCn (y) = sˆ K j . We suppose that sK ∈ is a set of source outputs, and ϕCn (sK ) is sent from the encoder, and output sequence y j is observed at the jth terminal, according to a probability Wnj for j ∈ J. If (sK j , y j ) holds the following both conditions: SnK
(K )
1) (sK j , y j ) ∈ Tn j 2) There is no s¯ K j ∈ SnK j such that s¯ K j sK j and (K j )
(s¯ K j , y j ) ∈ Tn
decoding error. Let us define the probability of error Pe( nj) (Cn ) at the jth terminal for a encoder and decoder pair (ϕCn , ψC( j)n ) by n Pe( nj) (Cn ) ≡ Pr{S K ψC( j)n (Y nj )}, j
where
Qi( j) (Cn )
Qi( j) (Cn )
n Pr{(S K , Y nj ) j
Q1( j) (Cn )
=
∈
∩
∈ Q1( j) (Cn )c } n Pr{(S K , Y nj ) ∈ Q2( j) (Cn )c }, j
is defined for
≡ {(sK j , y j ) ∈
(ϕCn , ψC( j)n )
SnK j
×
sK c ∈SK c
=
sK ∈SnK , Cn y j ∈Ynj s.t. (K j )
(sK j ,y j )Tn
j
,Y nj (sK j , y j )
j
(K j )
(sK j ,y j )Tn
(K j )
n , Y nj ) Tn = Pr{(S K j
}.
(14)
For the second term of the right-hand side of (13), we repeat the same technique, and we have the following chain: =
PCn (Cn )P
PCn (Cn )
sK ∈SnK , y j ∈Ynj
(sK j ,y j )∈Q2 (Cn )c
PS Kn (sK j ) j
PS Kn c |S Kn (sK cj |sK j )W nj (y j |ϕCn (sK )) j
j
j
Cn s.t.∃ s¯ K j ∈SnK :
PCn (Cn )PS Kn (sK )W nj (y j |ϕCn (sK ))
j
s¯ K j sK j ,
(K j )
≤
sK ∈SnK , y j ∈Ynj
(s¯ K j ,y j )∈Tn
s¯ K j ∈SnK j
x∈Xn
s.t.
PS Kn (sK )W nj (y j |x)
s¯ K j sK j (K j )
(s¯ K j ,y j )∈Tn
PCn (Cn )
Cn s.t. x=ϕCn (sK )
=
sK ∈SnK , y j ∈Ynj
s¯ K j ∈SnK j
s.t. x∈Xn
PS Kn (sK )W nj (y j |x)
s¯ K j sK j (K j )
(s¯ K j ,y j )∈Tn
=
j
(sK )W nj (y j |ϕCn (sK ))
· PX n |S Kn (x|sK )
PS Kn Y nj (sK , y j )
sK ∈SnK , s¯ K j ∈SnK s.t. j y j ∈Ynj s¯ K j sK j
(K j )
j
n SK
n PCn (Cn ) Pr{(S K , Y nj ) ∈ Q2( j) (Cn )c } j
Cn
(13)
Ynj
j
PS Kn X n Y nj (sK , x, y j )
y j ∈Ynj , s.t.
PS Kn c |S Kn (sK cj |sK j )W nj (y j |ϕCn (sK )) j
PS Kn
sK j ∈SnK ,
=
by
PS Kn (sK j )
(sK j ,y j )∈Q(1j) (Cn )c
j
PCn (Cn )
Cn
=
j
for i = 1, 2. We now evaluate the expectation of (13) with respect to the random selection over codebooks. Let PCn denote the probability distribution on the set of Cn . We estimate the expectation of the decoding error probability with respect to the code. For the first term on the right-hand side of (13), we have n PCn (Cn ) Pr{(S K , Y nj ) ∈ Q1( j) (Cn )c } j
x∈Xn
(K j )
|(sK j , y j ) satisfies condition i)}
Cn
sK ∈SnK , y j ∈Ynj , s.t.
sK c ∈SK c
Q2( j) (Cn )}
n Pr{(S K , Y nj ) j
+
(sK j ,y j )Tn
Then, the Pe( nj) (Cn ) can be bounded, and we have ≤
=
Cn
where Y nj denotes the corresponding channel output at the n jth terminal via Wnj due to the channel input ϕCn (S K ). ≤1−
PS Kn (sK )W nj (y j |x)PX n |S Kn (x|sK )
,
then the value sK j is uniquely determined on SnK j without
Pe( nj) (Cn )
=
=
(s¯ K j ,y j )∈Tn
sK ∈SnK , A j ⊂K j , s¯ K j ∈SnK s.t. j y j ∈Ynj A j ∅ (s¯ k sk )k∈A , j s¯ Ac =sAc , j
j
(K j )
(s¯ K j ,y j )∈Tn
PS Kn Y nj (sK , y j )
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.10 OCTOBER 2005
2814
≤
sK ∈SnK , y j ∈Ynj
A j ⊂K j , A j ∅
( j) Lemma 3: ∀ n ∈ N, ∀ γ > 0, ∀ (ϕn , (ψn ) j∈J , ( j) n n (εn ) j∈J )-code for S K and W , we have n n n n |S ) ≤ h(S |S ) − γ εn( j) ≥ Pr i(Y nj ; S A A j Ac Ac j
PS Kn Y nj (sK , y j )
s¯ K j ∈SnK j
s.t. s¯ Ac =sAc , j
j
(K j )
=
(s¯ K j ,y j )∈Tn
A j ⊂K j , sAc ∈SnAc , s¯ K j ∈SnK s.t. j j j A j ∅ s¯ Ac =sAc , y j ∈Ynj j j
PS An c Y nj (sAcj , y j ) j
=
(s¯ K j ,y j )∈Tn
A j ⊂K j , sAc ∈SnAc , s¯ K j ∈SnK s.t. j j j A j ∅ s¯ Ac =sAc , y j ∈Ynj j j
PS An c (sAcj )PY nj |S An c (y j |sAcj ). j
j
(K j )
(s¯ K j ,y j )∈Tn
(15) For (s¯ K j , y j ) ∈
(K ) Tn j
with s¯
Acj
for ∀ j ∈ J, where Y nj denotes the random variable set of outputs on the jth output terminal of the channel Wn induced n ), and Y nj can take values on Ynj . the channel input ϕn (S K [Proof of Lemma 3]: Let us define (A ) Ln j , Ln( j) ≡ A j ⊂K j , A j ∅
= s , we have Acj
(A j )
Ln
PY nj |S An c (y j |sAcj ) j
n (s ¯ A j |sAcj )PY nj |S¯ nA S n c (y j |s¯ A j , sAcj )e |S A c A j
j
−nγ
j
j
(A )
≤ e−nγ
A j ⊂K j , sAc ∈SnAc , s¯ K j ∈SnK s.t. j j j A j ∅ s¯ Ac =sAc , y j ∈Ynj j j
· PS An
j
PS An c (sAcj ) j
j
j
for A j ⊂ K j , A j ∅, ≡ y j ∈ Ynj |(sK j , y j ) ∈ Ln( j) , (A ) (A ) Ln j (sK j ) ≡ y j ∈ Ynj |(sK j , y j ) ∈ Ln j
Ln( j) (sK j )
for each j ∈ J. We also denote a decoding region Dn( j) (sK j ) for sK j ∈ SnK j on the output alphabet for the jth output terminal of the channel by Dn( j) (sK j ) = {y j ∈ Ynj |ψn( j) (y j ) = sK j }
n (s ¯ A j |sAcj )PY nj |S¯ nA S n c (y j |s¯ A j , sAcj ) |S A c A j
∀
from the definition of Tn j . Substitution of the above inequality into (15) gives n PCn (Cn ) Pr{(S K , Y nj ) ∈ Q2( j) (Cn )c } j Cn
≡ (sK j , y j ) ∈ SnK j × Ynj | n n n n i(Y nj ; S A |S A c ) ≤ h(S A |S Ac ) − γ j j
j
≤ PS An
j
− (2|K j | − 1)e−nγ
(K j )
j
A j ⊂K j , A j ∅
j
= (2|K j | − 1)e−nγ .
(16)
using the given ψn( j) for each j ∈ J. Then we have Dn( j) (sK j ) ⊂ Ynj ,
(18)
sK j ∈SK n
Using (13), (14), and (16), we obtain PCn (Cn )Pe( nj) (Cn )
j
Dn( j) (sK j )
Cn
∩ Dn( j) (s¯ K j ) = ∅ for sK j s¯ K j .
(19)
Using the above notation, we have (K j )
n , Y nj ) Tn ≤ Pr{(S K j
} + (2|K j | − 1)e−nγ .
Since Cn PCn (Cn )Pe( nj) (Cn ) is bounded for each j ∈ J, the above random coding argument shows the existence of at least one good (ϕn , (ψn( j) ) j∈J , (εn( j) ) j∈J )-code that satisfies the desired performance εn( j) (K ) n ≤ Pr{S K Y nj Tn j } + (2|K j | − 1)e−nγ j |J|
n , Y nj ) ∈ Ln( j) } Pr{(S K j = PS Kn Y nj (sK j , Ln( j) (sK j )) sK j ∈SnK
=
sK j ∈SnK j
(17)
for all j ∈ J, simultaneously. This can be understood in the following way. If there is no (ϕn , (ψn( j) ) j∈J , (εn( j) ) j∈J )-code that satisfies (17) for all j ∈ J, then we have a contradiction. Lemma 2 is proved.
+
PS Kn
sK j ∈SnK j
j
( j) Y nj (sK j , Ln (sK j )
PS Kn
≤ εn( j) +
sK j ∈SnK j
≤ εn( j) +
j
∩ (Dn( j) (sK j ))c )
( j) Y nj (sK j , Ln (sK j )
PS Kn
j
∩ Dn( j) (sK j ))
( j) Y nj (sK j , Ln (sK j )
∩ Dn( j) (sK j ))
A j ⊂K, sK j ∈SnK j A j ∅
5.2 Proof of Theorem 2 Theorem 2 is obtained from the following Lemma 3.
j
j
PS Kn
j
(A j ) Y nj (sK j , Ln (sK j )
∩ Dn( j) (sK j ))
(20)
IWATA and OOHAMA: INFORMATION-SPECTRUM CHARACTERIZATION OF BROADCAST CHANNEL WITH SOURCE
2815
for each j ∈ J, where the first inequality follows from the definition of (ϕn , (ψn( j) ) j∈J , (εn( j) ) j∈J )-code. On the second term on the right-hand side of (20), we define βn( j) (A j ) by (A ) PS Kn Y nj (sK j , Ln j (sK j ) ∩ Dn( j) (sK j )) βn( j) (A j ) ≡ sK j ∈SnK
j
j
for j ∈ J, A j ⊂ K j and A j ∅. Then βn( j) (A j ) is rewritten by βn( j) (A j ) = PS Kn (sK j ) j
sK j ∈SnK j
PY nj |S Kn (y j |sK j ). j
(A ) y j ∈Ln j (sK j )∩D(nj) (sK j )
e−nγ PY nj |S An c (y j |sAcj ) j
PS An
j
(A j )
for y j ∈ Ln
j
n (s |S A A j |sAcj ) c j
(sK j ), j ∈ J. Therefore, we have
βn( j) (A j ) ≤ e−nγ
sK j ∈SnK
≤ e−nγ
sK j ∈SnK
j
PY nj |S An c (y j |sAcj )
PS Kn (sK j )
j
y j ∈D(nj) (sK j )
j
PS An c (sAcj ) j
If Acj = ∅ then we have βn( j) (A j ) ≤ e−nγ
PS An
|S n c j Aj
(sA j |sAcj )
j
PY nj (y j )
= e−nγ ,
(21)
where the last inequality comes from (18) and (19). On the other hand, if A j ∅, Acj ∅, then we have PS An c (sAcj ) βn( j) (A j ) ≤ e−nγ j
j
j
PY nj |S An c (∪sA j ∈SA j Dn( j) (sK j )|sAcj ) j −nγ ≤e PS An c (sAcj ) sAc ∈SnAc j
j
j
= e−nγ , where the second inequality is due to (18), (19), and the fact that ∪sA j ∈SA j Dn( j) (sK j ) is subset of ∪sK j ∈SA j Dn( j) (sK j ) when sAcj is given. Combining this and (21), we have βn( j) (A j ) ≤ (2|K j | − 1)e−nγ A j ⊂K j , A j ∅
5.3 Proof of Theorem 3 Theorem 3 is established by proving both direct part and converse part. Let us show the proof of the direct part at first. It suffices to show that any rate (R1 , R2 , R3 ) satisfying J j (R j , R3 |U j , U3 , X) ≥ ε j /2, j = 1, 2 is achievable. To this end, we use Lemma 4 below.
εn( j) 1 ≤ Pr i(Y nj ; X n |U nj ) ≤ log |Mn( j) | + γ 2 n 1 n n ∨ i(Y j ; U3 ) ≤ log |M(3) | + γ + 2e−nγ , n n (22)
[Proof of Lemma 4]: In order to prove Lemma 4, it is sufficient to show that there exist a block ( j) (n, (|M(k) n |)k=1,2,3 , (εn ) j=1,2 )-code satisfies (22) for j = 1, 2.
sK j ∈SnK y j ∈D(nj) (sK ) j j
sAc ∈SnAc
for all j ∈ J, and Lemma 3 is proved.
for Wn , n ∈ N, and j = 1, 2 where γ > 0 is an arbitrary constant.
PY nj |S An c (y j |sAcj ).
y j ∈D(nj) (sK j )
j
n Pr{(S K , Y nj ) ∈ Ln( j) } ≤ εn( j) + (2|K j | − 1)e−nγ j
Lemma 4: For a channel W with two output terminals, let X be a channel input sequence, and let (U1 , U2 , U3 , X) satisfy property b). and let (Y1 , Y2 ) be output sequence from the channel W corresponding to X, Then there exists a (ϕn , (ψn( j) ) j=1,2 , (εn( j) ) j=1,2 )-code satisfying
We have PY nj |S Kn (y j |sK j ) ≤
for each j ∈ J. Finally, substitution of the above into (20) provides us
Generation of codebook: For all m(k) ∈ M(k) n , k = 1, 2, 3, we independently generate um(k) ∈ Ukn according to the distribution PUkn (·). For each (um(1) , um(2) , um(3) ), m(k) ∈ M(k) n , k = 1, 2, 3 , we independently generate the codewords ϕn (um(1) , um(2) , um(3) ) according to the distribution PX n |U1n U2n U3n (·|um(1) , um(2) , um(3) ). Both the encoder and every decoder are also assumed to know the codebook. We denote the codeword set Cn ⊂ Xn as Cn = {ϕn (um(1) , um(2) , um(3) )}m(k) ∈M(k) . n ,k=1,2,3 (2) (3) → Cn . To transEncoding ϕn : M(1) n × Mn × Mn (2) (1) (2) mit the message set (m , m , m(3) ) ∈ M(1) n × Mn × (3) Mn , the encoder sends the corresponding codeword ϕn (um(1) , um(2) , um(3) ). We define the encoder ϕCn : (2) (3) (1) (2) (3) M(1) n × Mn × Mn → Cn by ϕCn (m , m , m ) ≡ (k) ϕn (um(1) , um(2) , um(3) ), and define χCn (m ) ≡ um(k) for k = 1, 2, 3. Decoding ψC( j)n : Ynj → Mn( j) × M(3) n for j = 1, 2. To define
the decoding set for y j , we define the set Tn( j) , j = 1, 2 by
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.10 OCTOBER 2005
2816
Tn( j) ≡ Tn( j,a) ∩ (Tn( j,b) × U3n ), Tn( j,a) ≡ (u j , u3 , y j ) ∈ U nj × U3n × Ynj
1 log |Mn( j) | + γ , n n ≡ (u3 , y j ) ∈ U3 × Ynj 1 i(y j ; u3 ) > log |M(3) n |+γ . n i(y j ; u j |u3 ) >
Tn( j,b)
On receiving an output y j ∈ Ynj from the channel Wn , then the decoder ψC( j)n finds the unique set m(3) ∈ M(3) n
for i = 3, 4. We now evaluate the expectation of (23) with respect to the random selection over codebooks. Let PCn denote the probability distribution on the set of Cn . We estimate the expectation of the decoding error probability with respect to the code. For the first term on the right-hand side of (23), we repeat the same technique to obtain (14), and we have the following: PCn (Cn ) Pr{(χCn (Mn( j) ), χCn (Mn(3) ), Y nj ) ∈ Q3( j) (Cn )c } Cn
=
Tn( j,b) .
such that (χCn (m ), y j ) ∈ If there are none such m(3) or more than one such m(3) , then we determines ψC( j)n (y j ) = (m ˆ ( j) , m ˆ (3) ) with an arbitrary (3)
PCn (Cn )
Cn
Mn( j)
× M(3) n (3)
finds the unique set (m , m ) ∈ such that (χCn (m( j) ), χCn (m(3) ), y j ) ∈ Tn( j) using m , which is decided in the previous process. If there are none such m( j) or more than one such m( j) , then we determines ˆ ( j) , m ˆ (3) ) with an arbitrary (m ˆ ( j) , m ˆ (3) ) ∈ ψC( j)n (y j ) = (m (3)
Mn( j) × M(3) n .
We now evaluate the expectation of the decoding error probability. We can assume the message set (2) (3) (m(1) , m(2) , m(3) ) ∈ M(1) n × Mn × Mn was message to be (1) (2) (3) sent, and ϕCn (m , m , m ) was the codewords that had actually been sent at encoder, and let y1 and y2 are the corresponding channel outputs for the input ϕCn (m(1) , m(2) , m(3) ). If (u j , u3 , y j ) holds the following both conditions:
Pe( nj) (Cn )
≡
Pr{(Mn( j) ,
Mn(3) )
ψC( j)n (Y nj )},
Y nj
denotes the corresponding channel output at where the jth terminal via Wnj due to the channel input ϕCn (Mn(1) , Mn(2) , Mn(3) ). Then, the Pe( nj) (Cn ) can be bounded, and we have Pe( nj) (Cn ) ≤ Pr{(χCn (Mn( j) ), χCn (Mn(3) ), Y nj ) ∈ Q3( j) (Cn )c } + Pr{(χCn (Mn( j) ), χCn (Mn(3) ), Y nj ) ∈ Q4( j) (Cn )c }, where Qi( j) (Cn ) is defined for (ϕCn , ψC( j)n ) by Qi( j) (Cn ) ≡ {(u j , u3 , y j ) ∈ U nj × U3n × Ynj |(u j , u1 , y j ) satisfies condition i)}
(23)
P Mn(k) (m(k) )
W nj (y j |ϕCn (m(1) , m(2) , m(3) ))
y j ∈Ynj s.t. ( j) c (χCn (m(nj) ),χCn (m(3) n ),y j )∈Q3 (Cn )
= Pr{(U nj , U3n , Y nj ) Tn( j) }.
(24)
For the second term on the right-hand side of (23), we repeat the similar technique to obtain (15) and (16), and we have the following: PCn (Cn ) Pr{(χCn (Mn( j) ), χCn (Mn(3) ), Y nj ) ∈ Q4( j) (Cn )c } Cn
=
PCn (Cn )
Cn
3) (χCn (m( j) ), χCn (m(3) ), y j ) ∈ Tn( j) 4) There is no (χCn (m ¯ ( j) ), χCn (m ¯ (3) ) ∈ U nj × U3n such that (m ¯ ( j) , m ¯ (3) ) (m( j) , m(3) ) and (χCn (m ¯ ( j) ), χCn (m ¯ (3) ), y j ) ∈ ( j) Tn for j = 1, 2, then the value (m( j) , m(3) ) is uniquely determined on Mn( j) × M(3) n without decoding error. Let us define the probability of error Pe( nj) (Cn ) at the jth terminal for a encoder and decoder pair (ϕCn , ψC( j)n ) by
3
k=1 m(1) ∈M(1) n , m(2) ∈M(2) , n m(3) ∈M(3) n
( j) (m ˆ ( j) , m ˆ (3) ) ∈ Mn( j) × M(3) n . Next, the decoder ψCn ( j)
3
P Mn(k) (m(k) )
k=1 m(1) ∈M(1) n , m(2) ∈M(2) n , m(3) ∈M(3) n
W nj (y j |ϕCn (m(1) , m(2) , m(3) ))
y j ∈Ynj s.t. ( j) c (χCn (m(nj) ),χCn (m(3) n ),y j )∈Q4 (Cn )
≤ 2e−nγ . From (23), (24), and the above relation, we obtain PCn (Cn )Pe( nj) (Cn ) ≤ Pr{(U nj , U3n , Y nj ) Tn( j) } + 2e−nγ . Cn
Since Cn PCn (Cn )Pe( nj) (Cn ) is bounded for j = 1, 2, the above random coding argument shows the existence of at least one ( j) good (n, (|M(k) n |)k=1,2,3 , (εn ) j=1,2 )-code that satisfies the desired performance εn( j) ≤ Pr{(U nj , U3n , Y nj ) Tn( j) } + 2e−nγ 2 for j = 1, 2, simultaneously, and Lemma 4 is proved. We go to the proof of the direct part of Theorem 3. We consider an arbitrary (U1 , U2 , U3 , X) ∈ P∗ for given broadcast channel W with one input terminal and two output terminals. For an arbitrary rate (R1 , R2 , R3 ) satisfying (R1 , R2 , R3 ) ∈ RW ( ε21 , ε22 |U1 , U2 , U3 , X). We set n(Rk −2γ) |M(k) n | = e
IWATA and OOHAMA: INFORMATION-SPECTRUM CHARACTERIZATION OF BROADCAST CHANNEL WITH SOURCE
2817
for k = 1, 2, 3 and an arbitrarily small constant γ > 0. Then, by virtue of Lemma 4, there exists an ( j) (n, (|M(k) n |)k=1,2,3 , (εn ) j=1,2 )-code satisfying εn( j) ≤ Pr i(Y nj ; U nj |U3n ) ≤ R j − γ 2 ∨ i(Y nj ; U3n ) ≤ R3 − γ + 2e−nγ
(25)
for j = 1, 2. By taking lim supn→∞ of the above relations, we have lim sup
εn( j) 2
n→∞
≤ J j (R j − γ, R3 − γ)
for j = 1, 2. Therefore, noting that γ > 0 is arbitrarily small, the above relations means that any rate (R1 , R2 , R3 ) ∈ RW ( ε21 , ε22 |U1 , U2 , U3 , X) is (ε1 , ε2 )-achievable for broadcast channel W . We now consider the any rate (R1 , R2 , R3 ) satisfying (10). From (25), we have εn( j) ≤ Pr i(Y nj ; U nj |U 3n ) ≤ R j − γ 2 + Pr i(Y nj ; U3n ) ≤ R3 − γ + 2e−nγ ≤ Pr i(Y nj ; U nj |U 3n ) ≤ I(Y j ; U j |U3 ) − γ + Pr i(Y nj ; U3n ) ≤ I(Y j ; U3 ) − γ + 2e−nγ . From the definitions of the spectral (conditional) inf-mutual information rates, all the term on the right-hand side of the above converge to zero as n → ∞. Therefore, the error probabilities satisfy lim εn( j) = 0
n→∞
for j = 1, 2 from (10) and (22). As a consequence, by choosing a sequence {γn }n∈N which satisfies (8), and we establish the direct part of Theorem 3 for (ε1 , ε2 ) = (0, 0). Now, let us prove the converse part of Theorem 3 with Lemma 5 below. Lemma code for
∀
∀
∀
5: n ∈ N, γ > 0, (n, (|M(k) n |)k=1,2,3 , Wn , ∃ (U1n , U2n , U3n , X n ) ∈ P∗ such that
(εn( j) ) j=1,2 )-
1 εn( j) ≥ Pr i(Y nj ; U nj |U 3n ) ≤ log |Mn( j) | − γ n 1 −nγ ∨ i(Y nj ; U3n ) ≤ log |M(3) n | − γ − 2e n
for k = 1, 2, 3. We suppose that Ukn is the random variable uniformly distributed over M(k) n . For every fixed value (m(1) , m(2) , m(3) ) of (U1n , U2n , U3n ), a codeword ϕn (m(1) , m(2) , m(3) ) is chosen. Then (U1n , U2n , U3n , X n ) holds property b). Let us define subsets Ln( j) ⊂ U nj × U3n × Ynj , j = 1, 2 by Ln( j) ≡ Ln( j,a) ∪ (L(nj,b) × U3n ), L(nj,a) ≡ (u j , u3 , y j ) ∈ U nj × U3n × Ynj
PY nj |U nj U3n (y j |u j , u3 ) ≤ |Mn( j) |e−nγ PY nj |U3n (y j |u3 ) , L(nj,b) ≡ (u3 , y j ) ∈ U3n × Ynj −nγ PY nj |U3n (y j |u3 ) ≤ |M(3) PY nj (y j ) . n |e We can obtain Pr{(U nj , U3n , Y nj ) ∈ Ln( j) } ≤ εn( j) + 2e−nγ for j = 1, 2 by using the similar argument used in the proof of Lemma 3, and Lemma 5 is proved. We return to prove the converse part of Theorem 3. Suppose that a rate set (R1 , R2 , R3 ) is (ε1 , ε2 )-achievable, and ( j) there must exist a sequence of (n, (|M(k) n |)k=1,2,3 , (εn ) j=1,2 )code satisfying 1 log |M(k) (27) n | ≥ Rk − γ, n for k = 1, 2, 3, all sufficiently large n, an arbitrarily small constant γ > 0, and lim sup εn( j) ≤ ε j , for j = 1, 2.
(28)
n→∞
Let us define Ukn , X n , k = 1, 2, 3, n ∈ N for given broadcast channel W as are defined in Lemma 5, then (U1n , U2n , U3n , X n ) ∈ P∗ , and let Y nj , j = 1, 2 be the channel outputs corresponding to X n . By substituting (27) into (26), we obtain εn( j) ≥ Pr{[i(Y nj ; U nj |U3n ) ≤ R j − 2γ] ∨ [i(Y nj ; U3n ) ≤ R3 − 2γ]} − 2e−nγ
(29)
for j = 1, 2. By taking lim supn→∞ of the above relations, we have J j (R j − 2γ, R3 − 2γ|U j , U3 ) ≤ lim sup εn( j) n→∞
(26)
for j = 1, 2, where Y nj , j = 1, 2 denote the random variables of outputs on the jth output terminal of the channel Wn induced the channel input ϕn (Mn(1) , Mn(2) , Mn(3) ). [Proof of Lemma 5]: For given (n, (|M(k) n |)k=1,2,3 , ( j) (εn ) j=1,2 )-code, , Cn = ϕn (m(1) , m(2) , m(3) ) (k) (k) m ∈Mn ,k=1,2,3
Let us introduce random variables Ukn ranging over M(k) n
for j = 1, 2. From (28), we have J j (R j − 2γ, R3 − 2γ|U j , U3 ) ≤ ε j for j = 1, 2. Since γ > 0 is arbitrary, the above relations means (R1 , R2 , R3 ) ∈ RW (ε1 , ε2 |U , X). We now suppose that a rate (R1 , R2 , R3 ) is achievable, then we have limn→∞ εn( j) = 0 for j = 1, 2. From (29), we have, for j = 1, 2, εn( j) ≥ Pr{i(Y nj ; U nj |U3n ) ≤ R j − 2γ} − 2e−nγ ,
(30)
εn( j) ≥ Pr{i(Y nj ; U3n ) ≤ R j − 2γ} − 2e−nγ .
(31)
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.10 OCTOBER 2005
2818
We show by the contradiction argument that (10) must be satisfied. First, suppose that R j ≤ I(Y j ; U j |U3 ) does not hold. Then, letting γ > 0 be small enough, we have R j −3γ ≥ I(Y j ; U j |U3 ), and by substituting this relation into (30), we have
[9] J. K¨orner and K. Marton, “General broadcast channels with degraded message sets,” IEEE Trans. Inf. Theory, vol.23, no.1, pp.60–64, Jan. 1977.
εn( j) ≥ Pr{i(Y nj ; U nj |U3n ) ≤ I(Y j ; U j |U3 ) + γ} − 2e−nγ . However, by definition of the spectral conditional infmutual information rate I(Y j ; U j |U3 ), the probability on the right-hand side of the above relation cannot vanish asymptotically, therefore this contradicts the assumption limn→∞ εn( j) = 0. Hence we obtain R j ≤ I(Y j ; U j |U3 ). Similarly, R j ≤ I(Y j ; U3 ) can be shown in a similar manner by using (31). We establish the converse part of Theorem 3 for (ε1 , ε2 ) = (0, 0). 6.
Ken-ichi Iwata received the B.Ed. degree from Wakayama University in 1993, and the M.Sc. degree in Information Science from Japan Advanced Institute of Science and Technology in 1995. Since 2001, he has been with the University of Electro-Communications as research associate. He is a member of the SITA of Japan and the IEEE.
Conclusions
This paper derived a necessary condition and a sufficient condition for a set of general sources to be transmissible without distortion over the broadcast channel by a class of block codes. This is an extension of several previous works. Moreover, if we send independent private and common messages to receivers over the broadcast channel then we obtain the upper and lower bound solutions for the capacity region of broadcast channels with arbitrarily fixed error probabilities in Sect. 4, we also derive the capacity region of mixed broadcast channel. But, for general broadcast channel, to bound the range of the auxiliary random variables is still unsolved problem. Acknowledgment The authors would like to thank the anonymous referees for their helpful comments and suggestions which helped to correct the paper. The first author would like to thank Prof. Shuichi Itoh for his encouragement. References [1] C. Shannon, “Two-way communication channels,” Proc. 4th Berkeley Symp. Math. Stat. Prob., vol.1, pp.611–644, Univ. California Press, 1961. [2] T.M. Cover, “Broadcast channels,” IEEE Trans. Inf. Theory, vol.18, no.1, pp.2–13, Jan. 1972. [3] T.M. Cover, “Comments on broadcast channels,” IEEE Trans. Inf. Theory, vol.44, no.6, pp.2524–2530, Oct. 1998. [4] T.S. Han and M.H. Costa, “Broadcast channels with arbitrarily correlated sources,” IEEE Trans. Inf. Theory, vol.IT–33, no.5, pp.641–650, Sept. 1987. [5] A. Feinstein, “A new basic theorem of information theory,” IRE Trans. Inf. Theory, vol.IT-4, no.1, pp.2–22, 1954. [6] S. Verd´u and T.S. Han, “A general formula for channel capacity,” IEEE Trans. Inf. Theory, vol.40, no.4, pp.1147–1157, July 1994. [7] T.S. Han, Information-Spectrum Methods in Information Theory, Springer-Verlag, Berlin, New York, 2002. The Japanese edition was published by Baifukan-publisher, Tokyo, 1998. [8] S. Vembu, S. Verd´u, and Y. Steinberg, “The source-channel separation theorem revisited,” IEEE Trans. Inf. Theory, vol.41, no.1, pp.44–54, Jan. 1995.
Yasutada Oohama was born in Tokyo, Japan, on March 7, 1963. He received the B.Eng., M.Eng., and D.Eng. degrees in mathematical engineering in 1987, 1989, and 1992, respectively, from University of Tokyo, Tokyo, Japan. Since 1992, he has been with Kyushu University, Fukuoka Japan. He is currently an associate professor at the Department of Computer Science and Communication Engineering. From September 1996 to March 1997 he was a visiting scholar at the Information Systems Laboratory, Stanford University, Stanford CA. His current research interests include basic problems in information theory and related areas, Shannon Theory, multi-user information theory, nonlinear theory and its application to information science.