On the Multiple Access Channel with ... - Semantic Scholar

1

On the Multiple Access Channel with Asynchronous Cognition

arXiv:1503.08413v1 [cs.IT] 29 Mar 2015

Michal Yemini∗ , Anelia Somekh-Baruch∗ and Amir Leshem∗

Abstract In this paper we introduce the two-user asynchronous cognitive multiple access channel (ACMAC). This channel model includes two transmitters, an uninformed one, and an informed one which knows prior to the beginning of a transmission the message which the uninformed transmitter is about to send. We assume that the channel from the uninformed transmitter to the receiver suffers a fixed but unknown delay. We further introduce a modified model, referred to as the ACC-MAC, which differs from the ACMAC in that the informed user knows the signal that is to be transmitted by the other user, rather than the message that it is about to transmit. We state single-letter inner and outer bounds on the ACMAC and the ACC-MAC capacity regions, and we specialize the results to the Gaussian case. Further, we characterize the capacity regions of these channels in terms of multi-letter expressions. Finally, we provide an example which instantiates the difference between message side-information and codeword side-information.

I. I NTRODUCTION In recent years, due to the scarcity of free static spectrum resources, a new concept coined as “Cognitive Radio” [2]–[4] has emerged. ”Cognitive radio networks” may refer to several models and setups, however, generally speaking, the common assumption for the different interpretations of this term is the existence of users that can sense their surroundings and are able to change their configurations accordingly, these users are referred to as cognitive users. Though the knowledge that the cognitive users may acquire about the network may vary from one model network to This research was partially supported by Israel Science Foundation (ISF) grant 2013/919. The results of this paper were partially presented in [1]. ∗ Faculty of Engineering, Bar-Ilan University, Ramat-Gan, 52900, Israel. Email: [email protected], [email protected], [email protected].

DRAFT

2

the other, the common goal is to improve spectrum utilization by giving the opportunity to more users to transmit while limiting their interference to non-cognitive users in the network. Further, in some models, the cognitive users can even help the non-cognitive users to improve their reliable communication rates. One possible model of cognition assumes that the cognitive users possess knowledge of the codewords or messages that licensed users transmit. The information theoretic analysis of these models is closely related to the Gel’fand Pinsker channel [5] and the cognitive MAC [4], [6], hence our motivation to further broaden our knowledge of these channel models. Channels with side-information at the transmitter have been widely studied from the informationtheoretic perspective. One of the earliest models was introduced by Shannon [7]. In [7], Shannon analyzed the point-to-point state-dependent memoryless channel with causal side-information at the transmitter, and established a single-letter formula for its capacity. Another well known model is the point-to-point state-dependent memoryless channel with noncausal side-information at the transmitter, which is also known as the Gel’and-Pinsker (GP) channel. The capacity of this channel was found in [5]. Side-information also plays a role in multi-user channels, such as the multiple access channel (MAC). The capacity region of the discrete memoryless MAC was found in terms of a multi-letter expression in [8] and was further characterized by a single-letter expression in [9]. Additionally, the MAC with correlated sources [10] in which each transmitter has two codewords that it wishes to send, a private message and a common message which both transmitters share. The capacity region of this channel is achievable by superposition techniques [10]. For other related models see [11]–[15]. The classical MAC model assumes that the channel is synchronous, however, this is not necessarily the case in practical channels. Several extensions of the MAC to the asynchronous setup have been studied, see e.g., [16]–[18]. It was shown that the capacity region of the discrete memoryless MAC depends on the nature of the delay that may occur in the channel. It was shown [16] that if the delay is finite or grows slowly relatively to the block length, the capacity region remains the same as if there is no delay in the channel. Hui and Humblet [17] proved that the capacity region may be smaller if the delay is of the same order of the block length, since time sharing cannot be used. Asynchronism in MAC with memory was considered by Verd´u [19] under the assumption that the asynchronism is not bounded. Channels with side-information at the transmitter may also assume that the side-information at DRAFT

3

the transmitter is synchronized with the channel. However, this assumption is not always realistic and asynchronism is present in many practical communication systems. One example for such a practical setups is a cellular network in which coordinated multipoint (CoMP) techniques are used (see for example [20], [21]). These techniques can be used in the downlink and can involve different schemes for cooperation and coordination of base-stations. Additionally, one can also take advantage of base-stations cooperation in the uplink, for example several base-stations can jointly decode received signals. While in optimal scenarios all the cooperative nodes are synchronized there can be synchronization issues in these schemes (see for example [21]). An example for asynchronous CoMP is discussed in [22], in this setup two remote radio equipments (RREs) serve two user equipments (UEs) via joint transmissions. The two RREs are connected to the same eNodeB by high quality optical fiber channels (therefore, no delays are present in these channels). It is assumed that the channels from the RREs to the UEs suffer from random time offsets due to continuously varying multi-path environment. This scheme is depicted in Fig. 1 which is presented in [22].

Fig. 1.

An asynchronous CoMP [22]

In previous works, [23], [24], we analyzed several point-to-point state dependent channels with asynchronous side information. In this paper, we inspect how asynchronous knowledge affects the performance of cognitive multi-user channels. The asynchronous cognitive MAC is composed DRAFT

4

of a receiver and two transmitters, an uninformed transmitter that wishes to send a message, and a cognitive one which is informed of the other transmitter’s message and/or codeword. It is assumed that the MAC is asynchronous, that is, the channel from the uninformed user to the receiver suffers an unknown but bounded delay. We characterize the capacity region of this channel in terms of a multi-letter expression, and state single-letter inner and outer bounds on its capacity region. In this paper, we consider two variations of the asynchronous cognitive MAC. The first setup we consider is the asynchronous cognitive MAC with message-cognition at one transmitter (ACMAC) [1]. An additional setup we consider is the asynchronous cognitive MAC codeword-cognition at one encoder (ACC-MAC), depicted in Fig. 3. The difference between the ACC-MAC model and the ACMAC model is that in the former, the informed encoder knows prior to transmission the uninformed encoder’s codeword, whereas in the latter model it knows the uninformed encoder’s message and consequently its codeword. Thus, by definition, in the ACMAC model the informed encoder can send some of the information bits of the uninformed user’s message, whereas in the ACC-MAC model this is no longer possible. Consequently, the capacity region of the ACC-MAC is contained in the capacity region of the ACMAC, where the inclusion is usually strict. We note that the results of this paper were partially presented in [1]. The rest of the paper is organized as follows. Section II includes several notations and definitions and also the ACMAC and ACC-MAC models. Section III states the capacity region of the ACMAC in terms of a multi-letter expression and also includes single-letter inner and outer bounds on its capacity region. In Section IV we address the Gaussian ACMAC and state inner and outer bounds on its capacity region. Further, Section V presents the capacity region of the ACC-MAC in terms of a multi-letter expression and additionally establishes inner and outer bounds on its capacity region. In Section VI we present an example for a channel in which the ACC-MAC’s capacity region is strictly smaller than the ACMAC’s capacity region. Finally, Section VII concludes the paper. II. N OTATIONS AND D EFINITIONS We use the following notations and definitions: A vector (a1 , . . . , an ) is denoted by an , whereas the vector (ai , . . . , aj ) is denoted by aji . If an is a sequence of vectors, then the notation ai,j signifies the j entry of the vector ai . The probability law of a random variable X is denoted by PX and P(X ) denotes the set of distributions on the alphabet X . The set of all n vectors DRAFT

5

xn ∈ X n that are -strongly typical [25, p. 326] with respect to PX ∈ P(X ) is denoted by Tn (X). Additionally, we denote by Tn (X|y n ) the set of all n vectors xn that are -strongly jointly typical with the vector y n with respect to a probability mass function (p.m.f.) PX,Y . Further, 1{A} denotes the indicator function, i.e., 1{A} equals 1 if the statement A holds and 0 otherwise. In addition, D is a set of integers, and D = |D| denotes its cardinality. Further, let P be a conditional p.m.f. from X to Y. For xD ∈ X D denote by {Pd (y|xD 1 )} a set of conditional p.m.f.’s n from X D to Y, that depend on the value of d, where d ∈ D. We use the notation Td, (X, Y )

to make the underlying p.m.f. Pd (xD 1 , y) explicit where d ∈ D. Similarly, we use the notation n Tp, (X) to make the underlying p.m.f. p explicit. Finally, we denote the closure of a subset, A,

of a metric space by closure(A). A. The Asynchronous Cognitive Multiple Access Channel Model The asynchronous cognitive multiple access channel (ACMAC), PY |X1 ,X2 , which is depicted in Fig. 2, is a stationary discrete memoryless channel, defined by the channel transition probabilities Pd (y

n

|xn1 , xn2 )

=

n Y

PY |X1 ,X2 (yi |x1,i−d , x2,i ),

d∈D

(1)

i=1

from X1n × X2n to Y n , where for all i ∈ {1, . . . , n} such that i − d ∈ / {1, . . . , n}, x1,i−d are arbitrary. We consider unidirectional knowledge where the transmitter of user 2 knows in advance the message of user 1 (the uninformed user), and consequently its codeword too.

Encoder 1

X1,i

d

X1,i−d

MAC PY |X1 ,X2

M1

M2

Fig. 2.

Encoder 2

Yi

Decoder

ˆ 1, M ˆ 2) (M

X2,i

Asynchronous cognitive multiple access channel.

DRAFT

6

Let M1 = {1, 2, . . . , 2nR1 }, and M2 = {1, 2, . . . , 2nR2 }, and assume that the messages M1 and M2 are independent random variables uniformly distributed over the sets M1 and M2 , respectively. A (2nR1 , 2nR2 , n)-code for the ACMAC channel consists of the encoding functions f1,n : M1 → X1n

(2)

f2,n : M1 × M2 → X2n

(3)

gn : Y n → M1 × M2 .

(4)

and a decoding function

Define the average probability of error for d ∈ D as nR

P¯e,d =

nR

2X1 2X2 1 2nR1 2nR2 m =1 m =1 1

2

X y n :g

n

(y n )6=(m

Pd (y n |f1,n (m1 ), f2,n (m1 , m2 )) ,

(5)

1 ,m2 )

where Pd (y n |f1,n (m1 ), f2,n (m1 , m2 )) is defined in (1), and for all i ∈ {1, . . . , n} such that i−d∈ / {1, . . . , n}, x1,i−d are arbitrary. A (2nR1 , 2nR2 , n)-code is said to be a (2nR1 , 2nR2 , n, )-code, if P¯e,d ≤  for all d ∈ D. A rate-pair (R1 , R2 ) is said to be achievable for the ACMAC channel, if there exists a sequence
of 2nR1 , 2nR2 , n, n -codes with n → 0 as n → ∞. The capacity region of the ACMAC is defined as the closure of the set of all achievable rate-pairs. B. The Channel Model of the Asynchronous Cognitive Multiple Access Channel with Codeword Knowledge at One Encoder The definitions for the asynchronous cognitive MAC with codeword side-information at one encoder (ACC-MAC) are similar to those of the ACMAC, with the following modification: We consider unidirectional knowledge where transmitter 2 knows the codeword of user 1 (the uninformed user) prior to the beginning of transmission. This difference can be seen in Fig. 3 which depicts the ACC-MAC.

DRAFT

7

M1

Encoder 1

X1,i

d

X1,i−d

MAC PY |X1 ,X2

X1n

M2

Fig. 3.

Yi

Decoder

ˆ 1, M ˆ 2) (M

X2,i

Encoder 2

Asynchronous cognitive MAC with codeword knowledge at one encoder.

The encoder of the informed user is therefore defined by the mapping: f2,n : X1n × M2 → X2n ,

(6)

and the average probability of error takes on the form: P¯e,d =

1 2n(R1 +R2 )

nR1 2nR2 2X X

X

Pd (y n |f1,n (m1 ), f2,n (f1,n (m1 ), m2 )) ,

(7)

m1 =1 m2 =1 y n :gn (y n )6=(m1 ,m2 )

where Pd (y n |f1,n (m1 ), f2,n (f1,n (m1 ), m2 )) is defined in (1). The capacity region of the ACCMAC is defined similarly to that of the ACMAC. In the ACC-MAC model, as opposed to the ACMAC model, the informed encoder knows the uninformed encoder’s codeword, but not necessarily its message. Therefore, in the ACMAC model the mapping between the message to the codeword is not necessarily reversible, and the uninformed encoder may rely on the informed encoder to transmit the remaining information. In other words, in the ACMAC setup, the first user’s message M1 can be split into two distinct parts M1,1 , M1,2 , where user i transmits M1,i . C. The Set of Possible Delays For simplicity of the presentation, throughout this paper, we assume that the set of possible delays in the aforementioned channels is D = {−dmin , −dmin +1, . . . , dmax }, where 0 ≤ dmin , dmax , it follows that D = dmax + dmin + 1. Additionally, throughout this paper we assume that all transmitters and receivers know a-priori the (finite) values dmin and dmax . We note that the results DRAFT

8

which are derived in this paper can be easily generalized to arbitrary finite sets of delays, and hold in the general case in which the delay is randomly distributed over a finite set. D. Known Delay at the Receiver In the above channel models, i.e., ACMAC and ACC-MAC, we assume that the decoder does not know the actual delay in the channel before decoding the message. However, since the set of delays D is finite, by sending predefined training sequences in the first o(n) bits, the decoder can deduce the delay with probability of error that vanishes as n tends to infinity. Therefore, we assume hereafter that the decoder knows the delay d prior to the decoding stage. We will however include transmission of the training sequence in our coding schemes. III. B OUNDS ON THE C APACITY R EGION OF THE ACMAC This section is devoted to the ACMAC (see Fig. 2 and Section II-A). We present the capacity region of the ACMAC in terms of a multi-letter expression and derive single-letter outer and inner bounds on its capacity region. A. A Multi-letter Expression for the Capacity Region of the ACMAC Even though multi-letter expressions are usually not tractable, they can yield significant results and in certain cases even computable formulae (see for example [26]–[28]). We next provide a multi-letter formula for the capacity region of the ACMAC. Let d ∈ D, and let Pd (xn1 , xn2 , y n ) = P (xn1 , xn2 )Pd (y n |xn1 , xn2 ) where Pd (y n |xn1 , xn2 ) is defined in (1). We denote the information theoretic functionals of Pd (xn1 , xn2 , y n ) by the subscript d, e.g., Id (X1n , X2n ; Y n ). Define the region of rate-pairs (R1 , R2 )   1 n n n [ \ R1 + R2 ≤ n Id (X1 , X2 ; Y ),  Rn = , (R1 , R2 ) : 1 n n n   R ≤ I (X ; Y |X ) n n 2 d d∈D 2 1 P (x1 ,x2 ) n and additionally define the region   n−d 1  n n min [ \ R1 + R2 ≤ n Id (X1 , X2 ; Ydmax +1 ),  Qn = (R1 , R2 ) : min  R2 ≤ n1 Id (X2n ; Ydn−d |X1n )  n ) d∈D P (xn ,x max +1 1 2 Q min min where Pd (ydn−d |xn1 , xn2 ) = n−d i=dmax +1 P (yi |x1,i−d , x2,i ). max +1

(8)

(9)

DRAFT

9

Recall that, as noted in Section II, D = D, we next derive the capacity region of the ACMAC in terms of a multi-letter expression. Theorem 1: Let PY |X1 ,X2 be the channel transition probability of an ACMAC with a finite set of possible delays D. The capacity region of the ACMAC is given by ! [ C = closure Qn = closure(lim sup Rn ) = closure(lim inf Rn ). n→∞

n→∞

n≥D

(10)

The proof of this theorem appears in Appendix B. Corollary 1: Let d ∈ D. The capacity region of the ACMAC is not affected by the transition probability of the first and last D symbols. We note that the capacity region of the ACMAC is closed and convex. The region is closed by definition,and the convexity follows by standard arguments of time sharing between two codebooks operating in two different rate-pairs. B. Single-Letter Inner and Outer Bounds on the Capacity Region of the ACMAC Denote by X1n , X2n the codewords (X1,1 , X1,2 , . . . , X1,n ) and (X2,1 , X2,2 , . . . , X2,n ) of users 1 and 2, respectively, additionally, let vi = (x1,i−dmax , x1,i−dmax +1 , . . . , x1,i+dmin ),

i∈N

(11)

where x1,i ∈ X1 , and PV (v) =

D Y

PX1 (vi ).

(12)

i=1

We use the notation Vi,k to denote the k th entry of the ith sequence, that is, Vi,k = X1,i−dmax +k−1 . Denote the information theoretic functionals of Pd (v, x1 , x2 , y) = P (v)1{x1 =vdmax −d+1 } P (x2 |v)P (y|x1 , x2 )

(13)

by the subscript d, e.g., Id (X1 ; Y ). We next present a single-letter achievable region for the ACMAC. Theorem 2: Let PY |X1 ,X2 be an ACMAC with a finite set of possible delays D. Let (V, X1 , X2 , Y ) be distributed according to (13). Denote,    [ R1 + R2 ≤ mind∈D [Id (X1 ; Y ) + Id (X2 ; Y |V )] ,  . R= (R1 , R2 ) :   R2 ≤ mind∈D Id (X2 ; Y |V )

(14)

P (x1 ),P (x2 |v)

DRAFT

10

The closure convex of R is an achievable rate region for the ACMAC. The proof of this theorem appears in Appendix C. An outer bound o the capacity region of the ACMAC is presented next. Theorem 3: Let PY |X1 ,X2 be an ACMAC with a finite set of possible delays D. Denote,    [ R1 + R2 ≤ mind∈D Id (X1 , X2 ; Y )  (15) R= (R1 , R2 ) :  R2 ≤ mind∈D Id (X2 ; Y |V )  P (x1 ),P (x2 |v) where V is distributed according to (12), and Pd (v, x1 , x2 , y) = P (v)1{x1 =vdmax −d+1 } P (x2 |v)P (y|x1 , x2 ).

(16)

The closure convex of R includes the achievable region of the ACMAC. The proof of this theorem appears in Appendix D. Note that if dmin = dmax = 0, that is, there is no delay in the channel, the inner and outer bounds coincide and are equal to the capacity region of the cognitive MAC. Further, one can consider a different setup in which the delay d symbolizes the presence of a jitter. The jitter is modeled by a delay that randomly changes every sub-block of a sufficiently large size that allows the decoder to find the delay in the sub-block with an error probability that decays with the block length. It can be shown that in this setup, if the delays are i.i.d. random variables distributed over the set {−dmin , . . . , dmax }, the minimizations over the delay d in Theorems 1, 2 and 3 can be replaced with an expectation over the delay d. Remark 1: Consider the binary ACMAC defined by the following inputs-output relation, Yi = X1,i−d ⊕ X2,i ⊕ Zi

(17)

where Zi ∼ Bernoulli (p), and d ∈ {−dmin , . . . , dmax }. Assume that the informed encoder (encoder 2) knows in advance the message of the non-cognitive user, which can be regarded as a common message. We note that under the synchronous case, the outer and inner regions (Theorems 2 and 3) coincide and the resulting expression is the capacity region of the cognitive MAC. It is easy to verify that in the binary setup (17), the capacity regions of the synchronous cognitive and noncognitive MACs are equal, that is, the side information does not enlarge the capacity region. Consequently, the capacity regions of the binary ACMAC and binary MAC are equal. That is, the capacity region of the binary ACMAC is the union of all rate-pairs (R1 , R2 ) such that R1 + R2 ≤ 1 − H(Z). DRAFT

11

IV. T HE G AUSSIAN ACMAC

WITH I NDIVIDUAL

P OWER C ONSTRAINTS

Consider the Gaussian ACMAC defined by the inputs-output relation, Yi = X1,i−d + X2,i + Zi

(18)

where Zi ∼ N (0, N ), d ∈ {0, 1}, and each transmitter obeys an individual power constraint, n n 1X 2 1X 2 X ≤ P1 , X ≤ P2 . (19) n i=1 1,i n i=1 2,i It is assumed that the informed encoder knows in advance the message of the non-cognitive user (common message). We next present outer and inner bounds on the capacity region of the Gaussian ACMAC. The Outer Bound: The following theorem states an outer bound on the capacity region of the Gaussian ACMAC. Proposition 1: The union of all the rate-pairs (R1 , R2 ) satisfying √   1 P1 + P2 + 2ρ P1 P2 R1 + R2 ≤ log 1 + 2 N   P2 1 2 R2 ≤ log 1 + (1 − 2ρ ) 2 N i h for some ρ ∈ 0, √12 , is an outer bound on the capacity of the Gaussian ACMAC.

(20)

Proof: Theorem 3 can be generalized to the Gaussian ACMAC, using standard techniques [29]. Consequently, the convex hull of all rate-pairs satisfying (15) for a joint distribution that satisfies (16), under the constraints E[X12 ] ≤ P1 ,

E[X22 ] ≤ P2 ,

(21)

includes the capacity region of the Gaussian ACMAC. We will deduce an outer bound on the region (15), by upper bounding the expressions Id (X1 , X2 ; Y ) and Id (X2 ; Y |V ). Let d ∈ {0, 1} be a given delay. Denote V = (V1 , V2 ), where X1 = Vd+1 = V1 · 1{d=0} + V2 ·

1{d=1} , and V1 and V2 are i.i.d. In addition, let the covariance matrix CV1 ,V2 ,X2 of V1 , V2 , X2 be given by 

CV1 ,V2 ,X2

 ˜ P1 0 σ1    =  0 P˜1 σ2  ,   σ1 σ2 P˜2

(22)

DRAFT

12

where E(V12 ) = E(V22 ) = P˜1 ≤ P1 , and E(X22 ) = P˜2 ≤ P2 . Furthermore, the covariance between X1 and X2 as a function of the delay is d is E[X1 X2 ] = E[(V1 · 1{d=0} + V2 · 1{d=1} )X2 ] = σ1 · 1{d=0} + σ2 · 1{d=1} .

(23)

First, Id (X1 , X2 ; Y ) = hd (Y ) − hd (Y |X1 , X2 ) = hd (X1 + X2 + Z) − hd (Z|X1 , X2 ) (a)

= hd (X1 + X2 + Z) − h(Z)

P˜1 + P˜2 + 2σ1 · 1{d=0} + 2σ2 · 1{d=1} ≤ log 1 + N

(b)

! (24)

where (a) follows since Z is independent of (X1 , X2 ), and (b) follows since the differential entropy of a continuous random variable with variance σ 2 is upper bounded by

1 2

log(2πeσ 2 )

(with equality if the random variable is Gaussian). Next, Id (X2 ; Y |V ) = hd (Y |V ) − hd (Y |V, X2 ) = hd (X1 + X2 + Z|V ) − hd (X1 + X2 + Z|V, X2 ) (a)

= hd (X2 + Z − E[X2 + Z|V ]|V ) − hd (Z|V, X2 )

(b)

≤ hd (X2 + Z − E[X2 + Z|V ]) − h(Z) = hd (X2 + Z − E[X2 |V ]) − h(Z)

(25)

where (a) follows since E[X2 + Z|V ] is a function of V , and since X1 = Vd+1 , i.e., X1 is a function of (d, V ), and (b) follows since conditioning reduces entropy, and since Z is independent of (V, X2 ). To upper bound hd (X2 +Z −E[X2 |V ]), note that since the differential entropy of a continuous random variable with variance σ 2 is upper bounded by 1 log(2πe) + 2 1 = log(2πe) + 2

hd (X2 + Z − E[X2 |V ]) ≤

1 2

log(2πeσ 2 ) it follows that

  1 log E (X2 + Z − E[X2 |V ])2 2   1 log N + E (X2 − E[X2 |V ])2 . 2

(26) DRAFT

13

Now, since E[X2 |V ] is the minimum mean square error estimator (MMSE) of X2 given V , it follows that 1 hd (X2 + Z − E[X2 |V ]) ≤ log(2πe) + 2 1 = log(2πe) + 2

 2     1 T log N + min E X2 − a, b V a,b 2     2  1 T log N + E X2 − aopt , bopt V 2

(27)

where  

aopt bopt

T  = Cov(X2 , V )Var(V )−1

 T  −1  T ˜ ˜ σ1 P1 0 σ /P  =  1 1 =   σ2 0 P˜1 σ2 /P˜1

(28)

are the optimal values of the minimization, i.e., the coefficients of the linear MMSE of X2 given V = (V1 , V2 ). Consequently, "  2 # σ2 1 σ1 1 hd (X2 + Z − E[X2 |V ]) ≤ log(2πe) + log N + E X2 − V1 − V2 2 2 P˜1 P˜1   1 1 σ12 + σ22 ˜ = log(2πe) + log N + P2 − . 2 2 P˜1

(29)

Therefore,  1 Id (X2 ; Y |V ) ≤ log 1 + 2

P˜2 −

σ12 +σ22 P˜1

N



2 2 ˜ ˜  = 1 log 1 + P1 P2 − (σ1 + σ2 ) 2 N P˜1

! .

(30)

Let ρ1 = √σ˜1 ˜ , and ρ2 = √σ˜2 ˜ , it follows that P1 P2

P1 P2

P˜1 + P˜2 + 2 1 Id (X1 , X2 ; Y ) ≤ log 1 + 2 Id (X2 ; Y |V ) ≤

p
! P˜1 P˜2 ρ1 · 1{d=0} + ρ2 · 1{d=1} N !

1 P˜2 log 1 + (1 − (ρ21 + ρ22 )) . 2 N

(31)

Denote by R(P˜1 , P˜2 , ρ1 , ρ2 ) the following region,    √ P˜1 +P˜2 +2 P˜1 P˜2 (ρ1 ·1{d=0} +ρ2 ·1{d=1} )  1  R1 + R2 ≤ mind∈{0,1} 2 log 1 + N (R1 , R2 ) :    ˜  R2 ≤ mind∈{0,1} 1 log 1 + P2 (1 − (ρ2 + ρ2 )) 2

N

1

2

  

. (32)

 

Since the covariance matrix (22) is non-negative definite, the following inequality must hold P˜1 (P˜1 P˜2 − σ22 ) − P˜1 σ12 ≥ 0.

(33)

DRAFT

14

Substituting σ1 = ρ1

p p P˜1 P˜2 and σ2 = ρ2 P˜1 P˜2 yields ρ21 + ρ22 ≤ 1.

(34)

We first note that if ρ1 and ρ2 satisfy (34) then |ρ1 | and |ρ2 | satisfy (34) as well. Therefore, by definition of R(P˜1 , P˜2 , ρ1 , ρ2 ), it follows that R(P˜1 , P˜2 , ρ1 , ρ2 ) ⊆ R(P˜1 , P˜2 , |ρ1 |, |ρ2 |).

(35)

Consequently, it suffices to consider only ρ1 , ρ2 ≥ 0. In addition, if ρ1 , ρ2 ≥ 0 satisfy (34), then R(P˜1 , P˜2 , ρ1 , ρ2 ) ⊆ R(P1 , P2 , ρ1 , ρ2 ).

(36)

Therefore, it suffices to consider P˜1 = P1 and P˜2 = P2 . Further, suppose that ρ2 ≥ ρ1 ≥ 0, such that ρ21 + ρ22 ≤ 1, then, for every ρ1 ≤ ρ˜2 ≤ ρ2 it follows that ρ21 + ρ˜22 ≤ 1. In addition, by definition of R(P1 , P2 , ρ1 , ρ2 ) one has R(P1 , P2 , ρ1 , ρ2 ) ⊆ R(P1 , P2 , ρ1 , ρ˜2 ) ⊆ R(P1 , P2 , ρ1 , ρ1 ).

(37)

Similarly, if ρ1 ≥ ρ2 ≥ 0, then R(P1 , P2 , ρ1 , ρ2 ) ⊆ R(P1 , P2 , ρ2 , ρ2 ).

(38)

Consequently, let ρ1 , ρ2 ≥ 0 be such that ρ21 + ρ22 ≤ 1, and denote ρ = min{ρ1 , ρ2 }, it follows that R(P1 , P2 , ρ1 , ρ2 ) ⊆ R(P1 , P2 , ρ, ρ). h i Therefore, it suffices to consider ρ1 = ρ2 = ρ, where ρ ∈ 0, √12 . Consequently, by Theorem 3, the convex hull of all the rate-pairs (R1 , R2 ) satisfying √   1 P1 + P2 + 2ρ P1 P2 R1 + R2 ≤ log 1 + 2 N   1 P2 2 R2 ≤ log 1 + (1 − 2ρ ) 2 N h i for some ρ ∈ 0, √12 , is an outer bound on the capacity of the Gaussian ACMAC. Denote,

√   1 P1 + P2 + 2ρ P1 P2 g1 (ρ) = log 1 + 2 N   1 P2 2 g2 (ρ) = log 1 + (1 − 2ρ ) . 2 N

(39)

(40)

(41)

DRAFT

15

Since g1 (ρ) and g2 (ρ) are concave functions with respect to ρ, the union of all rate-pairs (R1 , R2 ) i h satisfying (40) for some ρ ∈ 0, √12 is a convex region. Therefore, the convex hull operation can be omitted. The Inner Bound: The following theorem states an inner bound on the capacity region of the Gaussian ACMAC. Proposition 2: The convex hull of the rate-pairs (R1 , R2 ) satisfying ! ! p 1 1 N + P1 + P˜2 + 2ρ P1 P˜2 P˜2 + log 1 + (1 − 2ρ2 ) R1 + R2 ≤ log 2 2 N N + P˜2 (1 − ρ2 ) ! P˜2 1 (42) R2 ≤ log 1 + (1 − 2ρ2 ) 2 N i h for some P˜2 ∈ [0, P2 ] and ρ ∈ 0, √12 , is an inner bound on the capacity of the Gaussian ACMAC. Proof: We will deduce an inner bound on the region (14), by choosing (V, X2 ) which are jointly Gaussian. Let d ∈ {0, 1} be a given delay and let V = (V1 , V2 ) where X1 = Vd+1 = V1 ·1{d=0} +V2 ·1{d=1} . Further, let (V1 , V2 , X2 ) be jointly Gaussian with zero mean, and the following covariance matrix:   ˜1 0 σ1 P     CV1 ,V2 ,X2 =  0 P˜1 σ2  , (43)   σ1 σ2 P˜2 where P˜1 ≤ P1 , and P˜2 ≤ P2 (see (21)). It is easy to verify that for this choice of random variables one has ! P˜1 P˜2 − (σ12 + σ22 ) 1 . Id (X2 ; Y |V ) = log 1 + 2 N P˜1

(44)

DRAFT

16

We now consider Id (X1 ; Y ) Id (X1 ; Y ) = hd (Y ) − hd (Y |X1 ) = hd (X1 + X2 + Z) − hd (X1 + X2 + Z|X1 ) = hd (X1 + X2 + Z) − hd (X2 + Z|X1 ) (a)

= hd (X1 + X2 + Z) − hd (X2 + Z − E[X2 + Z|X1 ]|X1 )

(b)

= hd (X1 + X2 + Z) − hd (X2 + Z − E[X2 + Z|X1 ])

= hd (X1 + X2 + Z) − hd (X2 + Z − E[X2 |X1 ])

(45)

where (a) follows since E[X2 + Z|X1 ] is a function of X1 , and (b) follows since E[(X2 + Z − E[X2 + Z|X1 ])X1 ] = 0, X1 = V1 · 1{d=0} + V2 · 1{d=1} , and since (V1 , V2 , X2 , Z) are jointly Gaussian. Now, let Λd (σ1 , σ2 ) = σ1 · 1{d=0} + σ2 · 1{d=1} . Since X1 = V1 · 1{d=0} + V2 · 1{d=1} and V1 , V2 , X2 are jointly Gaussian, E[X2 |X1 ] = Λd (σ1 , σ2 )P˜1−1 X1 . Therefore, hd (X2 + Z − E[X2 |X1 ]) = hd (X2 + Z − Λd (σ1 , σ2 )P˜1−1 X1 )   2 2 1 Λ (σ , σ ) Λ (σ , σ ) 1 d 1 2 d 1 2 −2 . = log(2πe) + log N + P˜2 + 2 2 P˜1 P˜1 (46) In addition, hd (X1 + X2 + Z) =

  1 1 log(2πe) + log N + P˜1 + P˜2 + 2Λd (σ1 , σ2 ) . 2 2

(47)

Therefore, N + P˜1 + P˜2 + 2Λd (σ1 , σ2 ) 2 N + P˜2 − Λd (σP˜1 ,σ2 ) 1 ! 1 P˜1 P˜2 − (σ12 + σ22 ) Id (X2 ; Y |V ) = log 1 + . 2 N P˜1 1 Id (X1 ; Y ) = log 2

!

(48)

Now, let ρ1 = √σ˜1 ˜ , and ρ2 = √σ˜2 ˜ , it follows that P1 P2

P1 P2

! p N + P˜1 + P˜2 + 2 P˜1 P˜2 Λd (ρ1 , ρ2 ) N + P˜2 [1 − Λd (ρ1 , ρ2 )2 ] ! 1 P˜2 [1 − (ρ21 + ρ22 )] Id (X2 ; Y |V ) = log 1 + . 2 N

1 Id (X1 ; Y ) = log 2

(49)

DRAFT

17

Hence, the following region, denoted R(P˜1 , P˜2 , ρ1 , ρ2 )     √  N +P˜1 +P˜2 +2 P˜1 P˜2 Λd (ρ1 ,ρ2 ) 1  R1 + R2 ≤ mind∈{0,1} 2 log   N +P˜2 [1−Λd (ρ1 ,ρ2 )2 ]   (R1 , R2 ) :   P˜2 [1−(ρ21 +ρ22 )] 1 + 2 log 1 + N        ˜  R2 ≤ mind∈{0,1} 21 log 1 + PN2 (1 − (ρ21 + ρ22 ))

      

.

     

is achievable. i h Now, one can pick1 P˜1 = P1 , and ρ1 = ρ2 ∈ 0, √12 , and consequently, the convex hull of all rate-pairs (R1 , R2 ) satisfying ! ! p N + P1 + P˜2 + 2ρ P1 P˜2 1 P˜2 + log 1 + (1 − 2ρ2 ) 2 ˜ 2 N N + P2 (1 − ρ ) ! 1 P˜2 R2 ≤ log 1 + (1 − 2ρ2 ) (50) 2 N i h 1 ˜ √ for some P2 ∈ [0, P2 ] and ρ ∈ 0, 2 , is an inner bound on the capacity of the Gaussian 1 R1 + R2 ≤ log 2

ACMAC. 1

It can be shown, similarly to the outer bound, that this choice does not reduce the convex-hull of all the rate-pairs (R1 , R2 ) such that (R1 , R2 ) ∈ R(P˜1 , P˜2 , ρ1 , ρ2 ), for some ρ1 , ρ2 ∈ [−1, 1] such that ρ21 + ρ22 ≤ 1, P˜1 ∈ [0, P1 ], and P˜2 ∈ [0, P2 ].

DRAFT

18

Fig. 4.

Outer and inner bounds on the capacity of the Gaussian ACMAC with d ∈ {0, 1} compared with the capacity of the

Gaussian cognitive MAC with parameters P1 = 0.5, P2 = N = 1.

Fig. 4 compares the capacity region of the synchronous Gaussian cognitive MAC, whose capacity is derived in [11, Theorem 7], and the outer and inner bounds on the capacity regions of the ACMAC with d ∈ {0, 1}, P1 = 0.5, and P2 = N = 1. V. B OUNDS ON THE C APACITY R EGION OF THE ACC-MAC In this section we derive the capacity region of the ACC-MAC in terms of a multi-letter expression and state single-letter inner and outer bounds on its capacity region. While when message cognition is concerned the informed encoder can help the uninformed encoder to achieve higher rates via rate-splitting, when codeword side-information is considered the informed encoder finds the uninformed encoder’s message by decoding its codeword, and therefore rate-splitting is not effective.

DRAFT

19

A. A Multi-letter Expression for the Capacity Region of the ACC-MAC Recall the definition of Pd (y n |xn1 , xn2 ) (see (1)). We denote information theoretic functionals of Pd (xn1 , xn2 , y n ) = P (xn1 , xn2 )Pd (y n |xn1 , xn2 ) by the subscript d, e.g.,   X Pd (xn1 , xn2 , y n ) n n n n n n Id (X1 , X2 ; Y ) = Pd (x1 , x2 , y ) log , n n n) )P (y , x P (x d 2 1 n n n x1 ,x2 ,y P where Pd (y n ) = xn ,xn Pd (xn1 , xn2 , y n ). 1

(51)

2

Define the region Rn of rate pairs (R1 , R2 ):    R1 ≤ n1 H(X1n ),  [ \  (R1 , R2 ) : ˇn = R R2 ≤ n1 Id (X2n ; Y n |X1n ),   n P (xn 1 ,x2 ) d∈D   R1 + R2 ≤ n1 Id (X1n , X2n ; Y n )

    

,

(52)

   

and the region Qn of rate pairs (R1 , R2 ):    R1 ≤ n1 H(X1n ),  [ \  (R1 , R2 ) : ˇn = min Q |X1n ), R2 ≤ n1 Id (X2n ; Ydn−d max +1   n P (xn 1 ,x2 ) d∈D  min  ) R1 + R2 ≤ n1 Id (X1n , X2n ; Ydn−d max +1 Q min min |xn1 , xn2 ) = n−d where Pd (ydn−d i=dmax +1 P (yi |x1,i−d , x2,i ). max +1

    

(53)

   

We can now state the capacity region of the ACC-MAC in terms of a multi-letter expression.

Theorem 4: Let PY |X1 ,X2 be an ACC-MAC with a finite set of possible delays D. The capacity region of the ACC-MAC is given by: ! [ ˇ n = closure(lim sup R ˇ n ) = closure(lim inf R ˇ n ). C = closure Q n≥D

n→∞

n→∞

(54)

The outline of the proof of achievability part of Theorem 4 appears in Appendix E. In the converse, the additional inequality compared to the multi-letter expression of the capacity region of the ACMAC, Eq. (10), R1 ≤ n1 H(X1n ) follows since nR1 = H(M1 ) = H(X1n , M1 ) = H(X1n ) + H(M |X1n ),

(55)

where n1 H(M |X1n ) → 0 as n → ∞ by Fano’s inequality and since the informed encoder possess only codeword side-information. It is easy to see that, as expected, CACC−M AC ⊆ CACM AC where CACM AC and CACC−M AC are the capacity regions of the ACMAC and ACC-MAC, respectively. DRAFT

20

B. Single-Letter Inner and Outer Bounds on the Capacity Region of the ACC-MAC We proceed to present single-letter inner and outer bounds on the capacity region of the ACCMAC. In this section we use the notations v n , PV (vi ), Pd (v, x1 , x2 , y) which are defined in Eq. (11)-(13). We denote information theoretic functionals of Pd (v, x1 , x2 , y) by the subscript d, e.g., Id (X1 ; Y ). Theorem 5: Let PY |X1 ,X2 be an ACC-MAC and let (V, X1 , X2 , Y ) be distributed according to (13). Denote,

R=

[

    

    

R1 ≤ H(X1 ), (R1 , R2 ) :



P (x1 ),P (x2 |v)   

R2 ≤ mind∈D Id (X2 ; Y |V ),

.

(56)

   R1 + R2 ≤ mind∈D [Id (X1 ; Y ) + Id (X2 ; Y |V )] 

The closure convex of R is an achievable rate region for the ACC-MAC. We note that the coding scheme of Theorem 2 is not suited to this theorem. It is due to the fact that the informed encoder knows the uninformed encoder’s codeword but not necessarily its message. In this case simultaneous decoding yields better results than successive decoding. The proof of Theorem 5 appears in Appendix F. We next provide a single-letter outer bound on the capacity region of the ACC-MAC. Theorem 6: Let PY |X1 ,X2 be an ACC-MAC. Denote,    R1 ≤ H(X1 ),   [ R= (R1 , R2 ) : R2 ≤ mind∈D Id (X2 ; Y |V ),  P (x1 ),P (x2 |v)    R1 + R2 ≤ mind∈D Id (X1 , X2 ; Y )

    

(57)

   

where V is distributed according to (12) and (V, X1 , X2 , Y ) is distributed according to (13) The closure convex of R includes the achievable region of the ACC-MAC. The proof of this theorem is similar to the proof of Theorem 3. The additional inequality R1 ≤ H(X1 ) follows since nR1 = H(M1 ) = H(X1n , M1 ) = nH(X1 ) + H(M |X1n ). Note that as before

1 H(M |X1n ) n

(58)

→ 0 as n → ∞ by Fano’s inequality and since the informed

encoder possesses only codeword side-information. We further remark that the regions (54), (56) and (57) differ from the regions of Theorem 1, 2 and 3, respectively, in the additional inequality on the rate of the uniformed user that is added in the ACC-MAC model. DRAFT

21

Consider the Gaussian ACC-MAC with the same characteristics as those of the ACMAC which is presented in Section IV. The uninformed encoder can use its first symbol in each codeword to notify the cognitive user about its message while consuming a negligible amount of power. Thus, the capacity regions of the Gaussian ACC-MAC and the Gaussian ACMAC (Section IV) coincide. Further, the outer and inner bounds on the capacity region of the Gaussian ACMAC, i.e., Propositions 1 and 2, hold for the Gaussian ACC-MAC. VI. A N E XAMPLE - C OMPARISON BETWEEN THE ACC-MAC AND THE ACMAC M ODELS Let X1 = {2, 4}, and let X2 = Y = {0, 1, 2, 3}. Define the channel by the following inputsoutput relation: Y = X2

(mod X1 ).

(59)

We first analyze the case of no delay (d = 0) as an example which demonstrates that the capacity region of the ACC-MAC can be strictly smaller than that of the ACMAC. Since the channel is synchronous, we can use the results of [10] and the capacity region is given by:   [  R2 ≤ I(X2 ; Y |X1 ),  (60) CACM AC = (R1 , R2 ) :  R1 + R2 ≤ I(X1 , X2 ; Y )  P (x1 ,x2 )

It follows that: I(X1 , X2 ; Y ) = H(Y ) ≤ log |Y| = 2 bits I(X2 ; Y |X1 ) = H(Y |X1 ) ≤ log |Y| = 2 bits, with equalities if Pr(x1 = 4) = 1, and Pr(x2 ) =

1 4

(61)

for all x2 ∈ X2 . Consequently, the capacity

of the proposed channel is the triangle: n o CACM AC = (R1 , R2 ) : R1 + R2 ≤ 2 .

(62)

A coding scheme that achieves every rate-pair in the capacity region lets the informed encoder send the messages of both encoders. Therefore, the capacity region of the channel given in (59) under the ACMAC model is unaffected by asynchronism regardless of the delay D.

DRAFT

22

Under the synchronous ACC-MAC model, the inner and outer bounds, i.e., the regions (56) and (57), respectively, coincide with the capacity region:    R1 ≤ H(X1 ),   [ CACC−M AC = (R1 , R2 ) : R2 ≤ I(X2 ; Y |X1 ),  P (x1 ),P (x2 |x1 )    R1 + R2 ≤ I(X1 , X2 ; Y )

    

.

(63)

   

Now, I(X2 ; Y |X1 ) = H(Y |X1 ) = Pr(x1 = 4)H(X2 ) + Pr(x1 = 2)H(X2

(mod 2)).

(64)

It can be shown that the capacity region is the following trapezoid    R1 ≤ 1,  CACC−M AC = (R1 , R2 ) : .  R1 + R2 ≤ 2  Its two corner points are achieved by the p.m.f.’s Pr(x1 = 4) = Pr(x1 = 2) = 2) = Pr(x2 = 3) =

1 , 2

and Pr(x1 = 4) = 1, Pr(x2 ) =

1 4

(65) 1 , Pr(x2 2

=

∀x2 ∈ X2 . The region (65) is

equivalent to the capacity region of the channel given in (59) with no side-information at both transmitters. Therefore, under ACC-MAC, the side-information in the this channel does not enlarge the capacity region, and we obtain CACC−M AC ⊂ CACM AC . Moreover, since the region (65) can be achieved by a coding scheme that does not use the sideinformation, we can use [16] to deduce that bounded asynchronism does not affect the channel capacity region. Consequently, CACC−M AC ⊂ CACM AC for every set D of bounded delays. VII. C ONCLUSION In this paper we presented the asynchronous cognitive MAC with message and/or codeword cognition at one encoder, denoted ACMAC and ACC-MAC, respectively. We characterized the capacity regions of the ACMAC and ACC-MAC in terms of multi-letter expressions. We presented single-letter inner an outer bounds on capacity regions these channels. Further, we analyzed the Gaussian ACMAC and derived inner and outer bounds on it capacity region. We noted that in the Gaussian case the capacity regions of the ACMAC and the ACC-MAC are equal. Finally, we presented an example for a channel in which the ACC-MAC capacity region is strictly smaller than the capacity region of the ACMAC.

DRAFT

23

A PPENDIX A. An auxiliary lemma The following lemma will be used in the proof of Theorem 1 (see Appendix B). Lemma 1: Let X n , Y n and Z n be random vectors whose symbols belong to the finite alphabets X , Y and Z, respectively. Additionally, let Θ be a finite set, let θ ∈ Θ, and denote Pθ,X n ,Y n ,Z n = PX n ,Y n Pθ,Z n |X n ,Y n Qθ,X n ,Y n ,Z n = PX n ,Y n Qθ,Z n |X n ,Y n

(66)

where n Pθ,Z n |X n ,Y n = Pθ,Z D1 |X n ,Y n Pθ,Zn−D 1

1

1

2 +1

n Qθ,Z n |X n ,Y n = Qθ,Z D1 |X n ,Y n Qθ,Zn−D 1

1

|X n ,Y n Pθ,Z n−D2 |X n ,Y n

2 +1

1

D1 +1

|X n ,Y n Pθ,Z n−D2 |X n ,Y n

(67)

D1 +1

and D1 , D2 are nonnegative finite integers. Denote by RP,n and RQ,n the following regions  [ \  R1 + R2 ≤ n1 IPθ (X n , Y n ; Z n ) RP,n = (R1 , R2 ) :  R2 ≤ n1 IPθ (X n ; Z n |Y n ) n P (xn 1 ,x2 ) θ∈Θ  [ \ R1 + R2 ≤ n1 IQθ (X n , Y n ; Z n ) RQ,n = (R1 , R2 ) :  R2 ≤ 1 IQ (X n ; Z n |Y n ) n n P (x1 ,x2 ) θ∈Θ

n

θ

    

.

(68)



Then, lim inf RP,n = lim inf RQ,n n→∞

n→∞

lim sup RP,n = lim sup RQ,n . n→∞

(69)

n→∞

Proof: IPθ (X n , Y n ; Z n ) n−D2 n−D2 = IPθ (X n , Y n ; ZD ) + IPθ (X n , Y n ; Z D1 |ZD ) 1 +1 1 +1 n + IPθ (X n , Y n ; Zn−D |Z n−D2 ) 2 +1 n−D2 n ≤ IPθ (X n , Y n ; ZD ) + HPθ (Z D1 ) + HPθ (Zn−D ) 2 +1 1 +1 n−D2 ≤ IPθ (X n , Y n ; ZD ) + (D1 + D2 ) log |Z| 1 +1 (a)

≤ IQθ (X n , Y n ; Z n ) + (D1 + D2 ) log |Z|.

(70) DRAFT

24

where (a) follows since n−D2 n−D2 IPθ (X n , Y n ; ZD ) = IQθ (X n , Y n ; ZD ). 1 +1 1 +1

(71)

Similarly, n−D2 IPθ (Y n ; Z n |X n ) ≤ IPθ (Y n ; ZD |X n ) + (D1 + D2 ) log |Z| 1 +1 (a)

≤ IQθ (Y n ; Z n |X n ) + (D1 + D2 ) log |Z|,

n−D2 IQθ (X n , Y n ; Z n ) ≤ IQθ (X n , Y n ; ZD ) + (D1 + D2 ) log |Z| 1 +1 (b)

≤ IPθ (X n , Y n ; Z n ) + (D1 + D2 ) log |Z|,

n−D2 IQθ (Y n ; Z n |X n ) ≤ IQθ (Y n ; ZD |X n ) + (D1 + D2 ) log |Z| 1 +1 (c)

≤ IPθ (Y n ; Z n |X n ) + (D1 + D2 ) log |Z|,

(72)

where (a) and (c) follows since n−D2 n−D2 |X n ), |X n ) = IQθ (Y n ; ZD IPθ (Y n ; ZD 1 +1 1 +1

(73)

n−D2 n−D2 IPθ (X n , Y n ; ZD ) = IQθ (X n , Y n ; ZD ). 1 +1 1 +1

(74)

and (b) follows since

Consequently, 1 (D1 + D2 ) log |Z| · U n 1 ⊂ RP,n + (D1 + D2 ) log |Z| · U n

RP,n ⊂ RQ,n + RQ,n

(75)

where U is the unit square {(x, y) : 0 ≤ x ≤ 1, 0 ≤ y ≤ 1}. One can see that equation (75) results in, lim inf RP,n = lim inf RQ,n , n→∞

n→∞

lim sup RP,n = lim sup RQ,n . n→∞

(76)

n→∞

DRAFT

25

B. Proof of Theorem 1 Achievability: Let n be the transmission block length, and let n ˜,

n k

. Denote by a ˜i the

hyper-symbol of length k ≥ D that consists of the k consecutive symbols (a(i−1)·k+1 , . . . , ai·k ) of the vector an . Partitioning each vector xn1 , xn2 into vectors that consist of hyper-symbols of length k, yields ˜ }. In addition, denote by y i the vectors x˜n1˜ , x˜n2˜ , where x˜1,i ∈ X1k , x˜2,i ∈ X2k for all i ∈ {1, . . . , n the hyper-symbol (y(i−1)·k+1+dmax , . . . , yi·k−dmin ) that is, y i ∈ Y k−D+1 Codebook Generation: The codebooks C (1) and C (2) (l), 1 ≤ l ≤ |C (1) | are produced in the following manner: ˜ 1 and R ˜ 2 . Let C (1) be the codebook of the common Set PX˜1 , PX˜2 |X˜1 , and fix the rates R ˜

message M1 , which consists of 2n˜ R1 codewords, each of these codewords is generated according Q˜ PX˜1 (˜ x1,i ). to P (˜ xn1˜ ) = ni=1 ˜

For every x˜n1˜ (l) ∈ C (1) generate randomly and independently 2n˜ R2 codewords {˜ xn2˜ (l, 1), . . . , Qn˜ ˜ x˜n2˜ (l, 2n˜ R2 )} according to P (˜ x2,i |˜ x1,i ). We denote {˜ xn2˜ (l, 1), . . . , x˜n2˜ (l, xn2˜ |˜ xn1˜ ) = ˜ 2 |X ˜ 1 (˜ i=1 PX ˜

2n˜ R2 )} by C (2) (l). Encoding: To send the messages m1 , m2 , encoder 1 sends x˜n1˜ (m1 ), and encoder 2 sends x˜n2˜ (m1 , m2 ). Decoding: Let y k = (y(i−1)·k+1+dmax , . . . , yi·k−dmin ). Suppose that the actual delay in the channel is d ∈ D. Denote the set of all vectors (˜ xn1˜ ,˜ xn2˜ , y n˜ ) that are -strongly typical with ˜1, X ˜ 2 , Y ), where respect to a p.m.f. Pd (˜ x1 , x˜2 , y) by T n (X d,

Pd (˜ x1 , x˜2 , y) = P (˜ x1 , x˜2 )Pd (y|˜ x1 , x˜2 ) Pd (y|˜ x1 , x˜2 ) =

k−d min Y

PY |X1 ,X2 (y j−dmax |˜ x1,j−d , x˜2,j ).

(77)

j=dmax +1

Given that the decoder knows the delay d, it looks for m ˆ 1 ∈ {1, . . . , 2nR1 } and m ˆ 2 ∈ {1, . . . , 2nR2 } such that n ˜ ˜ 2 , Y ). (˜ xn1˜ (m ˆ 1 ), x˜n2˜ (m ˆ 2 ), y n˜ ) ∈ Td, (X1 , X

(78)

Analysis of Probability of Error: Suppose that the pair of messages (m1 , m2 ) = (1, 1) is sent,

DRAFT

26

and that the delay is d ∈ D. An error is made if one or more of the following events occur: ˜ 1 )} / Tn˜ (X E1 = {˜ xn1˜ (1) ∈ ˜2, X ˜ 1 )} E2 = {(˜ xn2˜ (1, 1), x˜n1˜ (1)) ∈ / Tn˜ (X n o n ˜ ˜1, X ˜2, Y ) E3 = (˜ xn1˜ (1), x˜n2˜ (1, 1), y n˜ ) ∈ / Td, (X   ˜2  ∃m  n ˜R ˆ 1 6= 1 and m ˆ 2 ∈ {1, . . . , 2 } s.t. E4 = n ˜ (˜ ˜1, X ˜ 2 , Y ) xn1˜ (m ˆ 1 ), x˜n2˜ (m ˆ 1, m ˆ 2 ), y n˜ ) ∈ Td, (X     ∃m ˆ 2 6= 1 s.t. E5 = . (˜ ˜1, X ˜ 2 , Y ) xn˜ (1), x˜n˜ (1, m ˆ 2 ), y n˜ ) ∈ T n˜ (X 1

2

(79)

d,

By the union bound, Pr(E) = Pr

5 [

! Ei

≤ Pr(E1 ) + Pr(E1c ∩ E2 ) + Pr(E2c ∩ E3 ) + Pr(E4 ) + Pr(E5 ).

(80)

i=1

First, by the law of large numbers (LLN) Pr(E1 ) → 0 as n → ∞. Second, the conditional typicality lemma [30, p. 27] dictates that Pr(E1c ∩ E2 ) → 0 as n ˜ → ∞. Q˜ Pd (y i |˜ x1,i , x˜2,i ), therefore, The sequence y n˜ is generated given x˜k1 and x˜k2 according to ni=1 from the LLN we have that Pr(E2c ∩ E3 ) vanishes as n ˜ tends to infinity. Finally, by the packing lemma [30, p. 46], Pr(E4 ) → 0 as n ˜ → ∞ if ˜1 + R ˜ 2 ≤ Id (X ˜1, X ˜ 2 ; Y ), k(R1 + R2 ) = R

(81)

and Pr(E5 ) → 0 as k → ∞ if ˜ 2 ≤ Id (X ˜ 2 ; Y |X ˜ 1 ). kR2 = R Denote for every k, P (xk1 , xk2 ), and d ∈ D,   k−dmin 1  k k
R1 + R2 ≤ k Id (X1 , X2 ; Ydmax +1 ),  k k Qk,d P (x1 , x2 ) = . (R1 , R2 ) : k min   R2 ≤ k1 Id (X2k ; Ydk−d |X ) 1 max +1

(82)

(83)

Since the encoder does not know the delay d ∈ D, a rate-pair is achievable given P (xk1 , xk2 ) if it lies in the intersection of all the regions Rk,d (P (xk1 , xk2 )). Therefore, \ Qk (P (xk1 , xk2 )) = Qk,d (P (xk1 , xk2 ))

(84)

d∈D

is an achievable rate region. DRAFT

27

Consequently, the closure of the region

S

Qk is achievable.

k≥D

By definition lim inf Qn ⊆ lim sup Qn ⊆ n→∞

n→∞

[

Qn ,

(85)

n≥D

by Lemma 1 (see (66)-(69)), it follows that: lim inf Qn = lim inf Rn

(86)

lim sup Qn = lim sup Rn ,

(87)

n→∞

n→∞

n→∞

n→∞

where Rn is defined in (8). Therefore, lim inf Rn ⊆ lim sup Rn ⊆ n→∞

n→∞

[

Qn

(88)

n≥D

are achievable rate regions. Converse: Let, Pd (m1 , m2 , xn1 , xn2 , y n ) = P (m1 )P (m2 )P (xn1 |m1 )P (xn2 |xn1 )Pd (y n |xn1 , xn2 ),

(89)

where P (m1 ) = 2−nR1 , Pd (y n |xn1 , xn2 ) =

P (m2 ) = 2−nR2 n Y

P (yi |x1,i−d , x2,i ).

(90) (91)

i=1

We denote Information-Theoretic functionals of Pd (m1 , m2 , xn1 , xn2 , y n ) by the subscript d, e.g., Hd (M1 , M2 |Y n ). We first upper bound the sum-rate R1 + R2 . Let δn = n1 Hd (M1 , M2 |Y n ). For every sequence (n)

of (2nR1 , 2nR2 , n)-codes with probability of error Pe

that vanishes as n tends to infinity for

DRAFT

28

every d ∈ D, n(R1 + R2 ) = H(M1 , M2 ) = H(M1 , M2 ) − Hd (M1 , M2 |Y n ) + Hd (M1 , M2 |Y n ) = Id (M1 , M2 ; Y n ) + nδn = Hd (Y n ) − Hd (Y n |M1 , M2 ) + nδn (a)

= Hd (Y n ) − Hd (Y n |M1 , M2 , X1n , X2n ) + nδn

(b)

= Hd (Y n ) − Hd (Y n |X1n , X2n ) + nδn

= Id (X1n , X2n ; Y n ) + nδn

(92)

where (a) follows since X1n is a function of M1 , X2n is a function of M1 and X1n , and from the fact that conditioning reduces entropy, and (b) follows from the fact that (M1 , M2 ) − (X1n , X2n , (n)

d) − Y n is a Markov chain. Additionally, since the error probability Pe

vanishes as n tends to

infinity, Fano’s Inequality yields that δn vanishes as n tends to infinity. nR2 = H(M2 |M1 ) = H(M2 |M1 ) − Hd (M2 |M1 , Y n ) + Hd (M2 |M1 , Y n ) (a)

≤ Id (M2 ; Y n |M1 ) + nδn

= Hd (Y n |M1 ) − Hd (Y n |M1 , M2 ) + nδn (b)

= Hd (Y n |M1 , X1n ) − Hd (Y n |M1 , M2 , X1n , X2n ) + nδn

(c)

= Hd (Y n |X1n ) − Hd (Y n |X1n , X2n ) + nδn

= Id (X2n ; Y n |X1n ) + nδn

(93)

where (a) follows since conditioning reduces entropy, (b) follows since X1n is a function of M1 , X2n is a function of M2 and X1n , and from the fact that conditioning reduces entropy, and (c) follows from the fact that (M1 , M2 ) − (X1n , X2n , d) − Y n and M1 − (X1n , d) − Y n are a Markov chains. Since both encoders do not know the delay d, X1n and X2n do not depend on the delay d. Therefore, we can write the following outer rate region as a function of P (xn1 , xn2 ), d ∈ D and

DRAFT

29

n,  + δn  R1 + R2 ≤ n n . Rn,d (P (x1 , x2 )) = (R1 , R2 ) :  R2 ≤ n1 Id (X2n ; Y n |X1n ) + δn   

1 I (X1n , X2n ; Y n ) n d

(94)

In addition, the fact that both encoders do not know the delay d means that a rate-pair is achievable only if it lies in the intersection over d, of all the regions Rn,d (P (xn1 , xn2 )). Denote \ (95) Rn,d (P (xn1 , xn2 )) . Rn (P (xn1 , xn2 )) = d∈D

The union over all p.m.f. P (xn1 , xn2 ) yields the outer rate region [ Rn = Rn (P (xn1 , xn2 )) .

(96)

n P (xn 1 ,x2 )

Finally, taking n → ∞, and noting that δn vanishes as n tends to infinity, yields that the capacity region is included in the region [

lim inf n→∞

 \

n P (xn 1 ,x2 ) d∈D

 R1 + R2 ≤ n1 Id (X1n , X2n ; Y n )  . (R1 , R2 ) :  R2 ≤ 1 Id (X n ; Y n |X n )  n

2

(97)

1

C. Proof of Theorem 2 As mentioned before, by sending predefined training sequences in the first o(n) bits, the decoder can deduce the delay with probability of error that vanishes as n tends to infinity. Therefore, we can assume that the decoder knows the delay d. In addition, we ignore the end effects in our notations, since the first/last symbols do not affect the asymptotic performance in terms of the reliably transmitted rates. 0

Codebook Generation: The codebooks C (1) , C (1 ) and C (2) (l), 1 ≤ l ≤ |C (1) | are produced in the following manner: Set PX1 (x1 ), PX2 |V (x2 |v). Let C (1) be the codebook of the common message M1 , which conQ sists of 2nR1 codewords, each of these codewords is generated according to P (xn1 ) = ni=1 PX1 (x1,i ). 0

The codebook C (1 ) is produced from C (1) by following the one-to-one mapping: let xn1 (l) be the lth codeword in C (1) , then for every i ∈ {1, . . . , n} define vi (l) = (x1,i−dmax (l), . . . , x1,i+dmin (l)).

(98)

DRAFT

30 0

The resulting codeword v n (l) = (v1 (l), . . . , vn (l)) is the l-th codeword in C (1 ) , that is, the 0

codewords in C (1) appear in C (1 ) as vectors that were produced by a sliding window of size D on the sequence xn1 . 0

Now, for every v n (l) ∈ C (1 ) generate randomly and independently 2nR2 codewords {xn2 (l, 1), Q . . . , xn2 (l, 2nR2 )} according to P (xn2 |v n ) = ni=1 PX2 |V (x2,i |vi ). We denote {xn2 (l, 1), . . . , xn2 (l, 2nR2 )} by C (2) (l). Encoding: To send the messages m1 , m2 encoder 1 sends xn1 (m1 ) and encoder 2 sends xn2 (m1 , m2 ). Decoding: Suppose that the actual delay in the channel is d ∈ D. Let σ(xn1 , d) be the function   (x1,n−d+1 , . . . , x1,n , x1,1 , . . . , x1,n−d ) if d ≥ 0 n (99) σ(x1 , d) =  (x1,|d| , . . . , x1,n , x1,1 , . . . , x1,|d|−1 ) if d < 0. Given that the decoder knows the delay d, it first looks for m ˆ 1 ∈ {1, . . . , 2nR1 } such that n (σ(xn1 (m ˆ 1 ), d), y n ) ∈ Td, (X1 , Y )

(100)

where Pd (x1 , y) =

X

P (v)1{vdmax −d+1 =x1 } P (x2 |v)P (y|x2 , x1 ),

(101)

v,x2 n and Td, (X1 , Y ) is the set of all vectors (xn1 , y n ) that are -strongly typical with respect to

Pd (x1 , y). ˆ 1 ), it can deduce the sequence v n (m ˆ 1 ) by Once the decoder recovers the sequence xn1 (m the one-to-one mapping which is stated by Eq. (98). Then, with the delay knowledge that, as mentioned before, exists at the decoder, it looks for m ˆ 2 ∈ {1, . . . , 2nR2 } such that n (v n (m ˆ 1 ), xn2 (m ˆ 1, m ˆ 2 ), y n ) ∈ Td, (V, X2 , Y )

(102)

Pd (v, x2 , y) = P (v)P (x2 |v)P (y|x2 , vdmax −d+1 ),

(103)

where

n and Td, (V, X2 , Y ) is the set of all vectors (v n , xn2 , y n ) that are -strongly typical with respect to

Pd (v, x2 , y).

DRAFT

31

Analysis of the probability of error: Suppose that the pair of messages (m1 , m2 ) = (1, 1) is sent, and that the delay is d ∈ D. An error is made if one or more of the following events occur: E1 = {xn1 (1) ∈ / Tn (X1 )} E2 = {v n (1) ∈ / Tn (V )} E3 = {(xn2 (1, 1), v n (1)) ∈ / Tn (X2 , V )}     n n n (x1 (1), y ) ∈ / Td, (X1 , Y ) or E4 = (v n (1), xn (1, 1), y n ) ∈ / T n (V, X2 , Y ) 2

d,

n E5 = {∃m ˆ 1 6= 1 s.t. (σ(xn1 (m ˆ 1 ), d), y n ) ∈ Td, (X1 , Y )} n E6 = {∃m ˆ 2 6= 1 s.t. (xn2 (1, m ˆ 2 ), y n ) ∈ Td, (X2 , Y |v n (1))}

(104)

where the function σ(·, ·) is defined in (99). By the union bound, ! 6 [ Pr(E) = Pr Ei ≤ Pr(E1 ) + Pr(E1c ∩ E2 ) + Pr(E2c ∩ E3 ) + Pr(E3c ∩ E4 ) + Pr(E5 ) + Pr(E6 ). i=1

(105) From the LLN Pr(E1 ) → 0 as n → ∞. In addition, from the stationarity and ergodicity of v n we infer that Pr(E2 ) → 0 as n → ∞. By the conditional typicality lemma [30, p. 46] Pr(E3 ) → 0 as n → ∞. Additionally, the conditional typicality lemma [30, p. 46] implies that Pr(E4 ) → 0 as n → ∞. Since all xn1 ∈ C (1) were generated according to an i.i.d. distribution, we can use the packing lemma [30, p. 46] to deduce that Pr(E5 ) → 0 as n → ∞ if R1 < Id (X1 ; Y ).

(106)

Now we notice that, Pd (y

n

|xn2 , v n )

=

n Y

P (yi |x2,i , vi,dmax −d+1 )

(107)

i=1

that is, y n is memoryless given the sequences xn2 , v n and the delay. Therefore, an additional use of the packing lemma yields Pr(E6 ) → 0 as n → ∞ if R2 < Id (X2 ; Y |V ).

(108) DRAFT

32

We can argue that if (R1 , R2 ) is an achievable rate, then (R1 + R2 , 0) is an achievable rate as well. This is true since we can decompose each common message m1 into two common sub-messages (m11 , m12 ) and let encoder 1 send m11 and encoder 2 send m12 in addition to m2 . The decoder finds (m11 , m12 ) and can assemble the message m1 . Therefore, we can write the following rate region for every P (x1 ), P (x2 |v), and d ∈ D,    R1 + R2 ≤ Id (X1 ; Y ) + Id (X2 ; Y |V ),  . (109) Rd (P (x1 ), P (x2 |v)) = (R1 , R2 ) :   R2 ≤ Id (X2 ; Y |V ) Since the encoder does not know the delay d ∈ D, a rate-pair is achievable for fixed P (x1 ), P (x2 |v) if it lies in the intersection of all the regions Rd (P (x1 ), P (x2 |v)). Therefore, \ Rd (P (x1 ), P (x2 |v)) R(P (x1 ), P (x2 |v)) = d∈D

 R1 + R2 ≤ mind∈D [Id (X1 ; Y ) + Id (X2 ; Y |V )] ,  . (110) (R1 , R2 ) :   R2 ≤ mind∈D Id (X2 ; Y |V )  

where the last equality follows since the set of all possible delays is finite. Consequently, the following rate region [ R=

R(P (x1 ), P (x2 |v)),

(111)

P (x1 ),P (x2 |v)

is achievable. Finally, since dmax , dmin < ∞ we can use time sharing arguments to infer that the closure convex of the rate region R is an achievable rate region for the ACMAC. D. Proof of Theorem 3 Let, Pd (m1 , m2 , xn1 , v n , xn2 , y n ) = P (m1 )P (m2 )P (xn1 |m1 )P (v n |xn1 )P (xn2 |xn1 )Pd (y n |xn1 , xn2 ), (112) where P (m1 ) = 2−nR1 , P (v n |xn1 ) =

n Y

P (m2 ) = 2−nR2 ,

1{vi =(x1,i−dmax ,...,x1,i+dmin )}

i=1

Pd (y

n

|xn1 , xn2 )

=

n Y

P (yi |x1,i−d , x2,i ).

(113)

i=1

DRAFT

33

We denote information theoretic functionals of Pd (m1 , m2 , xn1 , xn2 , y n ) by the subscript d, e.g., Hd (M1 , M2 |Y n ). We first upper bound the sum-rate R1 + R2 . For every sequence of (2nR1 , 2nR2 , n)-codes with (n)

probability of error Pe

that vanishes as n tends to infinity for every d ∈ D,

n(R1 + R2 ) = H(M1 , M2 ) = H(M1 , M2 ) − Hd (M1 , M2 |Y n ) + Hd (M1 , M2 |Y n ) = Id (M1 , M2 ; Y n ) + nδn

(114)

where the second equality follows since the messages of the users do not depend on the delay (n)

d. Additionally, since the error probability Pe

vanishes as n tends to infinity, Fano’s Inequality

yields that δn vanishes as n tends to infinity. We now bound the term Id (M1 , M2 ; Y n ). n

Id (M1 , M2 ; Y ) =

n X

Id (M1 , M2 ; Yi |Y i−1 )

i=1

= (a)

≤ (b)

≤ (c)

=

=

n X i=1 n X i=1 n X i=1 n X i=1 n X

Hd (Yi |Y

i−1

)−

n X

Hd (Yi |M1 , M2 , Y i−1 )

i=1

Hd (Yi ) − Hd (Yi ) − Hd (Yi ) −

n X i=1 n X i=1 n X

Hd (Yi |M1 , M2 , Y i−1 ) Hd (Yi |M1 , M2 , X2,i , X1,i−d , Y i−1 ) Hd (Yi |X2,i , X1,i−d )

i=1

Id (X1,i−d , X2,i ; Yi )

(115)

i=1

where (a) and (b) follow since conditioning reduces entropy, and (c) follows since (M1 , M2 , Y i−1 ) − (X1,i−d , X2,i , d) − Yi is a Markov chain for any given d and all i. Now it is left to bound the rate R2 . nR2 = H(M2 |M1 ) = H(M2 |M1 ) − Hd (M2 |M1 , Y n ) + Hd (M2 |M1 , Y n ) = Id (M2 ; Y n |M1 ) + nδn .

(116) DRAFT

34

Again, by Fano’s Inequality we have that δn vanishes as n tends to infinity. Let Vi = (X1,i−dmax , . . . , X1,i+dmin ), then n

Id (M2 ; Y |M1 ) =

n X

Id (M2 ; Yi |M1 , Y i−1 )

i=1

= (a)

≤

n X i=1 n X

Hd (Yi |M1 , Y

i−1

)−

n X

Hd (Yi |M1 , M2 , Y i−1 )

i=1

Hd (Yi |X1,i−dmax , . . . , X1,i+dmin )

i=1

− (b)

≤ (c)

=

=

n X

Hd (Yi |M1 , M2 , Y i−1 , X1,i−dmax , . . . , X1,i+dmin )

i=1 n X

n X

i=1 n X

i=1 n X

Hd (Yi |Vi ) −

i=1 n X

Hd (Yi |Vi ) −

Hd (Yi |M1 , M2 , X2,i , Vi , Y i−1 ) Hd (Yi |X2,i , Vi )

i=1

Id (X2,i ; Yi |Vi )

(117)

i=1

where (a) follows since (X1,i−dmax , . . . , X1,i+dmin ) is a function of M1 , and from the fact that conditioning reduces entropy, (b) follows from the fact that conditioning reduces entropy, and (c) follows since (M1 , M2 , Y i−1 ) − (Vi , X2,i , d) − Yi is a Markov chain for any given d and all i. Hence, we have that for every d ∈ D n(R1 + R2 ) ≤ nR2 ≤

n X i=1 n X

Id (X1,i−d , X2,i ; Yi ) + nδn Id (X2,i ; Yi |Vi ) + nδn .

(118)

i=1

In addition, note that the delay d is known only at the receiver, that is, X1n and X2n are not functions of the delay. Let T be a time sharing random variable which is distributed uniformly over {1, . . . , n} and

DRAFT

35

independent of X1n , V n , X2n , Y n , and let X1 = X1,T −d , X2 = X2,T and Y = YT . Then, n

1X R1 + R2 ≤ Id (X1,i−d , X2,i ; Yi ) + δn n i=1 = Id (X1 , X2 ; Y |T ) + δn = Hd (Y |T ) − Hd (Y |X1 , X2 , T ) + δn (a)

= Hd (Y |T ) − H(Y |X1 , X2 ) + δn

(b)

≤ Hd (Y ) − H(Y |X1 , X2 ) + δn = Id (X1 , X2 ; Y ) + δn

(119)

where (a) follows since Pd (y|x1 , x2 , t) = P (y|x1 , x2 ), and (b) follows since conditioning reduces entropy. Similarly, n

1X Id (X2,i ; Yi |Vi ) + δn R2 ≤ n i=1 = Id (X2 ; Y |T, V ) + δn = Hd (Y |V, T ) − H(Y |X1 , X2 , T ) + δn (a)

= Hd (Y |V, T ) − Hd (Y |X2 , V ) + δn

(b)

≤ Hd (Y |V ) − Hd (Y |X2 , V ) + δn = Id (X2 ; Y |V ) + δn

(120)

where (a) follows since Pd (y|x2 , v, t) = Pd (y|x2 , v), and (b) follows since conditioning reduces entropy. Since both encoders do not know the delay d in advance, X1 and X2 do not depend on the delay d. Therefore, after taking n → ∞, we can write the following outer rate region given P (x1 ), P (x2 |v), and d ∈ D,  

 R1 + R2 ≤ Id (X1 , X2 ; Y )  Rd (P (x1 ), P (x2 |v)) = . (R1 , R2 ) :  R2 ≤ Id (X2 ; Y |V ) 

(121)

In addition, the fact that both encoders do not know in advance the delay d means that a rate-pair is achievable only if it lies in the intersection of all the regions Rd (P (x1 ), P (x2 |v)). DRAFT

36

Denote R(P (x1 ), P (x2 |v)) =

\

Rd (P (x1 ), P (x2 |v)).

(122)

d∈D

Note that since the set of all possible delays is finite it follows that    R1 + R2 ≤ mind∈D Id (X1 , X2 ; Y )  R(P (x1 ), P (x2 |v)) = . (R1 , R2 ) :  R2 ≤ mind∈D Id (X2 ; Y |V )  The union over all p.m.f.’s P (x1 ) and P (x2 |v) yields the rate region [ R= R(P (x1 ), P (x2 |v)).

(123)

(124)

P (x1 ),P (x2 |v)

Finally, a time sharing argument implies that the closure convex of R includes all achievable rate regions for the ACMAC. E. Proof of Theorem 4 The proof of Theorem 4 is similar to the proof of Theorem 1, for the sake of brevity we only outline the proof. By sending predefined training sequences in the first o(n) bits, the decoder can deduce the delay with probability of error that vanishes as n tends to infinity. Therefore, we can assume that the decoder knows the delay d. The remaining of the coding scheme can be described in the following manner. Partition each of the input sequences to sequences of length k (hypersymbols). Generate the codebooks of the two encoders using superposition coding according to p.m.f.’s of the hyper symbols. That is, for each of the codewords in the uninformed encoder’s codebook the informed encoder generates a separate codebook. Let C (1) be the codebook of the uninformed encoder, and let C (1) (m) be the m-th codeword in C (1) . Denote by C (2) (m) the codebook of the informed encoder which is associated with the codeword C (1) (m). Further, the kth codeword in C (2) (m) is denoted by C (2) (m, k). Suppose that the uninformed and the informed encoders transmit the messages m and k, respectively. In the encoding stage, the uninformed encoder transmits C (1) (m) to the informed encoder. The informed encoder first verifies that the codeword to be transmitted by the uninformed encoder is unique (does not appear more than once in C (1) ). Then, the uninformed and the informed encoders transmit C (1) (m) and C (2) (m, k), respectively. Finally, when the decoder receives the output sequence, it partitions it to sequences of length k, and then discards the first dmax and the last dmin symbols of every hyper symbol. DRAFT

37

This process yields a modified output sequence whose hyper-symbols are statistically independent given the input hyper-symbols. Therefore, standard typicality techniques can be used to prove that the resulting average probability of error vanishes as n tends to infinity. F. Proof of Theorem 5 0

Codebook Generation: The codebooks C (1) , C (1 ) and C (2) (l), 1 ≤ l ≤ |C (1) | are produced in the following manner: Set PX1 (x1 ), PX2 |V (x2 |v). Let C (1) be the codebook of the common message M1 , which consists of 2nR1 codewords, each of these codewords is generated according Q to P (xn1 ) = ni=1 PX1 (x1,i ). 0

The codebook C (1 ) is produced from C (1) by the following one-to-one mapping: let xn1 (l) be the lth codeword in C (1) , then for every i ∈ {1, . . . , n} define vi (l) = (x1,i−dmax (l), . . . , x1,i+dmin (l)).

(125) 0

The resulting codeword v n (l) = (v1 (l), . . . , vn (l)) is the l-th codeword in C (1 ) , that is, the 0

codewords in C (1) appear in C (1 ) as vectors that were produced by a sliding window of size D on the sequence xn1 . 0

Now, for every v n (l) ∈ C (1 ) generate randomly and independently 2nR2 codewords {xn2 (l, 1), Q . . . , xn2 (l, 2nR2 )} according to P (xn2 |v n ) = ni=1 PX2 |V (x2,i |vi ). We denote {xn2 (l, 1), . . . , xn2 (l, 2nR2 )} by C (2) (l). Encoding: To send the messages m1 , m2 encoder 1 sends xn1 (m1 ). Encoder 2 checks if xn1 (m1 ) ˆ 1 ) = xn1 (m1 ). is unique in C (1) , that is, if there is a unique m ˆ 1 ∈ {1, . . . , 2nR } such that xn1 (m If there is, encoder 2 sends xn2 (m ˆ 1 , m2 ), otherwise an error is declared and encoder 2 sends a sequence of zeroes. Decoding: Suppose that the actual delay in the channel is d ∈ D, and let Pd (v, x1 , x2 , y) = P (v)1{x1 =vdmax −d+1 } P (x2 |v)P (y|x1 , x2 ),

(126)

n and Td, (V, X2 , Y ) be the set of all vectors (v n , xn2 , y n ) that are -strongly typical with respect

to Pd (v, x2 , y). Given that the decoder knows the delay d, it looks for a pair of messages (m ˆ 1, m ˆ 2 ) ∈ {1, . . . , 2nR1 } × {1, . . . , 2nR2 } such that n (v n (m ˆ 1 ), xn2 (m ˆ 1, m ˆ 2 ), y n ) ∈ Td, (V, X2 , Y ).

(127)

If there is no such pair or if there is more then one, an error is declared. DRAFT

38

Analysis of the Probability of Error: Suppose that the pair of messages (m1 , m2 ) = (1, 1) is sent, and that the delay is d ∈ D. An error is made if one or more of the following events occur: / Tn (X1 )} E0 = {xn1 (1) ∈ E1 = {∃m ˆ 1 6= 1 s.t. xn1 (m ˆ 1 ) = xn1 (1)} E2 = {v n (1) ∈ / Tn (V )} E3 = {(v n (1), xn2 (1, 1)) ∈ / Tn (V, X2 )} n o n n n n E4 = (v (1), x2 (1, 1), y ) ∈ / Td, (V, X2 , Y ) n o nR n n n n 2 E5 = ∃m ˆ 1 6= 1 and m ˆ 2 ∈ {1, . . . , 2 } s.t. (v (m ˆ 1 ), x2 (m ˆ 1, m ˆ 2 ), y ) ∈ Td, (V, X2 , Y ) n o nR2 n n n n E6 = ∃m (128) ˆ 2 ∈ {1, . . . , 2 } s.t. (v (1), x2 (1, m ˆ 2 ), y ) ∈ Td, (V, X2 , Y ) where the function σ(·, ·) is defined in (99). By the union bound, ! 6 [ Pr(E) = Pr Ei ≤ Pr(E0 ) + Pr(E1 ∩ E0c ) + Pr(E1c ∩ E2 ) i=0

+ Pr(E2c ∩ E3 ) + Pr(E3c ∩ E4 ) + Pr(E5 ∩ E1c ) + Pr(E6 )

(129)

From the LLN Pr(E0 ) → 0 as n → ∞. Additionally, every sequence in the codebook of encoder 1 is generated by a memoryless source with p.m.f. PX1 , therefore Pr(E1 ∩ E0c ) ≤ ˜

˜

enR1 2−nH(X1 ) . Thats is, Pr(E1 ∩E0c ) → 0 as n ˜ → ∞ if R1 < H(X1 ). Further, from the stationarity and ergodicity of v n we infer that Pr(E2 ) → 0 as n → ∞. By the conditional typicality lemma [30, p. 27], Pr(E3 ) → 0 and Pr(E4 ) → 0 as n → ∞. We now bound Pr(E5 , y n ). Since both (v n (m1 ), xn2 (m1 , m2 )) and y n are not generated according to an i.i.d. distribution, we cannot use the packing lemma. We therefore upper bound the probability that the output of the channel y n is accidently typical with the pair of typical sequences of the form (v n , xn2 ). We denote this probability by Pr(E5 , y n ). Let n o n A(y n ) = (xn1 , xn2 ) ∈ X1n × X2n : (v n , xn2 , y n ) ∈ Td, (V, X2 , Y ) . By definition of v n , it follows that     n n n n (v , x2 , y ) ∈ Td, (V, X2 , Y ) n n n n A(y n ) = . (x , x ) ∈ X1 × X2 : n  1 2 (σ(xn1 (1), d), xn2 (1, 1), y n ) ∈ Td, (X1 , X2 , Y ) 

(130)

(131)

DRAFT

39

Further, Pr(E5 , y n ) =

X

P (xn1 , xn2 ) =

X

P (xn1 )P (xn2 |xn1 ).

(132)

n n (xn 1 ,x2 )∈A(y )

n n (xn 1 ,x2 )∈A(y )

Now, we the sequence xn1 according to an i.i.d. distribution, that is, P (xn1 ) =

Qn

i=1

P (x1,i ).

Additionally, P (xn2 |xn1 )

=

n Y

P (x2,i |vi )

(133)

i=1

where vi = (x1,i−dmax , . . . , x1,i+dmin ). Therefore, X

n

Pr(E5 , y ) =

n Y

n n i=1 (xn 1 ,x2 )∈A(y )

P (x1,i )

n Y

P (x2,i |vi )

(134)

i=1

Now, since xn1 is typical there exists 1 (δ) such that 1 (δ) → 0 as δ → 0, and n Y

P (x1,i ) ≤ 2−n[H(X1 )−1 (δ)] .

(135)

i=1

In addition, since (v

n

, xn2 )

are jointly typical, there exists 2 (δ) such that 2 (δ) → 0 as δ → 0,

and n Y

P (x2,i |vi ) ≤ 2−n[H(X2 |V )−2 (δ)] .

(136)

i=1

We have that Pr(E5 , y n ) ≤

X

2−n[H(X1 )−1 (δ)] 2−n[H(X2 |V )−2 (δ)] .

(137)

n (xn 1 ,x2 )∈A

(138) By typicality, there are no more than 2Hd (X1 |Y )+3 (n) sequences xn1 in the set A(y n ), where 3 (n) vanishes as n tends to infinity. Additionally, for each xn1 there are at most 2Hd (X2 |Y,V )+4 (n) sequences xn2 in the set A(y n ), where 4 (n) vanishes as n tends to infinity. Therefore, Pr(E5 , y n ) ≤ 2Hd (X1 |Y )+3 (n)+Hd (X2 |Y,V )+4 (n) 2−n[H(X1 )−1 (δ)+H(X2 |V )−2 (δ)]

(139)

= 2−n[H(X1 )−Hd (X1 |Y )+H(X2 |V )−Hd (X2 |Y,V )−1 (δ)−2 (δ)−3 (δ)−4 (δ)] .

(140)

Therefore Pr(E5 ) → 0 as n → ∞ if R1 + R2 < Id (X1 ; Y ) + Id (X2 ; Y |V ).

(141)

DRAFT

40

Further, by the packing lemma [30, p. 46] Pr(E6 ) tends to 0 and n tends to infinity if R2 < Id (X2 ; Y |V ). Since the encoder does not know the delay d ∈ D, a rate-pair is achievable for fixed P (x1 ), P (x2 |v) if it lies in the intersection of all the regions Rd (P (x1 ), P (x2 |v)). Therefore, \ R(P (x1 ), P (x2 |v)) = Rd (P (x1 ), P (x2 |v)) d∈D

     (R , R ) : R1 + R2 ≤ mind∈D [Id (X1 ; Y ) + Id (X2 ; Y |V )] , 1 2 R2 ≤ mind∈D Id (X2 ; Y |V ),     R1 ≤ H(X1 )

    

. (142)

   

where the last equality follows sice the set of all possible delays is finite. Consequently, the following rate region [ R=

R(P (x1 ), P (x2 |v)),

(143)

P (x1 ),P (x2 |v)

is achievable. Finally, since dmax , dmin < ∞ we can use time sharing arguments to infer that the closure convex of the rate region R is an achievable rate region for the ACMAC. R EFERENCES [1] M. Yemini, A. Somekh-Baruch, and A. Leshem, “On the asynchronous cognitive MAC,” in Proc. IEEE Int. Symp. Information Theory (ISIT’14), Honolulu, HI, USA, June/July 2014, pp. 2929–2933. [2] J. Mitola, “Cognitive radio: An integrated agent architecture for software defined radio,” Ph.D. dissertation, KTH Royal Institute of Technology Stockholm, Sweden, 2000. [3] S. Haykin, “Cognitive radio: Brain-empowered wireless communications,” IEEE Journal on Selected Areas in Communication, vol. 23, no. 2, pp. 201–220, February 2005. [4] A. Goldsmith, A. Jafar, I. Mari´c, and S. Srinivasa, “Breaking spectrum gridlock with cognitive radio: An information theoretic perspective,” Proceedings of the IEEE, vol. 97, no. 5, pp. 894–914, May 2009. [5] S. I. Gelf’and and M. S. Pinsker, “Coding for channel with random parameters,” Problem of Control and Information Theory, vol. 9, no. I, pp. 19–31, 1980. [6] N. Devroye, P. Mitran, and V. Tarokh, “Achievable rates in cognitive radio channels,” IEEE Trans. Inf. Theory, vol. 52, no. 5, pp. 1813–1827, May 2006. [7] C. E. Shannon, “Channels with side information at the transmitter,” IBM Research and Development, vol. 2, pp. 289–293, 1958. [8] E. C. van der Meulen, “The discrete memoryless channel with two senders and one receiver,” Proc. 2nd Int. Symp. Inf. Theory, Tsahkadsor, Armenian S.S.R., pp. 103–135, 1971. DRAFT

41

[9] R. Ahlswede, “Multiway communication channels,” in Proceedings of 2nd International Symposium on Information Theory, Tsahkadsor, Armenian USSR, September 1971, pp. 23–52. [10] D. Slepian and J. K. Wolf, “A coding theorem for multiple access channels with correlated sources,” Bell System Technical Journal, vol. 52, pp. 1037–1076, September 1973. [11] A. Somekh-Baruch, S. Shamai (Shitz), and S. Verd´u, “Cooperative multiple access encoding with states available at one transmitter,” IEEE Trans. Inf. Theory, vol. 54, pp. 4448–4469, October 2008. [12] M. Li, O. Simeone, and A. Yener, “Multiple access channels with states causally known at transmitters,” IEEE Trans. Inf. Theory, vol. 59, pp. 1394–1404, March 2013. [13] A. Zaidi, P. Piantanida, and S. Shamai (Shitz), “Capacity region of cooperative multiple-access channel with states,” IEEE Trans. Inf. Theory, vol. 59, pp. 6153–6174, October 2013. [14] S. I. Bross, A. Lapidoth, and M. Wigger, “Dirty-paper coding for the Gaussian multiaccess channel with conferencing,” IEEE Trans. Inf. Theory, vol. 58, pp. 5640–5668, September 2012. [15] I. Mari´c, R. D. Yates, and G. Kramer, “Capacity of interference channels with partial transmitter cooperation,” IEEE Transaction on Information Theory, vol. 53, no. 10, pp. 3536–3548, October 2007. [16] T. M. Cover, R. J. McEliece, and E. C. Posner, “Asynchronous multiple-access channel capacity,” IEEE Trans. Inf. Theory, vol. 27, no. 4, pp. 409–413, July 1981. [17] J. Hui and P. A. Humblet, “The capacity region of the totally asynchronous multiple-access channel,” IEEE Trans. Inf. Theory, vol. 31, no. 2, pp. 207–216, March 1985. [18] S. Verd´u, “The capacity region of the symbol-asynchronous Gaussian multiple-access channel,” IEEE Trans. Inf. Theory, vol. 35, no. 4, pp. 733–751, July 1989. [19] ——, “Multiple-access channels with memory with and without frame synchronism,” IEEE Trans. Inf. Theory, vol. 35, no. 3, pp. 605–619, May 1989. [20] M. K. Karakayali, G. J. Foschini, and R. A. Valenzuela, “Network coordination for spectrally efficient communications in cellular systems,” IEEE Wireless Communications, vol. 13, pp. 56–61, August 2006. [21] R. Irmer, H. Droste, P. Marsch, M. Grieger, G. Fettweis, S. Brueck, H. P. Mayer, L. Thiele, and V. Jungnickel, “Coordinated multipoint: Concepts, performance, and field trial results,” IEEE Communications Magazine, vol. 49, pp. 102–111, February 2011. [22] Y. Liu, Y. Li, D. Li, and H. Zhang, “Space-time coding for time and frequency asynchronous CoMP transmissions,” IEEE Wireless Communications and Networking Conference (WCNC), pp. 2632–2637, April 2013. [23] M. Yemini, A. Somekh-Baruch, and A. Leshem, “On channels with asynchronous state information at the transmitter,” Electrical and Electronics Engineers in Israel (IEEEI), 2012 IEEE 27th Convention of, 2012. [24] ——, “Asynchronous transmission over single-user state-dependent channels,” Submitted to IEEE Trans. Inf. Theory, 2014. [25] T. M. Cover and J. A. Thomas, Elements of information theory, 2nd ed.

Wiley Interscience, 2006.

[26] A. S. Motahari and A. K. Khandani, “Capacity bounds for the Gaussian interference channel,” IEEE Trans. Inf. Theory, vol. 55, no. 2, pp. 620–643, February 2009. [27] X. Shang, G. Kramer, and B. Chen, “A new outer bound and the noisy-interference sum-rate capacity for Gaussian interference channels,” IEEE Trans. Inf. Theory, vol. 55, no. 2, pp. 689–699, February 2009. [28] V. S. Annapureddy and V. V. Veeravalli, “Gaussian interference networks: Sum capacity in the low-interference regime and new outer bounds on the capacity region,” IEEE Trans. Inf. Theory, vol. 55, no. 7, pp. 3032–3050, July 2009. [29] R. G. Gallager, Information Theory and Reliable Communication.

New York: Wiley, 1968.

DRAFT

42

[30] A. El-Gamal and Y. H. Kim, Network Information Theory.

Cambridge University Press, 2011.

DRAFT