Coding Theorems for the Shannon Cipher System With a Guessing ...
© 2008 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
2808
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Coding Theorems for the Shannon Cipher System With a Guessing Wiretapper and Correlated Source Outputs Yutaka Hayashi and Hirosuke Yamamoto, Senior Member, IEEE
the exponent of Pc , i.e., Y 0(log Pc )=N , in the case that the wiretapper gets the cryptogram Z of Y . Even when the wiretapper has no cryptogram Z , he can guess the correct plaintext with the probability given by
Pc3 Abstract—The security level of the Shannon cipher system is traditionally ( j ), where is a secret plaintext with measured by equivocation and is its cryptogram. But, Merhav and Arikan have considlength ered another security criterion, which is measured by the number of guesses needed for a wiretapper to uncover from . Merhav has also considered the third security criterion, which measured by the probability of correct guess of a wiretapper. On the other hand, in the case of the traditional security criterion, Yamamoto has treated a coding problem for correlated and such that only is secret against wiretappers source outputs and only must be transmitted to a legitimate receiver. In this correspondence, coding theorems are proved for the case that Yamamoto’s coding problem is applied to Merhav–Arikan’s security criterion or Merhav’s security criterion.
N
HYZ
Z
Y
X
Y
Y
Y
0Y where 0Y is defined by 0Y 0 log max P (y ); y
Index Terms—Coding theorem, correlated sources, guessing wiretapper, perfect secrecy, Shannon cipher system.
I. INTRODUCTION The security level of the Shannon cipher system [1] is traditionally measured by equivocation N1 H (Y jZ ), where Y = Y1 Y2 . . . YN and Z are a secret independent and identically distributed (i.i.d.) plaintext and its cryptogram, respectively, and N is the length of Y . It is known that if security level N1 H (Y jZ ) = (0 H (Y )) is required, the key rate R of the cipher system must satisfy R [2]. In the case of = H (Y ), perfect secrecy can be attained, but the key rate R must be as large as H (Y ). As a new security criterion instead of the equivocation, Merhav and Arikan [3] proposed to use the expected th moment of the number of guesses to uncover a plaintext Y from its cryptogram Z . In their security criterion, the cipher system becomes more secure as a wiretapper needs more guesses. They showed that the asymptotic optimal exponent of the expected th moment can be represented by a function CY (R; ). The function CY (R; ) is nondecreasing in R, and CY (R; ) is constant if R is larger than a certain value R3 . Hence, if R R3 , perfect secrecy can be attained in their security criterion. Merhav and Arikan also considered the probability of large deviation such that the number of guesses becomes larger than 2LN for a given L > 0, and they showed that the asymptotic optimal exponent of the probability can also be represented by another function FY (R; L). Furthermore, they proved that CY (R; ) and FY (R; L) are related by the Fenchel–Legendre transform and its inverse transform. As another security criterion, Merhav [4] proposed to evaluate the security of the Shannon cipher system by the exponent of the probability Pc that a wiretapper guesses a correct plaintext Y . Let Y be Manuscript received February 14, 2007; revised January 10, 2008. This work was supported in part by the JSPS and MEXT Grants-in-Aid for Scientific Research, 16656114, 17360174, 18656110. The material in this correspondence was presented in part at the IEEE International Symposium on Information Theory, Seattle, WA, July 2006. Y. Hayashi is with the Financial Systems Division, Nomura Research Institute, Ltd., Tower S Ridge, Kiba, Koto-ku, Tokyo, 135-0042, Japan (e-mail: [email protected]). H. Yamamoto is with the Department of Complexity Science and Engineering, School of Frontier Sciences, University of Tokyo, Kashiwanoha, Kashiwa-shi, Chiba, 277-8561, Japan (e-mail: [email protected]). Communicated by H. Imai, Guest Editor for Special Issue on Information Theoretic Security. Digital Object Identifier 10.1109/TIT.2008.921707
(1)
Hence, if a cipher attains Y
Z
X
= max P N (y ) = [max P (y )]N : y y
(2)
perfect secrecy can be attained in this security criterion because the cryptogram Z does not decrease the exponent of Pc . The security criteria proposed by Merhav and Arikan [3] or Merhav [4] are suited for the case that an adversary wiretaps an encrypted password or secret personal verification information in order to attack a bank account, a computer account, or a web site account via the Internet. Merhav–Arikan’s criteria can be used if the adversary can attack the account by trying log-in with guessed passwords repeatedly while Merhav’s criterion can be used if the adversary can try log-in only once or a few times. Secret information is often related to some other information. For instance, a password is apt to be determined based on name, address, phone number, birthday, and so forth. In this case, even if the secret information is encrypted securely, it might be possible for an adversary to get such relevant information. If the adversary attacks the secret information using the obtained relevant information, the security level of the secret might decrease significantly. Hence, when we send the relevant information even if it is not secret, we must encrypt it carefully to prevent adversaries from attacking its related secret information. This situation was treated by Yamamoto [5] under the security criterion of equivocation. More precisely, he considered a coding problem for the case such that a nonsecret Y must be transmitted to a legitimate receiver with sufficiently small error probability and a secret X correlated to Y must be protected against wiretappers. It is shown in [5] that key rate R must satisfy R j 0 H (X jY )j+ to achieve N1 H (X jZ ) = , where jtj+ maxft; 0g, and, hence, the perfect secrecy of X can be attained with R = I (X ; Y ) for = H (X ). In this correspondence, we treat the same communication system model as Yamamoto’s model with two correlated source outputs, a secret X and a nonsecret Y . But the security of the system is measured by Merhav and Arikan’s security criterion and Merhav’s security criterion. Note that the coding theorems of this case cannot be derived by combining simply the Merhav–Arikan’s results [3] (or Merhav’s results [4]) with Yamamoto’s results [5]. Although the communication system model is the same as Yamamoto’s system, the proof technique used in [5] cannot be applied to this problem because different security criteria are treated. In this correspondence, the coding theorems will be proved by using the so-called Type Theory developed by Csiszár and Körer [6] in the same way as [3] and [4]. But, the coding theorems cannot be estimated from the results shown in these papers, which treated the case of a single source output, because the roles of X and Y are asymmetric and are cumbersomely intertwined. The case of Merhav-Arikan’s security criterion is treated in Section II. The system model is described in Section II-A, and coding theorems are shown in Section II-B and are proved in Section II-C and the Appendix. We note that in order to prove the direct parts of the coding theorems, a new coding scheme is devised based on an extended packing lemma (Lemma 1) and multiple packing by all distinct sequences (Algorithm 1) in Section II-C. The case of Merhav’s security criterion is treated in Section III. The system model and a coding theorem are described in Section III-A, and
0018-9448/$25.00 © 2008 IEEE
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
2809
F3
Fig. 1. Shannon cipher system with correlated source outputs
X and Y .
A. System Model We assume that a source generates discrete memoryless random variables X and Y , which are mutually correlated. X is a secret and Y must be transmitted to a legitimate receiver without error.1 X and Y take values from finite alphabets X and Y , respectively. Y has a probability distribution P (y ) for y 2 Y and X has a conditional probability distribution W (xjy ) for x 2 X and y 2 Y . We represent the joint probability of (X; Y ) and the marginal probability of X by P 1 W (y; x) P (y )W (xjy ) and P W (x) y P (y )W (xjy ), respectively. In the Shannon cipher system shown in Fig. 1, an encoder f generates a cryptogram Z from source outputs X , Y , and key U , where X = (X1 ; X2 ; . . . ; XN ), Y = (Y1 ; Y2 ; . . . ; YN ), and U = (U1 ; U2 ; . . . ; UK ). Uj ’s are true binary random numbers. The key rate R of this system is defined by K R= : (3) N The decoder must reproduce Y from the cryptogram Z and the key U without error. On the other hand, a wiretapper guesses the secret X from Z without U . We assume that the wiretapper knows the encoding function f and has a strategy g = fx1 (z ); x2 (z ); x3 (z ); . . .g to guess the correct secret x as fast as possible. Let GN f;g (X jZ ) be the number of guesses needed for the wiretapper to uncover the secret X by the strategy g . Remark 1: In the case of X = Y , the above system model is reduced ~ to Merhav–Arikan’s system model. Furthermore, in the case of X = X ~ Y ~ ) where X ~ and Y ~ are correlated random variables, the and Y = (X; above model corresponds to the case that the decoder reproduces both ~ and Y ~ and the wiretapper guesses only X ~ . Similarly, in the case of X ~ ~ ~ X = (X; Y ) and Y = Y , it corresponds to the case that the decoder ~ and Y ~. reproduces only Y~ and the wiretapper guesses both X B. Coding Theorems For the system model shown in the previous subsection, we consider the following two security criteria: GN f;g (X jZ )
C (R; )
0 D(Q 1 V kP 1 W )] (6) h(Q; V; R) minfH (QV ); R + H (QV ) 0 I (Q; V )g: (7) Theorem 2: For any R 0 and > 0, it holds that F 3 = F (R; ),
where
In this section, we consider a coding problem for the case that the security criterion based on the number of wiretapper’s guesses, which was treated by Merhav–Arikan [3], is applied to the Shannon cipher system with correlated source outputs (X; Y ), which was treated for the security criterion of equivocation by Yamamoto [5].
f
(5)
Theorem 1: For any R 0 and > 0, it holds that C 3 = C (R; ), where
II. SECURITY CRITERION BASED ON THE NUMBER OF GUESSES
1 N!1 lim sup inf log E g N
NL GN f;g (X jZ ) 2
where E[A] represents the expectation of A, L > 0 is a parameter, and GN f;g (X jZ ) is the th moment of the number of guesses. Then, for X and Y with probability distributions P (x) and W (xjy ), the above C 3 and F 3 are determined by the following theorems.
the theorem is proved in Section III-B. An asymptotically optimal code in this security criterion is given in Section III-C. The notations used in this correspondence are explained in the Appendix.
C3
1 N!1 lim inf sup 0 log Pr f g N
(4)
1In [5], a sufficiently small decoding error probability is allowed because a fixed length code is used. But, in this correspondence, the zero decoding error probability can be attained because a variable length code is used.
max[h(Q; V; R) Q;V
F (R; L)
min Q;V h(Q;V;R)L
D (Q 1 V kP 1 W ):
(8)
In the above theorems, Q(y ) and V (xjy ) are arbitrary probability distributions, and Q 1 V (y; x) and QV (x) are the joint and marginal probability distributions, respectively, which are determined from Q and V . H (QV ), I (Q; V ), and D(Q 1 V kP 1 W ) are the entropy, mutual information, and relative entropy, respectively, of these probability distributions (See the Appendix ). Furthermore, C (R; ) and F (R; L) are related by the Fenchel–Legendre transform as shown in the next theorem.
0 and > 0, it holds that C (R; ) = sup[L 0 F (R; L)] L> F (R; L) = sup [L 0 C (R; )]: >
Theorem 3: For any R
0
0
(9) (10)
Remark 2: In the case of Y = X , Theorems 1–3 coincides with Merhav–Arikan’s results, which can be derived by restricting W and V as W (xjx) = V (xjx) = 1 for X = Y , as follows:
CY (R; ) = C (R; )jX =Y = max[hY (Q; R) 0 D (QkP )] Q
FY (R; ) = F (R; )jX =Y = min D (Q kP )
hY (Q; R)
h
Q
(11)
(12)
(Q;R)L
max
fH (Q ); R g:
(13)
Remark 3: C (R; ) defined by (6) can be represented as follows:
C (R; ) = max[H (QV ) 0 jI (Q; V ) 0 Rj+ 0 D(Q 1 V kP 1 W )]: Q;V (14) ^ V ^ ) where Q ^ and V ^ are the Hence, C (R; ) is constant for R I (Q; probability distributions that attain the following maximum: max[H (QV ) Q;V
0 D(Q 1 V kP 1 W )]:
(15)
^ V ^) This means that perfect secrecy can be attained with R = I (Q; in the sense of security criterion C (R; ) because C (R; ) cannot be ^ V ^ ). This result is similar to the fact that perfect increased for R I (Q; secrecy can be attained with R = I (X ; Y ) = I (P; W ) in the sense of ^ V ^ ) is not equal the equivocation security criterion [5]. However, I (Q; to I (P; W ) generally.
We give the proof of Theorem 1 in Section II-C. The proof of Theorem 2 can be proved by combining the techniques shown in Section II-C and [3, the proof of Theorem 2], and it is given in the Appendix. The proof of Theorem 3 is omitted because it can be proved in the same way as [3, Theorem 3].
2810
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
By averaging the th moment of GN f;g (xjz ) for all types Q and all conditional types V , we have from (73) that
C. Proof of Theorem 1 We define C + , C 0 as follows: C
+
lim sup sup infg N1 log E
N !1 f 1 0 C lim inf sup inf g N N !1 f
j
N
j
N
log E Gf;g (X Z )
:
(16)
E Gf;g (X Z )
(17)
Q;V
Cg
lim sup sup N 1
N !1 f 1 0 Cf lim inf inf N !1 g N
N log E Gf;g (X
jZ )
j
N
log E Gf;g (X Z )
j
C (R; )
H (V jQ)
jTV
(y i )
j
2
j
)
H (V jQ) NR jXkYj 2 1 (N + 1) 1 2NH (V jQ) N (R+H (QV )0I (Q;V )) =2 1 (N + 1)jXjjYj (21) 0 where TV (y i ) is the V -shell of x for y i , i.e., the set of all x ’s with the conditional typle V 0 for y , and the inequalities holds from (67) and y ;V :H (V Q
)
i
(69). x 11 ; x 12 ; x 13 ; . . . and strategy g2 For strategy g1 = x 21 ; x22 ; x 23 ; . . . , we define a new strategy g as
=
1 2 1 2 1 2 g = x1 ; x1 ; x2 ; x2 ; x3 ; x3 ; . . . :
Then, the number of guesses in the strategy g is not more than twice the smaller number of guesses in g1 and g2 . Hence, we have N
j
Gf;g (x z )
2(N + 1)jXjjYj 1 min(2NH QV ; 2N R H QV 0I Q;V jXjjYj 1 2N fH QV ;R H QV 0I Q;V g : = 2(N + 1) (
min
(
)
)
( +
+
(
(
)
)
(
(23)
N
!1
f
1
N
N
j
log E Gf;g (X Z )
C (R; ):
(24)
Proof of C (R; ) Cf0 : We construct an encoding function f based on the following lemma. Lemma 1 (Extended Packing Lemma): For any given r > 0, > 0, and 0 < " 1, let Q be an arbitrary type of Y such that H (Q) > r , and let A" be an arbitrary subset of TQ that satisfies (25). Then, for sufficiently large N , there exist distinct y i , 1 i 2N (r0) , in A" that satisfy (26) for every pair of conditional types V and V^ .
6
j =i
jA" j "jTQ j TV (y j ) jTV (y i )j20N jI Q;V 0rj ^
(
^)
(25) : (26)
(20)
j
NH (V Q
jXkYj+jXj 2NC (R;) :
Cg = lim sup sup
TV (y i )
NH (Q )
1 (N + 1)jXj
)
0D(Q1V kP 1W )]
) 0 +H (QV )
2
(19)
where TQ is the set of all x with type QX , and the inequalities holds by applying (66) and (68) to the case of QX . Strategy g2 : by using The wiretapper first decodes all possible y 1 ; y 2 ; . . . ; y 2 all keys u 1 ; u 2 ; . . . ; u2 , and guess x in ascending order of conditional entropy H (V 0 jQy ) where Qy is the type of y i and V 0 is the conditional type of x for y i . Then, the number of guesses is bounded by y ;V :H (V Q
(
Since (23) holds for any encoding function f , we finally obtain :
Q :H (Q ) H (QV ) NH (QV )
2
+
[ min
(18)
Proof of Cg+ C (R; ): Assume that source outputs are (x ; y ) such that the type of y is Q and the conditional type of x for y is V . Then, the type of x is QV . We now consider the following two strategies g1 and g2 . Strategy g1 : The wiretapper ignores the cryptogram z and guess x in ascending order of entropy H (QX ) where QX is the type of x . In this strategy, the number of guesses is bounded by 2
jz )
f;g (x
2(N + 1) jXkYj jXj fR0I Q;V ; g 1 2N
+
f
0ND(Q1V kP 1W ) GN
= 2(N + 1)
if there exist an encoder f and a strategy g such that Cg+ Cf0 , we can conclude that C 3 = C (R; ).
Q :H (Q ) H (QV )
2
max
Then, it holds obviously that C 0 C 0 C 3 C + Cg+ . Hence,
jTQ j
2
Furthermore, for given f and g , define Cg+ , Cf0 as follows: +
j
N
Gf;g (X Z )
(
)
))
)
(22)
This lemma can be proved in the same way as [6, Ch. 2, Lemma 5.1], and the proof is given in the Appendix. For the case of V^ = V , we apply Lemma 1 repeatedly as follows. Algorithm 1 (Multiple Packing by All Distinct Sequences) Step 1: Let M = 2N (I (Q;V )02) and j = 1. Apply Lemma 1 to A" = TQ . For obtained y i , 1 i M , let TQ1 = fy 1 ; y 2 ; . . . ; y M g. Let A" = A" n TQ1 . Step 2: If jA" j < "jTQ j, then go to Step 4. Step 3: Let j = j + 1. Apply Lemma 1 to A" . For obtained y i , 1 j j i M , let TQ = fy 1 ; y 2 ; . . . ; y M g. Let A" = A" n TQ . Go to Step 2. Step 4: Let s = j and TQ0 = A" .
Then, it is obvious that the constructed TQj , 0 j s, satisfy the following conditions. (a) TQj , 0 j s are disjoint. (b) TQj = 2N (I (Q;V )02) for 1 j s and TQ0 < "jTQ j. (c) For 1 j s, every y 2TQj sasifties (26) with r = I (Q; V ) 0 . (d) s is bounded by s (1 0 ")2N (H (Q)0I (Q;V )+2) + 1.
Now we define an encoding function f based on TQj constructed in the above algorithm. Assume that y 2 TQj and x 2 TV (y ). Then we encode (x; y ) as follows. First we encode (Q; V ) with length l1 = dlog jQj + log jVje, where Q and V are the set of all distinct types on Y and the set of all distinct conditional types on X 2 Y . Next we encode j with length l2 = dlog(s + 1)e. Finally, we encode the index of y 2 TQj with length l3 , which is given by l3 = dN (I (Q; V ) 0 2 )e for 1 j s and l3 dN H (Q)e for s = 0. We encipher the third part of the codeword by key u with length lk = N (minfR; I (Q; V )g 0 2 ). The cryptogram z encoded from (x; y ) is the concatenation of the above three parts.
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Next, for the preceding encoding function f , we derive the lower bound of E GN f;g (X jZ ) . Since the second part with length l2 of the cryptogram z is not randomized by the key u, the wiretapper can know j from z . Furthermore, from the above property (b) for TQ0 , we have Prfj = 0j(x ; y ) 2 TQ1V g < ", and hence
f
j
Pr Z = z (x ; y )
6
z :j =0
Since it holds that
j
2 TQ1V g 1 0 ":
2811
Every x is distinct in M2 . Hence, the number of wiretapper’s guesses can be bounded from (29), (35), and (36) as follows:
(27)
1
z
j
6
z :j =0
1E
j
N
j
f j
y ) (x ;y
2T
2T
y ) (x ;y
=
2 TQ1V ; Z = z
jM(Q; V )j =
(29)
M(Q; V ) = f(x; y ) : (x; y ) 2 TQ1V \ Z 01 (z)g:
(30)
M f(x; y ) 2 M(Q; V ) : (x; y 0 ) 62 M(Q; V ) if y 0 6= y g: (33) From Lemma 1, the ratio of jM j to jM(Q; V )j tends to zero as N becomes large and " becomes small. Hence, M and M satisfy jM j = "0 jM(Q; V )j (34) 0 jM j = (1 0 " )jM(Q; V )j (35) 0 where " ! 0 as N ! 1 and " ! 0. 2
1
1
=
=
2
2M
f j
y) (x;y
+
2M
y) (x;y
2M
y) (x;y
f j
f j
Pr x; y (x ; y )
(
)
g j 0 jXjjYj (N + 1)
(38)
j
1+
12
f
g j 0 jXkYj (N + 1)
N (min R;I (Q;V ) +H (V Q) 2)
1+
f
j
(39)
Pr Q;V
fTQ1V gE 0
0" ) (1 0 ")(1 1+
j
N
j
Gf;g (X Z ) (X ; Y )
2 TQ1V
1+
0
1 k 1 0
N (h(Q;V;R) D(Q V P W ) 2)
] 1 maxQ;V [2 (N + 1)jXjjYj 0 (1 0 ")(1 0 " ) = 1 2N C R; 0 : 1+
(
(1 + )(N + 1)jXjjYj(1+)
(
)
2
)
(40)
Since (40) holds for any strategy g and any small > 0, we have that
0
N 2 TQ1V ; zgGf;g (xjz )
N 2 TQ1V ; zgGf;g (xjz ) :
0
1+
(1+ )
N 2 TQ1V ; zgGf;g (xjz )
Pr x; y (x ; y )
2 )
N (min H (QV );R+H (QV ) I (Q;V )
=
2 TQ1V ; Z = z N Prfx; y j(x; y ) 2 TQ1V ; z gGf;g (xjz )
Pr x; y (x ; y )
)
0 g02) jXkYj (N + 1) 0 1+ 2N (h(Q;V;R)02) (1 0 ")(1 0 " ) 1 (N + 1)jXkYj : 1+ 2
N
E Gf;g (X Z ) (X ; Y )
2M(Q;V )
(
2 TQ1V PrfZ = z j(x ; y ) 2 TQ1V g
E Gf;g (X Z )
2
y) (x;y
2 )
Finally, by averaging E GN f;g (X jZ ) j(X ; Y ) 2 TQ1V for all joint types Q 1 V , we obtain from (73) that
1
=
6
z :j =0
1
2
(32)
j
)
1+
M , which are defined as fol-
1
f
(
N (min R;I (Q;V ) +H (V Q) 2)
0" ) (1 0 ")(1 1+ 0 (1 0 ")(1 0 " ) =
(31)
M f(x; y ) 2 M(Q; V ) : 9(x; y 0 ) 2 M(Q; V ) for some y 0 6= y g
j
j
1+
Z 01 (z) = f(x; y ) : z 2 Z (x; y )g;
N
(min
1 jM(Q; V )j (1 0 "0 )
1
Obviously it holds that
2
=
1
and
N (min
2N
N
1
1
1
E Gf;g (X Z ) (X ; Y )
(z )
M(Q; V ) into M
2 TQ1V \ Z 0 (z) fR;I Q;V g0 1 jTV (y )j fR;I Q;V g0 1 2NH V jQ (N + 1)jXkYj
where the inequality holds from (69). Since (37) and (38) do not depend on z , E GN f;g (X jZ ) j(X ; Y ) 2 TQ1V can be bounded by combining (27), (28), (37), and (38) as follows:
jf(x; y)j(x; y) 2 TQ1V \ Z 0 (z)gj
We divide lows:
(37)
(x; y ) : (x; y )
=2
(28)
jTQ1V j0 20l jTQ1V j0 20l
1 jM(Q; V )j
where
jM j
From lk = N (minfR; I (Q; V )g 0 2 ), we have
2 TQ1V ; zg f j 2 TQ1V gPrfzjx; y g Prfx0 ; y 0 j(x 0 ; y 0 ) 2 TQ1V gPrfz jx0 ; y 0 g \Z
i=1
1+
1
=
1+
Pr x; y (x ; y )
=
i
0
we consider only the case of j 6= 0 in the following. Let Z (x; y ) be the set of all possible cryptograms for (x; y ). Then, for z 2 Z (x; y ), we have Prfz jx; y g = 20l . Furthermore, using Bayes’ theorem, we obtain that Pr x; y (x ; y )
2M jM j
jM(Q; V )j u du jM(Q; V )j (1 0 "0 ) : =
Gf;g (X Z ) (X ; Y )
2 TQ1V ; Z = z N Gf;g (xjz )
y) (x;y
1
jZ ) j(X ; Y ) 2 TQ1V ; Z = z PrfZ = z j(X ; Y ) 2 TQ1V g
N E Gf;g (X
j
1 jM(Q; V )j
E Gf;g (X Z ) (X ; Y ) =
1 jM(Q; V )j
2 TQ1V PrfZ = z j(X ; Y ) 2 TQ1V g
N
j
N
E Gf;g (X Z ) (X ; Y )
Cf = lim inf inf
(36)
N
!1
g
1
N
N
j
log E Gf;g (X Z )
C (R; ):
(41)
Remark 4: We note that Cg+ C (R; ) holds even in the case when the encoder f can know only Y . Furthermore, C (R; ) Cf0 can be
2812
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
attained even in the case when the encoder f can know Y and only the conditional type V of X for Y . But, we need both X and Y to know the conditional type V . III. SECURITY CRITERION BASED ON PROBABILITY OF CORRECT GUESS
THE
In this section, we consider a coding problem for the case that the security criterion based on the probability of wiretapper’s correct guess, which was treated by Merhav [3], is applied to the Shannon cipher system with correlated source outputs (X; Y ).
obviously. Hence, in the following, we consider only Q and V that satisfy 0 < 0 H (V jQ) + H (V jQV ) 0 D(Q 1 V kP 1 W ). Since only y must be transmitted to a legitimate receiver, we do not need consider any key length K (x; y ) longer than dN log jYje. For s s 2 f0; 1; . . . ; dN log jYjeg, we define TQ1V as follows:
TQs1V TQ1V \ f(x; y) : K (x; y) = sg:
Let TN be the number of distinct TQs1V . Then, from (66) and (67), TN is bounded by TN
A. System Model In the same way as Section II-A, a discrete memoryless source generates mutually correlated random variables X and Y with probability distribution P 1 W (y; x). Y must be transmitted to a legitimate receiver without error while X must be protected against a wiretapper. In Section II, we assumed that key length K is fixed and a wiretapper tries guessing repeatedly until he gets the correct secret. But, in this section, we assume that each key u has a variable length K (x; y ), i.e., u = (u1 ; u2 ; . . . ; uK (x;y y) ) and a wiretapper guesses x only once. Since the wiretapper wants to guess the correct x with the highest probability, the optimal guess of the wiretapper who gets a cryptogram z is given by ^ = arg max Prfx jz g: (42) x
Pc
= z
Prfz g max Prfxjz g: x
(43)
In this section, we consider a coding problem to attain Pc 20N for a given . The key length necessary to attain Pc 20N is determined by the next theorem. Theorem 4: If Pc 20N for a given > 0, then the following inequality holds for any > 0, any type Q, and any conditional type V :
TQ1V \ f(x; y ) : K (x; y ) N (j 0 H (V jQ) + H (V jQV ) 0 D (Q 1 V kP 1 W )j + 0 )g 2O(log N )0N jTQ1V j (44) where V is the reverse conditional probability distribution of V , which is defined by V Q 1 V (x; y )=QV (x). The left-hand side of (44) represents the number of pairs (x; y ) 2 TQ1V such that their key lengths are not longer than N (j 0 H (V jQ)+ jQV ) 0 D(Q 1 V kP 1 W )j+ 0 ), and (44) means that such number H (V decreases exponentially compared with jTQ1V j. Hence, in order to attain Pc 20N , almost of all (x; y ) 2 TQ1V must have a key length longer than N (j 0 H (V jQ) + H (V jQV ) 0 D(Q 1 V kP 1 W )j+ 0 ) when N is sufficiently large. Remark 5: In the case of Y = X , (44) coincides with Merhav’s results, which can be derived by restricting W and V as W (xjx) = (xjx) = 1 for X = Y , as follows: V ( x jx ) = V
Pc
= z
= z
z
= z
B. Proof of Theorem 4
If the type Q and conditional type V satisfies > 0 H (V jQ) + j 0 D(Q 1 V kP 1 W ), then we have K (x; y ) < 0 in (44). Since there are no pairs (x; y ) with negative key length, (44) holds
QV ) H (V
(N log jYj + 2)(N + 1)jXj+jXjjYj :
(47)
Prfz g max Prfxjz g x
Prfz g max x
V
x
:Pr
T T jz
Prfz g
2T
Prfx; y jz g Prfx; y jz g
(x)
y
max
T T jz
:Pr
(x)
max
2T
x
2T
y
>0
Pr (x)
Ts j
x ; y ; Q1V z
Prfz g
z
1
2T
y V
Prfz g max max
T1N
max
2T
x
2T
y
(x)
Prfx; y ; TQs1V jz g
(48)
>0
where the last inequality holds because the average is not larger than the maximum. The probability in (48) can be written as follows:
Prfz gPr = Pr = Pr
Ts j
x; y ; Q1V z
T 1 T 1
s QV s QV
jTQs1V s x; y jTQ1V
Pr Pr
x; y
Pr z jx ; y ; TQs1V Prfz jx; y g:
(49)
Since all (x; y ) 2 TQs1V have the joint type Q 1 V and occur with the same probability, we have that Pr x ; y jTQs1V = jT 1 j . Further-
more, we have Prfz jx; y g = 20s because any key u with K (x; y ) = s is independent of (x; y ). Hence, Pc can be bounded as follows: Pc
T1N 1
T :Pr T jz 1 0s TQs1V 1 2
T1N
max
2T
x
z
z
:Pr
T T jz
s QV
=
max
2T
x
z
:Pr
(x)
T T jz
>0
2N H (V jQV ) (N + 1)jXkYj
>0
1 Pr TQs1V 1 T s 1 20s Q1V 1
PrfTQs1V g
2NH (V jQV ) (N + 1)jXkYj
1
1 TN
2T
y
>0
1 Pr T 1 1 jT s j 1 20s Q1V
TQ \fy : K (y ) N (j 0 D(QkP )j+ 0 )g 2O(log N )0N jTQ j:
(45)
= jQjjVj(dN log jYje + 1)
Now we derive the lower bound of Pc defined by (43)
x
Hence, if the wiretapper guesses x optimally, the probability of correct guess can be represented as
(46)
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
=
TN
T
:Pr
1 Pr T =
1 TN
2NH (V jQV ) (N + 1)jXkYj
1
T
s Q1V
T
z
jz
1
>0
Since the above bound holds for any s > 0, we obtain that
jTQ1V \ f(x; y) : K (x; y) kgj
1 20s
1 Ts Q1V s z : Pr TQ1V jz
2813
>
0
=
2NH (V jQV )
1 (N + 1)jXkYj
1 Pr TQs1V 1 T s1 1 20s Q1V
TQs1V jz
: Pr =
z
>
1 20N [0D(Q1V kP 1W )0H (V jQ)+H (V jQV )]
(50)
g
: Pr
=
z
TQs1V jz
>
(51)
0
: z = f (x; y ; u) for some (x; y ) 2 TQs1V and u 2 f0; 1gs
TQs1V : 2NH (V jQ)
(52)
By substituting (52) into (50), we have that
Pc
T1N
2NH (V jQV ) 1 Pr (N + 1)jXkYj
T
TQs1V 1 2NH1(V jQ) 1 20s
2NH (V jQV )
T1N Tmax (N + 1)jXkYj 1 Pr TQs1V 1 2NH1(V jQ) 1 20s =
1 TN
max
T
2NH (V jQV ) 1 PrfTQ1V g 1 (N + 1)jXkYj
where the last equality comes from (47). Finally, (44) is obtained by substituting k = N [ 0 D(Q 1 V kP 1 W )0H (V jQ)+ H (V jQV ) 0 ] into (55). C. Asymptotically Optimal Code In this subsection, we show an optimal code which can attain 20N asymptotically with key length K (x; y )= N j 0 D(Q 1 k 1 W ) 0 H (V jQ) + H (V jQV )j+ . We encode the type Q of y and the conditional type V of x with l1 (x; y ) bits, and the index of y in TQ with l2 (x; y ) bits. From (66)–(68), it suffices to set l1 (x; y ) = d(jYj + jXjjYj) log(N + 1)e and l2 (x; y ) = dN H (Q)e. If Q and V satisfy for a given that jQV ), we use a key u with > D (Q 1 V kP 1 W ) + H (V jQ) 0 H (V length K (x; y ) = N [ 0 D(Q 1 V kP 1 W ) 0 H (V jQ) + H (V jQV )], and we randomize K (x; y ) bits in the second part of the codeword with length l2 (x; y ) by taking bitwise XOR operation with the key u . This operation is feasible because K (x; y ) l2 (x; y ) holds as shown in the following. Since a wiretapper can know the type QV of x from the first part of the codeword and the number of x with type QV is bounded by 2NH (QV ) , can be bounded by H (QV ) in the case of this coding scheme. Hence, K (x; y ) satisfies that
Pc V P
K (x; y )
TQs1V jTQ1V j
1 2NH1(V jQ) 1 20s
s NH (V jQV ) 0ND(Q1V kP 1W ) T1N Tmax (N2 + 1)jXkYj 1 2 (N + 1)jXkYj 1 jTTQQ11VV j
1 2NH1(V jQ) 1 20s
(53)
where the equality holds because every (x; y ) 2 TQ1V has the same probability. The last inequality comes from (73). Therefore, in order to attain Pc 20N , the following inequality must be satisfied for any Q, V , and s:
TQs1V TN (N + 1)2jXjjYjjTQ1V j 120N [0D(Q1V kP 1W )0H (V jQ)+H (V jQV )] 1 2s :
2s
Since y must be uniquely decodable from cryptogram z and key u , the size of the cryptogram must be larger than the number of distinct s NH (V jQ) y included in TQ1V when u is fixed. There exist at most 2 distinct x for each y 2 TQ from (69). Hence, the following must hold: z
s=0
= 2k0N [0D(Q1V kP 1W )0H (V jQ)+H (V jQV )]+O(log N ) 1 jTQ1V j (55)
: z = f (x ; y ; u ) for some (x; y ) 2 TQs1V and u 2 f
bkc
TN (N + 1)2jXjjYjjTQ1V j 1 20N [0D(Q1V kP 1W )0H (V jQ)+H (V jQV )] 1 2k+1
0
0; 1 s :
s=0
TQs1V
TN (N + 1)2jXjjYjjTQ1V j
where the second inequality holds from (70). The set z : Pr TQs1V jz > 0 can be represented as z
bkc
(54)
= N [ 0 D(Q 1 V kP 1 W ) 0 H (V jQ) + H (V jQV )] N [H (QV ) 0 D(Q 1 V kP 1 W ) 0 H (V jQ)+ H (V jQV )] = N [H (Q 1 V ) 0 D(Q 1 V kP 1 W ) 0 H (V jQ)] = N [H (Q) 0 D(Q 1 V kP 1 W )] (56) N H (Q ):
In the case of D(Q 1 V kP 1 W ) + H (V jQ) 0 H (V jQV ), we do not randomize any part of the codeword. In the above coding scheme, s = K (x; y ) depends on only Q and s s V . Hence, TQ1V defined in (46) satisfies that TQ1V = TQ1V for s = N j 0 D (Q 1 V kP 1 W ) 0 H (V jQ) + H (V jQV )j+ . On the other hand, every inequality in (48), (50), (52), (53) is exponentially tight within 2O(log N ) . Hence, we have from (53) that Pc
1 2O(log N ) max 2NH (V jQV ) 1 20ND(Q1V kP 1W ) 1 NH (V jQ) 1 20s Q;V 2
= max 2O(log N )0N [D(Q1V kP 1W )+H (V jQ)0H (V jQV )]0s :
Q;V
1 k 1 1 k 1 0
j 0
j
(57)
If > D(Q V P W ) + H (V Q) H (V QV ), we have s = QV )]. Otherwise, we have N [ D (Q V P W ) H (V Q)+ H (V s = 0. Hence, in the both cases, Pc satisfies that Pc 2O(log N )0N .
0
j
j
2814
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Remark 6: In the same way as the case of one source output shown in (2), we can attain perfect secrecy in the security criterion of Pc if = 0, where 0 is defined as
0 0 log max P W (x): x
(58)
Then, the necessary key length to attain the perfect secrecy is given by
N [0 0 D(Q 1 V kP 1 W ) 0 H (V jQ) + H (V jQV )] for (x; y ) with type Q 1 V . APPENDIX
jVj (N + 1)jXjjYj 2NH (Q) jT j 2NH (Q) Q (N + 1)jYj
(67) (68)
2NH (V jQ) jT (y )j 2NH (V jQ) V (N + 1)jXjYj 2NH (V jQV ) jT (x)j 2NH (V jQV ) V (N + 1)jXjYj 2N [H (Q)0I (Q;V )] (N + 1)jXjjYj jfy 2 TQ : x 2 TV (y )gj 2N [H (Q)0I (Q;V )] ;
A. Notations For a probability distribution Q(y ) and a conditional probability distribution V (xjy ), we represent the joint and marginal probability distributions by Q 1 V (y; x) Q(y )V (xjy ) and QV (x) y Q(y)V (xjy), respectively. Then, the entropies H (Q), H (QV ), the conditional entropy H (V jQ), and the mutual information I (Q; V ) of these probability distributions are defined as follows:
H (Q ) 0 H (QV ) 0 H ( V jQ ) 0
y x y;x
Q(y) log Q(y)
(59)
QV (x) log QV (x)
(60)
Q 1 V (y; x) log V (xjy)
(61)
I (Q; V ) H (QV ) 0 H (V jQ)
=k
y;x
Q 1 V (y; x) log
Q 1 V (y; x) : Q(y)QV (x)
(62)
The base of logarithm is 2 in this correspondence. The reverse conditional probability of V (xjy ) is given by V (y jx)= Q 1 V (y; x)=QV (x) and the corresponding conditional entropy H (V jQV ) is defined as
H (V jQV ) 0
y;x
Q 1 V (y; x) log V (yjx):
(63)
For two probability distributions Q(y ) and P (y ) and two conditional probability distributions V (xjy ) and W (xjy ), relative entropies D(QkP ) and D(Q 1 V kP 1 W ) are defined as follows:
D (Q kP ) D (Q 1 V kP 1 W )
y
y;x
Q(y) log
Q (y ) P (y )
Q 1 V (y; x) log
Q 1 V (y; x) : P 1 W (y; x)
(65)
(70)
for x 2 TQV
(71)
for x 62 TQV
(72)
where j1j represents the cardinality of a set. Furthermore, if Q 1 V (y; x) is the type of output sequences (x; y ) of an i.i.d. source with a probability distribution P 1 W (y; x), it holds that
20ND(Q1V kP 1W ) (N + 1)jXjjYj
PrfTQ1V g 20ND(Q1V kP 1W ):
(73)
B. Proof of Lemma 1 If I (Q; V^ ) r , (26) holds obviously. Hence, in the following, we consider the case that I (Q; V^ ) > r . Let CM = fy 1 ; y 2 ; . . . ; y M g, where every y i has the type Q and the size of CM satisfies that
2N (r0) M 2
2N (r0=2) :
(74)
We first note that if any y i in CM satisfies (26) for any V and V^ , it must hold that y i 6= y j for i 6= j . This holds from the fact that if y i = y j for some i 6= j , we have from (26) that for V = V^
jTV^ (y j )j20N (I (Q;V^ )0r) TV^ (y i )
6
k=i
= jTV^ (y j )j
(64)
For y y1 y2 . . . yN 2 Y N , x x1 x2 . . . xN 2 X N , a 2 Y , and b 2 X , let N (ajy ) and N (a; bjx ; y ) be the number of occurrences of a in y and the number of occurrences of (a; b) in f(xi ; yi )giN=1 , respectively. Then, the type Q of y is defined by Q(a) N (ajy )=N , and x has a conditional type V for y if N (a; bjx ; y ) = N (ajy )V (bja). We define Q, V , TQ , TQV , TQ1V , TV (y ), and TV (x) as follows. Q is the set of all distinctN typesNover Y N , V is the set of all distinct conditional types over X 2 Y , TQ is the set of all y with type Q, TQV is the set of all x with type QV , TQ1V is the set of all (x; y ) with joint type Q 1 V , TV (y ) is the V -shell of y , i.e., the set of all x that have the conditional type V for y , and TV (x) is the V -shell of x . Then the following lemma holds from [6, Ch. 1, Lemmas 2.2,2.3,2.5,2.6].
TV^ (y k ) (75)
which contradicts I (Q; V^ ) > r . From (69), the following inequality holds for any y j and V :
1 NH (V jQ) jTV (y j )j (N + 1) : jXjjYj 2
(76)
Hence, representing the left-hand side of (26) by Li (CM ; V; V^ ) for simplicity, (26) holds if the following inequality is satisfied:
Li (CM ; V; V^ ) (N + 1)0jXjjYj 20N (0H (V jQ)+I (Q;V )0r) : (77) ^
Furthermore, (77) is satisfied if it holds that
Li (CM ) (N + 1)jXjjYj
V;V^ 2V
Li (CM ; V; V^ )
1 2N (0H (V jQ)+I (Q;V^ )0r)
Lemma 2:
jQj (N + 1)jYj
jfy 2 TQ : x 2 TV (y )gj = 0;
(69)
(66)
1:
(78)
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
We show by a random coding technique that there exists CM satisfying (78). Selecting each Y i in CM independently over A" with the uniform probability distribution, the expectation of Li (CM ; V; V^ ) is bounded as follows:
C
2815
Now we evaluate the expectation of
^ )] E[Li ( M ; V; V = E[
M =
jTV (Y i )\( TV (Y j ))j] ^
6
j =i
=
fx 2 TV (Y i )\( TV (Y j ))g
Pr
2X
x
=
^
6
j =i
1
2X j =6 i
x
0 1)
2
= (M
0 1)
3
= (M
Pr
"
1
2
^
2
(79)
2
(N + 1)
Pr
(N + 1)
(80)
1
jfy 2 A" : x 2 TV (y)gj jfy 2 TQ : x 2 TV (y)gj 2N H Q 0I Q;V : )
(
))
(81)
On the other hand, we have from (25) and (68) that
jA" j (N +"1)jYj 2NH Q : Hence, it holds from (80)–(82) that for any x 2 TQV (
)
(82)
fx 2 TV (Y )g 1" (N + 1)jYj 20NI Q;V : (83) If x 62 TQV , we have Prfx 2 TV (Y )g = 0 from (72). By substituting Pr
(
1
2V
0N=2 (86)
)
M i=1
L i (C M )
1 2
:
(87)
Furthermore, there exist CM=2 CM such that every y i in CM=2 satisfies Li (CM ) 1. The reason comes from the fact that if it does not hold, we have 1
Since A" is a subset of TQ , the numerator of the right-hand side in (80) can be bounded from (71) as follows:
(
jXkYj+2jYj 20N=2 :
2
3
M
: x 2 TV (y )gj fx 2 TV (Y )g = jfy 2 A" jA : "j
(
V;V^
1
Since Y 1 is uniformly distributed over A" , we have for any fixed x 2
X N that
C
^ )] E[Li ( M ; V; V
jXkYj+2jYj
i=1
Y 1 and Y 2 are selected independently:
= :
2V
^ )) +I (Q;V
M
the probability does not depend on i and j;
3
)
V;V^
Since the right-hand side in (86) tends to zero as N goes to infinity, there exists CM for sufficiently large N that satisfies
the union bound;
:
2
= :
jXjjYj
i=1
2
2
^
where the numbered equalities and inequality hold because 1
(N + 1)
fx 2 TV (Y )gPrfx 2 TV (Y )g
Pr
2X
x
1
M
" 2M
1
C
i=1 M
fx 2 TV (Y ) \ TV (Y )g
Pr
L i (C M ) .
E[Li ( M )]
M
(
^
2X
M
1
M i=1
L i (C M )
i=1
1 2N (0H V jQ 0r
fx 2 TV (Y i ) \ TV (Y j )g
x
M
1
E
1
M
M
M
Li (CM ) > i=1
1
M
2 M2 2 1 = 21
which contradicts (87). Since the size of CM=2 is not less than 2N (r0) from (74), Lemma 1 holds. Remark 7: In the case of " = 1, Lemma 1 coincides with the usual Packing Lemma [6, Ch. 3, Lemma 5.1]. But, since Lemma 1 holds for any small " > 0, we can select 2N (r0) sequences y i in A" even if A" is much smaller than TQ . C. Proof of Theorem 2
We difine F + , Ff+ , Fg0 , and F 0 as follows:
1
these results into (79), the following bound is obtained:
C
^ )] E[Li ( M ; V; V
(M 0 1)
(M 0 1) = (M
Pr
2X
x
fx 2 TV (Y )gPrfx 2 TV (Y )g 1
2T
x
0 1)jTQV j
F0 (84)
Furthermore, applying (68), (74), and I (Q; V ) = H (QV ) 0 H (V jQ) to (84), E[Li (CM ; V; V^ )] is finally bounded as follows:
C
^ )] E[Li ( M ; V; V
2 1 2N r0= 2NH QV (N +"1) 1 20N H QV 0H V jQ I Q;V (
2)
(
)
=
(
2(N + 1)
"2
)
jYj
2
(
)+ (
jYj
2
^ ))
N H (V jQ)+r0 2 (
0=2) :
I (Q;V^ )
N N
!1
!1
1 lim inf inf 0 log Pr N !1 f N
NL GN f;g (X jZ ) 2
NL GN f;g (X jZ ) 2 NL GN f;g (X jZ ) 2
1 lim inf inf sup 0 log Pr N !1 f N g
NL GN f;g (X jZ ) 2
(88) (89) (90)
: (91)
Since it holds obviously from the above definition that Fg0 F 0 F 3 F + Ff+ , it suffices to show that there exists an encoder f and a strategy g that can attain Ff+ F (R; L) Fg0 .
Proof of F (R; L) Fg0 : We first show that the strategy g defined in the part (Proof of Cg+ C (R; )) of Section II-C satisfies F (R; L) Fg0 . From (22), we have that
2
(
lim sup sup 0 N1 log Pr g
jYj 0N (I (Q;V )+I (Q;V^ )) 2 :
2
2
"2
Ff+ Fg0
2
(N + 1)
2
1 lim sup inff sup 0 log Pr N g
jYj 0N (I (Q;V )+I (Q;V^ )) 2
(N + 1)
"
^
F+
jXjjYj 1 2N minfH (QV );R+H (QV )0I (Q;V )g jXjjYj 1 2Nh(Q;V;R) : = 2(N + 1) (92)
GN f;g (xjz ) 2(N (85)
+ 1)
2816
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Pr
NL GN j(X ; Y ) 2 TQ1V ; Z f;g (X jZ ) 2
=
z
0;
jM j0(
G
Pr
Pr
2(N + 1)
= Pr
2
h(Q; V; R) +
= Pr
2 Q;V h(Q;V;R)+O (log N )=N
(N + 1)jYj jXkYj 1 max = (N + 1)
jYj+jXkYj
1 exp 0N
L
0N D(Q1V kP 1W )
Q;V
h(Q;V;R)+O (log N )=N
L
Q;V
h(Q;V;R)+O (log N )=N
L
= lim inf inf N
!1
f
D(Q 1 V kP 1 W )
lim!1 inf inf N
f
F (R; L):
(93)
0 jM2(Q; V )j
O(log N ) : (94) N
Pr
min Q;V
L
z
:
(98)
fT 1 gPr
Pr
=
Pr
fT 1 g Q V
Q;V
jM(Q;V )j2
1 PrfG
X jZ ) 2N L j(X ; Y ) 2 TQ1V g
N f;g (
1
Pr
fT 1 g(1 0 ") Q V
Q;V
2
NL
1
0 jM2(Q; V )j NL
fT 1 g(1 0 ")
1
0 jM2(Q; V )j
fT 1 g(1 0 ")
1
0 2 21 2
Pr
Q V
Q;V
jM(Q;V )j212
N) 0 O(log N
Pr
Q V
Q;V
(95)
(99)
NL GN j(X ; Y ) 2 TQ1V f;g (X jZ ) 2
Q V
jM(Q;V )j2
D (Q 1 V kP 1 W )
= 0:
NL GN f;g (X jZ ) 2
NL GN f;g (X jZ ) 2
h(Q;V;R)+O (log N )=N
=
Hence, averaging all types Q and all conditional types V , we have the following bound:
Q;V
Proof of Ff F (R; L): Next we show that the encoder f defined in the part (Proof of C (R; ) Cf0 ) of Section II-C satisfies Ff+ F (R; L). When a wiretapper gets a cryptogram z encoded by f , he can know the fact that (x; y ) is included in M(Q; V ), which is defined by (31). Since (28) holds, we consider only the case of j 6= 0, i.e., y 62 TQ0 . If (x; y ) is included in M1 defined by (32), then x is paired with two or more y . But, if (x; y ) is included in M2 defined by (33), then x is paired with only one y . Since (29) holds for any (x; y ) 2 M(Q; V ), all x 2 M1 must be guessed before any x 2 M2 in the best strategy for this encoder f . This means that the best strategy, and hence any strategy, satisfies (96) at the top of the page. Therefore, for any Q +
1
NL GN j(X ; Y ) 2 TQ1V f;g (X jZ ) 2
Pr
=
D (Q 1 V kP 1 W ) 0
0 N1 log Pr
NL
On the other hand, it holds for any Q and V satisfying jM(Q; V )j < NL 2 that
Since (94) holds for any encoder f , we obtain the following relation:
Fg0
Q V
NL GN j(X ; Y ) 2 TQ1V ; Z f;g (X jZ ) 2
(1 0 ")
NL GN f;g (X jZ ) 2
min
fZ = zj(X ; Y ) 2 T 1 g
1 Pr
where the third inequality holds from (73) and the last inequality holds from (66) and (67). By taking logarithm of (93), we have
0 N1 log Pr
Q V
Pr
6
z :j =0
0N D(Q1V kP 1W ) 2
min
2
NL
+
Q;V
(96)
:
NL GN j(X ; Y ) 2 TQ1V f;g (X jZ ) 2
Pr
L
h(Q;V;R)+O (log N )=N
NL
2
Q V
L
2
have from (27), (28), (97) that
L PrfT 1 g
Q;V
NL
O(log N ) N
h(Q;V;R)+O (log N )=N
1
NL
NL
jXkYj 1 2N h(Q;V;R) 2N L
2
2
NL
N f;g
NL
N h(Q;V;R)+O (log N )
+
1
jM(Q; V )j 2 , we have from the relation jM j + jM j = jM(Q; V )j that Pr G (X jZ ) 2 j( X ; Y ) 2 T 1 ; Z = z 1 0 jM2(Q; V )j : (97) Since the size of M(Q; V ), i.e., jM(Q; V )j, does not depend on z , we 1
X jZ ) 2
2
and V satisfying
Hence, the following relation holds: N f;g (
jM j + jM j < 2 = 1 0 jM j jM j ; jM j + jM j 2
0jM j) jM j+jM j 2
jM(Q;V )j2
2
3 Q;V
jM(Q;V )j2
NL NL
0N D(Q1V kP 1W ) (1 0 ") (N + 1)jXjjYj 2
2
0N D(Q1V kP 1W )0O(log N )
2
0N D(Q1V kP 1W )0O(log N )
Q;V
jM(Q;V )j2 max Q;V
jM(Q;V )j2
= exp2
0N
min Q;V
jM(Q;V )j2
D(Q 1 V kP 1 W ) 0 O(log N ) (100)
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Since (101) holds for any strategy g and any small > 0 if we use the encoder f defined in the part (Proof of C (R; ) Cf0 ) of Section II-C, we finally obtain that
where the numbered inequalities hold because
1
:
(98);
2
:
the range of summation is restricted from
3
:
jM(Q; V )j 2NL
to
jM(Q; V )j 2 1 2NL ;
+
(73).
0 N log Pr
jM
N Gf;g (X
jZ ) 2
h(Q;V;R) L
1 k 1
Q;V (Q;V ) 2
j
min
Q;V h(Q;V;R) 2 O(log N )=N L
0 0
= F (R; L):
O (log N ) N
1 k 1
D (Q V P W ) +
REFERENCES O (log N ) N
(101) where the last inequality holds because we have from (38) that
g j 0 jXkYj (N + 1) N [minfR;I (Q;V )g+H (V jQ)020O(log N )=N ] =2 N [h(Q;V;R)020O(log N )=N ] =2 : (102)
jM(Q; V )j 2
f
(103)
NL
D (Q V P W ) +
min
1 N NL 0 log Pr Gf;g (X jZ ) 2 N g N !1 min D (Q 1 V kP 1 W ) Q;V
Ff = lim sup sup
By taking logarithm of the above inequality, we obtain that 1
2817
N (min R;I (Q;V ) +H (V Q) 2)
[1] C. E. Shannon, “Communication theory of secrecy systems,” Bell Syst. Tech. J., vol. 28, no. 3, pp. 565–715, Oct. 1949. [2] H. Yamamoto, “Information theory in cryptology,” IEIEC Trans., vol. E74, no. 9, pp. 2456–2464, Sep. 1991. [3] N. Merhav and E. Arikan, “The Shannon cipher system with a guessing wiretapper,” IEEE Trans. Inf. Theory, vol. 45, no. 6, pp. 1860–1866, Sep. 1999. [4] N. Merhav, “A large-deviations notion of perfect secrecy,” IEEE Trans. Inf. Theory, vol. 49, no. 2, pp. 506–508, Feb. 2003. [5] H. Yamamoto, “Coding theorems for Shannon’s cipher system with correlated source outputs, and common information,” IEEE Trans. Inf. Theory, vol. 40, no. 1, pp. 85–95, Jan. 1994. [6] I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
2808
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Coding Theorems for the Shannon Cipher System With a Guessing Wiretapper and Correlated Source Outputs Yutaka Hayashi and Hirosuke Yamamoto, Senior Member, IEEE
the exponent of Pc , i.e., Y 0(log Pc )=N , in the case that the wiretapper gets the cryptogram Z of Y . Even when the wiretapper has no cryptogram Z , he can guess the correct plaintext with the probability given by
Pc3 Abstract—The security level of the Shannon cipher system is traditionally ( j ), where is a secret plaintext with measured by equivocation and is its cryptogram. But, Merhav and Arikan have considlength ered another security criterion, which is measured by the number of guesses needed for a wiretapper to uncover from . Merhav has also considered the third security criterion, which measured by the probability of correct guess of a wiretapper. On the other hand, in the case of the traditional security criterion, Yamamoto has treated a coding problem for correlated and such that only is secret against wiretappers source outputs and only must be transmitted to a legitimate receiver. In this correspondence, coding theorems are proved for the case that Yamamoto’s coding problem is applied to Merhav–Arikan’s security criterion or Merhav’s security criterion.
N
HYZ
Z
Y
X
Y
Y
Y
0Y where 0Y is defined by 0Y 0 log max P (y ); y
Index Terms—Coding theorem, correlated sources, guessing wiretapper, perfect secrecy, Shannon cipher system.
I. INTRODUCTION The security level of the Shannon cipher system [1] is traditionally measured by equivocation N1 H (Y jZ ), where Y = Y1 Y2 . . . YN and Z are a secret independent and identically distributed (i.i.d.) plaintext and its cryptogram, respectively, and N is the length of Y . It is known that if security level N1 H (Y jZ ) = (0 H (Y )) is required, the key rate R of the cipher system must satisfy R [2]. In the case of = H (Y ), perfect secrecy can be attained, but the key rate R must be as large as H (Y ). As a new security criterion instead of the equivocation, Merhav and Arikan [3] proposed to use the expected th moment of the number of guesses to uncover a plaintext Y from its cryptogram Z . In their security criterion, the cipher system becomes more secure as a wiretapper needs more guesses. They showed that the asymptotic optimal exponent of the expected th moment can be represented by a function CY (R; ). The function CY (R; ) is nondecreasing in R, and CY (R; ) is constant if R is larger than a certain value R3 . Hence, if R R3 , perfect secrecy can be attained in their security criterion. Merhav and Arikan also considered the probability of large deviation such that the number of guesses becomes larger than 2LN for a given L > 0, and they showed that the asymptotic optimal exponent of the probability can also be represented by another function FY (R; L). Furthermore, they proved that CY (R; ) and FY (R; L) are related by the Fenchel–Legendre transform and its inverse transform. As another security criterion, Merhav [4] proposed to evaluate the security of the Shannon cipher system by the exponent of the probability Pc that a wiretapper guesses a correct plaintext Y . Let Y be Manuscript received February 14, 2007; revised January 10, 2008. This work was supported in part by the JSPS and MEXT Grants-in-Aid for Scientific Research, 16656114, 17360174, 18656110. The material in this correspondence was presented in part at the IEEE International Symposium on Information Theory, Seattle, WA, July 2006. Y. Hayashi is with the Financial Systems Division, Nomura Research Institute, Ltd., Tower S Ridge, Kiba, Koto-ku, Tokyo, 135-0042, Japan (e-mail: [email protected]). H. Yamamoto is with the Department of Complexity Science and Engineering, School of Frontier Sciences, University of Tokyo, Kashiwanoha, Kashiwa-shi, Chiba, 277-8561, Japan (e-mail: [email protected]). Communicated by H. Imai, Guest Editor for Special Issue on Information Theoretic Security. Digital Object Identifier 10.1109/TIT.2008.921707
(1)
Hence, if a cipher attains Y
Z
X
= max P N (y ) = [max P (y )]N : y y
(2)
perfect secrecy can be attained in this security criterion because the cryptogram Z does not decrease the exponent of Pc . The security criteria proposed by Merhav and Arikan [3] or Merhav [4] are suited for the case that an adversary wiretaps an encrypted password or secret personal verification information in order to attack a bank account, a computer account, or a web site account via the Internet. Merhav–Arikan’s criteria can be used if the adversary can attack the account by trying log-in with guessed passwords repeatedly while Merhav’s criterion can be used if the adversary can try log-in only once or a few times. Secret information is often related to some other information. For instance, a password is apt to be determined based on name, address, phone number, birthday, and so forth. In this case, even if the secret information is encrypted securely, it might be possible for an adversary to get such relevant information. If the adversary attacks the secret information using the obtained relevant information, the security level of the secret might decrease significantly. Hence, when we send the relevant information even if it is not secret, we must encrypt it carefully to prevent adversaries from attacking its related secret information. This situation was treated by Yamamoto [5] under the security criterion of equivocation. More precisely, he considered a coding problem for the case such that a nonsecret Y must be transmitted to a legitimate receiver with sufficiently small error probability and a secret X correlated to Y must be protected against wiretappers. It is shown in [5] that key rate R must satisfy R j 0 H (X jY )j+ to achieve N1 H (X jZ ) = , where jtj+ maxft; 0g, and, hence, the perfect secrecy of X can be attained with R = I (X ; Y ) for = H (X ). In this correspondence, we treat the same communication system model as Yamamoto’s model with two correlated source outputs, a secret X and a nonsecret Y . But the security of the system is measured by Merhav and Arikan’s security criterion and Merhav’s security criterion. Note that the coding theorems of this case cannot be derived by combining simply the Merhav–Arikan’s results [3] (or Merhav’s results [4]) with Yamamoto’s results [5]. Although the communication system model is the same as Yamamoto’s system, the proof technique used in [5] cannot be applied to this problem because different security criteria are treated. In this correspondence, the coding theorems will be proved by using the so-called Type Theory developed by Csiszár and Körer [6] in the same way as [3] and [4]. But, the coding theorems cannot be estimated from the results shown in these papers, which treated the case of a single source output, because the roles of X and Y are asymmetric and are cumbersomely intertwined. The case of Merhav-Arikan’s security criterion is treated in Section II. The system model is described in Section II-A, and coding theorems are shown in Section II-B and are proved in Section II-C and the Appendix. We note that in order to prove the direct parts of the coding theorems, a new coding scheme is devised based on an extended packing lemma (Lemma 1) and multiple packing by all distinct sequences (Algorithm 1) in Section II-C. The case of Merhav’s security criterion is treated in Section III. The system model and a coding theorem are described in Section III-A, and
0018-9448/$25.00 © 2008 IEEE
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
2809
F3
Fig. 1. Shannon cipher system with correlated source outputs
X and Y .
A. System Model We assume that a source generates discrete memoryless random variables X and Y , which are mutually correlated. X is a secret and Y must be transmitted to a legitimate receiver without error.1 X and Y take values from finite alphabets X and Y , respectively. Y has a probability distribution P (y ) for y 2 Y and X has a conditional probability distribution W (xjy ) for x 2 X and y 2 Y . We represent the joint probability of (X; Y ) and the marginal probability of X by P 1 W (y; x) P (y )W (xjy ) and P W (x) y P (y )W (xjy ), respectively. In the Shannon cipher system shown in Fig. 1, an encoder f generates a cryptogram Z from source outputs X , Y , and key U , where X = (X1 ; X2 ; . . . ; XN ), Y = (Y1 ; Y2 ; . . . ; YN ), and U = (U1 ; U2 ; . . . ; UK ). Uj ’s are true binary random numbers. The key rate R of this system is defined by K R= : (3) N The decoder must reproduce Y from the cryptogram Z and the key U without error. On the other hand, a wiretapper guesses the secret X from Z without U . We assume that the wiretapper knows the encoding function f and has a strategy g = fx1 (z ); x2 (z ); x3 (z ); . . .g to guess the correct secret x as fast as possible. Let GN f;g (X jZ ) be the number of guesses needed for the wiretapper to uncover the secret X by the strategy g . Remark 1: In the case of X = Y , the above system model is reduced ~ to Merhav–Arikan’s system model. Furthermore, in the case of X = X ~ Y ~ ) where X ~ and Y ~ are correlated random variables, the and Y = (X; above model corresponds to the case that the decoder reproduces both ~ and Y ~ and the wiretapper guesses only X ~ . Similarly, in the case of X ~ ~ ~ X = (X; Y ) and Y = Y , it corresponds to the case that the decoder ~ and Y ~. reproduces only Y~ and the wiretapper guesses both X B. Coding Theorems For the system model shown in the previous subsection, we consider the following two security criteria: GN f;g (X jZ )
C (R; )
0 D(Q 1 V kP 1 W )] (6) h(Q; V; R) minfH (QV ); R + H (QV ) 0 I (Q; V )g: (7) Theorem 2: For any R 0 and > 0, it holds that F 3 = F (R; ),
where
In this section, we consider a coding problem for the case that the security criterion based on the number of wiretapper’s guesses, which was treated by Merhav–Arikan [3], is applied to the Shannon cipher system with correlated source outputs (X; Y ), which was treated for the security criterion of equivocation by Yamamoto [5].
f
(5)
Theorem 1: For any R 0 and > 0, it holds that C 3 = C (R; ), where
II. SECURITY CRITERION BASED ON THE NUMBER OF GUESSES
1 N!1 lim sup inf log E g N
NL GN f;g (X jZ ) 2
where E[A] represents the expectation of A, L > 0 is a parameter, and GN f;g (X jZ ) is the th moment of the number of guesses. Then, for X and Y with probability distributions P (x) and W (xjy ), the above C 3 and F 3 are determined by the following theorems.
the theorem is proved in Section III-B. An asymptotically optimal code in this security criterion is given in Section III-C. The notations used in this correspondence are explained in the Appendix.
C3
1 N!1 lim inf sup 0 log Pr f g N
(4)
1In [5], a sufficiently small decoding error probability is allowed because a fixed length code is used. But, in this correspondence, the zero decoding error probability can be attained because a variable length code is used.
max[h(Q; V; R) Q;V
F (R; L)
min Q;V h(Q;V;R)L
D (Q 1 V kP 1 W ):
(8)
In the above theorems, Q(y ) and V (xjy ) are arbitrary probability distributions, and Q 1 V (y; x) and QV (x) are the joint and marginal probability distributions, respectively, which are determined from Q and V . H (QV ), I (Q; V ), and D(Q 1 V kP 1 W ) are the entropy, mutual information, and relative entropy, respectively, of these probability distributions (See the Appendix ). Furthermore, C (R; ) and F (R; L) are related by the Fenchel–Legendre transform as shown in the next theorem.
0 and > 0, it holds that C (R; ) = sup[L 0 F (R; L)] L> F (R; L) = sup [L 0 C (R; )]: >
Theorem 3: For any R
0
0
(9) (10)
Remark 2: In the case of Y = X , Theorems 1–3 coincides with Merhav–Arikan’s results, which can be derived by restricting W and V as W (xjx) = V (xjx) = 1 for X = Y , as follows:
CY (R; ) = C (R; )jX =Y = max[hY (Q; R) 0 D (QkP )] Q
FY (R; ) = F (R; )jX =Y = min D (Q kP )
hY (Q; R)
h
Q
(11)
(12)
(Q;R)L
max
fH (Q ); R g:
(13)
Remark 3: C (R; ) defined by (6) can be represented as follows:
C (R; ) = max[H (QV ) 0 jI (Q; V ) 0 Rj+ 0 D(Q 1 V kP 1 W )]: Q;V (14) ^ V ^ ) where Q ^ and V ^ are the Hence, C (R; ) is constant for R I (Q; probability distributions that attain the following maximum: max[H (QV ) Q;V
0 D(Q 1 V kP 1 W )]:
(15)
^ V ^) This means that perfect secrecy can be attained with R = I (Q; in the sense of security criterion C (R; ) because C (R; ) cannot be ^ V ^ ). This result is similar to the fact that perfect increased for R I (Q; secrecy can be attained with R = I (X ; Y ) = I (P; W ) in the sense of ^ V ^ ) is not equal the equivocation security criterion [5]. However, I (Q; to I (P; W ) generally.
We give the proof of Theorem 1 in Section II-C. The proof of Theorem 2 can be proved by combining the techniques shown in Section II-C and [3, the proof of Theorem 2], and it is given in the Appendix. The proof of Theorem 3 is omitted because it can be proved in the same way as [3, Theorem 3].
2810
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
By averaging the th moment of GN f;g (xjz ) for all types Q and all conditional types V , we have from (73) that
C. Proof of Theorem 1 We define C + , C 0 as follows: C
+
lim sup sup infg N1 log E
N !1 f 1 0 C lim inf sup inf g N N !1 f
j
N
j
N
log E Gf;g (X Z )
:
(16)
E Gf;g (X Z )
(17)
Q;V
Cg
lim sup sup N 1
N !1 f 1 0 Cf lim inf inf N !1 g N
N log E Gf;g (X
jZ )
j
N
log E Gf;g (X Z )
j
C (R; )
H (V jQ)
jTV
(y i )
j
2
j
)
H (V jQ) NR jXkYj 2 1 (N + 1) 1 2NH (V jQ) N (R+H (QV )0I (Q;V )) =2 1 (N + 1)jXjjYj (21) 0 where TV (y i ) is the V -shell of x for y i , i.e., the set of all x ’s with the conditional typle V 0 for y , and the inequalities holds from (67) and y ;V :H (V Q
)
i
(69). x 11 ; x 12 ; x 13 ; . . . and strategy g2 For strategy g1 = x 21 ; x22 ; x 23 ; . . . , we define a new strategy g as
=
1 2 1 2 1 2 g = x1 ; x1 ; x2 ; x2 ; x3 ; x3 ; . . . :
Then, the number of guesses in the strategy g is not more than twice the smaller number of guesses in g1 and g2 . Hence, we have N
j
Gf;g (x z )
2(N + 1)jXjjYj 1 min(2NH QV ; 2N R H QV 0I Q;V jXjjYj 1 2N fH QV ;R H QV 0I Q;V g : = 2(N + 1) (
min
(
)
)
( +
+
(
(
)
)
(
(23)
N
!1
f
1
N
N
j
log E Gf;g (X Z )
C (R; ):
(24)
Proof of C (R; ) Cf0 : We construct an encoding function f based on the following lemma. Lemma 1 (Extended Packing Lemma): For any given r > 0, > 0, and 0 < " 1, let Q be an arbitrary type of Y such that H (Q) > r , and let A" be an arbitrary subset of TQ that satisfies (25). Then, for sufficiently large N , there exist distinct y i , 1 i 2N (r0) , in A" that satisfy (26) for every pair of conditional types V and V^ .
6
j =i
jA" j "jTQ j TV (y j ) jTV (y i )j20N jI Q;V 0rj ^
(
^)
(25) : (26)
(20)
j
NH (V Q
jXkYj+jXj 2NC (R;) :
Cg = lim sup sup
TV (y i )
NH (Q )
1 (N + 1)jXj
)
0D(Q1V kP 1W )]
) 0 +H (QV )
2
(19)
where TQ is the set of all x with type QX , and the inequalities holds by applying (66) and (68) to the case of QX . Strategy g2 : by using The wiretapper first decodes all possible y 1 ; y 2 ; . . . ; y 2 all keys u 1 ; u 2 ; . . . ; u2 , and guess x in ascending order of conditional entropy H (V 0 jQy ) where Qy is the type of y i and V 0 is the conditional type of x for y i . Then, the number of guesses is bounded by y ;V :H (V Q
(
Since (23) holds for any encoding function f , we finally obtain :
Q :H (Q ) H (QV ) NH (QV )
2
+
[ min
(18)
Proof of Cg+ C (R; ): Assume that source outputs are (x ; y ) such that the type of y is Q and the conditional type of x for y is V . Then, the type of x is QV . We now consider the following two strategies g1 and g2 . Strategy g1 : The wiretapper ignores the cryptogram z and guess x in ascending order of entropy H (QX ) where QX is the type of x . In this strategy, the number of guesses is bounded by 2
jz )
f;g (x
2(N + 1) jXkYj jXj fR0I Q;V ; g 1 2N
+
f
0ND(Q1V kP 1W ) GN
= 2(N + 1)
if there exist an encoder f and a strategy g such that Cg+ Cf0 , we can conclude that C 3 = C (R; ).
Q :H (Q ) H (QV )
2
max
Then, it holds obviously that C 0 C 0 C 3 C + Cg+ . Hence,
jTQ j
2
Furthermore, for given f and g , define Cg+ , Cf0 as follows: +
j
N
Gf;g (X Z )
(
)
))
)
(22)
This lemma can be proved in the same way as [6, Ch. 2, Lemma 5.1], and the proof is given in the Appendix. For the case of V^ = V , we apply Lemma 1 repeatedly as follows. Algorithm 1 (Multiple Packing by All Distinct Sequences) Step 1: Let M = 2N (I (Q;V )02) and j = 1. Apply Lemma 1 to A" = TQ . For obtained y i , 1 i M , let TQ1 = fy 1 ; y 2 ; . . . ; y M g. Let A" = A" n TQ1 . Step 2: If jA" j < "jTQ j, then go to Step 4. Step 3: Let j = j + 1. Apply Lemma 1 to A" . For obtained y i , 1 j j i M , let TQ = fy 1 ; y 2 ; . . . ; y M g. Let A" = A" n TQ . Go to Step 2. Step 4: Let s = j and TQ0 = A" .
Then, it is obvious that the constructed TQj , 0 j s, satisfy the following conditions. (a) TQj , 0 j s are disjoint. (b) TQj = 2N (I (Q;V )02) for 1 j s and TQ0 < "jTQ j. (c) For 1 j s, every y 2TQj sasifties (26) with r = I (Q; V ) 0 . (d) s is bounded by s (1 0 ")2N (H (Q)0I (Q;V )+2) + 1.
Now we define an encoding function f based on TQj constructed in the above algorithm. Assume that y 2 TQj and x 2 TV (y ). Then we encode (x; y ) as follows. First we encode (Q; V ) with length l1 = dlog jQj + log jVje, where Q and V are the set of all distinct types on Y and the set of all distinct conditional types on X 2 Y . Next we encode j with length l2 = dlog(s + 1)e. Finally, we encode the index of y 2 TQj with length l3 , which is given by l3 = dN (I (Q; V ) 0 2 )e for 1 j s and l3 dN H (Q)e for s = 0. We encipher the third part of the codeword by key u with length lk = N (minfR; I (Q; V )g 0 2 ). The cryptogram z encoded from (x; y ) is the concatenation of the above three parts.
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Next, for the preceding encoding function f , we derive the lower bound of E GN f;g (X jZ ) . Since the second part with length l2 of the cryptogram z is not randomized by the key u, the wiretapper can know j from z . Furthermore, from the above property (b) for TQ0 , we have Prfj = 0j(x ; y ) 2 TQ1V g < ", and hence
f
j
Pr Z = z (x ; y )
6
z :j =0
Since it holds that
j
2 TQ1V g 1 0 ":
2811
Every x is distinct in M2 . Hence, the number of wiretapper’s guesses can be bounded from (29), (35), and (36) as follows:
(27)
1
z
j
6
z :j =0
1E
j
N
j
f j
y ) (x ;y
2T
2T
y ) (x ;y
=
2 TQ1V ; Z = z
jM(Q; V )j =
(29)
M(Q; V ) = f(x; y ) : (x; y ) 2 TQ1V \ Z 01 (z)g:
(30)
M f(x; y ) 2 M(Q; V ) : (x; y 0 ) 62 M(Q; V ) if y 0 6= y g: (33) From Lemma 1, the ratio of jM j to jM(Q; V )j tends to zero as N becomes large and " becomes small. Hence, M and M satisfy jM j = "0 jM(Q; V )j (34) 0 jM j = (1 0 " )jM(Q; V )j (35) 0 where " ! 0 as N ! 1 and " ! 0. 2
1
1
=
=
2
2M
f j
y) (x;y
+
2M
y) (x;y
2M
y) (x;y
f j
f j
Pr x; y (x ; y )
(
)
g j 0 jXjjYj (N + 1)
(38)
j
1+
12
f
g j 0 jXkYj (N + 1)
N (min R;I (Q;V ) +H (V Q) 2)
1+
f
j
(39)
Pr Q;V
fTQ1V gE 0
0" ) (1 0 ")(1 1+
j
N
j
Gf;g (X Z ) (X ; Y )
2 TQ1V
1+
0
1 k 1 0
N (h(Q;V;R) D(Q V P W ) 2)
] 1 maxQ;V [2 (N + 1)jXjjYj 0 (1 0 ")(1 0 " ) = 1 2N C R; 0 : 1+
(
(1 + )(N + 1)jXjjYj(1+)
(
)
2
)
(40)
Since (40) holds for any strategy g and any small > 0, we have that
0
N 2 TQ1V ; zgGf;g (xjz )
N 2 TQ1V ; zgGf;g (xjz ) :
0
1+
(1+ )
N 2 TQ1V ; zgGf;g (xjz )
Pr x; y (x ; y )
2 )
N (min H (QV );R+H (QV ) I (Q;V )
=
2 TQ1V ; Z = z N Prfx; y j(x; y ) 2 TQ1V ; z gGf;g (xjz )
Pr x; y (x ; y )
)
0 g02) jXkYj (N + 1) 0 1+ 2N (h(Q;V;R)02) (1 0 ")(1 0 " ) 1 (N + 1)jXkYj : 1+ 2
N
E Gf;g (X Z ) (X ; Y )
2M(Q;V )
(
2 TQ1V PrfZ = z j(x ; y ) 2 TQ1V g
E Gf;g (X Z )
2
y) (x;y
2 )
Finally, by averaging E GN f;g (X jZ ) j(X ; Y ) 2 TQ1V for all joint types Q 1 V , we obtain from (73) that
1
=
6
z :j =0
1
2
(32)
j
)
1+
M , which are defined as fol-
1
f
(
N (min R;I (Q;V ) +H (V Q) 2)
0" ) (1 0 ")(1 1+ 0 (1 0 ")(1 0 " ) =
(31)
M f(x; y ) 2 M(Q; V ) : 9(x; y 0 ) 2 M(Q; V ) for some y 0 6= y g
j
j
1+
Z 01 (z) = f(x; y ) : z 2 Z (x; y )g;
N
(min
1 jM(Q; V )j (1 0 "0 )
1
Obviously it holds that
2
=
1
and
N (min
2N
N
1
1
1
E Gf;g (X Z ) (X ; Y )
(z )
M(Q; V ) into M
2 TQ1V \ Z 0 (z) fR;I Q;V g0 1 jTV (y )j fR;I Q;V g0 1 2NH V jQ (N + 1)jXkYj
where the inequality holds from (69). Since (37) and (38) do not depend on z , E GN f;g (X jZ ) j(X ; Y ) 2 TQ1V can be bounded by combining (27), (28), (37), and (38) as follows:
jf(x; y)j(x; y) 2 TQ1V \ Z 0 (z)gj
We divide lows:
(37)
(x; y ) : (x; y )
=2
(28)
jTQ1V j0 20l jTQ1V j0 20l
1 jM(Q; V )j
where
jM j
From lk = N (minfR; I (Q; V )g 0 2 ), we have
2 TQ1V ; zg f j 2 TQ1V gPrfzjx; y g Prfx0 ; y 0 j(x 0 ; y 0 ) 2 TQ1V gPrfz jx0 ; y 0 g \Z
i=1
1+
1
=
1+
Pr x; y (x ; y )
=
i
0
we consider only the case of j 6= 0 in the following. Let Z (x; y ) be the set of all possible cryptograms for (x; y ). Then, for z 2 Z (x; y ), we have Prfz jx; y g = 20l . Furthermore, using Bayes’ theorem, we obtain that Pr x; y (x ; y )
2M jM j
jM(Q; V )j u du jM(Q; V )j (1 0 "0 ) : =
Gf;g (X Z ) (X ; Y )
2 TQ1V ; Z = z N Gf;g (xjz )
y) (x;y
1
jZ ) j(X ; Y ) 2 TQ1V ; Z = z PrfZ = z j(X ; Y ) 2 TQ1V g
N E Gf;g (X
j
1 jM(Q; V )j
E Gf;g (X Z ) (X ; Y ) =
1 jM(Q; V )j
2 TQ1V PrfZ = z j(X ; Y ) 2 TQ1V g
N
j
N
E Gf;g (X Z ) (X ; Y )
Cf = lim inf inf
(36)
N
!1
g
1
N
N
j
log E Gf;g (X Z )
C (R; ):
(41)
Remark 4: We note that Cg+ C (R; ) holds even in the case when the encoder f can know only Y . Furthermore, C (R; ) Cf0 can be
2812
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
attained even in the case when the encoder f can know Y and only the conditional type V of X for Y . But, we need both X and Y to know the conditional type V . III. SECURITY CRITERION BASED ON PROBABILITY OF CORRECT GUESS
THE
In this section, we consider a coding problem for the case that the security criterion based on the probability of wiretapper’s correct guess, which was treated by Merhav [3], is applied to the Shannon cipher system with correlated source outputs (X; Y ).
obviously. Hence, in the following, we consider only Q and V that satisfy 0 < 0 H (V jQ) + H (V jQV ) 0 D(Q 1 V kP 1 W ). Since only y must be transmitted to a legitimate receiver, we do not need consider any key length K (x; y ) longer than dN log jYje. For s s 2 f0; 1; . . . ; dN log jYjeg, we define TQ1V as follows:
TQs1V TQ1V \ f(x; y) : K (x; y) = sg:
Let TN be the number of distinct TQs1V . Then, from (66) and (67), TN is bounded by TN
A. System Model In the same way as Section II-A, a discrete memoryless source generates mutually correlated random variables X and Y with probability distribution P 1 W (y; x). Y must be transmitted to a legitimate receiver without error while X must be protected against a wiretapper. In Section II, we assumed that key length K is fixed and a wiretapper tries guessing repeatedly until he gets the correct secret. But, in this section, we assume that each key u has a variable length K (x; y ), i.e., u = (u1 ; u2 ; . . . ; uK (x;y y) ) and a wiretapper guesses x only once. Since the wiretapper wants to guess the correct x with the highest probability, the optimal guess of the wiretapper who gets a cryptogram z is given by ^ = arg max Prfx jz g: (42) x
Pc
= z
Prfz g max Prfxjz g: x
(43)
In this section, we consider a coding problem to attain Pc 20N for a given . The key length necessary to attain Pc 20N is determined by the next theorem. Theorem 4: If Pc 20N for a given > 0, then the following inequality holds for any > 0, any type Q, and any conditional type V :
TQ1V \ f(x; y ) : K (x; y ) N (j 0 H (V jQ) + H (V jQV ) 0 D (Q 1 V kP 1 W )j + 0 )g 2O(log N )0N jTQ1V j (44) where V is the reverse conditional probability distribution of V , which is defined by V Q 1 V (x; y )=QV (x). The left-hand side of (44) represents the number of pairs (x; y ) 2 TQ1V such that their key lengths are not longer than N (j 0 H (V jQ)+ jQV ) 0 D(Q 1 V kP 1 W )j+ 0 ), and (44) means that such number H (V decreases exponentially compared with jTQ1V j. Hence, in order to attain Pc 20N , almost of all (x; y ) 2 TQ1V must have a key length longer than N (j 0 H (V jQ) + H (V jQV ) 0 D(Q 1 V kP 1 W )j+ 0 ) when N is sufficiently large. Remark 5: In the case of Y = X , (44) coincides with Merhav’s results, which can be derived by restricting W and V as W (xjx) = (xjx) = 1 for X = Y , as follows: V ( x jx ) = V
Pc
= z
= z
z
= z
B. Proof of Theorem 4
If the type Q and conditional type V satisfies > 0 H (V jQ) + j 0 D(Q 1 V kP 1 W ), then we have K (x; y ) < 0 in (44). Since there are no pairs (x; y ) with negative key length, (44) holds
QV ) H (V
(N log jYj + 2)(N + 1)jXj+jXjjYj :
(47)
Prfz g max Prfxjz g x
Prfz g max x
V
x
:Pr
T T jz
Prfz g
2T
Prfx; y jz g Prfx; y jz g
(x)
y
max
T T jz
:Pr
(x)
max
2T
x
2T
y
>0
Pr (x)
Ts j
x ; y ; Q1V z
Prfz g
z
1
2T
y V
Prfz g max max
T1N
max
2T
x
2T
y
(x)
Prfx; y ; TQs1V jz g
(48)
>0
where the last inequality holds because the average is not larger than the maximum. The probability in (48) can be written as follows:
Prfz gPr = Pr = Pr
Ts j
x; y ; Q1V z
T 1 T 1
s QV s QV
jTQs1V s x; y jTQ1V
Pr Pr
x; y
Pr z jx ; y ; TQs1V Prfz jx; y g:
(49)
Since all (x; y ) 2 TQs1V have the joint type Q 1 V and occur with the same probability, we have that Pr x ; y jTQs1V = jT 1 j . Further-
more, we have Prfz jx; y g = 20s because any key u with K (x; y ) = s is independent of (x; y ). Hence, Pc can be bounded as follows: Pc
T1N 1
T :Pr T jz 1 0s TQs1V 1 2
T1N
max
2T
x
z
z
:Pr
T T jz
s QV
=
max
2T
x
z
:Pr
(x)
T T jz
>0
2N H (V jQV ) (N + 1)jXkYj
>0
1 Pr TQs1V 1 T s 1 20s Q1V 1
PrfTQs1V g
2NH (V jQV ) (N + 1)jXkYj
1
1 TN
2T
y
>0
1 Pr T 1 1 jT s j 1 20s Q1V
TQ \fy : K (y ) N (j 0 D(QkP )j+ 0 )g 2O(log N )0N jTQ j:
(45)
= jQjjVj(dN log jYje + 1)
Now we derive the lower bound of Pc defined by (43)
x
Hence, if the wiretapper guesses x optimally, the probability of correct guess can be represented as
(46)
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
=
TN
T
:Pr
1 Pr T =
1 TN
2NH (V jQV ) (N + 1)jXkYj
1
T
s Q1V
T
z
jz
1
>0
Since the above bound holds for any s > 0, we obtain that
jTQ1V \ f(x; y) : K (x; y) kgj
1 20s
1 Ts Q1V s z : Pr TQ1V jz
2813
>
0
=
2NH (V jQV )
1 (N + 1)jXkYj
1 Pr TQs1V 1 T s1 1 20s Q1V
TQs1V jz
: Pr =
z
>
1 20N [0D(Q1V kP 1W )0H (V jQ)+H (V jQV )]
(50)
g
: Pr
=
z
TQs1V jz
>
(51)
0
: z = f (x; y ; u) for some (x; y ) 2 TQs1V and u 2 f0; 1gs
TQs1V : 2NH (V jQ)
(52)
By substituting (52) into (50), we have that
Pc
T1N
2NH (V jQV ) 1 Pr (N + 1)jXkYj
T
TQs1V 1 2NH1(V jQ) 1 20s
2NH (V jQV )
T1N Tmax (N + 1)jXkYj 1 Pr TQs1V 1 2NH1(V jQ) 1 20s =
1 TN
max
T
2NH (V jQV ) 1 PrfTQ1V g 1 (N + 1)jXkYj
where the last equality comes from (47). Finally, (44) is obtained by substituting k = N [ 0 D(Q 1 V kP 1 W )0H (V jQ)+ H (V jQV ) 0 ] into (55). C. Asymptotically Optimal Code In this subsection, we show an optimal code which can attain 20N asymptotically with key length K (x; y )= N j 0 D(Q 1 k 1 W ) 0 H (V jQ) + H (V jQV )j+ . We encode the type Q of y and the conditional type V of x with l1 (x; y ) bits, and the index of y in TQ with l2 (x; y ) bits. From (66)–(68), it suffices to set l1 (x; y ) = d(jYj + jXjjYj) log(N + 1)e and l2 (x; y ) = dN H (Q)e. If Q and V satisfy for a given that jQV ), we use a key u with > D (Q 1 V kP 1 W ) + H (V jQ) 0 H (V length K (x; y ) = N [ 0 D(Q 1 V kP 1 W ) 0 H (V jQ) + H (V jQV )], and we randomize K (x; y ) bits in the second part of the codeword with length l2 (x; y ) by taking bitwise XOR operation with the key u . This operation is feasible because K (x; y ) l2 (x; y ) holds as shown in the following. Since a wiretapper can know the type QV of x from the first part of the codeword and the number of x with type QV is bounded by 2NH (QV ) , can be bounded by H (QV ) in the case of this coding scheme. Hence, K (x; y ) satisfies that
Pc V P
K (x; y )
TQs1V jTQ1V j
1 2NH1(V jQ) 1 20s
s NH (V jQV ) 0ND(Q1V kP 1W ) T1N Tmax (N2 + 1)jXkYj 1 2 (N + 1)jXkYj 1 jTTQQ11VV j
1 2NH1(V jQ) 1 20s
(53)
where the equality holds because every (x; y ) 2 TQ1V has the same probability. The last inequality comes from (73). Therefore, in order to attain Pc 20N , the following inequality must be satisfied for any Q, V , and s:
TQs1V TN (N + 1)2jXjjYjjTQ1V j 120N [0D(Q1V kP 1W )0H (V jQ)+H (V jQV )] 1 2s :
2s
Since y must be uniquely decodable from cryptogram z and key u , the size of the cryptogram must be larger than the number of distinct s NH (V jQ) y included in TQ1V when u is fixed. There exist at most 2 distinct x for each y 2 TQ from (69). Hence, the following must hold: z
s=0
= 2k0N [0D(Q1V kP 1W )0H (V jQ)+H (V jQV )]+O(log N ) 1 jTQ1V j (55)
: z = f (x ; y ; u ) for some (x; y ) 2 TQs1V and u 2 f
bkc
TN (N + 1)2jXjjYjjTQ1V j 1 20N [0D(Q1V kP 1W )0H (V jQ)+H (V jQV )] 1 2k+1
0
0; 1 s :
s=0
TQs1V
TN (N + 1)2jXjjYjjTQ1V j
where the second inequality holds from (70). The set z : Pr TQs1V jz > 0 can be represented as z
bkc
(54)
= N [ 0 D(Q 1 V kP 1 W ) 0 H (V jQ) + H (V jQV )] N [H (QV ) 0 D(Q 1 V kP 1 W ) 0 H (V jQ)+ H (V jQV )] = N [H (Q 1 V ) 0 D(Q 1 V kP 1 W ) 0 H (V jQ)] = N [H (Q) 0 D(Q 1 V kP 1 W )] (56) N H (Q ):
In the case of D(Q 1 V kP 1 W ) + H (V jQ) 0 H (V jQV ), we do not randomize any part of the codeword. In the above coding scheme, s = K (x; y ) depends on only Q and s s V . Hence, TQ1V defined in (46) satisfies that TQ1V = TQ1V for s = N j 0 D (Q 1 V kP 1 W ) 0 H (V jQ) + H (V jQV )j+ . On the other hand, every inequality in (48), (50), (52), (53) is exponentially tight within 2O(log N ) . Hence, we have from (53) that Pc
1 2O(log N ) max 2NH (V jQV ) 1 20ND(Q1V kP 1W ) 1 NH (V jQ) 1 20s Q;V 2
= max 2O(log N )0N [D(Q1V kP 1W )+H (V jQ)0H (V jQV )]0s :
Q;V
1 k 1 1 k 1 0
j 0
j
(57)
If > D(Q V P W ) + H (V Q) H (V QV ), we have s = QV )]. Otherwise, we have N [ D (Q V P W ) H (V Q)+ H (V s = 0. Hence, in the both cases, Pc satisfies that Pc 2O(log N )0N .
0
j
j
2814
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Remark 6: In the same way as the case of one source output shown in (2), we can attain perfect secrecy in the security criterion of Pc if = 0, where 0 is defined as
0 0 log max P W (x): x
(58)
Then, the necessary key length to attain the perfect secrecy is given by
N [0 0 D(Q 1 V kP 1 W ) 0 H (V jQ) + H (V jQV )] for (x; y ) with type Q 1 V . APPENDIX
jVj (N + 1)jXjjYj 2NH (Q) jT j 2NH (Q) Q (N + 1)jYj
(67) (68)
2NH (V jQ) jT (y )j 2NH (V jQ) V (N + 1)jXjYj 2NH (V jQV ) jT (x)j 2NH (V jQV ) V (N + 1)jXjYj 2N [H (Q)0I (Q;V )] (N + 1)jXjjYj jfy 2 TQ : x 2 TV (y )gj 2N [H (Q)0I (Q;V )] ;
A. Notations For a probability distribution Q(y ) and a conditional probability distribution V (xjy ), we represent the joint and marginal probability distributions by Q 1 V (y; x) Q(y )V (xjy ) and QV (x) y Q(y)V (xjy), respectively. Then, the entropies H (Q), H (QV ), the conditional entropy H (V jQ), and the mutual information I (Q; V ) of these probability distributions are defined as follows:
H (Q ) 0 H (QV ) 0 H ( V jQ ) 0
y x y;x
Q(y) log Q(y)
(59)
QV (x) log QV (x)
(60)
Q 1 V (y; x) log V (xjy)
(61)
I (Q; V ) H (QV ) 0 H (V jQ)
=k
y;x
Q 1 V (y; x) log
Q 1 V (y; x) : Q(y)QV (x)
(62)
The base of logarithm is 2 in this correspondence. The reverse conditional probability of V (xjy ) is given by V (y jx)= Q 1 V (y; x)=QV (x) and the corresponding conditional entropy H (V jQV ) is defined as
H (V jQV ) 0
y;x
Q 1 V (y; x) log V (yjx):
(63)
For two probability distributions Q(y ) and P (y ) and two conditional probability distributions V (xjy ) and W (xjy ), relative entropies D(QkP ) and D(Q 1 V kP 1 W ) are defined as follows:
D (Q kP ) D (Q 1 V kP 1 W )
y
y;x
Q(y) log
Q (y ) P (y )
Q 1 V (y; x) log
Q 1 V (y; x) : P 1 W (y; x)
(65)
(70)
for x 2 TQV
(71)
for x 62 TQV
(72)
where j1j represents the cardinality of a set. Furthermore, if Q 1 V (y; x) is the type of output sequences (x; y ) of an i.i.d. source with a probability distribution P 1 W (y; x), it holds that
20ND(Q1V kP 1W ) (N + 1)jXjjYj
PrfTQ1V g 20ND(Q1V kP 1W ):
(73)
B. Proof of Lemma 1 If I (Q; V^ ) r , (26) holds obviously. Hence, in the following, we consider the case that I (Q; V^ ) > r . Let CM = fy 1 ; y 2 ; . . . ; y M g, where every y i has the type Q and the size of CM satisfies that
2N (r0) M 2
2N (r0=2) :
(74)
We first note that if any y i in CM satisfies (26) for any V and V^ , it must hold that y i 6= y j for i 6= j . This holds from the fact that if y i = y j for some i 6= j , we have from (26) that for V = V^
jTV^ (y j )j20N (I (Q;V^ )0r) TV^ (y i )
6
k=i
= jTV^ (y j )j
(64)
For y y1 y2 . . . yN 2 Y N , x x1 x2 . . . xN 2 X N , a 2 Y , and b 2 X , let N (ajy ) and N (a; bjx ; y ) be the number of occurrences of a in y and the number of occurrences of (a; b) in f(xi ; yi )giN=1 , respectively. Then, the type Q of y is defined by Q(a) N (ajy )=N , and x has a conditional type V for y if N (a; bjx ; y ) = N (ajy )V (bja). We define Q, V , TQ , TQV , TQ1V , TV (y ), and TV (x) as follows. Q is the set of all distinctN typesNover Y N , V is the set of all distinct conditional types over X 2 Y , TQ is the set of all y with type Q, TQV is the set of all x with type QV , TQ1V is the set of all (x; y ) with joint type Q 1 V , TV (y ) is the V -shell of y , i.e., the set of all x that have the conditional type V for y , and TV (x) is the V -shell of x . Then the following lemma holds from [6, Ch. 1, Lemmas 2.2,2.3,2.5,2.6].
TV^ (y k ) (75)
which contradicts I (Q; V^ ) > r . From (69), the following inequality holds for any y j and V :
1 NH (V jQ) jTV (y j )j (N + 1) : jXjjYj 2
(76)
Hence, representing the left-hand side of (26) by Li (CM ; V; V^ ) for simplicity, (26) holds if the following inequality is satisfied:
Li (CM ; V; V^ ) (N + 1)0jXjjYj 20N (0H (V jQ)+I (Q;V )0r) : (77) ^
Furthermore, (77) is satisfied if it holds that
Li (CM ) (N + 1)jXjjYj
V;V^ 2V
Li (CM ; V; V^ )
1 2N (0H (V jQ)+I (Q;V^ )0r)
Lemma 2:
jQj (N + 1)jYj
jfy 2 TQ : x 2 TV (y )gj = 0;
(69)
(66)
1:
(78)
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
We show by a random coding technique that there exists CM satisfying (78). Selecting each Y i in CM independently over A" with the uniform probability distribution, the expectation of Li (CM ; V; V^ ) is bounded as follows:
C
2815
Now we evaluate the expectation of
^ )] E[Li ( M ; V; V = E[
M =
jTV (Y i )\( TV (Y j ))j] ^
6
j =i
=
fx 2 TV (Y i )\( TV (Y j ))g
Pr
2X
x
=
^
6
j =i
1
2X j =6 i
x
0 1)
2
= (M
0 1)
3
= (M
Pr
"
1
2
^
2
(79)
2
(N + 1)
Pr
(N + 1)
(80)
1
jfy 2 A" : x 2 TV (y)gj jfy 2 TQ : x 2 TV (y)gj 2N H Q 0I Q;V : )
(
))
(81)
On the other hand, we have from (25) and (68) that
jA" j (N +"1)jYj 2NH Q : Hence, it holds from (80)–(82) that for any x 2 TQV (
)
(82)
fx 2 TV (Y )g 1" (N + 1)jYj 20NI Q;V : (83) If x 62 TQV , we have Prfx 2 TV (Y )g = 0 from (72). By substituting Pr
(
1
2V
0N=2 (86)
)
M i=1
L i (C M )
1 2
:
(87)
Furthermore, there exist CM=2 CM such that every y i in CM=2 satisfies Li (CM ) 1. The reason comes from the fact that if it does not hold, we have 1
Since A" is a subset of TQ , the numerator of the right-hand side in (80) can be bounded from (71) as follows:
(
jXkYj+2jYj 20N=2 :
2
3
M
: x 2 TV (y )gj fx 2 TV (Y )g = jfy 2 A" jA : "j
(
V;V^
1
Since Y 1 is uniformly distributed over A" , we have for any fixed x 2
X N that
C
^ )] E[Li ( M ; V; V
jXkYj+2jYj
i=1
Y 1 and Y 2 are selected independently:
= :
2V
^ )) +I (Q;V
M
the probability does not depend on i and j;
3
)
V;V^
Since the right-hand side in (86) tends to zero as N goes to infinity, there exists CM for sufficiently large N that satisfies
the union bound;
:
2
= :
jXjjYj
i=1
2
2
^
where the numbered equalities and inequality hold because 1
(N + 1)
fx 2 TV (Y )gPrfx 2 TV (Y )g
Pr
2X
x
1
M
" 2M
1
C
i=1 M
fx 2 TV (Y ) \ TV (Y )g
Pr
L i (C M ) .
E[Li ( M )]
M
(
^
2X
M
1
M i=1
L i (C M )
i=1
1 2N (0H V jQ 0r
fx 2 TV (Y i ) \ TV (Y j )g
x
M
1
E
1
M
M
M
Li (CM ) > i=1
1
M
2 M2 2 1 = 21
which contradicts (87). Since the size of CM=2 is not less than 2N (r0) from (74), Lemma 1 holds. Remark 7: In the case of " = 1, Lemma 1 coincides with the usual Packing Lemma [6, Ch. 3, Lemma 5.1]. But, since Lemma 1 holds for any small " > 0, we can select 2N (r0) sequences y i in A" even if A" is much smaller than TQ . C. Proof of Theorem 2
We difine F + , Ff+ , Fg0 , and F 0 as follows:
1
these results into (79), the following bound is obtained:
C
^ )] E[Li ( M ; V; V
(M 0 1)
(M 0 1) = (M
Pr
2X
x
fx 2 TV (Y )gPrfx 2 TV (Y )g 1
2T
x
0 1)jTQV j
F0 (84)
Furthermore, applying (68), (74), and I (Q; V ) = H (QV ) 0 H (V jQ) to (84), E[Li (CM ; V; V^ )] is finally bounded as follows:
C
^ )] E[Li ( M ; V; V
2 1 2N r0= 2NH QV (N +"1) 1 20N H QV 0H V jQ I Q;V (
2)
(
)
=
(
2(N + 1)
"2
)
jYj
2
(
)+ (
jYj
2
^ ))
N H (V jQ)+r0 2 (
0=2) :
I (Q;V^ )
N N
!1
!1
1 lim inf inf 0 log Pr N !1 f N
NL GN f;g (X jZ ) 2
NL GN f;g (X jZ ) 2 NL GN f;g (X jZ ) 2
1 lim inf inf sup 0 log Pr N !1 f N g
NL GN f;g (X jZ ) 2
(88) (89) (90)
: (91)
Since it holds obviously from the above definition that Fg0 F 0 F 3 F + Ff+ , it suffices to show that there exists an encoder f and a strategy g that can attain Ff+ F (R; L) Fg0 .
Proof of F (R; L) Fg0 : We first show that the strategy g defined in the part (Proof of Cg+ C (R; )) of Section II-C satisfies F (R; L) Fg0 . From (22), we have that
2
(
lim sup sup 0 N1 log Pr g
jYj 0N (I (Q;V )+I (Q;V^ )) 2 :
2
2
"2
Ff+ Fg0
2
(N + 1)
2
1 lim sup inff sup 0 log Pr N g
jYj 0N (I (Q;V )+I (Q;V^ )) 2
(N + 1)
"
^
F+
jXjjYj 1 2N minfH (QV );R+H (QV )0I (Q;V )g jXjjYj 1 2Nh(Q;V;R) : = 2(N + 1) (92)
GN f;g (xjz ) 2(N (85)
+ 1)
2816
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Pr
NL GN j(X ; Y ) 2 TQ1V ; Z f;g (X jZ ) 2
=
z
0;
jM j0(
G
Pr
Pr
2(N + 1)
= Pr
2
h(Q; V; R) +
= Pr
2 Q;V h(Q;V;R)+O (log N )=N
(N + 1)jYj jXkYj 1 max = (N + 1)
jYj+jXkYj
1 exp 0N
L
0N D(Q1V kP 1W )
Q;V
h(Q;V;R)+O (log N )=N
L
Q;V
h(Q;V;R)+O (log N )=N
L
= lim inf inf N
!1
f
D(Q 1 V kP 1 W )
lim!1 inf inf N
f
F (R; L):
(93)
0 jM2(Q; V )j
O(log N ) : (94) N
Pr
min Q;V
L
z
:
(98)
fT 1 gPr
Pr
=
Pr
fT 1 g Q V
Q;V
jM(Q;V )j2
1 PrfG
X jZ ) 2N L j(X ; Y ) 2 TQ1V g
N f;g (
1
Pr
fT 1 g(1 0 ") Q V
Q;V
2
NL
1
0 jM2(Q; V )j NL
fT 1 g(1 0 ")
1
0 jM2(Q; V )j
fT 1 g(1 0 ")
1
0 2 21 2
Pr
Q V
Q;V
jM(Q;V )j212
N) 0 O(log N
Pr
Q V
Q;V
(95)
(99)
NL GN j(X ; Y ) 2 TQ1V f;g (X jZ ) 2
Q V
jM(Q;V )j2
D (Q 1 V kP 1 W )
= 0:
NL GN f;g (X jZ ) 2
NL GN f;g (X jZ ) 2
h(Q;V;R)+O (log N )=N
=
Hence, averaging all types Q and all conditional types V , we have the following bound:
Q;V
Proof of Ff F (R; L): Next we show that the encoder f defined in the part (Proof of C (R; ) Cf0 ) of Section II-C satisfies Ff+ F (R; L). When a wiretapper gets a cryptogram z encoded by f , he can know the fact that (x; y ) is included in M(Q; V ), which is defined by (31). Since (28) holds, we consider only the case of j 6= 0, i.e., y 62 TQ0 . If (x; y ) is included in M1 defined by (32), then x is paired with two or more y . But, if (x; y ) is included in M2 defined by (33), then x is paired with only one y . Since (29) holds for any (x; y ) 2 M(Q; V ), all x 2 M1 must be guessed before any x 2 M2 in the best strategy for this encoder f . This means that the best strategy, and hence any strategy, satisfies (96) at the top of the page. Therefore, for any Q +
1
NL GN j(X ; Y ) 2 TQ1V f;g (X jZ ) 2
Pr
=
D (Q 1 V kP 1 W ) 0
0 N1 log Pr
NL
On the other hand, it holds for any Q and V satisfying jM(Q; V )j < NL 2 that
Since (94) holds for any encoder f , we obtain the following relation:
Fg0
Q V
NL GN j(X ; Y ) 2 TQ1V ; Z f;g (X jZ ) 2
(1 0 ")
NL GN f;g (X jZ ) 2
min
fZ = zj(X ; Y ) 2 T 1 g
1 Pr
where the third inequality holds from (73) and the last inequality holds from (66) and (67). By taking logarithm of (93), we have
0 N1 log Pr
Q V
Pr
6
z :j =0
0N D(Q1V kP 1W ) 2
min
2
NL
+
Q;V
(96)
:
NL GN j(X ; Y ) 2 TQ1V f;g (X jZ ) 2
Pr
L
h(Q;V;R)+O (log N )=N
NL
2
Q V
L
2
have from (27), (28), (97) that
L PrfT 1 g
Q;V
NL
O(log N ) N
h(Q;V;R)+O (log N )=N
1
NL
NL
jXkYj 1 2N h(Q;V;R) 2N L
2
2
NL
N f;g
NL
N h(Q;V;R)+O (log N )
+
1
jM(Q; V )j 2 , we have from the relation jM j + jM j = jM(Q; V )j that Pr G (X jZ ) 2 j( X ; Y ) 2 T 1 ; Z = z 1 0 jM2(Q; V )j : (97) Since the size of M(Q; V ), i.e., jM(Q; V )j, does not depend on z , we 1
X jZ ) 2
2
and V satisfying
Hence, the following relation holds: N f;g (
jM j + jM j < 2 = 1 0 jM j jM j ; jM j + jM j 2
0jM j) jM j+jM j 2
jM(Q;V )j2
2
3 Q;V
jM(Q;V )j2
NL NL
0N D(Q1V kP 1W ) (1 0 ") (N + 1)jXjjYj 2
2
0N D(Q1V kP 1W )0O(log N )
2
0N D(Q1V kP 1W )0O(log N )
Q;V
jM(Q;V )j2 max Q;V
jM(Q;V )j2
= exp2
0N
min Q;V
jM(Q;V )j2
D(Q 1 V kP 1 W ) 0 O(log N ) (100)
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 54, NO. 6, JUNE 2008
Since (101) holds for any strategy g and any small > 0 if we use the encoder f defined in the part (Proof of C (R; ) Cf0 ) of Section II-C, we finally obtain that
where the numbered inequalities hold because
1
:
(98);
2
:
the range of summation is restricted from
3
:
jM(Q; V )j 2NL
to
jM(Q; V )j 2 1 2NL ;
+
(73).
0 N log Pr
jM
N Gf;g (X
jZ ) 2
h(Q;V;R) L
1 k 1
Q;V (Q;V ) 2
j
min
Q;V h(Q;V;R) 2 O(log N )=N L
0 0
= F (R; L):
O (log N ) N
1 k 1
D (Q V P W ) +
REFERENCES O (log N ) N
(101) where the last inequality holds because we have from (38) that
g j 0 jXkYj (N + 1) N [minfR;I (Q;V )g+H (V jQ)020O(log N )=N ] =2 N [h(Q;V;R)020O(log N )=N ] =2 : (102)
jM(Q; V )j 2
f
(103)
NL
D (Q V P W ) +
min
1 N NL 0 log Pr Gf;g (X jZ ) 2 N g N !1 min D (Q 1 V kP 1 W ) Q;V
Ff = lim sup sup
By taking logarithm of the above inequality, we obtain that 1
2817
N (min R;I (Q;V ) +H (V Q) 2)
[1] C. E. Shannon, “Communication theory of secrecy systems,” Bell Syst. Tech. J., vol. 28, no. 3, pp. 565–715, Oct. 1949. [2] H. Yamamoto, “Information theory in cryptology,” IEIEC Trans., vol. E74, no. 9, pp. 2456–2464, Sep. 1991. [3] N. Merhav and E. Arikan, “The Shannon cipher system with a guessing wiretapper,” IEEE Trans. Inf. Theory, vol. 45, no. 6, pp. 1860–1866, Sep. 1999. [4] N. Merhav, “A large-deviations notion of perfect secrecy,” IEEE Trans. Inf. Theory, vol. 49, no. 2, pp. 506–508, Feb. 2003. [5] H. Yamamoto, “Coding theorems for Shannon’s cipher system with correlated source outputs, and common information,” IEEE Trans. Inf. Theory, vol. 40, no. 1, pp. 85–95, Jan. 1994. [6] I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
