Polar Codes are Optimal for Write-Efficient Memories - Semantic Scholar
Polar Codes are Optimal for Write-Efficient Memories Qing Li
Anxiao (Andrew) Jiang
Department of Computer Science and Engineering Texas A & M University College Station, TX 77840 [email protected]
Department of Computer Science and Engineering Texas A & M University College Station, TX 77840 [email protected]
Abstract—Write-efficient memories (WEM) [1] were introduced by Ahswede and Zhang as a model for storing and updating information on a rewritable medium with constraints. A coding scheme for WEM using recently proposed polar codes is presented. The coding scheme achieves rewriting capacity, and the rewriting and decoding operation can be done in time O(N log N ), where N is the length of the codeword.
I. I NTRODUCTION Write-efficient memories (WEM) are models for storing and updating information on a rewritable medium with constraints. WEM is widely used in data storage area: in flash memories, write-once memories (WOM) [11], and the recently proposed compressed rank modulation (CRM) [9] are examples of WEM; codes based on WEM were proposed for phase change memories [8]. The recently proposed scheme, that polar codes are constructed for WOM codes achieving capacity [3], motivates us to construct codes for WEM. A. WEM with a maximal rewriting cost constraint Let X = {0, 1, ..., q − 1} be the storage alphabet. R+ = [0, +∞), and ϕ : X × X → R+ is the rewriting cost function. Suppose that a memory def consists of N cells. Given one cell state, x0N −1 = (x0 , x1 , ..., xN −1 ) ∈ X N , and another cell state, y0N −1 ∈ −1 X N , the rewriting cost of changing xN to y0N −1 is 0 NP −1 −1 N −1 ϕ(xN , y0 ) = ϕ(xi , yi ). 0 i=0
Let M ∈ N, and D = {0, 1, ..., M − 1}. Define the decoding function, D : X N → D, i.e., D(x0N −1 ) = i. −1 Given the current cell state xN , and data to rewrite, j, 0 the rewriting function is defined as R : X N × D → X N −1 such that D(R(xN , j)) = j. 0 Definition 1. [5] An (N, M, q, d) WEM code is a collection of subsets, C = {Ci : i ∈ D}, where Ci ⊆ X N , −1 −1 and ∀xN ∈ Ci D(xN ) = i, such that 0 0 T Cj = ∅. • ∀i 6= j, Ci
•
∀j, ∀x0N −1 ∈ C, ∃y0N −1 = R(x0N −1 , j) such that −1 N −1 ϕ(xN , y0 ) ≤ N d. 0
The rewriting rate of an (N, M, q, d) WEM code is R = logN2 M , and the rewriting capacity function, R(q, d), is the largest d-admissible rate when N → ∞. Let P(X × X ) be the set of joint probability distributions over X × X . For a pair of random variables (X, Y ) ∈ (X , X ), let PXY denote the joint probability distribution, PY denotes the marginal distribution, and PY |X denotes the conditional probability distribution. E(·) denotes the expectation operator. If X is uniformly distributed over {0, 1, ..., q − 1}, denote it as X ∼ U (q). Let P(q, d) = {PXY ∈ P(X × X ) : PX = PY , E(ϕ(X, Y )) ≤ d}. R(q, d) is determined as [5]: R(q, d) = max H(Y |X). PXY ∈P(q,d)
For WOM codes, the cell state can only increase but not decrease. WOM codes are special cases of WEM codes: for a WOM cell if we update it from x ∈ X to y ∈ X , the cost is measured by ϕ(x, y) = 0 if y ≥ x, and ∞ otherwise. Therefore, WOM codes are such WEM codes with ϕ(·) defined previously, and d is equal to 0. In this paper, we focus on symmetric rewriting capacity function Rs (q, d): Definition 2. For X, Y ∈ X with PX , PY and PXY , and ϕ : X × X → R+ , Rs (q, d) = max H(Y |X), s PXY ∈P (q,d)
def
where P s (q, d) = {PXY ∈ P(X × X ) : PX = PY , X ∼ U (q), E(ϕ(X, Y )) ≤ d}. We present an example of WEM with Rs (q, d) below. Sq,m is the set of (mq)! permutations over m!q m m z }| { z }| { {1, 1, ..., 1, ..., q, q, ..., q}. We abuse the notation of def = [u0 , u1 , ..., uqm−1 ] to denote an element of uqm−1 0 Sq,m , which denotes the mapping i → ui . Example 3. A rewriting code for CRM with a maximal
rewriting cost constraint, (q, m, M, d), is defined by replacing X N by Sq,m , ϕ(·) by Chebyshev distance def between uqm−1 , v0qm−1 ∈ Sq,m , d∞ (uqm−1 , v0qm−1 ) = 0 0 max |uj − vj | , and N d by d in definition 1.
j∈{0,1,...,qm−1}
The above rewriting code is actually an instance of WEM: for x, y ∈ X , let ϕ(x, y) = 0 if |x − y| ≤ d, and ∞ otherwise. Now the (q, m, M, d) CRM is an (qm, M, q, 0) WEM with X N replaced by Sq,m , and ϕ(·) is defined previously. B. WEM with an average rewriting cost constraint −1 D(xN ) 0
R(x0N −1 , i),
With deterministic and suppose that message sequences M1 , M2 , ..., Mt (t → ∞) are written into the memory medium, where Mi ∈ D −1 is uniformly distributed, represented by xN (i), and it 0 forms a markov chain. Its transition probability matrix −1 µ(y0N −1 |xN )yN −1 ,xN −1 ∈X N is given by 0 0 0 ( −1 1 if ∃i s.t.y0N −1 = R(xN , i), −1 0 µ(y0N −1 |xN )= M 0 0 otherwise. −1 Denote the stationary distribution of xN by π(x0N −1 ). 0 ¯ is determined as follows: The average rewriting cost D t
¯ D
1 X −1 −1 E(ϕ(xN (i), xN (i + 1))), 0 0 t→∞ N t i=1 X 1 X −1 −1 µ(y0N −1 |xN ) π(xN ) = 0 0 N N −1 N −1 =
lim
x0
y0
−1 N −1 ϕ(xN , y0 ), X0 X −1 ¯ j (xN −1 ), π(xN ) D 0 0 N −1 j x
The binary polar WEM codes with an average rewriting cost constraint and a maximal rewriting cost constraint are presented in subsection A and B of section III, respectively; The q-ary polar WEM codes, based on the recently proposed q-ary polar codes [10], are presented in subsection A and B of section IV for an average rewriting cost constraint and a maximal rewriting cost constraint, respectively; The conclusion is obtained in section V. II. L OSSY SOURCE CODING AND ITS DUALITY WITH WEM In this section, we present briefly background of lossy source coding and its duality with WEM, which inspires code constructions for WEM. Let X also denote the variable space, and Y denotes the reconstruction space. Let d : Y × X → R+ denote the distortion function, and the distortion among a vector x0N −1 and its reconstructed vector y0N −1 is NP −1 −1 N −1 d(xi , yi ). d(xN , y0 ) = N1 0 i=0
A (q N R , N ) rate distortion code consists of a encoding function fN : X N → {0, 1, ..., q N R − 1} and a reproduction function gN : {0, 1, ..., q N R − 1} → Y N . The associated distortion is defined as E(d(X0N −1 , gN (fN (X0N −1 )))), where the expectation is with respect to the probability distribution on X N . R(q, D) is the infimum of rates R such that E(d(X0N −1 , gN (fN (X0N −1 )))) is at most D as N → ∞. def
(1)
Let P (q, D) = {PXY ∈ P(X × Y) : E(d(X, Y )) ≤ D}, and R(q, D) is min I(X; Y ) [4].
¯ j (xN −1 ) is the average rewriting cost of updatwhere D 0 −1 ing from xN to data j. 0
We focus on the double symmetric rate-distortion for (X, Y ) ∈ (X × X ), and d(x, y), RS (q, D), which is Rs (q, D) = min I(Y ; X), where s
=
0
PXY ∈P (q,D)
PXY ∈P (q,D)
Definition 4. An (N, M, q, d)ave WEM code is a collection of subsets, C = {Ci : i ∈ D}, where Ci ⊆ X N , −1 −1 and ∀xN ∈ Ci D(xN ) = i, such that 0 0 T • ∀i 6= j, Ci Cj = ∅; ¯ ≤ d. • The average rewriting cost D
P (q, D) = {PXY ∈ P(X × X ) : PX = PY , X ∼ U (q), E(d(X, Y )) ≤ D}. The duality between Rs (q, D) and Rs (q, D) is captured by the following lemma, the proof of which is ommited due to being straightforward.
The rewriting rate of an (N, M, q, d)ave code is Rave = logN2 M , and its rewriting capacity function, Rave (q, d), is the largest d-admissible rate when N → ∞. It is proved that Rave (q, d) = R(q, d) [1]. Similarly, we focus on the symmetric rewriting capacity function, Rsave (q, d), as defined in definition 2.
Lemma 5. With the same d(·) and ϕ(·),
C. The outline The connection between rate-distortion theorem and rewriting capacity theorem is presented in section II;
s
def
Rs (q, D) + Rs (q, D) = log2 q.
(2)
III. P OLAR CODES ARE OPTIMAL FOR BINARY WEM Inspired by lemma 5, we show that polar codes can be used to construct binary WEM codes with Rs (2, D) in a way related to the code construction for lossy source coding of [7] in this section.
A. A code construction for binary WEM with an average rewriting cost constraint 1) Background on polar codes [2]: Let W : {0, 1} → Y be a binary-input discrete memoryless channel for some output alphabet Y. Let I(W ) ∈ [0, 1] denote the mutual information between the input and output of W with a uniform distribution on the input. Let GN denote 1 0 . Let the Bhattacharyya n-th Kronecker product of P p 11 WY |X (y|0)WY |X (y|1). parameter Z(W ) = y∈X
The polar code, CN (F, uF ), ∀F ⊆ {0, 1, ..., N − 1}, uF denotes the subvector ui : i ∈ F , and uF ∈ {0, 1}|F | , is a linear code given by CN (F, uF ) = c −1 −1 {xN = uN GN : uF c ∈ {0, 1}|F | }. The polar 0 0 code ensemble, CN (F ), ∀F ⊆ {0, 1, ..., N − 1}, is CN (F ) = {CN (F, uF ), ∀uF ∈ {0, 1}|F | }. The secret of polar codes achieving I(W ) lies (i) in how to select F : define WN : {0, 1} → N i Y × {0, 1} as sub-channel i with input ui , output (y0N −1 , ui−1 and transition probabilities 0 ) P def (i) N −1 i−1 W N (y0N −1 |u0N −1 ), WN (y0 , u0 |ui ) = 2N1−1 −1 uN i+1 N −1 Q def
−1 where W N (y0N −1 |uN ) = 0
i=0
−1 W (yi |(uN GN )i ), 0
−1 and (uN GN )i denotes the i-th element of u0N −1 GN ; 0 (i) The fraction of {WN } that are approaching noiseless, β (i) i.e., Z(WN ) ≤ 2−N for 0 ≤ β ≤ 12 , approaches (i) I(W ); The F is chozen as indecies with large Z(WN ), β def (i) that is F = {i ∈ {0, 1..., N − 1} : Z(WN ) ≥ 2−N } for β ≤ 1/2. The encoding is done by a linear transformation, and the decoding is done by successive cancellation (SC). 2) Polar codes on lossy source coding: We sketch the binary polar code construction for lossy source coding [7] as follows. Note that Rs (2, D) can be obtained through the following optimization function:
min s.t.
: I(Y ; X), X1 X1 1 : P (y|x) = P (x|y) = , 2 2 2 x y XX 1 P (y|x)d(y, x) ≤ D. 2 x y
(3)
Let P ∗ (y|x) be the probability distribution minimizing the objective function of (3). P ∗ (y|x) plays the role of a channel. By convention, we call P ∗ (y|x) as test channel, and denote it as W (y|x). Now, we construct the source code with Rs (2, D) using the polar code for W (y|x), and denote the rate of
the source code by R: set F as N (1 − R) sub-channels (i) with the highest Z(WN ), set F c as the remaining N R sub-channels, and set uF to an arbitrary value. A source codeword y0N −1 is mapped to a codeword N −1 −1 x0 ∈ CN (F, uF ), and xN is described by the index 0 N −1 −1 uF c = (x0 GN )F c . The reproduction process is done as follows: we do −1 ˆ (y N −1 , uF ), that is for SC encoding scheme, u ˆN =U 0 0 each k in the range 0 till N − 1: 1) If k ∈ F , set u ˆk = uk ; 2) Else, set u ˆk = m with the posterior N −1 P (m|ˆ ui−1 , y ). 0 0 The reproduction codeword is u ˆ0N −1 GN . Thus, the average distortion DN (F, uF ) (over the source codeword y0N −1 and the encoder randomness for the code CN (F, uF )) is : X Y X N −1 P (y0N −1 ) P (ˆ ui |ˆ ui−1 ) 0 , y0 u ˆF c i∈F c
y0N −1
−1 d(y0N −1 , u ˆN GN ), 0
where u ˆF = uF . The expectation of DN (F, uF ) P over the uniform 1 D (F, uF ). choice of uF , DN (F ), is DN (F ) = 2|F | N uF
Let QU N −1 ,Y N −1 denote the distribution defined by 0 0 QY N −1 (y0N −1 ) = P (y0N −1 ), and QU N −1 |Y N −1 by 0 0 0 ( 1 , if i ∈ F, Q(ui |u0i−1 , y0N −1 ) = 2 i−1 N −1 P (ui |u0 , y0 ), otherwise. Thus, DN (F ) is equivalent to EQ (d(y0N −1 , u0N −1 GN )), where EQ (·) denotes the expectation with respect to the distribution QU N −1 ,Y N −1 . 0
0
β
It is proved that DN (F ) ≤ D+O(2−N )cfor 0 ≤ β ≤ |F | 1 2 , the rate of the above scheme is R = N , and polar codes achieve the rate-distortion bound by Theorem 3 of [7]. On the other hand, Theorem 2 of [3] further states the strong converse result of the rate distortion theory. More precisely, if y0N −1 is uniformly distributed over {0, 1}N , then ∀δ > 0, 0 < β < 21 , N sufficiently large, and with the above SC encoding process and the induced Q, β Q(d(u0N −1 GN , y0N −1 ) ≥ D + δ) < 2−N . That is, for N −1 −1 ∀y0 , the above reproduction process obtains xN = 0 N −1 N −1 N −1 u0 GN such that the distortion d(x0 , y0 ) is less than D almost by sure. 3) The code construction: We focus on the code construction with symmetric rewriting cost function, which satisfies ∀x, y, z ∈ {0, 1}, ϕ(x, y) = ϕ(x + z, y + z), where + is over GF(2).
To construct codes for WEM with Rs (2, D), we utilize its related form Rs (2, D) in (2) and the test channel W (y|x) for Rs (2, D). Note that W (y|x) is a binary symmetric channel. The code construction for (N, M, 2, D)ave with rate R is presented in Algorithm III.1: Algorithm III.1 A code construction for (N, M, 2, D)ave WEM 1: Set F as N R sub-channels with the highest (i) Z(WN ), and set F c as the remaining N (1 − R) sub-channels. 2: The (N, M, 2, D)ave code is C = {Ci : Ci = CN (F, uF (i))}, where uF (i) is the binary representation form of i for i ∈ {0, 1, ..., M − 1}. Clearly, since TGN is of full rank [2], ∀uF (i) 6= uF (j), CN (F, uF (i)) CN (F, uF (j)) = ∅. The rewriting operation and the decoding operation are defined in Algorithm III.2 and Algorithm III.3, respectively, where the dither g0N −1 is inspired by [3]. Algorithm III.2 The rewriting operation y0N −1 = −1 R(xN , i). 0 −1 Let v0N −1 = xN + g0N −1 , where g0N −1 is known 0 both to the decoding function and to the rewriting function, it is chosen such that v0N −1 is uniformly distributed over {0, 1}N , and + is over GF(2). N −1 2: SC encoding v0 , and this results u0N −1 = N −1 −1 ˆ (v U , uF (i)) and yˆ0N −1 = uN GN . 0 0 N −1 N −1 N −1 3: y0 = yˆ0 + g0 .
1:
ˆ (xN −1 + g N −1 , uF (i))GN + g N −1 ), which ϕ(x0N −1 , U 0 0 0 N −1 ˆ (xN −1 + g N −1 , uF (i))GN ) due to is ϕ(x0 + g0N −1 , U 0 0 ϕ(·) being symmetric. Denote w0N −1 = x0N −1 + g0N −1 , ˆ (wN −1 , uF (i))GN ). thus ϕ(x0N −1 , y0N −1 ) = ϕ(w0N −1 , U 0 ¯ The average rewriting cost D t
=
1 X −1 E(ϕ(xN (i), x0N −1 (i + 1))), 0 t→∞ N t i=1
=
1 X E(ϕ(w0N −1 , t→∞ N t i=1
lim
t
lim
ˆ (wN −1 , uF (Mi+1 ))GN )), U X0 X ¯ j (wN −1 ). = π(w0N −1 ) D 0 w0N −1
(4)
j
¯ j (wN −1 ), which is the average Let us focus on D 0 (in this case, over the probability of rewriting to data j and the randomness of the encoder) rewriting cost of updating w0N −1 to a codeword representing j. Thus ¯ j (wN −1 ) = D 0 1 X Y N −1 −1 P (ui |ui−1 )ϕ(w0N −1 , uN GN ). 0 , w0 0 |F 2 | u c i∈F c F
¯ is actually the Therefore, interpreting ϕ(·) as d(·), D average distortion over the ensemble CN (F ), DN (F ). ¯ The following lemma from [7] is to bound D: Lemma 7. [7] Let β < 12 be a constant and let σN = 1 −N β . When the polar code for the source code with 2N 2 Rs (2, D) is constructed with F : (i)
2 F = {i ∈ {0, 1, ..., N − 1} : Z(WN ) ≥ 1 − 2σN }, β
then DN (F ) ≤ D + O(2−N ), where D is the average rewriting cost constraint. Algorithm III.3 The decoding operation uF (i) = −1 D(xN ). 0 −1 y0N −1 = xN + g0N −1 . 0 N −1 −1 2: uF (i) = (y0 GN )F .
1:
Therefore, with the same β, σN , F, and polar code ¯ ≤ D + O(2−N β ). ensemble CN (F ), D |F c | According to [2], lim N = n N =2 ,n→∞
(i)
Lemma 6. D(R(y0N −1 , i)) = i holds for each rewriting. Proof: From the rewriting operation, −1 y0N −1 = yˆ0N −1 + g0N −1 = uN G + g0N −1 = N 0 N −1 N −1 ˆ (v U , uF (i))GN + g0 , from the decoding 0 ˆ (v N −1 , uF (i))GN + g N −1 + g N −1 , which is function U 0 0 0 N −1 ˆ (v U , uF (i))GN , thus the decoding result is i. 0 4) The average rewriting cost analysis: −1 From y0N −1 = R(xN , i), we know that 0 N −1 N −1 ˆ (v ˆ (xN −1 + y0 = U , u (i))G + g0N −1 = U F N 0 0 N −1 N −1 N −1 N −1 g0 , uF (i))GN + g0 , thus ϕ(x0 , y0 ) is
nβ
|{i ∈ {0, 1, ..., N } : Z(WN ) ≤ 2−2 }| N →∞ N I(W ) = Rs (2, D), lim
=
thus this implies that for N sufficiently large ∃ a set F |F c | such that N ≥ Rs (2, D) − , ∀ > 0. In other words, | the rate of the constructed WEM code, R = |F N = 1− |F c | s N ≤ R (2, D) + . The complexity of the decoding and the rewriting operation is of the order O(N log N ) according to [2]. We conclude the theoretical performance of the above polar WEM code as follows:
Theorem 8. For a binary symmetric rewriting cost function ϕ : X × X → R+ . Fix a rewriting cost D and 0 < β < 12 . For any rate R < Rs (2, D), there exists a sequence of polar WEM codes of length N and rate ¯ R ≤ R, so that under the above rewriting operation, D −N β ¯ satisfies D ≤ D+O(2 ). The decoding and rewriting operation complexity of the codes is O(N log N ). B. A code construction for binary WEM with a maximal rewriting cost constraint The code construction, the rewriting operation and the decoding operation are exactly the same as Algorithm III.1, Algorithm III.2, and Algorithm III.3, respectively. The rewriting capacity is guaranteed by Lemma 5, the decoding and rewriting operation complexity is the same as polar codes, and the rewriting cost is obtained due to the strong converse result of rate distortion theory, i.e., Theorem 2 of [3]. Thus, we have:
y ∈ Y, u ∈ {0, 1}r−k , and xP=p (x0 , x1 , ..., xr−1 ) ∈ X ; Define Z(W{x,x0 } ) = W (y|x)W (y|x0 ), let y∈Y P Zv (W ) = 21r Z(W{x,x+v} ) for v ∈ X \{0}, and x∈X P Zi (W ) = 21i Zv (W ), where i = 0, 1, ..., r − 1, and v∈Xi
Xi = {v ∈ X : i = arg max vj 6= 0} with the binary 0≤j≤r−1
representation of v, (v0 , v1 , ..., vr−1 ). (j) Zi (WN ) ∀i ∈ {0, 1, ..., r − 1} and j ∈ {0, 1, ..., N − 1} converges to Zi,∞ ∈ {0, 1}, and with probability one (Z0,∞ , Z1,∞ , ..., Zr−1,∞ ) takes one of the values (Z0,∞ = 1, ..., Zk−1,∞ = 1, Zk,∞ = 0, ..., Zr−1,∞ = 0) ∀k = 0, 1, ..., r − 1, i.e., Theorem 1.b of [10]. j ∈ Ak,n (j) (j) (j) iff (Z0 (WN ), Z1 (WN ), ...., Zr−1 (WN )) ∈ Rk (), r−1 Q Q def k−1 where Rk () = ( D1 ) × ( D0 ), D0 = [0, ), i=0
i∈{0,1,...,N −1}
Theorem 9. For a binary symmetric rewriting cost function ϕ : X × X → R+ . Fix a rewriting cost D, δ, and 0 < β < 12 . For any rate R < Rs (2, D), there exists a sequence of polar WEM codes of length N and rate R ≤ R, so that under the above rewriting operation and the induced probability distribution Q, the rewriting cost between a current codeword ∀y0N −1 and its updated −1 −1 codeword xN satisfies Q(ϕ(y0N −1 , xN ) ≥ D+δ) < 0 0 −N β 2 . The decoding and rewriting operation complexity of the codes is O(N log N ). IV. P OLAR CODES ARE OPTIMAL FOR q- ARY WEM, q = 2r In this section, we extend the previous binary scheme to q-ary WEM (q = 2r ), considering the length of polar codes, which is N = 2n . A. A code construction for q-ary WEM with an average rewriting cost constraint, q = 2r 1) Background of q-ary polar codes, q = 2r [10]: The storage alphabet is X , |X | = q, and for x ∈ X , (x0 , x1 , ..., xr−1 ) is its binary representation. Let W : X → Y be a discrete memoryless channel. I(W ) is P P 1 W (y|x) P 1 . q W (y|x) log2 W (y|x0 )
x∈X y∈Y
x0 ∈X
q
(i)
The sub-channel i, WN , is defined by P def (i) 1 WN (y0N −1 , ui−1 W N (y0N −1 |u0N −1 ), 0 |ui ) = q N −1 −1 uN i+1 N −1 def Q
−1 where W N (y0N −1 |uN ) = 0
i=0
−1 W (yi |(uN GN )i ). 0
Consider the following bit channel and its Bhattacharyya: Fix k P ∈ {0, 1, ..., r}, and 1 W [r−k] (y|u) = W (y|x), where 2k r−1 x:xk =u
i=k
D1 =P(1 − , 1]. The channel polarizes in the sense that |Aki ,n |
→ I(W ) as N → ∞. rN For u0N −1 ∈ X N , we also represent it by its binary form, that is u0N −1 = (uI(0,0) , ..., uI(0,r−1) , ..., uI(N −1,0) , ..., uI(N −1,r−1) ) ∈ {0, 1}rN , where uI(i,0) , ..., uI(i,r−1) is the binary representation of ui ∈ X , and I(i, j) = i × r + j. ∀i ∈ Aki ,n , let the frozen bit set be determined by F = {I(i, j) : i ∈ {0, 1, ..., N − 1}, j ∈ {0, 1, ..., ki − 1}} ⊆ {0, 1, ..., rN − 1}. Frozen bits for u0N −1 are defined as uF ∈ {0, 1}|F | with the subvector ui : i ∈ F . Finally, the polar code, CN (F, uF ), ∀F ⊆ {0, 1, ..., rN − 1} and uF ∈ {0, 1}|F | , is a linear code given by CN (F, uF ) = {x0N −1 = u0N −1 GN : c ∀uF c ∈ {0, 1}|F | }. The polar code ensemble, CN (F ), ∀F ⊆ {0, 1, ..., rN − 1}, is CN (F ) = {CN (F, uF ) : ∀uF ∈ {0, 1}|F | }. 2) The code construction: We focus on the q-ary WEM code construction with a symmetric rewriting cost function, which satisfies ∀x, y, z ∈ {0, 1, ..., q − 1}, ϕ(x, y) = ϕ(x + z, y + z), where + is over GF(q). Similar to the binary case, to construct codes for WEM with Rs (q, D), we utilize its related form Rs (q, D) in Lemma 5 and its test channel W (y|x). The code construction is presented in Algorithm IV.1. Algorithm IV.1 A code construction for (N, M, q, D)ave WEM 1: ∀i ∈ Aki ,n , F = {I(i, j) : i ∈ {0, 1, ..., N − 1}, j ∈ {0, 1, ..., ki − 1}}. 2: The (N, M, q, D)ave code is C = {Ci : Ci = CN (F, uF (i))}, where uF (i) is the binary representation form of i for i ∈ {0, 1, ..., M − 1}.
|F | , and the polar The WEM code rate is R = rN c |F | code rate is R = rN . Similarly, when R approaches Rs (q, D), R approaches Rs (q, D) based on Lemma 5. The rewriting operation and the decoding operation are defined in Algorithm IV.2 and Algorithm IV.3, respectively, where the SC encoding is a generalization of q-ary lossy source coding, and the dither g0N −1 is inspired by [3] to make sure the uniform distribution of v0N −1 .
Algorithm IV.2 The rewriting operation y0N −1 −1 R(xN , i). 0
=
−1 Let v0N −1 = xN + g0N −1 , where g0N −1 is known 0 both to the decoding function and to the rewriting function, it is chosen such that v0N −1 is uniformly distributed, and + is over GF(q). N −1 −1 ˆ (v N −1 , uF (i)), that 2: SC encoding v0 ,u ˆN =U 0 0 is for each k in the range 0 till N − 1: ( uj if j ∈ Ar,n , u ˆj = N −1 m with the posterior P (m|ˆ uj−1 ), 0 , v0
1:
where in the above m = 0, 1, ..., q − 1 and if j ∈ Ak,n for 0 ≤ k ≤ r − 1, the first k bits of u ˆj are −1 fixed, and let yˆ0N −1 = u ˆN G . N 0 N −1 3: y0 = yˆ0N −1 − g0N −1 , and − is over GF(q).
Ar,n is equal to if Aˆr,n 6= Ar,n , 0 Q i−1 N −1 P (ui |u0 , v0 ) otherwise, S i∈r−1 Ak,n k=0
where in the second case ∀i ∈ Ak,n , the first k bits of ui are fixed. Therefore, the average (in this case, over the probability of rewriting to data j and the randomness of the encoder) rewriting cost of updating w0N −1 to a codeword ¯ j (wN −1 ), is representing j, D 0 Y 1 X N −1 P (ui |ui−1 = ) 0 , w0 2|F | u c Sr−1 F
i∈
k=0
Ak,n
ϕ(w0N −1 , u0N −1 GN ), c
where uF = uF (j), uF c ∈ {0, 1}|F | , and the summation over uF c takes care of Ar,n and the fact that i ∈ Ak,n , the first k bits of ui are fixed. ¯ Thus, we obtain that D X X ¯ j (wN −1 ), D π(w0N −1 ) = 0 w0N −1
=
X
j
π(w0N −1 )
w0N −1
0, −1 N −1 |C(a, b|xN , y0 )/N − p(a, b)| < . Second, ∀a, b ∈ 0 −1 N −1 X × Y with p(a, b) = 0, C(a, b|xN , y0 ) = 0. 0 N −1 N −1 In our case y0 = u0 GN . Due to the full rank of −1 GN , there is a one-to-one correspondence between uN 0 ∗(N ) and y0N −1 . We say that u0N −1 , x0N −1 ∈ A (U, X) ∗(N ) −1 −1 if xN , uN GN ∈ A (X, Y ) with respect to 0 0 1 q W (y|x), where W (y|x) is the test channel. The first conclusion is that for N sufficiently large, β ∗(N ) Q(A (U, X)) > 1 − 2−N for ∀0 < β < 21 , > 0, which is a generalization of Theorem 4 ∗(N ) of [3], and where Q(A (U, X)) = Q(∀a, b :
−1 −1 GN , xN ) − 1q W (a|b)| ≤ ). The out| N1 C(a, b|uN R EFERENCES 0 0 line is that based on Lemma 10 and Lemma 11 we obtain [1] R. Ahlswede and Z. Zhang, “Coding for write-efficient memory,” Inform. Comput, vol. 83, no. 1, pp. 80–97, Oct 1989. that [2] E. Arikan, “Channel polarization: a method for constructing X −1 −1 capacity-achieving codes for symmetric binary-input memoryless |Q(uN , xN )−P (u0N −1 , x0N −1 )| 0 0 ∗(N )
−1 N −1 uN ,x0 ∈A 0
(U,X)
[3]
≤ |Ar,n |σN dmax . Thus, we obtain X −1 −1 −1 Q(uN , xN ) − P (uN , x0N −1 )| | 0 0 0 −1 N −1 uN ,x0 0
≤
X
−1 −1 −1 −1 |Q(uN , xN ) − P (uN , xN )|, 0 0 0 0
−1 N −1 uN ,x0 0
≤
[4] [5] [6] [7]
|Ar,n |σN dmax , ∗(N )
−1 −1 where in the above uN , xN ∈ A (U, X). 0 0 ∗(N ) By lower bounding P (A (U, X)) = 1 − P (∃a, b : −1 −1 | N1 C(a, b|uN GN , xN ) − 1q W (a|b)| ≥ ) ≥ 0 0 2 1 − 2q 2 e−2N based on Hoeffding’s inequality, and ∗(N ) ∗(N ) Q(A (U, X)) ≥ P (A (U, X)) − |Ar,n |σN dmax ,
[8]
[9]
−N β
2 we obtain the desired result by setting σN = 2N dmax . N −1 The second conclusion is that let y0 = R(x0N −1 , i), then ∀δ > 0, 0 < β < 12 and N sufficiently large, −1 Q(ϕ(y0N −1 , xN )/N ≥ D + δ) < 2−N β , which is a 0 generalization of Theorem 5 of [3]. The outline is that −1 N −1 Q(ϕ(xN , y0 )/N ≥ D + δ) 0
≤
−1 N −1 Q(ϕ(xN , y0 )/N ≥ D + δ 0 \ N −1 N −1 ) x0 , y0 ∈ A∗(N (X, Y ))
+
−1 N −1 ) Q(xN , y0 6∈ A∗(N (X, Y )), 0
≤
2−N β ,
where the last inequality is based on the conclusion just ∗(N ) −1 N −1 obtained Q(xN , y0 6∈ A (X, Y )) < 2−N β , and 0 ∗(N ) N −1 N −1 that when x0 , y0 ∈ A (X, Y ), for sufficiently small and N sufficiently large, ϕ(y0N −1 , x0N −1 )/N ≤ D + δ. V. C ONCLUSION Code constructions for WEM using recently proposed polar codes have been presented. Future work focuses on exploring error-correcting codes for WEM. VI. ACKNOWLEDGEMENT This work was supported in part by the National Science Foundation (NSF) CAREER Award CCF-0747415 and the NSF Grant CCF-1217944.
[10] [11]
channels,” IEEE Trans on Inf Theory, vol. 55, no. 7, pp. 3051– 3073, July 2009. D. Burshtein and A. Strugatski, “Polar write once memory codes,” in Proceedings of the 2012 IEEE international conference on Symposium on Information Theory (ISIT’12), Boston, MA, July 2012, pp. 1972–1975. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991. F. Fu and R. W. Yeung, “On the capacity and error-correcting codes of write-efficient memories,” IEEE Trans on Inf Theory, vol. 46, no. 7, pp. 2299–2314, Nov 2000. M. Karzand and E. Teletar, “Polar codes for q-ary source coding,” in Proc. IEEE International Symposium on Information Theory (ISIT), Austin, Texas, June 2010, pp. 909–912. S. B. Korada and R. Urbanke, “Polar codes are optimal for lossy source coding,” IEEE Trans. Inf. Theory, vol. 56, no. 4, pp. 1751– 1768, April 2010. L. A. Lastras-Montano, M. Franceschini, T. Mittelholzer, J. Karidis, and M. Wegman, “On the lifetime of multilevel memories,” in Proceedings of the 2009 IEEE international conference on Symposium on Information Theory (ISIT’09), Coex, Seoul, Korea, 2009, pp. 1224–1228. Q. Li, “Compressed rank modulation,” in Proc. 50th Annual Allerton Conference on Communication, Control and Computing (Allerton), Monticello, IL, October 2012. W. Park and A. Barg, “Polar codes for q-ary channels, q = 2r ,” in Proc. IEEE International Symposium on Information Theory (ISIT), Boston, MA, June 2012, pp. 2142–2146. R. L. Rivest and A. Shamir, “How to reuse a write-once memory,” Inf. Contr, vol. 55, pp. 1–19, 1982.
Anxiao (Andrew) Jiang
Department of Computer Science and Engineering Texas A & M University College Station, TX 77840 [email protected]
Department of Computer Science and Engineering Texas A & M University College Station, TX 77840 [email protected]
Abstract—Write-efficient memories (WEM) [1] were introduced by Ahswede and Zhang as a model for storing and updating information on a rewritable medium with constraints. A coding scheme for WEM using recently proposed polar codes is presented. The coding scheme achieves rewriting capacity, and the rewriting and decoding operation can be done in time O(N log N ), where N is the length of the codeword.
I. I NTRODUCTION Write-efficient memories (WEM) are models for storing and updating information on a rewritable medium with constraints. WEM is widely used in data storage area: in flash memories, write-once memories (WOM) [11], and the recently proposed compressed rank modulation (CRM) [9] are examples of WEM; codes based on WEM were proposed for phase change memories [8]. The recently proposed scheme, that polar codes are constructed for WOM codes achieving capacity [3], motivates us to construct codes for WEM. A. WEM with a maximal rewriting cost constraint Let X = {0, 1, ..., q − 1} be the storage alphabet. R+ = [0, +∞), and ϕ : X × X → R+ is the rewriting cost function. Suppose that a memory def consists of N cells. Given one cell state, x0N −1 = (x0 , x1 , ..., xN −1 ) ∈ X N , and another cell state, y0N −1 ∈ −1 X N , the rewriting cost of changing xN to y0N −1 is 0 NP −1 −1 N −1 ϕ(xN , y0 ) = ϕ(xi , yi ). 0 i=0
Let M ∈ N, and D = {0, 1, ..., M − 1}. Define the decoding function, D : X N → D, i.e., D(x0N −1 ) = i. −1 Given the current cell state xN , and data to rewrite, j, 0 the rewriting function is defined as R : X N × D → X N −1 such that D(R(xN , j)) = j. 0 Definition 1. [5] An (N, M, q, d) WEM code is a collection of subsets, C = {Ci : i ∈ D}, where Ci ⊆ X N , −1 −1 and ∀xN ∈ Ci D(xN ) = i, such that 0 0 T Cj = ∅. • ∀i 6= j, Ci
•
∀j, ∀x0N −1 ∈ C, ∃y0N −1 = R(x0N −1 , j) such that −1 N −1 ϕ(xN , y0 ) ≤ N d. 0
The rewriting rate of an (N, M, q, d) WEM code is R = logN2 M , and the rewriting capacity function, R(q, d), is the largest d-admissible rate when N → ∞. Let P(X × X ) be the set of joint probability distributions over X × X . For a pair of random variables (X, Y ) ∈ (X , X ), let PXY denote the joint probability distribution, PY denotes the marginal distribution, and PY |X denotes the conditional probability distribution. E(·) denotes the expectation operator. If X is uniformly distributed over {0, 1, ..., q − 1}, denote it as X ∼ U (q). Let P(q, d) = {PXY ∈ P(X × X ) : PX = PY , E(ϕ(X, Y )) ≤ d}. R(q, d) is determined as [5]: R(q, d) = max H(Y |X). PXY ∈P(q,d)
For WOM codes, the cell state can only increase but not decrease. WOM codes are special cases of WEM codes: for a WOM cell if we update it from x ∈ X to y ∈ X , the cost is measured by ϕ(x, y) = 0 if y ≥ x, and ∞ otherwise. Therefore, WOM codes are such WEM codes with ϕ(·) defined previously, and d is equal to 0. In this paper, we focus on symmetric rewriting capacity function Rs (q, d): Definition 2. For X, Y ∈ X with PX , PY and PXY , and ϕ : X × X → R+ , Rs (q, d) = max H(Y |X), s PXY ∈P (q,d)
def
where P s (q, d) = {PXY ∈ P(X × X ) : PX = PY , X ∼ U (q), E(ϕ(X, Y )) ≤ d}. We present an example of WEM with Rs (q, d) below. Sq,m is the set of (mq)! permutations over m!q m m z }| { z }| { {1, 1, ..., 1, ..., q, q, ..., q}. We abuse the notation of def = [u0 , u1 , ..., uqm−1 ] to denote an element of uqm−1 0 Sq,m , which denotes the mapping i → ui . Example 3. A rewriting code for CRM with a maximal
rewriting cost constraint, (q, m, M, d), is defined by replacing X N by Sq,m , ϕ(·) by Chebyshev distance def between uqm−1 , v0qm−1 ∈ Sq,m , d∞ (uqm−1 , v0qm−1 ) = 0 0 max |uj − vj | , and N d by d in definition 1.
j∈{0,1,...,qm−1}
The above rewriting code is actually an instance of WEM: for x, y ∈ X , let ϕ(x, y) = 0 if |x − y| ≤ d, and ∞ otherwise. Now the (q, m, M, d) CRM is an (qm, M, q, 0) WEM with X N replaced by Sq,m , and ϕ(·) is defined previously. B. WEM with an average rewriting cost constraint −1 D(xN ) 0
R(x0N −1 , i),
With deterministic and suppose that message sequences M1 , M2 , ..., Mt (t → ∞) are written into the memory medium, where Mi ∈ D −1 is uniformly distributed, represented by xN (i), and it 0 forms a markov chain. Its transition probability matrix −1 µ(y0N −1 |xN )yN −1 ,xN −1 ∈X N is given by 0 0 0 ( −1 1 if ∃i s.t.y0N −1 = R(xN , i), −1 0 µ(y0N −1 |xN )= M 0 0 otherwise. −1 Denote the stationary distribution of xN by π(x0N −1 ). 0 ¯ is determined as follows: The average rewriting cost D t
¯ D
1 X −1 −1 E(ϕ(xN (i), xN (i + 1))), 0 0 t→∞ N t i=1 X 1 X −1 −1 µ(y0N −1 |xN ) π(xN ) = 0 0 N N −1 N −1 =
lim
x0
y0
−1 N −1 ϕ(xN , y0 ), X0 X −1 ¯ j (xN −1 ), π(xN ) D 0 0 N −1 j x
The binary polar WEM codes with an average rewriting cost constraint and a maximal rewriting cost constraint are presented in subsection A and B of section III, respectively; The q-ary polar WEM codes, based on the recently proposed q-ary polar codes [10], are presented in subsection A and B of section IV for an average rewriting cost constraint and a maximal rewriting cost constraint, respectively; The conclusion is obtained in section V. II. L OSSY SOURCE CODING AND ITS DUALITY WITH WEM In this section, we present briefly background of lossy source coding and its duality with WEM, which inspires code constructions for WEM. Let X also denote the variable space, and Y denotes the reconstruction space. Let d : Y × X → R+ denote the distortion function, and the distortion among a vector x0N −1 and its reconstructed vector y0N −1 is NP −1 −1 N −1 d(xi , yi ). d(xN , y0 ) = N1 0 i=0
A (q N R , N ) rate distortion code consists of a encoding function fN : X N → {0, 1, ..., q N R − 1} and a reproduction function gN : {0, 1, ..., q N R − 1} → Y N . The associated distortion is defined as E(d(X0N −1 , gN (fN (X0N −1 )))), where the expectation is with respect to the probability distribution on X N . R(q, D) is the infimum of rates R such that E(d(X0N −1 , gN (fN (X0N −1 )))) is at most D as N → ∞. def
(1)
Let P (q, D) = {PXY ∈ P(X × Y) : E(d(X, Y )) ≤ D}, and R(q, D) is min I(X; Y ) [4].
¯ j (xN −1 ) is the average rewriting cost of updatwhere D 0 −1 ing from xN to data j. 0
We focus on the double symmetric rate-distortion for (X, Y ) ∈ (X × X ), and d(x, y), RS (q, D), which is Rs (q, D) = min I(Y ; X), where s
=
0
PXY ∈P (q,D)
PXY ∈P (q,D)
Definition 4. An (N, M, q, d)ave WEM code is a collection of subsets, C = {Ci : i ∈ D}, where Ci ⊆ X N , −1 −1 and ∀xN ∈ Ci D(xN ) = i, such that 0 0 T • ∀i 6= j, Ci Cj = ∅; ¯ ≤ d. • The average rewriting cost D
P (q, D) = {PXY ∈ P(X × X ) : PX = PY , X ∼ U (q), E(d(X, Y )) ≤ D}. The duality between Rs (q, D) and Rs (q, D) is captured by the following lemma, the proof of which is ommited due to being straightforward.
The rewriting rate of an (N, M, q, d)ave code is Rave = logN2 M , and its rewriting capacity function, Rave (q, d), is the largest d-admissible rate when N → ∞. It is proved that Rave (q, d) = R(q, d) [1]. Similarly, we focus on the symmetric rewriting capacity function, Rsave (q, d), as defined in definition 2.
Lemma 5. With the same d(·) and ϕ(·),
C. The outline The connection between rate-distortion theorem and rewriting capacity theorem is presented in section II;
s
def
Rs (q, D) + Rs (q, D) = log2 q.
(2)
III. P OLAR CODES ARE OPTIMAL FOR BINARY WEM Inspired by lemma 5, we show that polar codes can be used to construct binary WEM codes with Rs (2, D) in a way related to the code construction for lossy source coding of [7] in this section.
A. A code construction for binary WEM with an average rewriting cost constraint 1) Background on polar codes [2]: Let W : {0, 1} → Y be a binary-input discrete memoryless channel for some output alphabet Y. Let I(W ) ∈ [0, 1] denote the mutual information between the input and output of W with a uniform distribution on the input. Let GN denote 1 0 . Let the Bhattacharyya n-th Kronecker product of P p 11 WY |X (y|0)WY |X (y|1). parameter Z(W ) = y∈X
The polar code, CN (F, uF ), ∀F ⊆ {0, 1, ..., N − 1}, uF denotes the subvector ui : i ∈ F , and uF ∈ {0, 1}|F | , is a linear code given by CN (F, uF ) = c −1 −1 {xN = uN GN : uF c ∈ {0, 1}|F | }. The polar 0 0 code ensemble, CN (F ), ∀F ⊆ {0, 1, ..., N − 1}, is CN (F ) = {CN (F, uF ), ∀uF ∈ {0, 1}|F | }. The secret of polar codes achieving I(W ) lies (i) in how to select F : define WN : {0, 1} → N i Y × {0, 1} as sub-channel i with input ui , output (y0N −1 , ui−1 and transition probabilities 0 ) P def (i) N −1 i−1 W N (y0N −1 |u0N −1 ), WN (y0 , u0 |ui ) = 2N1−1 −1 uN i+1 N −1 Q def
−1 where W N (y0N −1 |uN ) = 0
i=0
−1 W (yi |(uN GN )i ), 0
−1 and (uN GN )i denotes the i-th element of u0N −1 GN ; 0 (i) The fraction of {WN } that are approaching noiseless, β (i) i.e., Z(WN ) ≤ 2−N for 0 ≤ β ≤ 12 , approaches (i) I(W ); The F is chozen as indecies with large Z(WN ), β def (i) that is F = {i ∈ {0, 1..., N − 1} : Z(WN ) ≥ 2−N } for β ≤ 1/2. The encoding is done by a linear transformation, and the decoding is done by successive cancellation (SC). 2) Polar codes on lossy source coding: We sketch the binary polar code construction for lossy source coding [7] as follows. Note that Rs (2, D) can be obtained through the following optimization function:
min s.t.
: I(Y ; X), X1 X1 1 : P (y|x) = P (x|y) = , 2 2 2 x y XX 1 P (y|x)d(y, x) ≤ D. 2 x y
(3)
Let P ∗ (y|x) be the probability distribution minimizing the objective function of (3). P ∗ (y|x) plays the role of a channel. By convention, we call P ∗ (y|x) as test channel, and denote it as W (y|x). Now, we construct the source code with Rs (2, D) using the polar code for W (y|x), and denote the rate of
the source code by R: set F as N (1 − R) sub-channels (i) with the highest Z(WN ), set F c as the remaining N R sub-channels, and set uF to an arbitrary value. A source codeword y0N −1 is mapped to a codeword N −1 −1 x0 ∈ CN (F, uF ), and xN is described by the index 0 N −1 −1 uF c = (x0 GN )F c . The reproduction process is done as follows: we do −1 ˆ (y N −1 , uF ), that is for SC encoding scheme, u ˆN =U 0 0 each k in the range 0 till N − 1: 1) If k ∈ F , set u ˆk = uk ; 2) Else, set u ˆk = m with the posterior N −1 P (m|ˆ ui−1 , y ). 0 0 The reproduction codeword is u ˆ0N −1 GN . Thus, the average distortion DN (F, uF ) (over the source codeword y0N −1 and the encoder randomness for the code CN (F, uF )) is : X Y X N −1 P (y0N −1 ) P (ˆ ui |ˆ ui−1 ) 0 , y0 u ˆF c i∈F c
y0N −1
−1 d(y0N −1 , u ˆN GN ), 0
where u ˆF = uF . The expectation of DN (F, uF ) P over the uniform 1 D (F, uF ). choice of uF , DN (F ), is DN (F ) = 2|F | N uF
Let QU N −1 ,Y N −1 denote the distribution defined by 0 0 QY N −1 (y0N −1 ) = P (y0N −1 ), and QU N −1 |Y N −1 by 0 0 0 ( 1 , if i ∈ F, Q(ui |u0i−1 , y0N −1 ) = 2 i−1 N −1 P (ui |u0 , y0 ), otherwise. Thus, DN (F ) is equivalent to EQ (d(y0N −1 , u0N −1 GN )), where EQ (·) denotes the expectation with respect to the distribution QU N −1 ,Y N −1 . 0
0
β
It is proved that DN (F ) ≤ D+O(2−N )cfor 0 ≤ β ≤ |F | 1 2 , the rate of the above scheme is R = N , and polar codes achieve the rate-distortion bound by Theorem 3 of [7]. On the other hand, Theorem 2 of [3] further states the strong converse result of the rate distortion theory. More precisely, if y0N −1 is uniformly distributed over {0, 1}N , then ∀δ > 0, 0 < β < 21 , N sufficiently large, and with the above SC encoding process and the induced Q, β Q(d(u0N −1 GN , y0N −1 ) ≥ D + δ) < 2−N . That is, for N −1 −1 ∀y0 , the above reproduction process obtains xN = 0 N −1 N −1 N −1 u0 GN such that the distortion d(x0 , y0 ) is less than D almost by sure. 3) The code construction: We focus on the code construction with symmetric rewriting cost function, which satisfies ∀x, y, z ∈ {0, 1}, ϕ(x, y) = ϕ(x + z, y + z), where + is over GF(2).
To construct codes for WEM with Rs (2, D), we utilize its related form Rs (2, D) in (2) and the test channel W (y|x) for Rs (2, D). Note that W (y|x) is a binary symmetric channel. The code construction for (N, M, 2, D)ave with rate R is presented in Algorithm III.1: Algorithm III.1 A code construction for (N, M, 2, D)ave WEM 1: Set F as N R sub-channels with the highest (i) Z(WN ), and set F c as the remaining N (1 − R) sub-channels. 2: The (N, M, 2, D)ave code is C = {Ci : Ci = CN (F, uF (i))}, where uF (i) is the binary representation form of i for i ∈ {0, 1, ..., M − 1}. Clearly, since TGN is of full rank [2], ∀uF (i) 6= uF (j), CN (F, uF (i)) CN (F, uF (j)) = ∅. The rewriting operation and the decoding operation are defined in Algorithm III.2 and Algorithm III.3, respectively, where the dither g0N −1 is inspired by [3]. Algorithm III.2 The rewriting operation y0N −1 = −1 R(xN , i). 0 −1 Let v0N −1 = xN + g0N −1 , where g0N −1 is known 0 both to the decoding function and to the rewriting function, it is chosen such that v0N −1 is uniformly distributed over {0, 1}N , and + is over GF(2). N −1 2: SC encoding v0 , and this results u0N −1 = N −1 −1 ˆ (v U , uF (i)) and yˆ0N −1 = uN GN . 0 0 N −1 N −1 N −1 3: y0 = yˆ0 + g0 .
1:
ˆ (xN −1 + g N −1 , uF (i))GN + g N −1 ), which ϕ(x0N −1 , U 0 0 0 N −1 ˆ (xN −1 + g N −1 , uF (i))GN ) due to is ϕ(x0 + g0N −1 , U 0 0 ϕ(·) being symmetric. Denote w0N −1 = x0N −1 + g0N −1 , ˆ (wN −1 , uF (i))GN ). thus ϕ(x0N −1 , y0N −1 ) = ϕ(w0N −1 , U 0 ¯ The average rewriting cost D t
=
1 X −1 E(ϕ(xN (i), x0N −1 (i + 1))), 0 t→∞ N t i=1
=
1 X E(ϕ(w0N −1 , t→∞ N t i=1
lim
t
lim
ˆ (wN −1 , uF (Mi+1 ))GN )), U X0 X ¯ j (wN −1 ). = π(w0N −1 ) D 0 w0N −1
(4)
j
¯ j (wN −1 ), which is the average Let us focus on D 0 (in this case, over the probability of rewriting to data j and the randomness of the encoder) rewriting cost of updating w0N −1 to a codeword representing j. Thus ¯ j (wN −1 ) = D 0 1 X Y N −1 −1 P (ui |ui−1 )ϕ(w0N −1 , uN GN ). 0 , w0 0 |F 2 | u c i∈F c F
¯ is actually the Therefore, interpreting ϕ(·) as d(·), D average distortion over the ensemble CN (F ), DN (F ). ¯ The following lemma from [7] is to bound D: Lemma 7. [7] Let β < 12 be a constant and let σN = 1 −N β . When the polar code for the source code with 2N 2 Rs (2, D) is constructed with F : (i)
2 F = {i ∈ {0, 1, ..., N − 1} : Z(WN ) ≥ 1 − 2σN }, β
then DN (F ) ≤ D + O(2−N ), where D is the average rewriting cost constraint. Algorithm III.3 The decoding operation uF (i) = −1 D(xN ). 0 −1 y0N −1 = xN + g0N −1 . 0 N −1 −1 2: uF (i) = (y0 GN )F .
1:
Therefore, with the same β, σN , F, and polar code ¯ ≤ D + O(2−N β ). ensemble CN (F ), D |F c | According to [2], lim N = n N =2 ,n→∞
(i)
Lemma 6. D(R(y0N −1 , i)) = i holds for each rewriting. Proof: From the rewriting operation, −1 y0N −1 = yˆ0N −1 + g0N −1 = uN G + g0N −1 = N 0 N −1 N −1 ˆ (v U , uF (i))GN + g0 , from the decoding 0 ˆ (v N −1 , uF (i))GN + g N −1 + g N −1 , which is function U 0 0 0 N −1 ˆ (v U , uF (i))GN , thus the decoding result is i. 0 4) The average rewriting cost analysis: −1 From y0N −1 = R(xN , i), we know that 0 N −1 N −1 ˆ (v ˆ (xN −1 + y0 = U , u (i))G + g0N −1 = U F N 0 0 N −1 N −1 N −1 N −1 g0 , uF (i))GN + g0 , thus ϕ(x0 , y0 ) is
nβ
|{i ∈ {0, 1, ..., N } : Z(WN ) ≤ 2−2 }| N →∞ N I(W ) = Rs (2, D), lim
=
thus this implies that for N sufficiently large ∃ a set F |F c | such that N ≥ Rs (2, D) − , ∀ > 0. In other words, | the rate of the constructed WEM code, R = |F N = 1− |F c | s N ≤ R (2, D) + . The complexity of the decoding and the rewriting operation is of the order O(N log N ) according to [2]. We conclude the theoretical performance of the above polar WEM code as follows:
Theorem 8. For a binary symmetric rewriting cost function ϕ : X × X → R+ . Fix a rewriting cost D and 0 < β < 12 . For any rate R < Rs (2, D), there exists a sequence of polar WEM codes of length N and rate ¯ R ≤ R, so that under the above rewriting operation, D −N β ¯ satisfies D ≤ D+O(2 ). The decoding and rewriting operation complexity of the codes is O(N log N ). B. A code construction for binary WEM with a maximal rewriting cost constraint The code construction, the rewriting operation and the decoding operation are exactly the same as Algorithm III.1, Algorithm III.2, and Algorithm III.3, respectively. The rewriting capacity is guaranteed by Lemma 5, the decoding and rewriting operation complexity is the same as polar codes, and the rewriting cost is obtained due to the strong converse result of rate distortion theory, i.e., Theorem 2 of [3]. Thus, we have:
y ∈ Y, u ∈ {0, 1}r−k , and xP=p (x0 , x1 , ..., xr−1 ) ∈ X ; Define Z(W{x,x0 } ) = W (y|x)W (y|x0 ), let y∈Y P Zv (W ) = 21r Z(W{x,x+v} ) for v ∈ X \{0}, and x∈X P Zi (W ) = 21i Zv (W ), where i = 0, 1, ..., r − 1, and v∈Xi
Xi = {v ∈ X : i = arg max vj 6= 0} with the binary 0≤j≤r−1
representation of v, (v0 , v1 , ..., vr−1 ). (j) Zi (WN ) ∀i ∈ {0, 1, ..., r − 1} and j ∈ {0, 1, ..., N − 1} converges to Zi,∞ ∈ {0, 1}, and with probability one (Z0,∞ , Z1,∞ , ..., Zr−1,∞ ) takes one of the values (Z0,∞ = 1, ..., Zk−1,∞ = 1, Zk,∞ = 0, ..., Zr−1,∞ = 0) ∀k = 0, 1, ..., r − 1, i.e., Theorem 1.b of [10]. j ∈ Ak,n (j) (j) (j) iff (Z0 (WN ), Z1 (WN ), ...., Zr−1 (WN )) ∈ Rk (), r−1 Q Q def k−1 where Rk () = ( D1 ) × ( D0 ), D0 = [0, ), i=0
i∈{0,1,...,N −1}
Theorem 9. For a binary symmetric rewriting cost function ϕ : X × X → R+ . Fix a rewriting cost D, δ, and 0 < β < 12 . For any rate R < Rs (2, D), there exists a sequence of polar WEM codes of length N and rate R ≤ R, so that under the above rewriting operation and the induced probability distribution Q, the rewriting cost between a current codeword ∀y0N −1 and its updated −1 −1 codeword xN satisfies Q(ϕ(y0N −1 , xN ) ≥ D+δ) < 0 0 −N β 2 . The decoding and rewriting operation complexity of the codes is O(N log N ). IV. P OLAR CODES ARE OPTIMAL FOR q- ARY WEM, q = 2r In this section, we extend the previous binary scheme to q-ary WEM (q = 2r ), considering the length of polar codes, which is N = 2n . A. A code construction for q-ary WEM with an average rewriting cost constraint, q = 2r 1) Background of q-ary polar codes, q = 2r [10]: The storage alphabet is X , |X | = q, and for x ∈ X , (x0 , x1 , ..., xr−1 ) is its binary representation. Let W : X → Y be a discrete memoryless channel. I(W ) is P P 1 W (y|x) P 1 . q W (y|x) log2 W (y|x0 )
x∈X y∈Y
x0 ∈X
q
(i)
The sub-channel i, WN , is defined by P def (i) 1 WN (y0N −1 , ui−1 W N (y0N −1 |u0N −1 ), 0 |ui ) = q N −1 −1 uN i+1 N −1 def Q
−1 where W N (y0N −1 |uN ) = 0
i=0
−1 W (yi |(uN GN )i ). 0
Consider the following bit channel and its Bhattacharyya: Fix k P ∈ {0, 1, ..., r}, and 1 W [r−k] (y|u) = W (y|x), where 2k r−1 x:xk =u
i=k
D1 =P(1 − , 1]. The channel polarizes in the sense that |Aki ,n |
→ I(W ) as N → ∞. rN For u0N −1 ∈ X N , we also represent it by its binary form, that is u0N −1 = (uI(0,0) , ..., uI(0,r−1) , ..., uI(N −1,0) , ..., uI(N −1,r−1) ) ∈ {0, 1}rN , where uI(i,0) , ..., uI(i,r−1) is the binary representation of ui ∈ X , and I(i, j) = i × r + j. ∀i ∈ Aki ,n , let the frozen bit set be determined by F = {I(i, j) : i ∈ {0, 1, ..., N − 1}, j ∈ {0, 1, ..., ki − 1}} ⊆ {0, 1, ..., rN − 1}. Frozen bits for u0N −1 are defined as uF ∈ {0, 1}|F | with the subvector ui : i ∈ F . Finally, the polar code, CN (F, uF ), ∀F ⊆ {0, 1, ..., rN − 1} and uF ∈ {0, 1}|F | , is a linear code given by CN (F, uF ) = {x0N −1 = u0N −1 GN : c ∀uF c ∈ {0, 1}|F | }. The polar code ensemble, CN (F ), ∀F ⊆ {0, 1, ..., rN − 1}, is CN (F ) = {CN (F, uF ) : ∀uF ∈ {0, 1}|F | }. 2) The code construction: We focus on the q-ary WEM code construction with a symmetric rewriting cost function, which satisfies ∀x, y, z ∈ {0, 1, ..., q − 1}, ϕ(x, y) = ϕ(x + z, y + z), where + is over GF(q). Similar to the binary case, to construct codes for WEM with Rs (q, D), we utilize its related form Rs (q, D) in Lemma 5 and its test channel W (y|x). The code construction is presented in Algorithm IV.1. Algorithm IV.1 A code construction for (N, M, q, D)ave WEM 1: ∀i ∈ Aki ,n , F = {I(i, j) : i ∈ {0, 1, ..., N − 1}, j ∈ {0, 1, ..., ki − 1}}. 2: The (N, M, q, D)ave code is C = {Ci : Ci = CN (F, uF (i))}, where uF (i) is the binary representation form of i for i ∈ {0, 1, ..., M − 1}.
|F | , and the polar The WEM code rate is R = rN c |F | code rate is R = rN . Similarly, when R approaches Rs (q, D), R approaches Rs (q, D) based on Lemma 5. The rewriting operation and the decoding operation are defined in Algorithm IV.2 and Algorithm IV.3, respectively, where the SC encoding is a generalization of q-ary lossy source coding, and the dither g0N −1 is inspired by [3] to make sure the uniform distribution of v0N −1 .
Algorithm IV.2 The rewriting operation y0N −1 −1 R(xN , i). 0
=
−1 Let v0N −1 = xN + g0N −1 , where g0N −1 is known 0 both to the decoding function and to the rewriting function, it is chosen such that v0N −1 is uniformly distributed, and + is over GF(q). N −1 −1 ˆ (v N −1 , uF (i)), that 2: SC encoding v0 ,u ˆN =U 0 0 is for each k in the range 0 till N − 1: ( uj if j ∈ Ar,n , u ˆj = N −1 m with the posterior P (m|ˆ uj−1 ), 0 , v0
1:
where in the above m = 0, 1, ..., q − 1 and if j ∈ Ak,n for 0 ≤ k ≤ r − 1, the first k bits of u ˆj are −1 fixed, and let yˆ0N −1 = u ˆN G . N 0 N −1 3: y0 = yˆ0N −1 − g0N −1 , and − is over GF(q).
Ar,n is equal to if Aˆr,n 6= Ar,n , 0 Q i−1 N −1 P (ui |u0 , v0 ) otherwise, S i∈r−1 Ak,n k=0
where in the second case ∀i ∈ Ak,n , the first k bits of ui are fixed. Therefore, the average (in this case, over the probability of rewriting to data j and the randomness of the encoder) rewriting cost of updating w0N −1 to a codeword ¯ j (wN −1 ), is representing j, D 0 Y 1 X N −1 P (ui |ui−1 = ) 0 , w0 2|F | u c Sr−1 F
i∈
k=0
Ak,n
ϕ(w0N −1 , u0N −1 GN ), c
where uF = uF (j), uF c ∈ {0, 1}|F | , and the summation over uF c takes care of Ar,n and the fact that i ∈ Ak,n , the first k bits of ui are fixed. ¯ Thus, we obtain that D X X ¯ j (wN −1 ), D π(w0N −1 ) = 0 w0N −1
=
X
j
π(w0N −1 )
w0N −1
0, −1 N −1 |C(a, b|xN , y0 )/N − p(a, b)| < . Second, ∀a, b ∈ 0 −1 N −1 X × Y with p(a, b) = 0, C(a, b|xN , y0 ) = 0. 0 N −1 N −1 In our case y0 = u0 GN . Due to the full rank of −1 GN , there is a one-to-one correspondence between uN 0 ∗(N ) and y0N −1 . We say that u0N −1 , x0N −1 ∈ A (U, X) ∗(N ) −1 −1 if xN , uN GN ∈ A (X, Y ) with respect to 0 0 1 q W (y|x), where W (y|x) is the test channel. The first conclusion is that for N sufficiently large, β ∗(N ) Q(A (U, X)) > 1 − 2−N for ∀0 < β < 21 , > 0, which is a generalization of Theorem 4 ∗(N ) of [3], and where Q(A (U, X)) = Q(∀a, b :
−1 −1 GN , xN ) − 1q W (a|b)| ≤ ). The out| N1 C(a, b|uN R EFERENCES 0 0 line is that based on Lemma 10 and Lemma 11 we obtain [1] R. Ahlswede and Z. Zhang, “Coding for write-efficient memory,” Inform. Comput, vol. 83, no. 1, pp. 80–97, Oct 1989. that [2] E. Arikan, “Channel polarization: a method for constructing X −1 −1 capacity-achieving codes for symmetric binary-input memoryless |Q(uN , xN )−P (u0N −1 , x0N −1 )| 0 0 ∗(N )
−1 N −1 uN ,x0 ∈A 0
(U,X)
[3]
≤ |Ar,n |σN dmax . Thus, we obtain X −1 −1 −1 Q(uN , xN ) − P (uN , x0N −1 )| | 0 0 0 −1 N −1 uN ,x0 0
≤
X
−1 −1 −1 −1 |Q(uN , xN ) − P (uN , xN )|, 0 0 0 0
−1 N −1 uN ,x0 0
≤
[4] [5] [6] [7]
|Ar,n |σN dmax , ∗(N )
−1 −1 where in the above uN , xN ∈ A (U, X). 0 0 ∗(N ) By lower bounding P (A (U, X)) = 1 − P (∃a, b : −1 −1 | N1 C(a, b|uN GN , xN ) − 1q W (a|b)| ≥ ) ≥ 0 0 2 1 − 2q 2 e−2N based on Hoeffding’s inequality, and ∗(N ) ∗(N ) Q(A (U, X)) ≥ P (A (U, X)) − |Ar,n |σN dmax ,
[8]
[9]
−N β
2 we obtain the desired result by setting σN = 2N dmax . N −1 The second conclusion is that let y0 = R(x0N −1 , i), then ∀δ > 0, 0 < β < 12 and N sufficiently large, −1 Q(ϕ(y0N −1 , xN )/N ≥ D + δ) < 2−N β , which is a 0 generalization of Theorem 5 of [3]. The outline is that −1 N −1 Q(ϕ(xN , y0 )/N ≥ D + δ) 0
≤
−1 N −1 Q(ϕ(xN , y0 )/N ≥ D + δ 0 \ N −1 N −1 ) x0 , y0 ∈ A∗(N (X, Y ))
+
−1 N −1 ) Q(xN , y0 6∈ A∗(N (X, Y )), 0
≤
2−N β ,
where the last inequality is based on the conclusion just ∗(N ) −1 N −1 obtained Q(xN , y0 6∈ A (X, Y )) < 2−N β , and 0 ∗(N ) N −1 N −1 that when x0 , y0 ∈ A (X, Y ), for sufficiently small and N sufficiently large, ϕ(y0N −1 , x0N −1 )/N ≤ D + δ. V. C ONCLUSION Code constructions for WEM using recently proposed polar codes have been presented. Future work focuses on exploring error-correcting codes for WEM. VI. ACKNOWLEDGEMENT This work was supported in part by the National Science Foundation (NSF) CAREER Award CCF-0747415 and the NSF Grant CCF-1217944.
[10] [11]
channels,” IEEE Trans on Inf Theory, vol. 55, no. 7, pp. 3051– 3073, July 2009. D. Burshtein and A. Strugatski, “Polar write once memory codes,” in Proceedings of the 2012 IEEE international conference on Symposium on Information Theory (ISIT’12), Boston, MA, July 2012, pp. 1972–1975. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991. F. Fu and R. W. Yeung, “On the capacity and error-correcting codes of write-efficient memories,” IEEE Trans on Inf Theory, vol. 46, no. 7, pp. 2299–2314, Nov 2000. M. Karzand and E. Teletar, “Polar codes for q-ary source coding,” in Proc. IEEE International Symposium on Information Theory (ISIT), Austin, Texas, June 2010, pp. 909–912. S. B. Korada and R. Urbanke, “Polar codes are optimal for lossy source coding,” IEEE Trans. Inf. Theory, vol. 56, no. 4, pp. 1751– 1768, April 2010. L. A. Lastras-Montano, M. Franceschini, T. Mittelholzer, J. Karidis, and M. Wegman, “On the lifetime of multilevel memories,” in Proceedings of the 2009 IEEE international conference on Symposium on Information Theory (ISIT’09), Coex, Seoul, Korea, 2009, pp. 1224–1228. Q. Li, “Compressed rank modulation,” in Proc. 50th Annual Allerton Conference on Communication, Control and Computing (Allerton), Monticello, IL, October 2012. W. Park and A. Barg, “Polar codes for q-ary channels, q = 2r ,” in Proc. IEEE International Symposium on Information Theory (ISIT), Boston, MA, June 2012, pp. 2142–2146. R. L. Rivest and A. Shamir, “How to reuse a write-once memory,” Inf. Contr, vol. 55, pp. 1–19, 1982.
