Multilevel Diversity Coding with Regeneration
Multilevel Diversity Coding with Regeneration Chao Tian and Tie Liu March 3, 2015
arXiv:1503.00013v1 [cs.IT] 27 Feb 2015
Abstract Digital contents in large scale distributed storage systems may have different reliability and access delay requirements, and for this reason, erasure codes with different strengths need to be utilized to achieve the best storage efficiency. At the same time, in such large scale distributed storage systems, nodes fail on a regular basis, and the contents stored on them need to be regenerated and stored on other healthy nodes, the efficiency of which is an important factor affecting the overall quality of service. In this work, we formulate the problem of multilevel diversity coding with regeneration to address these considerations, for which the storage vs. repair-bandwidth tradeoff is investigated. We show that the extreme point on this tradeoff corresponding to the minimum possible storage can be achieved by a simple coding scheme, where contents with different reliability requirements are encoded separately with individual regenerating codes without any mixing. On the other hand, we establish the complete storage-repair-bandwidth tradeoff for the case of four storage nodes, which reveals that codes mixing different contents can strictly improve this tradeoff over the separate coding solution.
1
Introduction
The importance of big data analytics has been widely recognized in recent years, however, efficient large scale distributed data storage systems have to be designed and implemented in order to support the complete pipeline of data collection, processing and archival on a scale never seen before, where advanced coding techniques have been shown to play an important role. Digital contents in large scale distributed storage systems usually have different reliability requirements. For example, although it is important to protect recent customer billing records with a very reliable code, it may be acceptable to allow the data loss probability of a 5-year-old office document backup to be higher by using a relatively weak code. Moreover, it was shown recently that erasure codes can also be used to reduce data access latency, whose behavior depends on the coding parameters [1]. Thus, different levels of latency can also be integrated into the same data storage system by adopting different coding parameters for different contents. Such flexibility can significantly reduce the cost of hardware infrastructure, and there is tremendous interest recently both in industry and in academia to design efficient software defined storage (SDS) systems utilizing flexible erasure codes. The theoretical framework of symmetrical multilevel diversity (MLD) coding [2, 3] is a natural fit for this scenario, where there are a total of k0 ≤ n independent messages (M1 , M2 , ..., Mk0 ) to be stored, and the overall system consists of n storage nodes situated in different network locations, each with α units of data. The messages are coded in such a way that by accessing any k ≤ k0 of these nodes, the first k messages (M1 , M2 , ..., Mk ) can be completely recovered. Disk or node failures occur regularly in a large scale storage system, and the overall quality of service is heavily affected by the efficiency of the repair process. Dimakis et al. [4] proposed the framework of regenerating codes to address the tradeoff between the storage and repair-bandwidth in (n, k) erasurecoded distributed storage systems. To repair a node, a new node replacing the failed one requests β units of data each from any of the d remaining nodes, and regenerates the α units of content to store on the new node; this code is referred to as the (n, k, d) regenerating code. There exists a natural tradeoff between 1
the storage α and the repair bandwidth β: the point corresponding to the minimum amount of the storage is referred to as the minimum storage regenerating (MSR) point, and the other extreme is referred to as the minimum repair-bandwidth regenerating (MBR) point. In [4], the content regenerated is allowed to be only functionally equivalent to the content on the failed node, thus the name “functional-repair” regenerating codes. In practice, requiring the content regenerated to be exactly the same as that on the failed node can simplify the system design significantly, and thus recent research effort has focused on “exact-repair” regenerating codes [5–11]. In the regenerating code framework, only a single message is allowed, and thus offering only a single level of reliability and access latency; on the other hand, in the MLD coding framework, the data repair process was not considered. In this work, we consider repair-efficient codes in systems with heterogeneous reliability and latency requirements, and investigate the optimal storage vs. repair-bandwidth tradeoff. Because of the connection to the MLD coding problem and the regenerating code problem, we refer to this problem as multilevel diversity coding with regeneration (MLD-R) in the sequel; see Fig. 1. We shall restrict our attention to the case of exact-repair, and furthermore to the case when d = n−1, because this is the most practically important case. Nevertheless, the framework can be straightforwardly generalized to other relevant settings. An intuitive and straightforward coding strategy is to use individual regenerating code for each message to satisfy the respective reliability and latency requirement (i.e., separate coding), and thus the most important question we must answer first is whether it is even beneficial to consider codes that “mix” the messages. Without the repair consideration, it was shown in [2, 3] that no mixing is necessary for the symmetrical MLD coding problem. We show that for the minimum storage point on the optimal tradeoff where α is minimized (analogous to the MSR point in regenerating codes), the aforementioned separate coding is again sufficient. On the other hand, we show for n = 4, by providing a novel code construction, that mixing can strictly improve upon the performance of the separate coding solution in terms of overall storage-repair-bandwidth tradeoff. In fact, we completely characterize this tradeoff by establishing its converse. It is worth noting that when n = 3, separate coding is in fact sufficient, thus n = 4 is the smallest non-trivial case where the benefit of mixing manifests. One difficulty in establishing these results is in deriving the outer bounds. For the minimum storage point, we utilize a recursive bounding technique which may be of independent interest. The converse for the tradeoff rate region when n = 4 is rather difficult to identify and derive manually, and thus we utilize the computational approach developed in [11]. This proof is thus presented in tables whose rows are simple known information inequalities, and the summation of the rows are precisely the outer bounds. Though this does not conform to the conventional proof approach of using chains of information inequalities in information theory literature, we believe that these tables are in a sense more fundamental. Indeed, we can write down many different versions of chains of inequalities with the help of these tables, by taking different orders when applying these individual inequalities. The rest of the paper is organized as follows. A formal problem formulation and some preliminaries are given in Section 2. In Section 3, the main results of the paper are presented together with the relevant discussion. The proofs are given in Section 4 and Section 5. Section 6 concludes the paper with a few future research directions. Several technical proofs are given in the Appendix.
2 2.1
Problem Formulation and Preliminaries Problem Formulation
An MLD-R code is defined below, where In is used to denote the set {1, 2, . . . , n}, and |A| is used to denote the cardinality of a set A. It can be assumed without loss of generality d = k0 , where k0 is the number of messages. This is because if d < k0 , then the messages (Md+1 , Md+2 , ..., Mk0 ) can be viewed as part of Md ; on the other hand if d > k0 , we can simply consider an alternative problem with k 0 0 = d and define the messages Mk0 , Mk0 +1 , ..., Md to be degenerate. Recall in this work we assume d = n − 1. 2
Node 1
α α
M
α α
Node 1
Node 1'
β Node 2
α α
β
α α
M1, M 2 ,..., M L Node 3
β
Node 1
α α
Node 2
M1, M 2 ,..., M L
k nodes Node 3
α α
Node 2
Node 4
M1, M2 ,...,Mk Node n
Node n
Node 4
M1, M2 ,...,Mk
...
α
...
...
α
β decoder
...
...
... α
β k nodes
Node 3
decoder Node 4
Node 1'
β
Node n
Figure 1: Multilevel diversity coding with regeneration. P Definition 1. An (N1 , N2 , ..., Nd , Kd , K) MLD-R code consists of n encoding functions fiE (·), di=1 ni E (·), and n repair decoding functions decoding functions fAD (·, ..., ·), n(n − 1) repair encoding functions Fi,j D Fi (·, ..., ·), where fiE : IN1 × IN2 × ... × INd → IKd ,
i ∈ In ,
each of which maps the message (M1 , M2 , ..., Md ) ∈ IN1 × IN2 × ... × INd to one piece of coded information, fAD : IKd × IKd × ... × IKd → IN1 × IN2 × ... × IN|A| , A ⊂ In
and
|A| = 1, 2, . . . , d,
each of which maps coded information stored on a set A of nodes to the first |A| messages (M1 , M2 , ..., M|A| ), E Fi,j : IKd → IK ,
j ∈ In ,
and
i ∈ In \ {j},
each of which maps a piece of coded information at node i to an index that is made available to node j to reconstruct the coded data at node j, and FjD : IK × IK × ... × IK → IKd ,
j ∈ In ,
each of which maps d such indices from the helper nodes to reconstruct the information stored at the failed node j. The functions must satisfy the data reconstruction conditions ! Y fAD fiE (M1 , M2 , ..., Md ) = (M1 , M2 , ..., M|A| ), i∈A
(M1 , M2 , ..., Md ) ∈ IN1 × IN2 × ... × INd , A ⊂ In and |A| = 1, 2, . . . , d,
(1)
and the repair conditions FjD
Y
E Fi,j fiE (M1 , M2 , ..., Md )
i∈In \{j}
= fjE (M1 , M2 , ..., Md ), (M1 , M2 , ..., Md ) ∈ IN1 × IN2 × ... × INd ,
j ∈ In .
(2)
Note α = log Kd is the storage node capacity, β = log K is the per-helper-node repair bandwidth, and Bi = log Ni is the rate of the i-th message; the base of log(·) is arbitrary, and we choose base 2 for convenience. Because of the linear scaling relation among them, we can alternatively consider the normalized version as follows. 3
¯ B ¯1 , B ¯2 , ..., B ¯d ) is said to Definition 2. A normalized storage-repair-bandwidth-message-rate tuple (¯ α, β, Pd ¯ be achievable with n nodes where j=1 Bj = 1, if there exists an (N1 , N2 , ..., Nd , Kd , K) MLD-R code such that log Kd log K α ¯ ≥ Pd , β¯ ≥ Pd , i=1 log Ni i=1 log Ni ¯j = P log Ni , j = 1, 2, ..., d. B d i=1 log Ni ¯ B ¯1 , B ¯2 , ..., B ¯d ) tuples is the achievable storage-repairThe convex closure of all the achievable (¯ α, β, ¯ ¯ ¯d ) tuple, the achievable storage-repairbandwidth-message-rate tradeoff region R. For a fixed (B1 , B2 , ..., B ¯ ¯ B ¯1 , B ¯2 , ..., B ¯d ) ∈ R, which is bandwidth tradeoff region is the collection of all (¯ α, β) pairs such that (¯ α, β, ¯ ¯ ¯ denoted as Rn (B1 , B2 , ..., Bd ). The codes and the tradeoff regions do not involve any particular assumption on the distribution of the messages, however, without loss of generality we can assume that the messages M1 , M2 , . . . , Md are mutually independent and uniformly distributed, since otherwise we can perform a pre-coding to eliminate any dependence and non-uniformity. Sometimes it is convenient to use the accumulative sum rates instead of individual rates, and thus we define ¯+ , B k
k X
¯i , B
k = 1, 2, . . . , d.
(3)
i=1
The corresponding accumulative message (M1 , M2 , . . . , Mk ) is sometimes also written as Mk+ . Note that ¯ + = 1, however we often still write B ¯ + for convenience. this definition implies B d d The reconstruction condition (1) requires that there is no decoding error, i.e., the zero-error requirement is adopted. An alternative definition is to require instead the probability of decoding error to be asymptotically zero as Πdi=1 Ni → ∞. It will become clear that this does not cause any essential difference, and thus we do not give this alternative definition. When deriving outer bounds, we use Si→j to denote the helper (random) information sent from node-i to node-j, and Wi to denote the coded data stored on node-i. The random variable vector (X1 , X2 , . . . , Xm ) is sometimes written as X1m .
2.2
Separate Coding
One straightforward coding strategy is to encode each individual message using a regenerating code of the necessary strength. More precisely, the message Mk is encoded using an (n, k, d) regenerating code (i.e., any k nodes can recover the message Mk , and any new node obtains information from any d nodes for repair) of rate (αk , βk ), and the resultant code thus has storage rate and repair rate: α=
d X
αk ,
β=
k=1
d X
βk .
k=1
¯1 , B ¯2 , ..., B ¯d ), tradeoff ˜ n (B In order to characterize this separate coding tradeoff region, denoted as R region characterizations of individual regenerating codes are needed; e.g., for the case of n = 3, tradeoff region characterizations for (3, 1, 2) and (3, 2, 2) regenerating codes are needed. However, such characterizations for general parameters are still unknown, except for the case of k = 1, 2, and the special case (n, k, d) = (4, 3, 3) only recently established in [11]. Fortunately, using these existing results, we can provide a characterization of the separate coding tradeoff region for MLD-R when n = 3, denoted as ˜ 3 (B ¯1 , B ¯2 ), and that when n = 4, denoted as R ˜ 4 (B ¯1 , B ¯2 , B ¯3 ). R 4
˜ 3 (B ¯1 , B ¯2 ) achievable using separate coding Proposition 1. The storage-repair-bandwidth tradeoff region R ¯ is the set of (¯ α, β) pairs satisfying the following conditions ¯1 + α ¯≥B
¯2 B , 2
¯1 3B ¯2 , +B 2
α ¯ + β¯ ≥
¯1 B ¯2 B β¯ ≥ + . 2 3
˜ 4 (B ¯1 , B ¯2 , B ¯3 ) achievable using separate Proposition 2. The storage-repair-bandwidth tradeoff region R ¯ coding is the set of (¯ α, β) pairs satisfying the following conditions ¯ ¯ ¯1 + B2 + B3 α ¯≥B 2 3 ¯ 1 5B ¯2 7 B ¯3 2¯ α + β¯ ≥ + +B 3 4 ¯ ¯1 + 7B2 + 3B ¯3 4α ¯ + 6β¯ ≥ 6B 2 ¯1 5B ¯2 + 5 B ¯3 α ¯ + 2β¯ ≥ +B 3 6 ¯1 1 B ¯2 + 1 B ¯3 . β¯ ≥ + B 3 5 6
(4) (5) (6) (7) (8)
The proofs for these propositions are given in the Appendix.
3
Main Results
¯1 , B ¯2 , ..., B ¯d ) Our first main result is a characterization of the extreme point in the tradeoff rate region Rn (B where α ¯ is minimized, through the following theorem. ¯ ∈ Rn , we have Theorem 3. For any (¯ α, β) (n − 2)α ¯ + β¯ ≥
n−1 X
(n − 2)(n − k) + 1 ¯ Bk . k(n − k)
k=1
(9)
It was shown in [2] that for MLD coding without repair considerations, the minimum achievable rate is given as α ¯=
n−1 X k=1
¯k B , k
(10)
using separate coding. Thus Theorem 3 and the existence of (n, k, d) exact-repair regenerating codes at the MSR point [8] imply the following corollary. ¯ ∈ Rn Corollary 1. For the minimum storage point on the boundary of the tradeoff rate region (¯ α, β) where α ¯=
n−1 X
¯k B , k
(11)
¯k B , k(n − k)
(12)
k=1
the minimum repair bandwidth is given as β¯ =
n−1 X k=1
and it can be achieved by separately coding of each message Mk with an (n, k, d) exact-repair MSR code. 5
¯1 , B ¯2 , B ¯3 ). ¯1 , B ¯2 ) and R4 (B The next two theorems provide complete characterizations for R3 (B ¯1 , B ¯2 ) = R ˜ 3 (B ¯1 , B ¯2 ). Theorem 4. R3 (B ˜ 3 (B ¯1 , B ¯2 ) ⊆ R3 (B ¯1 , B ¯2 ) is straightforward, and the converse direction is The forward direction R proved in the Appendix. This theorem states that for the case of n = 3, the strategy of separate coding is optimal, and there is no need to mix the messages. However, our next result shows that this is in general not true, and mixing the messages is beneficial. For this purpose, we establish Theorem 5, whose proof is provided in Section 5. ¯ ¯1 , B ¯2 , B ¯3 ) is the collection of all (¯ Theorem 5. The storage-repair-bandwidth tradeoff region R4 (B α, β) pairs that satisfy the following constraints: ¯ ¯ ¯1 + B2 + B3 α ¯≥B 2 3 5 7 ¯1 + B ¯2 + B ¯3 2¯ α + β¯ ≥ B 3 4 3¯ 5¯ 4¯ α ¯ + β¯ ≥ B 1 + B2 + B3 3 4 8 5 3 ¯1 + B ¯2 + B ¯3 2α ¯ + 3β¯ ≥ 3B 3 2 5¯ 5¯ ¯ α ¯ + 2β¯ ≥ B 1 + B2 + B3 3 6 1¯ 1¯ 1¯ ¯ β ≥ B1 + B2 + B 3. 3 5 6
(13) (14) (15) (16) (17) (18)
˜ 4 (0, 1 , 2 ) are depicted in Fig. 2 for the case B ¯2 = 1 and B ¯3 = 2 . It can The regions R4 (0, 13 , 23 ) and R 3 3 3 3 ˜ 4 (0, 1 , 2 ) ⊂ R4 (0, 1 , 2 ) is strict, thus generally, mixing of contents in MLD-R be seen that the inclusion R 3 3 3 3 can be beneficial. We formally state it as it a corollary. ˜ 4 (B ¯1 , B ¯2 , B ¯3 ) ( R4 (B ¯1 , B ¯2 , B ¯3 ). Corollary 2. R
4
The Minimum Storage Point: Proof of Theorem 3
It can be shown that we only need to consider symmetric codes, where permutations of node indices do not change the joint entropy values; see [11] for more details about this symmetry. Thus without loss of generality, we can restrict the proof to symmetric codes only. Before presenting the proof of Theorem 3, we first give an auxiliary lemma. This lemma plays a similar instrumental role in the proof of Theorem 3 as Han’s inequality in the converse proof of symmetrical MLD coding problem given in [2]. Lemma 1. For any integer ` such that 1 ≤ ` ≤ d − 1 and any symmetric MLD-R code with a total of n nodes n−1 n−`−1 H(W1` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) `(n − `) n−` ≥ B`+1 + H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1`+1 ).
(19)
Proof. Let ` be an integer such that 1 ≤ ` ≤ d − 1. We start by writing H(W1` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) (s)
= H(W2`+1 |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` )
= H(S`+1→1 , W2`+1 |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) ≥ H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1` ) + H(S`+1→1 , W2` |M1` ) ≥ B`+1 + H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1`+1 ) + H(S`+1→1 , W2` |M1` ), 6
(20)
MLD−R rate region separate coding rate region
0.34 0.32 (7/18, 11/36)
0.3 0.28
β¯0.26
(5/12, 1/4 )
0.24 0.22
(1/2 , 7/36 ) (4/9, 2/9)
0.2 0.18
(8/15, 8/45 )
0.16 0.35
0.4
0.45
α ¯
0.5
0.55
0.6
˜ 4 (0, 1 , 2 ) using separate coding, and the tradeoff rate region R4 (0, 1 , 2 ). Figure 2: The rate region R 3 3 3 3 where we write (s) to denote for the reason of symmetry, the first inequality in the chain is because H(W`+1 |M1` , S`+1→1 , W2` ) + H(S`+2→1 , . . . , Sn→1 , W1 |M1` , S`+1→1 , W2` ) ≥ H(S`+2→1 , . . . , Sn→1 , W1 , W`+1 |M1` , S`+1→1 , W2` ) = H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1` , S`+1→1 , W2` ),
(21)
and the second inequality is by data reconstruction requirement (1). However notice that (n − `)H(S`+1→1 , W2` |M1` ) = (n − `)H(W2` |M1` ) + (n − `)H(S`+1→1 |M1` , W2` ) n X (s) = (n − `)H(W2` |M1` ) + H(Si→1 |M1` , W2` ) ≥ (n − = (n −
i=`+1 ` ` `)H(W2 |M1 ) + H(S`+1→1 , . . . , Sn→1 |M1` , W2` ) `)H(W2` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1 |M1` , W2` ) ` − 1)H(W2` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` )
= (n − (n − ` − 1)(` − 1) ≥ H(W1` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ), `
(22)
where in the last step we applied Han’s inequality, and also used the symmetry property. Putting (20) and (22) together gives H(W1` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) ≥ B`+1 + H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1`+1 ) + +
H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) , n−` 7
(n − ` − 1)(` − 1) H(W1` |M1` ) `(n − `) (23)
which has common terms on both sides that can be eliminated, and this leads to exactly the inequality in the lemma. In the proof of the lemma above, after several steps of derivation, the same terms in the original quantity reappear, albeit with different coefficients. The same terms are subtracted on both sides of the inequality, which can be conceptually viewed as recursively applying the same chains of inequalities. We are now ready to present the proof of Theorem 3. Proof of Theorem 3. The theorem is proved through an induction, where we show that for m = 1, 2, . . . , d, m X (n − 2)(n − k) + 1 n−2 m−1 (n − 2)α + β ≥ H(W1m |M1m ) Bk + − k(n − k) m m(n − m) k=1
+
H(Sm+1→1 , . . . , Sn→1 , W1m |M1m ) . n−m
(24)
The theorem isPthen simply a consequence of this statement when setting m = d = n − 1 and normalizing both sides by n−1 k=1 Bk , and taking into account of the facts n−2 m−1 = 0, (25) − m m(n − m) and H(Sm+1→1 , . . . , Sn→1 , W1m |M1m ) ≥ 0.
(26)
To show (24) is true for m = 1, we write the following chain of inequalities (n − 2)α + β ≥ (n − 2)H(W1 ) + H(S2→1 ) (n − 1)H(S2→1 ) = (n − 2)H(W1 ) + n−1 H(S2→1 , S3→1 , . . . , Sn→1 ) ≥ (n − 2)H(W1 ) + n−1 H(S2→1 , S3→1 , . . . , Sn→1 , W1 ) ≥ (n − 2)B1 + (n − 2)H(W1 |M1 ) + n−1 B1 H(S2→1 , S3→1 , . . . , Sn→1 , W1 |M1 ) = (n − 2)B1 + (n − 2)H(W1 |M1 ) + + n−1 n−1 (n − 2)(n − 1) + 1 H(S2→1 , S3→1 , . . . , Sn→1 , W1 |M1 ) = B1 + (n − 2)H(W1 |M1 ) + . n−1 n−1
(27)
Applying Lemma 1 now with ` = 1 gives (n − 2)(n − 1) + 1 1 B1 + (n − 2) − H(W1 |M1 ) (n − 2)α + β ≥ n−1 n−2 B2 H(S3→1 , . . . , Sn→1 , W1 , W2 |M1 , M2 ) + + . n−2 n−2
(28)
Thus (24) is true for m = 1. Next suppose (24) is true m = m0 , and we wish to show it is also true for m = m0 + 1. Notice that by Lemma 1 with ` = m0 , we have n−1 H(Sm0 +1→1 , . . . , Sn→1 , W1m0 |M1m0 ) H(W1m0 |M1m0 ) + m0 (n − m0 )(n − m0 − 1) n − m0 1 1 ≥ Bm0 +1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ). n − m0 − 1 n − m0 − 1 8
(29)
Thus (n − 2)α + β ≥
m0 X (n − 2)(n − k) + 1 k=1
k(n − k)
n−2 m0 − 1 Bk + − H(W1m0 |M1m0 ) m0 m0 (n − m0 )
H(Sm0 +1→1 , . . . , Sn→1 , W1m0 |M1m0 ) n − m0 m0 X (n − 2)(n − k) + 1 m0 − 1 n−2 n−1 ≥ − H(W1m0 |M1m0 ) Bk + − k(n − k) m0 m0 (n − m0 ) m0 (n − m0 )(n − m0 − 1) +
k=1
1 1 Bm0 +1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ) n − m0 − 1 n − m0 − 1 m0 X 1 (n − 2)(n − k) + 1 n−2 − H(W1m0 |M1m0 ) = Bk + k(n − k) m0 n − m0 − 1 +
k=1
1 1 Bm0 +1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ) n − m0 − 1 n − m0 − 1 m0 X (n − 2)(n − k) + 1 1 n−2 m0 ≥ Bk + − H(W1m0 +1 |M1m0 ) k(n − k) m0 n − m0 − 1 m0 + 1 +
k=1
1 1 Bm0 +1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ) n − m0 − 1 n − m0 − 1 m0 X (n − 2)(n − k) + 1 n−2 m0 1 = Bk + − + Bm0 +1 k(n − k) m0 + 1 (n − m0 − 1)(m0 + 1) n − m0 − 1 k=1 n−2 m0 + − H(W1m0 +1 |M1m0 +1 ) m0 + 1 (n − m0 − 1)(m0 + 1) 1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ), n − m0 − 1 +
(30)
which is precisely (24) for m = m0 + 1 after some elementary algebra. This completes the proof of Theorem 3. Readers familiar with the converse proof for the symmetric MLD coding problem may recognize certain similarity between the above proof and that in [2]. The key difference is that in the converse proof for MLD coding, one only needs to “peel” off the message rates by combining information in Wi ’s sequentially. Here the regeneration requirement necessitates a more elaborate peeling process, and Lemma 1 serves a similar role as Han’s inequality in the proof in [2].
5
¯1 , B ¯2 , B ¯3 ): Proof of Theorem 5 The Rate Region R4 (B
In this section the proof of Theorem 5 is presented. We start by providing a new code construction that achieves a particular operating point, which will play an instrumental role in the proof of Theorem 5.
5.1
A New Code
We first give a proposition, the proof of which requires a new code construction. Proposition 6. The rate pair ( 49 , 29 ) ∈ R4 (0, 31 , 23 ). Proof. We give a novel code construction where B1 = 0, B2 = 3, B3 = 6, α = 4 and β = 2. For concreteness, the code symbols and algebraic operations are assumed in GF(24 ). Let us denote the information symbols of message M2 as (x1 , x2 , x3 ), and the symbols of message M3 as (y1 , y2 , ..., y6 ). 9
Table 1: A code for n = 4 where (α, β) = (4, 2) and (B1 , B2 , B3 ) = (0, 3, 6). symbol 1 symbol 2 symbol 3 symbol 4 node 1 z1 z 5 + y1 y2 y3 node 2 z2 z 6 + y4 y1 y5 node 3 z3 z 7 + y2 y6 + z10 y4 node 4 z4 z 8 + y3 y5 + z9 y6 Encoding: First use a (10, 3) MDS erasure code (e.g., Reed-Solomon code) to encode (x1 , x2 , x3 ) into ten coded symbols (z1 , z2 , ..., z10 ), such that any three symbols can completely recover (x1 , x2 , x3 ). Then place linear combinations of (y1 , y2 , ..., y6 ) and (z1 , z2 , ..., z10 ) into the nodes as in Table 1, where the addition + is also in GF(24 ). Decoding M2 using any two nodes: To decode M2 , observe that any pair of nodes has two symbols involving the same yj , in the form of (zi + yj ) in one node and yj in the other node. For example node 2 has symbol z6 + y4 and node 3 has y4 . This implies zi can be recovered, and together with the first symbols stored in this pair of nodes, we have three distinct symbols in the set {z1 , z2 , ..., z10 }. Thus by the property of the MDS code, these three symbols can be used to recover (x1 , x2 , x3 ) and thus the message M2 . Decoding M2 and M3 using any three nodes: Recall using any two nodes we can recover the message M2 , and thus all the code symbols (z1 , z2 , ..., z10 ). This implies that when three nodes are available, we can eliminate all the zi symbols first in the linear combinations. However, it is clear that after this step all the symbols (y1 , y2 , ..., y6 ) are directly available, and thus the message M3 can be decoded. Repair using any three nodes: To regenerate the symbols in one node from the other three, each of the helper nodes sends the first symbol stored on the nodes as the initial step. Denote the y symbols on the failed node as (yi , yj , yk ), which may be stored in a form also involving z-symbols. The helper nodes each find in the symbols stored on it the one involving yi , yj and yk , respectively, and send these symbol combinations as the second step. The placement of the y-symbol guarantees that these symbols are stored on the three helper nodes respectively. Recall that from any three z-symbols available in the initial step, the message M2 can recovered, and thus any of the z-symbols. This implies that (yi , yj , yk ) can be recovered after eliminating the z symbols from the received symbol combinations in the second step, and thus all the symbols on the failed node can be successfully regenerated. Each helper node contributes exactly 2 symbols in this process. In the above code the linear combinations of z-symbols and y-symbols are not necessarily in GF(24 ), and it can even be in the base field GF(2). The only constraint on the alphabet size is through the MDS erasure code used when encoding (x1 , x2 , x3 ), and we have chosen GF(24 ) for simplicity. Readers familiar with the work [6] may recognize that if message M2 = (x1 , x2 , x3 ) does not exist, the remaining code involving y’s is a degenerate case of the repair-by-transfer code in [6]. Mixing improves the storage-repair-bandwidth tradeoff, however it has the side effect of possibly requiring downloading more symbols than the non-mixing solution when only a single message is required. Interestingly, in the above code, this side effect can be eliminated by downloading only the “non-mixing” symbols when all nodes are healthy.
5.2
Forward Proof of Theorem 5
¯1 , B ¯2 , B ¯3 ) for To prove the forward part of Theorem 5, we examine the extremal points of the region R4 (B ¯ ¯ ¯ ¯ ¯ ¯ fixed (B1 , B2 , B3 ). It should be noted that for different values of (B1 , B2 , B3 ), the extremal points may be different which causes certain complication. The intersecting points of any two inequalities in (13)-(18),
10
Table 2: Analysis of extremal points. α ¯ β¯ (14) (15) (13)
(16)
¯1 + B
¯2 B 2
+
¯3 B 3
(17) (18) (15) (14)
(16) (17) (18) (16)
(15)
(17) (18)
(16)
(17) (18)
(17)
(18)
¯1 + B¯2 + 3B¯3 B 2 8 ¯1 + 25B¯2 + 3B¯3 B 48
8
¯1 + B¯2 + 7B¯3 B 2 18 ¯2 ¯ 21B ¯ B1 + + 5B3 40
12
¯1 + 7B¯2 + 3B¯3 B 12 8 ¯1 + B¯2 + 5B¯3 B
2 12 ¯1 + 11B¯2 + 11B¯3 B 20 24 ¯1 + B¯2 + B¯3 B 3 2 ¯2 ¯ 8B B ¯ B1 + 15 + 23 ¯1 + 3B¯2 + B¯3 B 5 2
¯2 ¯3 ¯1 B B B 3 + 4 + 3 ¯1 ¯2 ¯3 B B 7B 3 + 4 + 24 ¯2 ¯3 ¯1 2B 5B B 3 + 9 + 18 ¯1 ¯2 ¯3 B B B 3 + 4 + 4 ¯1 ¯2 ¯3 B B B 3 + 5 + 6 ¯2 ¯3 ¯1 B B B 3 + 4 + 4 ¯1 ¯2 ¯3 B 5B B 3 + 24 + 4 ¯1 ¯2 ¯3 B B 2B 3 + 4 + 9 ¯1 ¯2 ¯3 B B B 3 + 5 + 6 ¯1 ¯2 ¯3 B B B 3 + 6 + 4 ¯1 ¯2 ¯3 B B 5B 3 + 4 + 24 ¯1 ¯2 ¯3 B B B 3 + 5 + 6 ¯1 ¯2 ¯3 B B B 3 + 3 + 6 ¯1 ¯2 ¯3 B B B 3 + 5 + 6 ¯1 ¯2 ¯3 B B B 3 + 5 + 6
remark X
(14)
X (15) (a) (b) (c) (d) (17) X
¯1 , B ¯2 , B ¯3 ). The first two columns specify when taken to be equality, are listed in Table 2 for any fixed (B which two inequalities induce the intersection. The intersecting points with X in the last column can be easily seen to be achievable using the separate coding scheme. The points labeled using (14), (15) and (17) in fact violate those constraints, and thus cannot be the extremal points. It remains to analyze the points (a)-(d), which may violate some ¯1 and B ¯2 values. constraints for certain B ¯2 ≤ B ¯3 . Due • For point (a): by the inequality (17), point (a) can be an extremal point only when 2B to the polytope structure of the region, we only need to check that the two extreme cases when ¯2 = B ¯3 , the rate point (a) is indeed achievable. The first case is trivial, since ¯2 = 0 and the 2B B it can be achieved by separate coding with only (4, 1, 3) and (4, 3, 3) components. For the second ¯1 , B ¯2 , 2B ¯2 ), however, we can encode the message M1 using a case, observe we need a code for (B B1 (4, 1, 3) code at rate point (α1 , β1 ) = (B1 , 3 ), and the messages M2 and M3 using the code given in Proposition 6, at (α2,3 , β2,3 ) = ( 4B3 2 , 2B3 2 ); superposition of the two codes leads to the the desired operating point. ¯2 ≥ B ¯3 . We • For point (b): by the inequality (16), point (b) can be an extremal point only when 2B ¯ ¯ ¯ only need to check that the two extreme cases when B3 = 0 and 2B2 = B3 , the rate point (b) is indeed achievable. The first case can be achieved by separate coding with only (4, 1, 3) and (4, 2, 3) components, while the second case reduces to the second case of point (a). • For point (c): by the inequality (16) and (17), point (c) can be an extremal point only when ¯2 ¯2 2B 6B ¯ ¯ ¯ 2 ≥ B3 ≥ 5 , which can hold only when B2 = B3 = 0. This is trivially achievable using a (4, 1, 3) code. ¯2 ≤ B ¯3 . The • For point (d): by inequality (16), point (d) can be an extremal point only when 2B ¯2 = 0 and 2B ¯2 = B ¯3 , which is again achievable by separate coding with two extreme cases are B only (4, 1, 3) and (4, 3, 3) components, and by the code given for the second case for point (a). 11
Table 3: Terms needed to prove Proposition 7 T1 H(S4→3 ) T2 H(S4→2 , S3→2 ) T3 H(S4→1 , S3→1 , S2→1 ) T4 H(W4 ) T5 H(S3→2 , W4 ) T6 H(S3→4 , S2→4 , W4 ) T7 H(W4 , W3 ) T8 H(S2→4 , W4 , W3 ) T9 H(S3→1 , S2→1 , W4 , W1 ) T10 H(S4→3 , M1+ ) T11 H(S4→2 , S3→2 , M1+ ) T12 H(W4 , M2+ ) T13 H(S3→2 , W4 , M2+ ) T14 H(S3→4 , W4 , M2+ ) T15 H(S3→2 , S2→4 , W4 , M2+ ) ¯+ T16 B 1 ¯+ T17 B 2 ¯+ T18 B 3 The proof is now complete.
5.3
Converse Proof of Theorem 5
In order to establish the converse of Theorem 5, we prove each inequality individually. It is clear that (13) is trivially true due to the result on multilevel diversity coding in [2], and (14) is true because of Theorem 3. It remains to show that inequalities (15)-(18) are true, and we prove each as a proposition. Instead of writing the proofs in the conventional approach as chains of inequalities, we utilize the computational approach developed in [11], and prove these inequalities by tabulation. Each inequality is given as two tables, the first of which lists the joint entropy terms in the proof, and the second of which gives the known sub-modular inequalities (Shannon-type inequalities) and the coefficients of these inequalities, whose last row, as the summation of all the other rows, is exactly the sought-after inequality. Note that each row in the second table is a sub-modular inequality (or an integer multiple of such an inequality), possibly after permutation of the indices for each entropy term. For example, the second line of Table 4 is 5H(S4→3 ) + 5H(S4→2 , S3→2 ) − 5H(S4→1 , S3→1 , S2→1 ) ≥ 0,
(31)
which is equivalent to the simple inequality 5[H(S2→1 ) + H(S4→1 , S3→1 ) − H(S4→1 , S3→1 , S2→1 )] ≥ 0,
(32)
after taking into account of the symmetry of the optimal solutions. Further note that for n = 4 we have ¯ + = 1, however we still write it as B ¯ + to make explicit its meaning of sum rate. B 3 3 ¯ ∈ R4 (B ¯1 , B ¯2 , B ¯3 ), we have Proposition 7. For any (α ¯ , β) 4¯ 3¯ 5¯ ¯ ¯ + + 3B ¯ + + 15B ¯ +. 24[¯ α + β] ≥ 24 B1 + B2 + B3 = 14B 1 2 3 3 4 8 Proof. See Table 3 and Table 4.
12
(33)
Table 4: Proof by Tabulation of Proposition 7, with terms defined in Table 3. T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16 T17 T18 16 −8 4 −4 4 −4 2 2 −2 12 −6 −6 2 2 −2 −2 6 6 −6 4 4 −4 −4 4 −4 4 −4 3 3 −3 −3 3 −3 3 −3 3 3 −3 −3 3 −3 3 −3 3 3 −3 −3 2 2 −2 −2 24 24 −14 −3 −15 ¯ ∈ R4 (B ¯1 , B ¯2 , B ¯3 ), we have Proposition 8. For any (¯ α, β) 3¯ 5¯ ¯ ¯ +. ¯ + + 9B ¯+ + B ¯ 6[2¯ α + 3β] ≥ 6 3B1 + B2 + B3 = 8B 3 2 1 3 2
(34)
Proof. See Table 5 and Table 6. ¯ ∈ R4 (B ¯1 , B ¯2 , B ¯3 ), we have Proposition 9. For any (¯ α, β) 5¯ 5¯ ¯ ¯ +. ¯ + + 10B ¯ ¯ + + 2B 12[¯ α + 2β] ≥ 12 B1 + B2 + B3 = 8B 3 2 1 3 6
(35)
Proof. See Table 7 and Table 8. ¯ ∈ R4 (B ¯1 , B ¯2 , B ¯3 ), we have Proposition 10. For any (¯ α, β) 1¯ 1¯ 1¯ ¯ ¯+ + B ¯ + + 5B ¯ +. 30β ≥ 30 B1 + B2 + B3 = 4B 1 2 3 3 5 6
(36)
Proof. See Table 9 and Table 10.
6
Conclusion
We considered multilevel diversity coding with regeneration, which addresses the storage vs. repairbandwidth tradeoff in distributed storage systems with heterogeneous reliability and access latency requirements. It is shown that for the minimum storage point on the optimal tradeoff, separate coding is sufficient, and there is no need to mix different contents. On the other hand, a complete characterization of the tradeoff region is provided for the case of four nodes, which reveals that mixing can strictly improve the general tradeoff. In other words, separate coding using individual regenerating code is not optimal, and mixing the contents can strictly improve the efficiency of the codes.
13
Table 5: Terms needed to prove Proposition 8 T1 H(S4→3 ) T2 H(S4→2 , S3→2 ) T3 H(S4→1 , S3→1 , S2→1 ) T4 H(W4 ) T5 H(S3→2 , W4 ) T6 H(S3→4 , W4 ) T7 H(S3→4 , S2→4 , W4 ) T8 H(W4 , W3 ) T9 H(S2→4 , W4 , W3 ) T10 H(S3→1 , S2→1 , W4 , W1 ) T11 H(S4→3 , M1+ ) T12 H(S4→2 , S3→2 , M1+ ) T13 H(S4→3 , S3→4 , M1+ ) T14 H(S4→3 , S3→4 , S2→4 , M1+ ) T15 H(W4 , M2+ ) T16 H(S3→2 , W4 , M2+ ) T17 H(S3→4 , W4 , M2+ ) T18 H(S3→2 , S2→4 , W4 , M2+ ) ¯+ B T19 1 ¯+ T20 B 2 ¯+ B T21 3
Table 6: Proof by Tabulation of Proposition 8, with terms defined in Table 5. T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16 T17 T18 T19 T20 12 −6 4 −4 4 −4 4 4 −4 4 −4 4 −4 2 2 −2 −2 2 2 −2 −2 2 2 −2 2 −2 −2 2 2 2 −2 −2 2 −2 2 2 2 −2 1 1 −1 1 −1 1 1 1 −1 1 −1 1 1 1 −1 18 12 −8 −1
14
T21
−2 −2 −1 −1 −1 −1 −1 −9
Table 7: Terms needed to prove Proposition 9 T1 H(S4→3 ) T2 H(S4→2 , S3→2 ) T3 H(S4→1 , S3→1 , S2→1 ) T4 H(W4 ) T5 H(S3→4 , W4 ) T6 H(S3→2 , W4 ) T7 H(S3→4 , S2→4 , W4 ) T8 H(W4 , W3 ) T9 H(S2→4 , W4 , W3 ) T10 H(S3→1 , S2→1 , W4 , W1 ) T11 H(S3→4 , S3→1 , S2→1 , W4 , W1 ) T12 H(S4→3 , M1+ ) T13 H(S4→3 , M2+ ) T14 H(S4→2 , S3→4 , M2+ ) T15 H(S4→2 , S3→2 , M1+ ) T16 H(S3→2 , W4 , M2+ ) T17 H(S3→4 , W4 , M2+ ) T18 H(S3→2 , S2→4 , W4 , M2+ ) T19 H(S3→2 , S2→3 , W4 , M2+ ) T20 H(S3→2 , S2→4 , S1→4 , W4 , M2+ ) ¯+ T21 B 1 ¯+ T22 B 2 ¯+ B T23 3
Table 8: T1 T2 T3 T4 T5 16 −8 4 4 4 −4 1 1 4 4 4 4 2
1
Proof by Tabulation of Proposition 9, with terms defined in Table 7. T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16 T17 T18 T19 T20 T21 T22 T23 −4
−4
4 −1
−1
−4 −4 −4
4 2 −2 2
−2 2 −1 −1
1
−4
4 −4
1
−2 −2 −1
1
−1 −1
1 1
−1 24
1
1 1 1 1 1 2 1
1
1
−1 −1 −1
1 −1
1 −1
1 −1
−1 −1 −1 −1 −1 −1 −1
−1 −8 −2 −10
12
15
Table 9: Terms needed to prove Proposition 10 T1 H(S4→3 ) T2 H(S4→2 , S3→2 ) T3 H(S4→1 , S3→1 , S2→1 ) T4 H(W4 ) T5 H(S3→2 , W4 ) T6 H(S3→1 , S2→1 , W4 , W1 ) T7 H(S4→2 , S4→1 , S3→2 , S3→1 , W2 , W1 ) T8 H(S4→3 , M1+ ) T9 H(S4→3 , M2+ ) T10 H(S4→2 , S3→2 , M1+ ) T11 H(S4→3 , S3→4 , M1+ ) T12 H(S4→2 , S3→4 , M2+ ) T13 H(S4→1 , S3→4 , S2→4 , M2+ ) T14 H(S4→1 , S3→4 , S3→2 , S2→4 , M2+ ) ¯+ T15 B 1 ¯+ T16 B 2 ¯+ T17 B 3
Table 10: Proof by Tabulation of Proposition T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 20 −10 5 5 −5 5 5 −5 1 −1 1 2 2 −2 −2 2 −2 2 1 −1 1 −1 1 1 −1 −1 1 1 1 2 −1 1 1 1 30
16
10, with terms defined in Table 9. T11 T12 T13 T14 T15 T16 T17
−1 −2 −1
1 −1
1 −1 −1 1
−1 −1
−1 −1
1
−1 −1 −1 −4 −1 −5
Another notable feature of this work is that we further developed the computational approach in [11] to identify and prove the converse. As a result, the converse proof is presented as tabulation, without translating the tables to the conventional form of proofs usually seen in information theory literature. It is our belief that this computational approach will be able to play an even more significant role in future studies. In order to help future researchers to utilize such data, we post the result in this paper as part of the online collection of “ Solutions of Computed Information Theoretic Limits (SCITL)” hosted at [12], which hopefully in the future can serve as a data depot for information theoretic limits obtained through computational approaches. There are several immediate research directions to follow. Firstly, the code construction given for n = 4 can be generalized to other parameters straightforwardly, and we shall address this issue in a later work. Secondly, the MLD-R problem can be extended to allow more general values of d not equal to n−1. Thirdly, it is important to understand how much improvement over separate coding can be attained by mixing the contents. Lastly, it may also be useful to consider the analogous requirement in the locally repairable code setting [13–15].
Appendix: Proofs of Proposition 1 and Proposition 2 Proof of Proposition 1. It is known that the rate region of (3, 1, 2) regenerating codes (before normalization) is α1 ≥ B1 ,
β1 ≥
B1 , 2
(37)
and the rate region of (3, 2, 2) regenerating codes is 2α2 ≥ B2 ,
α2 + β2 ≥ B2 ,
3β2 ≥ B2 .
(38)
It is straightforward to verify that separate coding indeed leads to the rate region given in Proposition 1 using Fourier-Motzkin elimination. Proof of Proposition 2. Using Fourier-Motzkin elimination directly for this case is time-consuming, and we instead consider inner bounding and outer bounding the region. It is straightforward to show that the inequalities in Proposition 2 have to hold in this separate coding based achievability scheme, e.g., 4α + 6β = 4(α1 + α2 + α3 ) + 6(β1 + β2 + β3 ) (a)
≥ 6B1 + 4(α2 + α3 ) + 6(β2 + β3 ) = 6B1 + α2 + 3(α2 + 2β2 ) + (4α3 + 6β3 ) (b)
7 ≥ 6B1 + B2 + (4α3 + 6β3 ) 2 (c) 7 ≥ 6B1 + B2 + 3B3 , 2
(39)
where (a) is because of (41), (b) because of (42), and (c) because of (44). Other inequalities can be shown in a similar manner. Next we show this outer bound is indeed achievable using the separate coding based scheme. Computing the intersections of any two linear bounds in (4)-(8) yields a total of 10 points, which are the possible extremal (corner) points of the region. However, some of them do not satisfy all the inequalities, and after eliminating them, the corner points of the region in Proposition 2 are thus given
17
as (see also Fig. 2) ¯2 B ¯3 B ¯1 B ¯2 B ¯3 B + , + + ) 2 3 3 4 3 ¯ ¯ ¯ ¯ ¯ ¯1 + B2 + 3B3 , B1 + B2 + B3 ) (B 2 8 3 4 4 ¯2 B ¯3 B ¯1 B ¯2 B ¯3 B ¯1 + (B + , + + ) 2 2 3 4 6 ¯ ¯ ¯ ¯ ¯ ¯1 + 3B2 + B3 , B1 + B2 + B3 ). (B (40) 5 2 3 5 6 Next we show that they are indeed achievable using the separate coding based scheme, which implies that the region given in (4)-(8) is achievable by a time-sharing argument. It is known that the rate region of (4, 1, 3) regenerating codes is ¯1 + (B
α1 ≥ B1 ,
β1 ≥
B1 , 3
(41)
and thus there is only one corner point of the rate region, which is (B1 , B31 ). The rate region of (4, 2, 3) regenerating codes is 2α2 ≥ B2 ,
α2 + 2β2 ≥ B2 ,
5β2 ≥ B2 ,
(42)
and there are two corner points in this rate region, which are 3B2 B2 B2 B2 , ), ( , ). 2 4 5 5 The rate region of (4, 3, 3) regenerating codes is given in [11] (
3α3 ≥ B3 , 2α3 + β3 ≥ B3 , 4α3 + 6β3 ≥ 3B3 , 6β3 ≥ B3 .
(43)
(44)
and there are three corner points in this rate region, which are B3 B3 3B3 B3 B3 B3 , ), ( , ), ( , ). (45) 3 3 8 4 2 6 It is easily seen that the rate pairs in (40) are all simple linear combinations of the corner point rate pairs of the (4, 1, 3), (4, 2, 3) and (4, 3, 3) codes after proper normalization, i.e., by dividing (B1 +B2 +B3 ), and thus achievable using the separate coding based scheme. The proof is now complete. (
Converse Proof of Theorem 4. It can be shown that we only need to consider symmetric codes, where permutations of node indices do not change joint entropy values; see [11] for more details about this symmetry. Thus without loss of generality, we can restrict the proof to symmetric codes only. Denote the information stored at node i as Wi , the helper information sent from node i to node j as Si,j , and the message to be decoded using any j nodes as Mj . First write the following inequalities (a)
α ≥ H(W1 ) = H(W1 , M1 ) ≥ H(W1 |M1 ) + B1 1 (s) = B1 + [H(W1 |M1 ) + H(W2 |M1 )] 2 1 ≥ B1 + H(W1 , W2 |M1 ) 2 1 (c) = B1 + H(W1 , W2 , M2 |M1 ) 2 1 1 ≥ B1 + H(M2 |M1 ) ≥ B1 + B2 , 2 2 18
(46)
where (a) is because M1 can be recovered from W1 alone, and (s) implies by the existence of symmetric solutions, (c) is because (W1 , W2 ) together can recover M2 . Dividing both sides of (46) by (B1 + B2 ) gives the result. From here, we use (s) to denote by the reason of the symmetry. Next we bound β by writing 1 β ≥ [H(S1→3 ) + H(S2→3 )] 2 1 ≥ [H(S1→3 , S2→3 )] 2 (a) 1 = [H(S1→3 , S2→3 , W3 , M1 )] 2 1 1 ≥ B1 + H(S1→3 , S2→3 , W3 |M1 ), 2 2
(47)
where (a) is because from the helper message S1→3 and S2→3 , we can regenerate W3 . To bound the second term above, we write H(S1→3 , S2→3 , W3 |M1 )] (a)
= H(S1→3 , S2→3 , W3 , S3→1 , S3→2 |M1 )
≥ H(S1→3 , S2→3 , S3→1 , S3→2 |M1 ) (s) 1 = [H(S1→3 , S2→3 , S3→1 , S3→2 |M1 ) 3 + H(S1→2 , S3→2 , S2→1 , S2→3 |M1 ) + H(S3→1 , S2→1 , S1→2 , S1→3 |M1 )] (b)
1 [H(S1→3 , S2→3 , S3→1 , S3→2 , S1→2 , S2→1 |M1 ) 3 + H(S3→2 , S2→3 |M1 ) + H(S3→1 , S2→1 , S1→2 , S1→3 |M1 )] (c) 1 = [B2 + H(S3→2 , S2→3 |M1 ) 3 + H(S3→1 , S2→1 , S1→2 , S1→3 |M1 )] 1 ≥ [B2 + H(S3→2 , S2→3 , S3→1 , S2→1 , S1→2 , S1→3 |M1 )] 3 1 2B2 ≥ [B2 + H(M2 |M1 )] ≥ , 3 3 ≥
(48)
where (a) is because S3→1 , S3→2 are functions of W3 , (b) is because H(S1→3 , S3→1 |S3→2 , S2→3 , M1 ) ≥ H(S1→3 , S3→1 |S1→2 , S3→2 , S2→1 , S2→3 , M1 ),
(49)
and (c) is because from (S1→3 , S2→3 , S3→1 , S2→1 ) we can regenerate (W1 , W3 ), subsequently decode M2 . It follows that 1 1 β ≥ B1 + B2 . 2 3
19
(50)
The remaining inequality in Theorem 4 can be proved by writing the following: α+β 1 ≥ B1 + H(W1 |M1 ) + [B1 + H(S1→3 , S2→3 , W3 |M1 )] 2 3 1 = B1 + H(W1 , S1→2 , S1→3 |M1 ) + H(S1→3 , S2→3 , W3 |M1 ) 2 2 1 3 = B1 + H(W1 , S1→2 , S1→3 |M1 ) 2 2 1 + [H(W1 , S1→2 , S1→3 |M1 ) + H(S1→3 , S2→3 , W3 |M1 )] 2 1 3 ≥ B1 + H(W1 , S1→2 , S1→3 |M1 ) 2 2 1 + [H(W1 , W3 , S1→2 , S1→3 , S2→3 |M1 ) + H(S1→3 |M1 )] 2 1 1 3 = B1 + B2 + [H(W1 , S1→2 , S1→3 |M1 ) + H(S1→3 |M1 )] 2 2 2 1 1 (s) 3 = B1 + B2 + [H(W1 , S1→2 , S1→3 |M1 ) + H(S2→3 |M1 )] 2 2 2 3 1 1 ≥ B1 + B2 + [H(W1 , S1→2 , S1→3 , S2→3 |M1 )] 2 2 2 3 ≥ B1 + B2 . 2
(51)
The proof is thus complete.
References [1] N. B. Shah, Kangwook Lee, and K. Ramchandran, “The MDS queue: Analysing the latency performance of erasure codes,” in Proceeds, IEEE International Symposim on Information Theory,, pp. 861–865, Hololulu HI, Jun.-Jul. 2014. [2] J. R. Roche, R. W. Yeung, and K. P. Hau, “Symmetrical multilevel diversity coding,” IEEE Trans. Information Theory, vol. 43, no. 5, pp. 1059–1064, May 1997. [3] R. W. Yeung and Z. Zhang, “On symmetrical multilevel diversity coding,” IEEE Trans. Information Theory, vol. 45, no. 2, pp. 609–621, Mar. 1999. [4] A. G. Dimakis, P. B. Godfrey, Y. Wu, M. Wainwright and K. Ramchandran, “Network coding for distributed storage systems,” IEEE Trans. Information Theory, vol. 56, no. 9, pp. 4539–4551, Sep. 2010. [5] A. G. Dimakis, K. Ramchandran, Y. Wu, C. Suh, “A survey on network codes for distributed storage,” Proceedings of the IEEE, vol. 99, no. 3, pp. 476–489, Mar. 2011. [6] N. B. Shah, K. V. Rashmi, P. V. Kumar and K. Ramchandran, “Distributed storage codes with repair-by-transfer and non-achievability of interior points on the storage-bandwidth tradeoff,” IEEE Trans. Information Theory, vol. 58, no. 3, pp. 1837-1852, Mar. 2012. [7] K. V. Rashmi, N. B. Shah, and P. V. Kumar, “Optimal exact-regenerating codes for distributed storage at the MSR and MBR points via a product-matrix construction,” IEEE Trans. Information Theory, vol. 57, no. 8, pp. 5227–5239, Aug. 2011.
20
[8] V. Cadambe, S. Jafar, H. Maleki, K. Ramchandran and C. Suh, “Asymptotic interference alignment for optimal repair of MDS codes in distributed storage,” IEEE Trans. Information Theory, vol. 59, no. 5, pp. 2974–2987 , May 2013. [9] I. Tamo, Z. Y. Wang and J. Bruck, “Zigzag codes: MDS array codes with optimal rebuilding,” IEEE Trans. Information Theory, vol. 59, no. 3, pp. 1597–1616, Mar. 2013. [10] D. S. Papailiopoulos, A. G. Dimakis and V. R. Cadambe, “Repair optimal erasure codes through Hadamard designs,” IEEE Trans. Information Theory, vol. 59, no. 5, pp. 3021–3037, May 2013. [11] C. Tian, “Characterizing rate region of the (4, 3, 3) exact-repair regenerating codes,” IEEE Journal on Selected Areas of Communications, Vol. 32, No. 5, pp. 967-975, May 2014. [12] Solutions of Computed Information Theoretic Limits (SCITL), ~ctian1/SCITL.html.
http://web.eecs.utk.edu/
[13] C. Huang, H. Simitci, Y. K. Xu, A. Ogus, B. Calder, P. Gopalan, J. Li, and S. Yekhanin, “Erasure coding in Windows Azure storage,” in Proceeds, 2012 USENIX Annual Technical Conference, San Jose CA. [14] D. S. Papailiopoulos and A. Dimakis, “Locally repairable codes,” IEEE Trans. on Information Theory, vol.60, no.9, pp. 5843-5855, Sep. 2014. [15] A. S. Rawat, O. O. Koyluoglu, N. Silberstein and S. Vishwanath, “Optimal locally repairable and secure codes for distributed storage systems,” IEEE Trans. on Information Theory, vol.60, no.1, pp. 212-236, Jan. 2014.
21
arXiv:1503.00013v1 [cs.IT] 27 Feb 2015
Abstract Digital contents in large scale distributed storage systems may have different reliability and access delay requirements, and for this reason, erasure codes with different strengths need to be utilized to achieve the best storage efficiency. At the same time, in such large scale distributed storage systems, nodes fail on a regular basis, and the contents stored on them need to be regenerated and stored on other healthy nodes, the efficiency of which is an important factor affecting the overall quality of service. In this work, we formulate the problem of multilevel diversity coding with regeneration to address these considerations, for which the storage vs. repair-bandwidth tradeoff is investigated. We show that the extreme point on this tradeoff corresponding to the minimum possible storage can be achieved by a simple coding scheme, where contents with different reliability requirements are encoded separately with individual regenerating codes without any mixing. On the other hand, we establish the complete storage-repair-bandwidth tradeoff for the case of four storage nodes, which reveals that codes mixing different contents can strictly improve this tradeoff over the separate coding solution.
1
Introduction
The importance of big data analytics has been widely recognized in recent years, however, efficient large scale distributed data storage systems have to be designed and implemented in order to support the complete pipeline of data collection, processing and archival on a scale never seen before, where advanced coding techniques have been shown to play an important role. Digital contents in large scale distributed storage systems usually have different reliability requirements. For example, although it is important to protect recent customer billing records with a very reliable code, it may be acceptable to allow the data loss probability of a 5-year-old office document backup to be higher by using a relatively weak code. Moreover, it was shown recently that erasure codes can also be used to reduce data access latency, whose behavior depends on the coding parameters [1]. Thus, different levels of latency can also be integrated into the same data storage system by adopting different coding parameters for different contents. Such flexibility can significantly reduce the cost of hardware infrastructure, and there is tremendous interest recently both in industry and in academia to design efficient software defined storage (SDS) systems utilizing flexible erasure codes. The theoretical framework of symmetrical multilevel diversity (MLD) coding [2, 3] is a natural fit for this scenario, where there are a total of k0 ≤ n independent messages (M1 , M2 , ..., Mk0 ) to be stored, and the overall system consists of n storage nodes situated in different network locations, each with α units of data. The messages are coded in such a way that by accessing any k ≤ k0 of these nodes, the first k messages (M1 , M2 , ..., Mk ) can be completely recovered. Disk or node failures occur regularly in a large scale storage system, and the overall quality of service is heavily affected by the efficiency of the repair process. Dimakis et al. [4] proposed the framework of regenerating codes to address the tradeoff between the storage and repair-bandwidth in (n, k) erasurecoded distributed storage systems. To repair a node, a new node replacing the failed one requests β units of data each from any of the d remaining nodes, and regenerates the α units of content to store on the new node; this code is referred to as the (n, k, d) regenerating code. There exists a natural tradeoff between 1
the storage α and the repair bandwidth β: the point corresponding to the minimum amount of the storage is referred to as the minimum storage regenerating (MSR) point, and the other extreme is referred to as the minimum repair-bandwidth regenerating (MBR) point. In [4], the content regenerated is allowed to be only functionally equivalent to the content on the failed node, thus the name “functional-repair” regenerating codes. In practice, requiring the content regenerated to be exactly the same as that on the failed node can simplify the system design significantly, and thus recent research effort has focused on “exact-repair” regenerating codes [5–11]. In the regenerating code framework, only a single message is allowed, and thus offering only a single level of reliability and access latency; on the other hand, in the MLD coding framework, the data repair process was not considered. In this work, we consider repair-efficient codes in systems with heterogeneous reliability and latency requirements, and investigate the optimal storage vs. repair-bandwidth tradeoff. Because of the connection to the MLD coding problem and the regenerating code problem, we refer to this problem as multilevel diversity coding with regeneration (MLD-R) in the sequel; see Fig. 1. We shall restrict our attention to the case of exact-repair, and furthermore to the case when d = n−1, because this is the most practically important case. Nevertheless, the framework can be straightforwardly generalized to other relevant settings. An intuitive and straightforward coding strategy is to use individual regenerating code for each message to satisfy the respective reliability and latency requirement (i.e., separate coding), and thus the most important question we must answer first is whether it is even beneficial to consider codes that “mix” the messages. Without the repair consideration, it was shown in [2, 3] that no mixing is necessary for the symmetrical MLD coding problem. We show that for the minimum storage point on the optimal tradeoff where α is minimized (analogous to the MSR point in regenerating codes), the aforementioned separate coding is again sufficient. On the other hand, we show for n = 4, by providing a novel code construction, that mixing can strictly improve upon the performance of the separate coding solution in terms of overall storage-repair-bandwidth tradeoff. In fact, we completely characterize this tradeoff by establishing its converse. It is worth noting that when n = 3, separate coding is in fact sufficient, thus n = 4 is the smallest non-trivial case where the benefit of mixing manifests. One difficulty in establishing these results is in deriving the outer bounds. For the minimum storage point, we utilize a recursive bounding technique which may be of independent interest. The converse for the tradeoff rate region when n = 4 is rather difficult to identify and derive manually, and thus we utilize the computational approach developed in [11]. This proof is thus presented in tables whose rows are simple known information inequalities, and the summation of the rows are precisely the outer bounds. Though this does not conform to the conventional proof approach of using chains of information inequalities in information theory literature, we believe that these tables are in a sense more fundamental. Indeed, we can write down many different versions of chains of inequalities with the help of these tables, by taking different orders when applying these individual inequalities. The rest of the paper is organized as follows. A formal problem formulation and some preliminaries are given in Section 2. In Section 3, the main results of the paper are presented together with the relevant discussion. The proofs are given in Section 4 and Section 5. Section 6 concludes the paper with a few future research directions. Several technical proofs are given in the Appendix.
2 2.1
Problem Formulation and Preliminaries Problem Formulation
An MLD-R code is defined below, where In is used to denote the set {1, 2, . . . , n}, and |A| is used to denote the cardinality of a set A. It can be assumed without loss of generality d = k0 , where k0 is the number of messages. This is because if d < k0 , then the messages (Md+1 , Md+2 , ..., Mk0 ) can be viewed as part of Md ; on the other hand if d > k0 , we can simply consider an alternative problem with k 0 0 = d and define the messages Mk0 , Mk0 +1 , ..., Md to be degenerate. Recall in this work we assume d = n − 1. 2
Node 1
α α
M
α α
Node 1
Node 1'
β Node 2
α α
β
α α
M1, M 2 ,..., M L Node 3
β
Node 1
α α
Node 2
M1, M 2 ,..., M L
k nodes Node 3
α α
Node 2
Node 4
M1, M2 ,...,Mk Node n
Node n
Node 4
M1, M2 ,...,Mk
...
α
...
...
α
β decoder
...
...
... α
β k nodes
Node 3
decoder Node 4
Node 1'
β
Node n
Figure 1: Multilevel diversity coding with regeneration. P Definition 1. An (N1 , N2 , ..., Nd , Kd , K) MLD-R code consists of n encoding functions fiE (·), di=1 ni E (·), and n repair decoding functions decoding functions fAD (·, ..., ·), n(n − 1) repair encoding functions Fi,j D Fi (·, ..., ·), where fiE : IN1 × IN2 × ... × INd → IKd ,
i ∈ In ,
each of which maps the message (M1 , M2 , ..., Md ) ∈ IN1 × IN2 × ... × INd to one piece of coded information, fAD : IKd × IKd × ... × IKd → IN1 × IN2 × ... × IN|A| , A ⊂ In
and
|A| = 1, 2, . . . , d,
each of which maps coded information stored on a set A of nodes to the first |A| messages (M1 , M2 , ..., M|A| ), E Fi,j : IKd → IK ,
j ∈ In ,
and
i ∈ In \ {j},
each of which maps a piece of coded information at node i to an index that is made available to node j to reconstruct the coded data at node j, and FjD : IK × IK × ... × IK → IKd ,
j ∈ In ,
each of which maps d such indices from the helper nodes to reconstruct the information stored at the failed node j. The functions must satisfy the data reconstruction conditions ! Y fAD fiE (M1 , M2 , ..., Md ) = (M1 , M2 , ..., M|A| ), i∈A
(M1 , M2 , ..., Md ) ∈ IN1 × IN2 × ... × INd , A ⊂ In and |A| = 1, 2, . . . , d,
(1)
and the repair conditions FjD
Y
E Fi,j fiE (M1 , M2 , ..., Md )
i∈In \{j}
= fjE (M1 , M2 , ..., Md ), (M1 , M2 , ..., Md ) ∈ IN1 × IN2 × ... × INd ,
j ∈ In .
(2)
Note α = log Kd is the storage node capacity, β = log K is the per-helper-node repair bandwidth, and Bi = log Ni is the rate of the i-th message; the base of log(·) is arbitrary, and we choose base 2 for convenience. Because of the linear scaling relation among them, we can alternatively consider the normalized version as follows. 3
¯ B ¯1 , B ¯2 , ..., B ¯d ) is said to Definition 2. A normalized storage-repair-bandwidth-message-rate tuple (¯ α, β, Pd ¯ be achievable with n nodes where j=1 Bj = 1, if there exists an (N1 , N2 , ..., Nd , Kd , K) MLD-R code such that log Kd log K α ¯ ≥ Pd , β¯ ≥ Pd , i=1 log Ni i=1 log Ni ¯j = P log Ni , j = 1, 2, ..., d. B d i=1 log Ni ¯ B ¯1 , B ¯2 , ..., B ¯d ) tuples is the achievable storage-repairThe convex closure of all the achievable (¯ α, β, ¯ ¯ ¯d ) tuple, the achievable storage-repairbandwidth-message-rate tradeoff region R. For a fixed (B1 , B2 , ..., B ¯ ¯ B ¯1 , B ¯2 , ..., B ¯d ) ∈ R, which is bandwidth tradeoff region is the collection of all (¯ α, β) pairs such that (¯ α, β, ¯ ¯ ¯ denoted as Rn (B1 , B2 , ..., Bd ). The codes and the tradeoff regions do not involve any particular assumption on the distribution of the messages, however, without loss of generality we can assume that the messages M1 , M2 , . . . , Md are mutually independent and uniformly distributed, since otherwise we can perform a pre-coding to eliminate any dependence and non-uniformity. Sometimes it is convenient to use the accumulative sum rates instead of individual rates, and thus we define ¯+ , B k
k X
¯i , B
k = 1, 2, . . . , d.
(3)
i=1
The corresponding accumulative message (M1 , M2 , . . . , Mk ) is sometimes also written as Mk+ . Note that ¯ + = 1, however we often still write B ¯ + for convenience. this definition implies B d d The reconstruction condition (1) requires that there is no decoding error, i.e., the zero-error requirement is adopted. An alternative definition is to require instead the probability of decoding error to be asymptotically zero as Πdi=1 Ni → ∞. It will become clear that this does not cause any essential difference, and thus we do not give this alternative definition. When deriving outer bounds, we use Si→j to denote the helper (random) information sent from node-i to node-j, and Wi to denote the coded data stored on node-i. The random variable vector (X1 , X2 , . . . , Xm ) is sometimes written as X1m .
2.2
Separate Coding
One straightforward coding strategy is to encode each individual message using a regenerating code of the necessary strength. More precisely, the message Mk is encoded using an (n, k, d) regenerating code (i.e., any k nodes can recover the message Mk , and any new node obtains information from any d nodes for repair) of rate (αk , βk ), and the resultant code thus has storage rate and repair rate: α=
d X
αk ,
β=
k=1
d X
βk .
k=1
¯1 , B ¯2 , ..., B ¯d ), tradeoff ˜ n (B In order to characterize this separate coding tradeoff region, denoted as R region characterizations of individual regenerating codes are needed; e.g., for the case of n = 3, tradeoff region characterizations for (3, 1, 2) and (3, 2, 2) regenerating codes are needed. However, such characterizations for general parameters are still unknown, except for the case of k = 1, 2, and the special case (n, k, d) = (4, 3, 3) only recently established in [11]. Fortunately, using these existing results, we can provide a characterization of the separate coding tradeoff region for MLD-R when n = 3, denoted as ˜ 3 (B ¯1 , B ¯2 ), and that when n = 4, denoted as R ˜ 4 (B ¯1 , B ¯2 , B ¯3 ). R 4
˜ 3 (B ¯1 , B ¯2 ) achievable using separate coding Proposition 1. The storage-repair-bandwidth tradeoff region R ¯ is the set of (¯ α, β) pairs satisfying the following conditions ¯1 + α ¯≥B
¯2 B , 2
¯1 3B ¯2 , +B 2
α ¯ + β¯ ≥
¯1 B ¯2 B β¯ ≥ + . 2 3
˜ 4 (B ¯1 , B ¯2 , B ¯3 ) achievable using separate Proposition 2. The storage-repair-bandwidth tradeoff region R ¯ coding is the set of (¯ α, β) pairs satisfying the following conditions ¯ ¯ ¯1 + B2 + B3 α ¯≥B 2 3 ¯ 1 5B ¯2 7 B ¯3 2¯ α + β¯ ≥ + +B 3 4 ¯ ¯1 + 7B2 + 3B ¯3 4α ¯ + 6β¯ ≥ 6B 2 ¯1 5B ¯2 + 5 B ¯3 α ¯ + 2β¯ ≥ +B 3 6 ¯1 1 B ¯2 + 1 B ¯3 . β¯ ≥ + B 3 5 6
(4) (5) (6) (7) (8)
The proofs for these propositions are given in the Appendix.
3
Main Results
¯1 , B ¯2 , ..., B ¯d ) Our first main result is a characterization of the extreme point in the tradeoff rate region Rn (B where α ¯ is minimized, through the following theorem. ¯ ∈ Rn , we have Theorem 3. For any (¯ α, β) (n − 2)α ¯ + β¯ ≥
n−1 X
(n − 2)(n − k) + 1 ¯ Bk . k(n − k)
k=1
(9)
It was shown in [2] that for MLD coding without repair considerations, the minimum achievable rate is given as α ¯=
n−1 X k=1
¯k B , k
(10)
using separate coding. Thus Theorem 3 and the existence of (n, k, d) exact-repair regenerating codes at the MSR point [8] imply the following corollary. ¯ ∈ Rn Corollary 1. For the minimum storage point on the boundary of the tradeoff rate region (¯ α, β) where α ¯=
n−1 X
¯k B , k
(11)
¯k B , k(n − k)
(12)
k=1
the minimum repair bandwidth is given as β¯ =
n−1 X k=1
and it can be achieved by separately coding of each message Mk with an (n, k, d) exact-repair MSR code. 5
¯1 , B ¯2 , B ¯3 ). ¯1 , B ¯2 ) and R4 (B The next two theorems provide complete characterizations for R3 (B ¯1 , B ¯2 ) = R ˜ 3 (B ¯1 , B ¯2 ). Theorem 4. R3 (B ˜ 3 (B ¯1 , B ¯2 ) ⊆ R3 (B ¯1 , B ¯2 ) is straightforward, and the converse direction is The forward direction R proved in the Appendix. This theorem states that for the case of n = 3, the strategy of separate coding is optimal, and there is no need to mix the messages. However, our next result shows that this is in general not true, and mixing the messages is beneficial. For this purpose, we establish Theorem 5, whose proof is provided in Section 5. ¯ ¯1 , B ¯2 , B ¯3 ) is the collection of all (¯ Theorem 5. The storage-repair-bandwidth tradeoff region R4 (B α, β) pairs that satisfy the following constraints: ¯ ¯ ¯1 + B2 + B3 α ¯≥B 2 3 5 7 ¯1 + B ¯2 + B ¯3 2¯ α + β¯ ≥ B 3 4 3¯ 5¯ 4¯ α ¯ + β¯ ≥ B 1 + B2 + B3 3 4 8 5 3 ¯1 + B ¯2 + B ¯3 2α ¯ + 3β¯ ≥ 3B 3 2 5¯ 5¯ ¯ α ¯ + 2β¯ ≥ B 1 + B2 + B3 3 6 1¯ 1¯ 1¯ ¯ β ≥ B1 + B2 + B 3. 3 5 6
(13) (14) (15) (16) (17) (18)
˜ 4 (0, 1 , 2 ) are depicted in Fig. 2 for the case B ¯2 = 1 and B ¯3 = 2 . It can The regions R4 (0, 13 , 23 ) and R 3 3 3 3 ˜ 4 (0, 1 , 2 ) ⊂ R4 (0, 1 , 2 ) is strict, thus generally, mixing of contents in MLD-R be seen that the inclusion R 3 3 3 3 can be beneficial. We formally state it as it a corollary. ˜ 4 (B ¯1 , B ¯2 , B ¯3 ) ( R4 (B ¯1 , B ¯2 , B ¯3 ). Corollary 2. R
4
The Minimum Storage Point: Proof of Theorem 3
It can be shown that we only need to consider symmetric codes, where permutations of node indices do not change the joint entropy values; see [11] for more details about this symmetry. Thus without loss of generality, we can restrict the proof to symmetric codes only. Before presenting the proof of Theorem 3, we first give an auxiliary lemma. This lemma plays a similar instrumental role in the proof of Theorem 3 as Han’s inequality in the converse proof of symmetrical MLD coding problem given in [2]. Lemma 1. For any integer ` such that 1 ≤ ` ≤ d − 1 and any symmetric MLD-R code with a total of n nodes n−1 n−`−1 H(W1` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) `(n − `) n−` ≥ B`+1 + H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1`+1 ).
(19)
Proof. Let ` be an integer such that 1 ≤ ` ≤ d − 1. We start by writing H(W1` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) (s)
= H(W2`+1 |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` )
= H(S`+1→1 , W2`+1 |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) ≥ H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1` ) + H(S`+1→1 , W2` |M1` ) ≥ B`+1 + H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1`+1 ) + H(S`+1→1 , W2` |M1` ), 6
(20)
MLD−R rate region separate coding rate region
0.34 0.32 (7/18, 11/36)
0.3 0.28
β¯0.26
(5/12, 1/4 )
0.24 0.22
(1/2 , 7/36 ) (4/9, 2/9)
0.2 0.18
(8/15, 8/45 )
0.16 0.35
0.4
0.45
α ¯
0.5
0.55
0.6
˜ 4 (0, 1 , 2 ) using separate coding, and the tradeoff rate region R4 (0, 1 , 2 ). Figure 2: The rate region R 3 3 3 3 where we write (s) to denote for the reason of symmetry, the first inequality in the chain is because H(W`+1 |M1` , S`+1→1 , W2` ) + H(S`+2→1 , . . . , Sn→1 , W1 |M1` , S`+1→1 , W2` ) ≥ H(S`+2→1 , . . . , Sn→1 , W1 , W`+1 |M1` , S`+1→1 , W2` ) = H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1` , S`+1→1 , W2` ),
(21)
and the second inequality is by data reconstruction requirement (1). However notice that (n − `)H(S`+1→1 , W2` |M1` ) = (n − `)H(W2` |M1` ) + (n − `)H(S`+1→1 |M1` , W2` ) n X (s) = (n − `)H(W2` |M1` ) + H(Si→1 |M1` , W2` ) ≥ (n − = (n −
i=`+1 ` ` `)H(W2 |M1 ) + H(S`+1→1 , . . . , Sn→1 |M1` , W2` ) `)H(W2` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1 |M1` , W2` ) ` − 1)H(W2` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` )
= (n − (n − ` − 1)(` − 1) ≥ H(W1` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ), `
(22)
where in the last step we applied Han’s inequality, and also used the symmetry property. Putting (20) and (22) together gives H(W1` |M1` ) + H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) ≥ B`+1 + H(S`+2→1 , . . . , Sn→1 , W1`+1 |M1`+1 ) + +
H(S`+1→1 , . . . , Sn→1 , W1` |M1` ) , n−` 7
(n − ` − 1)(` − 1) H(W1` |M1` ) `(n − `) (23)
which has common terms on both sides that can be eliminated, and this leads to exactly the inequality in the lemma. In the proof of the lemma above, after several steps of derivation, the same terms in the original quantity reappear, albeit with different coefficients. The same terms are subtracted on both sides of the inequality, which can be conceptually viewed as recursively applying the same chains of inequalities. We are now ready to present the proof of Theorem 3. Proof of Theorem 3. The theorem is proved through an induction, where we show that for m = 1, 2, . . . , d, m X (n − 2)(n − k) + 1 n−2 m−1 (n − 2)α + β ≥ H(W1m |M1m ) Bk + − k(n − k) m m(n − m) k=1
+
H(Sm+1→1 , . . . , Sn→1 , W1m |M1m ) . n−m
(24)
The theorem isPthen simply a consequence of this statement when setting m = d = n − 1 and normalizing both sides by n−1 k=1 Bk , and taking into account of the facts n−2 m−1 = 0, (25) − m m(n − m) and H(Sm+1→1 , . . . , Sn→1 , W1m |M1m ) ≥ 0.
(26)
To show (24) is true for m = 1, we write the following chain of inequalities (n − 2)α + β ≥ (n − 2)H(W1 ) + H(S2→1 ) (n − 1)H(S2→1 ) = (n − 2)H(W1 ) + n−1 H(S2→1 , S3→1 , . . . , Sn→1 ) ≥ (n − 2)H(W1 ) + n−1 H(S2→1 , S3→1 , . . . , Sn→1 , W1 ) ≥ (n − 2)B1 + (n − 2)H(W1 |M1 ) + n−1 B1 H(S2→1 , S3→1 , . . . , Sn→1 , W1 |M1 ) = (n − 2)B1 + (n − 2)H(W1 |M1 ) + + n−1 n−1 (n − 2)(n − 1) + 1 H(S2→1 , S3→1 , . . . , Sn→1 , W1 |M1 ) = B1 + (n − 2)H(W1 |M1 ) + . n−1 n−1
(27)
Applying Lemma 1 now with ` = 1 gives (n − 2)(n − 1) + 1 1 B1 + (n − 2) − H(W1 |M1 ) (n − 2)α + β ≥ n−1 n−2 B2 H(S3→1 , . . . , Sn→1 , W1 , W2 |M1 , M2 ) + + . n−2 n−2
(28)
Thus (24) is true for m = 1. Next suppose (24) is true m = m0 , and we wish to show it is also true for m = m0 + 1. Notice that by Lemma 1 with ` = m0 , we have n−1 H(Sm0 +1→1 , . . . , Sn→1 , W1m0 |M1m0 ) H(W1m0 |M1m0 ) + m0 (n − m0 )(n − m0 − 1) n − m0 1 1 ≥ Bm0 +1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ). n − m0 − 1 n − m0 − 1 8
(29)
Thus (n − 2)α + β ≥
m0 X (n − 2)(n − k) + 1 k=1
k(n − k)
n−2 m0 − 1 Bk + − H(W1m0 |M1m0 ) m0 m0 (n − m0 )
H(Sm0 +1→1 , . . . , Sn→1 , W1m0 |M1m0 ) n − m0 m0 X (n − 2)(n − k) + 1 m0 − 1 n−2 n−1 ≥ − H(W1m0 |M1m0 ) Bk + − k(n − k) m0 m0 (n − m0 ) m0 (n − m0 )(n − m0 − 1) +
k=1
1 1 Bm0 +1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ) n − m0 − 1 n − m0 − 1 m0 X 1 (n − 2)(n − k) + 1 n−2 − H(W1m0 |M1m0 ) = Bk + k(n − k) m0 n − m0 − 1 +
k=1
1 1 Bm0 +1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ) n − m0 − 1 n − m0 − 1 m0 X (n − 2)(n − k) + 1 1 n−2 m0 ≥ Bk + − H(W1m0 +1 |M1m0 ) k(n − k) m0 n − m0 − 1 m0 + 1 +
k=1
1 1 Bm0 +1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ) n − m0 − 1 n − m0 − 1 m0 X (n − 2)(n − k) + 1 n−2 m0 1 = Bk + − + Bm0 +1 k(n − k) m0 + 1 (n − m0 − 1)(m0 + 1) n − m0 − 1 k=1 n−2 m0 + − H(W1m0 +1 |M1m0 +1 ) m0 + 1 (n − m0 − 1)(m0 + 1) 1 + H(Sm0 +2→1 , . . . , Sn→1 , W1m0 +1 |M1m0 +1 ), n − m0 − 1 +
(30)
which is precisely (24) for m = m0 + 1 after some elementary algebra. This completes the proof of Theorem 3. Readers familiar with the converse proof for the symmetric MLD coding problem may recognize certain similarity between the above proof and that in [2]. The key difference is that in the converse proof for MLD coding, one only needs to “peel” off the message rates by combining information in Wi ’s sequentially. Here the regeneration requirement necessitates a more elaborate peeling process, and Lemma 1 serves a similar role as Han’s inequality in the proof in [2].
5
¯1 , B ¯2 , B ¯3 ): Proof of Theorem 5 The Rate Region R4 (B
In this section the proof of Theorem 5 is presented. We start by providing a new code construction that achieves a particular operating point, which will play an instrumental role in the proof of Theorem 5.
5.1
A New Code
We first give a proposition, the proof of which requires a new code construction. Proposition 6. The rate pair ( 49 , 29 ) ∈ R4 (0, 31 , 23 ). Proof. We give a novel code construction where B1 = 0, B2 = 3, B3 = 6, α = 4 and β = 2. For concreteness, the code symbols and algebraic operations are assumed in GF(24 ). Let us denote the information symbols of message M2 as (x1 , x2 , x3 ), and the symbols of message M3 as (y1 , y2 , ..., y6 ). 9
Table 1: A code for n = 4 where (α, β) = (4, 2) and (B1 , B2 , B3 ) = (0, 3, 6). symbol 1 symbol 2 symbol 3 symbol 4 node 1 z1 z 5 + y1 y2 y3 node 2 z2 z 6 + y4 y1 y5 node 3 z3 z 7 + y2 y6 + z10 y4 node 4 z4 z 8 + y3 y5 + z9 y6 Encoding: First use a (10, 3) MDS erasure code (e.g., Reed-Solomon code) to encode (x1 , x2 , x3 ) into ten coded symbols (z1 , z2 , ..., z10 ), such that any three symbols can completely recover (x1 , x2 , x3 ). Then place linear combinations of (y1 , y2 , ..., y6 ) and (z1 , z2 , ..., z10 ) into the nodes as in Table 1, where the addition + is also in GF(24 ). Decoding M2 using any two nodes: To decode M2 , observe that any pair of nodes has two symbols involving the same yj , in the form of (zi + yj ) in one node and yj in the other node. For example node 2 has symbol z6 + y4 and node 3 has y4 . This implies zi can be recovered, and together with the first symbols stored in this pair of nodes, we have three distinct symbols in the set {z1 , z2 , ..., z10 }. Thus by the property of the MDS code, these three symbols can be used to recover (x1 , x2 , x3 ) and thus the message M2 . Decoding M2 and M3 using any three nodes: Recall using any two nodes we can recover the message M2 , and thus all the code symbols (z1 , z2 , ..., z10 ). This implies that when three nodes are available, we can eliminate all the zi symbols first in the linear combinations. However, it is clear that after this step all the symbols (y1 , y2 , ..., y6 ) are directly available, and thus the message M3 can be decoded. Repair using any three nodes: To regenerate the symbols in one node from the other three, each of the helper nodes sends the first symbol stored on the nodes as the initial step. Denote the y symbols on the failed node as (yi , yj , yk ), which may be stored in a form also involving z-symbols. The helper nodes each find in the symbols stored on it the one involving yi , yj and yk , respectively, and send these symbol combinations as the second step. The placement of the y-symbol guarantees that these symbols are stored on the three helper nodes respectively. Recall that from any three z-symbols available in the initial step, the message M2 can recovered, and thus any of the z-symbols. This implies that (yi , yj , yk ) can be recovered after eliminating the z symbols from the received symbol combinations in the second step, and thus all the symbols on the failed node can be successfully regenerated. Each helper node contributes exactly 2 symbols in this process. In the above code the linear combinations of z-symbols and y-symbols are not necessarily in GF(24 ), and it can even be in the base field GF(2). The only constraint on the alphabet size is through the MDS erasure code used when encoding (x1 , x2 , x3 ), and we have chosen GF(24 ) for simplicity. Readers familiar with the work [6] may recognize that if message M2 = (x1 , x2 , x3 ) does not exist, the remaining code involving y’s is a degenerate case of the repair-by-transfer code in [6]. Mixing improves the storage-repair-bandwidth tradeoff, however it has the side effect of possibly requiring downloading more symbols than the non-mixing solution when only a single message is required. Interestingly, in the above code, this side effect can be eliminated by downloading only the “non-mixing” symbols when all nodes are healthy.
5.2
Forward Proof of Theorem 5
¯1 , B ¯2 , B ¯3 ) for To prove the forward part of Theorem 5, we examine the extremal points of the region R4 (B ¯ ¯ ¯ ¯ ¯ ¯ fixed (B1 , B2 , B3 ). It should be noted that for different values of (B1 , B2 , B3 ), the extremal points may be different which causes certain complication. The intersecting points of any two inequalities in (13)-(18),
10
Table 2: Analysis of extremal points. α ¯ β¯ (14) (15) (13)
(16)
¯1 + B
¯2 B 2
+
¯3 B 3
(17) (18) (15) (14)
(16) (17) (18) (16)
(15)
(17) (18)
(16)
(17) (18)
(17)
(18)
¯1 + B¯2 + 3B¯3 B 2 8 ¯1 + 25B¯2 + 3B¯3 B 48
8
¯1 + B¯2 + 7B¯3 B 2 18 ¯2 ¯ 21B ¯ B1 + + 5B3 40
12
¯1 + 7B¯2 + 3B¯3 B 12 8 ¯1 + B¯2 + 5B¯3 B
2 12 ¯1 + 11B¯2 + 11B¯3 B 20 24 ¯1 + B¯2 + B¯3 B 3 2 ¯2 ¯ 8B B ¯ B1 + 15 + 23 ¯1 + 3B¯2 + B¯3 B 5 2
¯2 ¯3 ¯1 B B B 3 + 4 + 3 ¯1 ¯2 ¯3 B B 7B 3 + 4 + 24 ¯2 ¯3 ¯1 2B 5B B 3 + 9 + 18 ¯1 ¯2 ¯3 B B B 3 + 4 + 4 ¯1 ¯2 ¯3 B B B 3 + 5 + 6 ¯2 ¯3 ¯1 B B B 3 + 4 + 4 ¯1 ¯2 ¯3 B 5B B 3 + 24 + 4 ¯1 ¯2 ¯3 B B 2B 3 + 4 + 9 ¯1 ¯2 ¯3 B B B 3 + 5 + 6 ¯1 ¯2 ¯3 B B B 3 + 6 + 4 ¯1 ¯2 ¯3 B B 5B 3 + 4 + 24 ¯1 ¯2 ¯3 B B B 3 + 5 + 6 ¯1 ¯2 ¯3 B B B 3 + 3 + 6 ¯1 ¯2 ¯3 B B B 3 + 5 + 6 ¯1 ¯2 ¯3 B B B 3 + 5 + 6
remark X
(14)
X (15) (a) (b) (c) (d) (17) X
¯1 , B ¯2 , B ¯3 ). The first two columns specify when taken to be equality, are listed in Table 2 for any fixed (B which two inequalities induce the intersection. The intersecting points with X in the last column can be easily seen to be achievable using the separate coding scheme. The points labeled using (14), (15) and (17) in fact violate those constraints, and thus cannot be the extremal points. It remains to analyze the points (a)-(d), which may violate some ¯1 and B ¯2 values. constraints for certain B ¯2 ≤ B ¯3 . Due • For point (a): by the inequality (17), point (a) can be an extremal point only when 2B to the polytope structure of the region, we only need to check that the two extreme cases when ¯2 = B ¯3 , the rate point (a) is indeed achievable. The first case is trivial, since ¯2 = 0 and the 2B B it can be achieved by separate coding with only (4, 1, 3) and (4, 3, 3) components. For the second ¯1 , B ¯2 , 2B ¯2 ), however, we can encode the message M1 using a case, observe we need a code for (B B1 (4, 1, 3) code at rate point (α1 , β1 ) = (B1 , 3 ), and the messages M2 and M3 using the code given in Proposition 6, at (α2,3 , β2,3 ) = ( 4B3 2 , 2B3 2 ); superposition of the two codes leads to the the desired operating point. ¯2 ≥ B ¯3 . We • For point (b): by the inequality (16), point (b) can be an extremal point only when 2B ¯ ¯ ¯ only need to check that the two extreme cases when B3 = 0 and 2B2 = B3 , the rate point (b) is indeed achievable. The first case can be achieved by separate coding with only (4, 1, 3) and (4, 2, 3) components, while the second case reduces to the second case of point (a). • For point (c): by the inequality (16) and (17), point (c) can be an extremal point only when ¯2 ¯2 2B 6B ¯ ¯ ¯ 2 ≥ B3 ≥ 5 , which can hold only when B2 = B3 = 0. This is trivially achievable using a (4, 1, 3) code. ¯2 ≤ B ¯3 . The • For point (d): by inequality (16), point (d) can be an extremal point only when 2B ¯2 = 0 and 2B ¯2 = B ¯3 , which is again achievable by separate coding with two extreme cases are B only (4, 1, 3) and (4, 3, 3) components, and by the code given for the second case for point (a). 11
Table 3: Terms needed to prove Proposition 7 T1 H(S4→3 ) T2 H(S4→2 , S3→2 ) T3 H(S4→1 , S3→1 , S2→1 ) T4 H(W4 ) T5 H(S3→2 , W4 ) T6 H(S3→4 , S2→4 , W4 ) T7 H(W4 , W3 ) T8 H(S2→4 , W4 , W3 ) T9 H(S3→1 , S2→1 , W4 , W1 ) T10 H(S4→3 , M1+ ) T11 H(S4→2 , S3→2 , M1+ ) T12 H(W4 , M2+ ) T13 H(S3→2 , W4 , M2+ ) T14 H(S3→4 , W4 , M2+ ) T15 H(S3→2 , S2→4 , W4 , M2+ ) ¯+ T16 B 1 ¯+ T17 B 2 ¯+ T18 B 3 The proof is now complete.
5.3
Converse Proof of Theorem 5
In order to establish the converse of Theorem 5, we prove each inequality individually. It is clear that (13) is trivially true due to the result on multilevel diversity coding in [2], and (14) is true because of Theorem 3. It remains to show that inequalities (15)-(18) are true, and we prove each as a proposition. Instead of writing the proofs in the conventional approach as chains of inequalities, we utilize the computational approach developed in [11], and prove these inequalities by tabulation. Each inequality is given as two tables, the first of which lists the joint entropy terms in the proof, and the second of which gives the known sub-modular inequalities (Shannon-type inequalities) and the coefficients of these inequalities, whose last row, as the summation of all the other rows, is exactly the sought-after inequality. Note that each row in the second table is a sub-modular inequality (or an integer multiple of such an inequality), possibly after permutation of the indices for each entropy term. For example, the second line of Table 4 is 5H(S4→3 ) + 5H(S4→2 , S3→2 ) − 5H(S4→1 , S3→1 , S2→1 ) ≥ 0,
(31)
which is equivalent to the simple inequality 5[H(S2→1 ) + H(S4→1 , S3→1 ) − H(S4→1 , S3→1 , S2→1 )] ≥ 0,
(32)
after taking into account of the symmetry of the optimal solutions. Further note that for n = 4 we have ¯ + = 1, however we still write it as B ¯ + to make explicit its meaning of sum rate. B 3 3 ¯ ∈ R4 (B ¯1 , B ¯2 , B ¯3 ), we have Proposition 7. For any (α ¯ , β) 4¯ 3¯ 5¯ ¯ ¯ + + 3B ¯ + + 15B ¯ +. 24[¯ α + β] ≥ 24 B1 + B2 + B3 = 14B 1 2 3 3 4 8 Proof. See Table 3 and Table 4.
12
(33)
Table 4: Proof by Tabulation of Proposition 7, with terms defined in Table 3. T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16 T17 T18 16 −8 4 −4 4 −4 2 2 −2 12 −6 −6 2 2 −2 −2 6 6 −6 4 4 −4 −4 4 −4 4 −4 3 3 −3 −3 3 −3 3 −3 3 3 −3 −3 3 −3 3 −3 3 3 −3 −3 2 2 −2 −2 24 24 −14 −3 −15 ¯ ∈ R4 (B ¯1 , B ¯2 , B ¯3 ), we have Proposition 8. For any (¯ α, β) 3¯ 5¯ ¯ ¯ +. ¯ + + 9B ¯+ + B ¯ 6[2¯ α + 3β] ≥ 6 3B1 + B2 + B3 = 8B 3 2 1 3 2
(34)
Proof. See Table 5 and Table 6. ¯ ∈ R4 (B ¯1 , B ¯2 , B ¯3 ), we have Proposition 9. For any (¯ α, β) 5¯ 5¯ ¯ ¯ +. ¯ + + 10B ¯ ¯ + + 2B 12[¯ α + 2β] ≥ 12 B1 + B2 + B3 = 8B 3 2 1 3 6
(35)
Proof. See Table 7 and Table 8. ¯ ∈ R4 (B ¯1 , B ¯2 , B ¯3 ), we have Proposition 10. For any (¯ α, β) 1¯ 1¯ 1¯ ¯ ¯+ + B ¯ + + 5B ¯ +. 30β ≥ 30 B1 + B2 + B3 = 4B 1 2 3 3 5 6
(36)
Proof. See Table 9 and Table 10.
6
Conclusion
We considered multilevel diversity coding with regeneration, which addresses the storage vs. repairbandwidth tradeoff in distributed storage systems with heterogeneous reliability and access latency requirements. It is shown that for the minimum storage point on the optimal tradeoff, separate coding is sufficient, and there is no need to mix different contents. On the other hand, a complete characterization of the tradeoff region is provided for the case of four nodes, which reveals that mixing can strictly improve the general tradeoff. In other words, separate coding using individual regenerating code is not optimal, and mixing the contents can strictly improve the efficiency of the codes.
13
Table 5: Terms needed to prove Proposition 8 T1 H(S4→3 ) T2 H(S4→2 , S3→2 ) T3 H(S4→1 , S3→1 , S2→1 ) T4 H(W4 ) T5 H(S3→2 , W4 ) T6 H(S3→4 , W4 ) T7 H(S3→4 , S2→4 , W4 ) T8 H(W4 , W3 ) T9 H(S2→4 , W4 , W3 ) T10 H(S3→1 , S2→1 , W4 , W1 ) T11 H(S4→3 , M1+ ) T12 H(S4→2 , S3→2 , M1+ ) T13 H(S4→3 , S3→4 , M1+ ) T14 H(S4→3 , S3→4 , S2→4 , M1+ ) T15 H(W4 , M2+ ) T16 H(S3→2 , W4 , M2+ ) T17 H(S3→4 , W4 , M2+ ) T18 H(S3→2 , S2→4 , W4 , M2+ ) ¯+ B T19 1 ¯+ T20 B 2 ¯+ B T21 3
Table 6: Proof by Tabulation of Proposition 8, with terms defined in Table 5. T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16 T17 T18 T19 T20 12 −6 4 −4 4 −4 4 4 −4 4 −4 4 −4 2 2 −2 −2 2 2 −2 −2 2 2 −2 2 −2 −2 2 2 2 −2 −2 2 −2 2 2 2 −2 1 1 −1 1 −1 1 1 1 −1 1 −1 1 1 1 −1 18 12 −8 −1
14
T21
−2 −2 −1 −1 −1 −1 −1 −9
Table 7: Terms needed to prove Proposition 9 T1 H(S4→3 ) T2 H(S4→2 , S3→2 ) T3 H(S4→1 , S3→1 , S2→1 ) T4 H(W4 ) T5 H(S3→4 , W4 ) T6 H(S3→2 , W4 ) T7 H(S3→4 , S2→4 , W4 ) T8 H(W4 , W3 ) T9 H(S2→4 , W4 , W3 ) T10 H(S3→1 , S2→1 , W4 , W1 ) T11 H(S3→4 , S3→1 , S2→1 , W4 , W1 ) T12 H(S4→3 , M1+ ) T13 H(S4→3 , M2+ ) T14 H(S4→2 , S3→4 , M2+ ) T15 H(S4→2 , S3→2 , M1+ ) T16 H(S3→2 , W4 , M2+ ) T17 H(S3→4 , W4 , M2+ ) T18 H(S3→2 , S2→4 , W4 , M2+ ) T19 H(S3→2 , S2→3 , W4 , M2+ ) T20 H(S3→2 , S2→4 , S1→4 , W4 , M2+ ) ¯+ T21 B 1 ¯+ T22 B 2 ¯+ B T23 3
Table 8: T1 T2 T3 T4 T5 16 −8 4 4 4 −4 1 1 4 4 4 4 2
1
Proof by Tabulation of Proposition 9, with terms defined in Table 7. T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16 T17 T18 T19 T20 T21 T22 T23 −4
−4
4 −1
−1
−4 −4 −4
4 2 −2 2
−2 2 −1 −1
1
−4
4 −4
1
−2 −2 −1
1
−1 −1
1 1
−1 24
1
1 1 1 1 1 2 1
1
1
−1 −1 −1
1 −1
1 −1
1 −1
−1 −1 −1 −1 −1 −1 −1
−1 −8 −2 −10
12
15
Table 9: Terms needed to prove Proposition 10 T1 H(S4→3 ) T2 H(S4→2 , S3→2 ) T3 H(S4→1 , S3→1 , S2→1 ) T4 H(W4 ) T5 H(S3→2 , W4 ) T6 H(S3→1 , S2→1 , W4 , W1 ) T7 H(S4→2 , S4→1 , S3→2 , S3→1 , W2 , W1 ) T8 H(S4→3 , M1+ ) T9 H(S4→3 , M2+ ) T10 H(S4→2 , S3→2 , M1+ ) T11 H(S4→3 , S3→4 , M1+ ) T12 H(S4→2 , S3→4 , M2+ ) T13 H(S4→1 , S3→4 , S2→4 , M2+ ) T14 H(S4→1 , S3→4 , S3→2 , S2→4 , M2+ ) ¯+ T15 B 1 ¯+ T16 B 2 ¯+ T17 B 3
Table 10: Proof by Tabulation of Proposition T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 20 −10 5 5 −5 5 5 −5 1 −1 1 2 2 −2 −2 2 −2 2 1 −1 1 −1 1 1 −1 −1 1 1 1 2 −1 1 1 1 30
16
10, with terms defined in Table 9. T11 T12 T13 T14 T15 T16 T17
−1 −2 −1
1 −1
1 −1 −1 1
−1 −1
−1 −1
1
−1 −1 −1 −4 −1 −5
Another notable feature of this work is that we further developed the computational approach in [11] to identify and prove the converse. As a result, the converse proof is presented as tabulation, without translating the tables to the conventional form of proofs usually seen in information theory literature. It is our belief that this computational approach will be able to play an even more significant role in future studies. In order to help future researchers to utilize such data, we post the result in this paper as part of the online collection of “ Solutions of Computed Information Theoretic Limits (SCITL)” hosted at [12], which hopefully in the future can serve as a data depot for information theoretic limits obtained through computational approaches. There are several immediate research directions to follow. Firstly, the code construction given for n = 4 can be generalized to other parameters straightforwardly, and we shall address this issue in a later work. Secondly, the MLD-R problem can be extended to allow more general values of d not equal to n−1. Thirdly, it is important to understand how much improvement over separate coding can be attained by mixing the contents. Lastly, it may also be useful to consider the analogous requirement in the locally repairable code setting [13–15].
Appendix: Proofs of Proposition 1 and Proposition 2 Proof of Proposition 1. It is known that the rate region of (3, 1, 2) regenerating codes (before normalization) is α1 ≥ B1 ,
β1 ≥
B1 , 2
(37)
and the rate region of (3, 2, 2) regenerating codes is 2α2 ≥ B2 ,
α2 + β2 ≥ B2 ,
3β2 ≥ B2 .
(38)
It is straightforward to verify that separate coding indeed leads to the rate region given in Proposition 1 using Fourier-Motzkin elimination. Proof of Proposition 2. Using Fourier-Motzkin elimination directly for this case is time-consuming, and we instead consider inner bounding and outer bounding the region. It is straightforward to show that the inequalities in Proposition 2 have to hold in this separate coding based achievability scheme, e.g., 4α + 6β = 4(α1 + α2 + α3 ) + 6(β1 + β2 + β3 ) (a)
≥ 6B1 + 4(α2 + α3 ) + 6(β2 + β3 ) = 6B1 + α2 + 3(α2 + 2β2 ) + (4α3 + 6β3 ) (b)
7 ≥ 6B1 + B2 + (4α3 + 6β3 ) 2 (c) 7 ≥ 6B1 + B2 + 3B3 , 2
(39)
where (a) is because of (41), (b) because of (42), and (c) because of (44). Other inequalities can be shown in a similar manner. Next we show this outer bound is indeed achievable using the separate coding based scheme. Computing the intersections of any two linear bounds in (4)-(8) yields a total of 10 points, which are the possible extremal (corner) points of the region. However, some of them do not satisfy all the inequalities, and after eliminating them, the corner points of the region in Proposition 2 are thus given
17
as (see also Fig. 2) ¯2 B ¯3 B ¯1 B ¯2 B ¯3 B + , + + ) 2 3 3 4 3 ¯ ¯ ¯ ¯ ¯ ¯1 + B2 + 3B3 , B1 + B2 + B3 ) (B 2 8 3 4 4 ¯2 B ¯3 B ¯1 B ¯2 B ¯3 B ¯1 + (B + , + + ) 2 2 3 4 6 ¯ ¯ ¯ ¯ ¯ ¯1 + 3B2 + B3 , B1 + B2 + B3 ). (B (40) 5 2 3 5 6 Next we show that they are indeed achievable using the separate coding based scheme, which implies that the region given in (4)-(8) is achievable by a time-sharing argument. It is known that the rate region of (4, 1, 3) regenerating codes is ¯1 + (B
α1 ≥ B1 ,
β1 ≥
B1 , 3
(41)
and thus there is only one corner point of the rate region, which is (B1 , B31 ). The rate region of (4, 2, 3) regenerating codes is 2α2 ≥ B2 ,
α2 + 2β2 ≥ B2 ,
5β2 ≥ B2 ,
(42)
and there are two corner points in this rate region, which are 3B2 B2 B2 B2 , ), ( , ). 2 4 5 5 The rate region of (4, 3, 3) regenerating codes is given in [11] (
3α3 ≥ B3 , 2α3 + β3 ≥ B3 , 4α3 + 6β3 ≥ 3B3 , 6β3 ≥ B3 .
(43)
(44)
and there are three corner points in this rate region, which are B3 B3 3B3 B3 B3 B3 , ), ( , ), ( , ). (45) 3 3 8 4 2 6 It is easily seen that the rate pairs in (40) are all simple linear combinations of the corner point rate pairs of the (4, 1, 3), (4, 2, 3) and (4, 3, 3) codes after proper normalization, i.e., by dividing (B1 +B2 +B3 ), and thus achievable using the separate coding based scheme. The proof is now complete. (
Converse Proof of Theorem 4. It can be shown that we only need to consider symmetric codes, where permutations of node indices do not change joint entropy values; see [11] for more details about this symmetry. Thus without loss of generality, we can restrict the proof to symmetric codes only. Denote the information stored at node i as Wi , the helper information sent from node i to node j as Si,j , and the message to be decoded using any j nodes as Mj . First write the following inequalities (a)
α ≥ H(W1 ) = H(W1 , M1 ) ≥ H(W1 |M1 ) + B1 1 (s) = B1 + [H(W1 |M1 ) + H(W2 |M1 )] 2 1 ≥ B1 + H(W1 , W2 |M1 ) 2 1 (c) = B1 + H(W1 , W2 , M2 |M1 ) 2 1 1 ≥ B1 + H(M2 |M1 ) ≥ B1 + B2 , 2 2 18
(46)
where (a) is because M1 can be recovered from W1 alone, and (s) implies by the existence of symmetric solutions, (c) is because (W1 , W2 ) together can recover M2 . Dividing both sides of (46) by (B1 + B2 ) gives the result. From here, we use (s) to denote by the reason of the symmetry. Next we bound β by writing 1 β ≥ [H(S1→3 ) + H(S2→3 )] 2 1 ≥ [H(S1→3 , S2→3 )] 2 (a) 1 = [H(S1→3 , S2→3 , W3 , M1 )] 2 1 1 ≥ B1 + H(S1→3 , S2→3 , W3 |M1 ), 2 2
(47)
where (a) is because from the helper message S1→3 and S2→3 , we can regenerate W3 . To bound the second term above, we write H(S1→3 , S2→3 , W3 |M1 )] (a)
= H(S1→3 , S2→3 , W3 , S3→1 , S3→2 |M1 )
≥ H(S1→3 , S2→3 , S3→1 , S3→2 |M1 ) (s) 1 = [H(S1→3 , S2→3 , S3→1 , S3→2 |M1 ) 3 + H(S1→2 , S3→2 , S2→1 , S2→3 |M1 ) + H(S3→1 , S2→1 , S1→2 , S1→3 |M1 )] (b)
1 [H(S1→3 , S2→3 , S3→1 , S3→2 , S1→2 , S2→1 |M1 ) 3 + H(S3→2 , S2→3 |M1 ) + H(S3→1 , S2→1 , S1→2 , S1→3 |M1 )] (c) 1 = [B2 + H(S3→2 , S2→3 |M1 ) 3 + H(S3→1 , S2→1 , S1→2 , S1→3 |M1 )] 1 ≥ [B2 + H(S3→2 , S2→3 , S3→1 , S2→1 , S1→2 , S1→3 |M1 )] 3 1 2B2 ≥ [B2 + H(M2 |M1 )] ≥ , 3 3 ≥
(48)
where (a) is because S3→1 , S3→2 are functions of W3 , (b) is because H(S1→3 , S3→1 |S3→2 , S2→3 , M1 ) ≥ H(S1→3 , S3→1 |S1→2 , S3→2 , S2→1 , S2→3 , M1 ),
(49)
and (c) is because from (S1→3 , S2→3 , S3→1 , S2→1 ) we can regenerate (W1 , W3 ), subsequently decode M2 . It follows that 1 1 β ≥ B1 + B2 . 2 3
19
(50)
The remaining inequality in Theorem 4 can be proved by writing the following: α+β 1 ≥ B1 + H(W1 |M1 ) + [B1 + H(S1→3 , S2→3 , W3 |M1 )] 2 3 1 = B1 + H(W1 , S1→2 , S1→3 |M1 ) + H(S1→3 , S2→3 , W3 |M1 ) 2 2 1 3 = B1 + H(W1 , S1→2 , S1→3 |M1 ) 2 2 1 + [H(W1 , S1→2 , S1→3 |M1 ) + H(S1→3 , S2→3 , W3 |M1 )] 2 1 3 ≥ B1 + H(W1 , S1→2 , S1→3 |M1 ) 2 2 1 + [H(W1 , W3 , S1→2 , S1→3 , S2→3 |M1 ) + H(S1→3 |M1 )] 2 1 1 3 = B1 + B2 + [H(W1 , S1→2 , S1→3 |M1 ) + H(S1→3 |M1 )] 2 2 2 1 1 (s) 3 = B1 + B2 + [H(W1 , S1→2 , S1→3 |M1 ) + H(S2→3 |M1 )] 2 2 2 3 1 1 ≥ B1 + B2 + [H(W1 , S1→2 , S1→3 , S2→3 |M1 )] 2 2 2 3 ≥ B1 + B2 . 2
(51)
The proof is thus complete.
References [1] N. B. Shah, Kangwook Lee, and K. Ramchandran, “The MDS queue: Analysing the latency performance of erasure codes,” in Proceeds, IEEE International Symposim on Information Theory,, pp. 861–865, Hololulu HI, Jun.-Jul. 2014. [2] J. R. Roche, R. W. Yeung, and K. P. Hau, “Symmetrical multilevel diversity coding,” IEEE Trans. Information Theory, vol. 43, no. 5, pp. 1059–1064, May 1997. [3] R. W. Yeung and Z. Zhang, “On symmetrical multilevel diversity coding,” IEEE Trans. Information Theory, vol. 45, no. 2, pp. 609–621, Mar. 1999. [4] A. G. Dimakis, P. B. Godfrey, Y. Wu, M. Wainwright and K. Ramchandran, “Network coding for distributed storage systems,” IEEE Trans. Information Theory, vol. 56, no. 9, pp. 4539–4551, Sep. 2010. [5] A. G. Dimakis, K. Ramchandran, Y. Wu, C. Suh, “A survey on network codes for distributed storage,” Proceedings of the IEEE, vol. 99, no. 3, pp. 476–489, Mar. 2011. [6] N. B. Shah, K. V. Rashmi, P. V. Kumar and K. Ramchandran, “Distributed storage codes with repair-by-transfer and non-achievability of interior points on the storage-bandwidth tradeoff,” IEEE Trans. Information Theory, vol. 58, no. 3, pp. 1837-1852, Mar. 2012. [7] K. V. Rashmi, N. B. Shah, and P. V. Kumar, “Optimal exact-regenerating codes for distributed storage at the MSR and MBR points via a product-matrix construction,” IEEE Trans. Information Theory, vol. 57, no. 8, pp. 5227–5239, Aug. 2011.
20
[8] V. Cadambe, S. Jafar, H. Maleki, K. Ramchandran and C. Suh, “Asymptotic interference alignment for optimal repair of MDS codes in distributed storage,” IEEE Trans. Information Theory, vol. 59, no. 5, pp. 2974–2987 , May 2013. [9] I. Tamo, Z. Y. Wang and J. Bruck, “Zigzag codes: MDS array codes with optimal rebuilding,” IEEE Trans. Information Theory, vol. 59, no. 3, pp. 1597–1616, Mar. 2013. [10] D. S. Papailiopoulos, A. G. Dimakis and V. R. Cadambe, “Repair optimal erasure codes through Hadamard designs,” IEEE Trans. Information Theory, vol. 59, no. 5, pp. 3021–3037, May 2013. [11] C. Tian, “Characterizing rate region of the (4, 3, 3) exact-repair regenerating codes,” IEEE Journal on Selected Areas of Communications, Vol. 32, No. 5, pp. 967-975, May 2014. [12] Solutions of Computed Information Theoretic Limits (SCITL), ~ctian1/SCITL.html.
http://web.eecs.utk.edu/
[13] C. Huang, H. Simitci, Y. K. Xu, A. Ogus, B. Calder, P. Gopalan, J. Li, and S. Yekhanin, “Erasure coding in Windows Azure storage,” in Proceeds, 2012 USENIX Annual Technical Conference, San Jose CA. [14] D. S. Papailiopoulos and A. Dimakis, “Locally repairable codes,” IEEE Trans. on Information Theory, vol.60, no.9, pp. 5843-5855, Sep. 2014. [15] A. S. Rawat, O. O. Koyluoglu, N. Silberstein and S. Vishwanath, “Optimal locally repairable and secure codes for distributed storage systems,” IEEE Trans. on Information Theory, vol.60, no.1, pp. 212-236, Jan. 2014.
21
