Generalized Sphere Packing Bound - Semantic Scholar
Generalized Sphere Packing Bound Arman Fazeli(1), Alexander Vardy(1), and Eitan Yaakobi(2) (1) - University of California San Diego
(2) - Technion Israel Institute of Technology
1
The Sphere Packing Bound • Upper bound on a code C with min dist 2r+1 –
• This bound is valid for other cases as well where the error graph is regular ( ) what happens if the graph is not regular? – Naïve solution: choose a ball in the graph
to be the minimum size of
2
What about other bounds? • The Gilbert-Varshamov lower bound: There exists a code with min dist r+1 of size • If the graph is not regular, it is possible to choose as the average size of a ball • There exists a code with min dist r+1 of size Does the same analogy hold for the sphere packing bound? Is an upper bound on a code with min dist 2r+1?
3
The Deletion Channel • An example of non-regular graph – 10010 -> 0010, 1010, 1000, 1001 – 11100 -> 1100, 1110 – 10101 -> 0101, 1101, 1001, 1011, 1010
• It is not possible to apply the sphere packing bound • Previous results – Levenshtein ‘66: asymptotic upper bound – Kulkarni & Kiyavash ‘12: a method to derive explicit nonasymptotic upper bound using tools from hypergraph theory
• Can this method be generalized for other graphs? – Grain errors:
• Kashyap & Zemor ‘13; Gabrys, Yaakobi, Dolecek ‘13
– Multi-permutations with the Kendall’s tau dist • Buzaglo, Yaakobi, Etzion, Bruck ‘13
4
Hypergraphs • Let H=(X,E) be a hypergraph, where – X={x1,…,xn} – set of vertices, E={E1,…,Em} – set of hyperedges – A is a binary n×m incidence matrix of H
• Matching - a collection of pairwise disjoint hyperedges – The matching number ν (H) - the size of the largest matching
• Transversal - a vertices subset that intersects every hyperedge – The transversal number τ(H) - the size of the smallest transversal
5
Hypergraphs • The matching number • The transversal number • These problems satisfy weak duality ν(H) ≤ τ(H) • The relaxation versions of these problems
satisfy strong duality
• Every vector w in τ*(H) is called a fractional transversal 6
The Deletion Channel – KK’12 • Define a hypergraph H(X,E): – X = {0,1}n-1 , E = {all 2n single-deletion balls}
• Every single-deletion correcting code of length n is a matching in the hypergraph H • Find the value of τ*(H) or any fractional transversal to get an explicit upper bound
7
The General Case • G=(X,E) is a graph describing an error channel graph – X = the set of all possible words (transmitted and received) – E = the set of vertices pairs of dist one • The distance d(x,y) b/w x and y is the length of the shortest path from x to y (not necessarily symmetric)
– Br(x) = {y ∊ X : d(x,y)≤r}; degr(x) = |Br(x)|
• For any r>0, H(G,r)=(Xr,Er) is a hypergraph for G – Xr=X, Er={Br(x) : x∊X}
• Every code C in G is a matching in H(G,r) • AG(n,d) - the max size of a code w/ min dist d in G For every r>0:
• Q: Does the following hold? is called the Average Sphere Packing Value: ASPV(G,r) 8
Example: The Z Channel • GZ=(XZ,EZ), XZ={0,1}n
BZ,1(10010)={10010,00010,10000}
• H(GZ,r) = (XZ,r,EZ,r), XZ,r=XZ={0,1}n
9
Example: The Z Channel • The average size of a ball with radius r
• The average sphere packing value
• For r=1: 10
So what is the problem? • A channel graph G=(X,E) w/ hypergraph H(G,r)=(Xr,Er) • A code with min dist 2r+1 in G is a matching in H(G,r) • An upper bound is given by
• The good news: there is an explicit upper bound! • The bad news: It is not necessarily easy to calculate it – Need to solve a linear programming… – Usually the number of variables and constraints in exponential
• Our goal: how to calculate the value of 11
Some General Results • Lemma: If for all x∊X, degrin(x) ≤ Δ then
Proof: Since degrin(x) ≤ Δ then ⇒ For any transversal w,
Therefore,
12
Some General Results • Lemma: If for all x∊X, degrin(x) ≤ Δ then
• Lemma: If for all x∊X, degr(x) ≥ Δ, then
Proof: The vector w=1/Δ is a fractional transversal Thus,
13
Some General Results • Lemma: If for all x∊X, degrin(x) ≤ Δ then
• Lemma: If for all x∊X, degr(x) ≥ Δ, then
• Corollary: If G is symmetric and regular then the generalized sphere packing bound and the sphere packing bound coincide, and
14
Monotonicity and Fractional Transversals • The vector w is a fractional transversal if w ≥ 0 and • Lemma: The vector w, given by is a fractional transversal Proof: If y∊Br(xi), then xi∊Brin(y) and Therefore,
15
Monotonicity and Fractional Transversals • The vector w is a fractional transversal if w ≥ 0 and • Lemma: The vector w, given by is a fractional transversal • Def: G is called monotone if for all x∊X and y∊Br(x) • Lemma: If G is monotone then the vector is a fractional transversal • Corollary: If G is monotone an upper bound on AG(n,2r+1) is called the monotonicity upper bound MB(G,r)
16
Cont. Ex: The Z Channel • x,y∊{0,1}n, if y∊BZ,r(x), wH(y)≤wH(x) and degr(y)≤degr(x) • The vector given by is a fractional transversal • The monotonicity upper bound for the Z channel: • For r=1: • The average sphere packing bound • Is it possible to do better…?
17
Can we find the optimal transversal? • w is a fractional transversal if w ≥ 0 and • For the Z channel: – 2n constraints: Ex, n=3: • • • •
111: w111+w110+w101+w011 ≥ 1 110: w110+w100+w010 ≥ 1, 101: w101+w100+w100 ≥ 1, 011: w011+w010+w001 ≥ 1 100: w100+w000 ≥ 1, 010: w010+w000 ≥ 1, 001: w001+w000 ≥ 1 000: w000 ≥ 1
– Probably vectors w/ the same weight will have the same value – If so, only n+1 constraints: • • • •
3: w3+3w2 ≥ 1 2: w2+2w1 ≥ 1 1: w1+w0 ≥ 1 0: w0 ≥ 1
• Is it still possible to find the value of
? 18
Automorphisms on Graphs • Given a graph G=(X,E), an automorphism is a permutation that preserves adjacency π:X->X s.t. (x,y)∊E iff (π(x),π(y))∊E • The automorphisms set Aut(G)={π∊Sn : π is an automorphism in G} is a subgroup of Sn under composition • Aut(G) induces an equivalence order R on X: (x,y)∊R iff there exists π∊ Aut(G) and π(x)=y and partitions X into n(G) equivalence classes • For a vector w and automorphism π, the vector wπ is (wπ)i = wπ(i) • Lemma: If w is a transversal and π an automorphism then wπ is a transversal as well 19
Automorphisms on Graphs • Given a graph G=(X,E), an automorphism is a permutation that preserves adjacency π:X->X s.t. (x,y)∊E iff (π(x),π(y))∊E Proof: • The automorphisms set Need to show: for 1 ≤ i ≤n Aut(G)={π∊Sn : π is an automorphism in G} y ∊ Br(xof ∊ Br(π(x i) iff i)) under composition is a subgroup Snπ(y) Therefore, • Aut(G) induces an equivalence order R on X: (x,y)∊R iff there exists π∊ Aut(G) and π(x)=y and partitions X into n(G) equivalence classes • For a vector w and automorphism π, the vector wπ is (wπ)i = wπ(i) • Lemma: If w is a transversal and π an automorphism then wπ is a transversal as well 20
Automorphisms on Graphs • Aut(G) = {π∊Sn : π is an automorphism in G} • An equivalence order R on X: (x,y)∊R iff there exists π∊ Aut(G) and π(x)=y • Lemma: If w is a transversal and π an automorphism then wπ is a transversal as well • Wc = {w : w is a transversal and Σwi=c} • Theorem: If Wc≠∅ then Wc contains a transversal which assigns the same weight to the equivalence classes of R • This result holds also for any subgroup of Aut(G)
21
Cont. Ex: The Z Channel • For every σ∊Sn define a permutation πσ:{0,1}n->{0,1}n for all x∊{0,1}n, (πσ(x))i=xσ(i) • The set K={πσ : σ∊Sn} is a subgroup of Aut(GZ) and partitions {0,1}n into n+1 equivalence classes XK(GZ) = {X0,X1,…,Xn}, Xi=all vectors of weight i • The generalized sphere packing bound now becomes:
22
Cont. Ex: The Z Channel • The generalized sphere packing bound now becomes:
• For r=1: where
,
and
– For example, n=3: w*3=0, w*2= (1-0)/3=1/3, w*1= (1-1/3)/2=1/3, w*0= 1
23
Best known upper bound by Weber, De Vroedt, and Boekee ‘88
24
Cont. Ex: The Z Channel • For arbitrary r, the optimal solution is given by
25
Comparison for r=2
Comparison for r=3
26
Limited Magnitude Channels • Asymmetric errors GA,q=(X,E): X=[q]n
• The graph is monotone • The monotonicity upper bound for r=1 is
• The average sphere packing bound 27
The Asymmetric Channel • The linear programming problem to find is given by
• By automorphisms as in the Z channel we can divide to the equivalence classes
28
The Asymmetric Channel • Theorem: the vector
given by
is a fractional transversal
29
Results for q=3
30
The Symmetric Channel • No longer monotonicity because the channel is symmetric • Can follow similar steps… Comparison for q=3
Comparison for q=4
31
Does the average sphere packing hold? • The average size of the ball is (1∙5+4∙1)/5=9/5 • The average sphere packing value 5/(9/5)=25/9 • But the min dist of the code {x2,x3,x4,x5} is infinity… • Another example
– n=k2 vertices into two groups k and n-k – Every vertex from the 1st group is connected to exactly k-1 vertices from the 2nd group – All n-k vertices in the second group are connected
– But there is a code with k vertices… 32
Back to the Deletion Channel… • GD=(XD,ED), XD={0,1}nU{0,1}n-1 and
• For x∊{0,1}n, • The hypergraph H(GD,1)=(XD,1,ED,1), XD,1={0,1}n-1, ED,1 = {BD,1(x) : x∊{0,1}n } • The generalized sphere packing bound
33
Back to the Deletion Channel… • For x∊{0,1}n, ρ(x)= the number of runs in x – x=001010010, ρ(x)=7
degD,1(x) = ρ(x) For every y∊BD,1(x), ρ(y)≤ρ(x)=degD,1(x) This graph satisfies a similar property to monotonicity The vector given by is a fractional transversal (KK’12) • The corresponding upper bound is • • • •
34
Is it possible to do better…? • A transversal is given by wx=1/ρ(x) • A vector x has ρ(x) neighbors – If they all have ρ(x) runs then we can’t do better • x=001100: BD(x)={01100(w=1/3), 00100(w=1/3), 00110(w=1/3)}
– But if many neighbors have less than ρ(x) runs…?
• x=000010: BD(x)={000010(w=1/3), 00000(w=1), 00001(w=1/2)}
• For x, μ(x) = # of middle runs of length 1 – x=001010010, μ(x)=4 – 0≤μ(x) ≤ρ(x)-2
• Nn(ρ, μ) = # of vectors with ρ runs and μ middle-1 runs – Nn(1,0)=2
35
Is it possible to do better…? • • • •
For x, μ(x) = # of middle runs of length 1 Nn(ρ, μ) = # of vectors with ρ runs and μ middle-1 runs Lemma: For 2≤ρ≤n and 0≤μ≤ρ-2, Theorem: The vector defined by
is a fractional transversal • Theorem: The value
satisfies
36
Comparison of the Different Bounds
37
Conclusion • • • •
A method to generalize the sphere packing bound The Deletion channel – Kulkarni & Kiyavash ’12 General Results Some specific channels – – – – –
The Z channel Asymmetric errors Deletion channel Grain errors Projective space
Thank You! 38
(2) - Technion Israel Institute of Technology
1
The Sphere Packing Bound • Upper bound on a code C with min dist 2r+1 –
• This bound is valid for other cases as well where the error graph is regular ( ) what happens if the graph is not regular? – Naïve solution: choose a ball in the graph
to be the minimum size of
2
What about other bounds? • The Gilbert-Varshamov lower bound: There exists a code with min dist r+1 of size • If the graph is not regular, it is possible to choose as the average size of a ball • There exists a code with min dist r+1 of size Does the same analogy hold for the sphere packing bound? Is an upper bound on a code with min dist 2r+1?
3
The Deletion Channel • An example of non-regular graph – 10010 -> 0010, 1010, 1000, 1001 – 11100 -> 1100, 1110 – 10101 -> 0101, 1101, 1001, 1011, 1010
• It is not possible to apply the sphere packing bound • Previous results – Levenshtein ‘66: asymptotic upper bound – Kulkarni & Kiyavash ‘12: a method to derive explicit nonasymptotic upper bound using tools from hypergraph theory
• Can this method be generalized for other graphs? – Grain errors:
• Kashyap & Zemor ‘13; Gabrys, Yaakobi, Dolecek ‘13
– Multi-permutations with the Kendall’s tau dist • Buzaglo, Yaakobi, Etzion, Bruck ‘13
4
Hypergraphs • Let H=(X,E) be a hypergraph, where – X={x1,…,xn} – set of vertices, E={E1,…,Em} – set of hyperedges – A is a binary n×m incidence matrix of H
• Matching - a collection of pairwise disjoint hyperedges – The matching number ν (H) - the size of the largest matching
• Transversal - a vertices subset that intersects every hyperedge – The transversal number τ(H) - the size of the smallest transversal
5
Hypergraphs • The matching number • The transversal number • These problems satisfy weak duality ν(H) ≤ τ(H) • The relaxation versions of these problems
satisfy strong duality
• Every vector w in τ*(H) is called a fractional transversal 6
The Deletion Channel – KK’12 • Define a hypergraph H(X,E): – X = {0,1}n-1 , E = {all 2n single-deletion balls}
• Every single-deletion correcting code of length n is a matching in the hypergraph H • Find the value of τ*(H) or any fractional transversal to get an explicit upper bound
7
The General Case • G=(X,E) is a graph describing an error channel graph – X = the set of all possible words (transmitted and received) – E = the set of vertices pairs of dist one • The distance d(x,y) b/w x and y is the length of the shortest path from x to y (not necessarily symmetric)
– Br(x) = {y ∊ X : d(x,y)≤r}; degr(x) = |Br(x)|
• For any r>0, H(G,r)=(Xr,Er) is a hypergraph for G – Xr=X, Er={Br(x) : x∊X}
• Every code C in G is a matching in H(G,r) • AG(n,d) - the max size of a code w/ min dist d in G For every r>0:
• Q: Does the following hold? is called the Average Sphere Packing Value: ASPV(G,r) 8
Example: The Z Channel • GZ=(XZ,EZ), XZ={0,1}n
BZ,1(10010)={10010,00010,10000}
• H(GZ,r) = (XZ,r,EZ,r), XZ,r=XZ={0,1}n
9
Example: The Z Channel • The average size of a ball with radius r
• The average sphere packing value
• For r=1: 10
So what is the problem? • A channel graph G=(X,E) w/ hypergraph H(G,r)=(Xr,Er) • A code with min dist 2r+1 in G is a matching in H(G,r) • An upper bound is given by
• The good news: there is an explicit upper bound! • The bad news: It is not necessarily easy to calculate it – Need to solve a linear programming… – Usually the number of variables and constraints in exponential
• Our goal: how to calculate the value of 11
Some General Results • Lemma: If for all x∊X, degrin(x) ≤ Δ then
Proof: Since degrin(x) ≤ Δ then ⇒ For any transversal w,
Therefore,
12
Some General Results • Lemma: If for all x∊X, degrin(x) ≤ Δ then
• Lemma: If for all x∊X, degr(x) ≥ Δ, then
Proof: The vector w=1/Δ is a fractional transversal Thus,
13
Some General Results • Lemma: If for all x∊X, degrin(x) ≤ Δ then
• Lemma: If for all x∊X, degr(x) ≥ Δ, then
• Corollary: If G is symmetric and regular then the generalized sphere packing bound and the sphere packing bound coincide, and
14
Monotonicity and Fractional Transversals • The vector w is a fractional transversal if w ≥ 0 and • Lemma: The vector w, given by is a fractional transversal Proof: If y∊Br(xi), then xi∊Brin(y) and Therefore,
15
Monotonicity and Fractional Transversals • The vector w is a fractional transversal if w ≥ 0 and • Lemma: The vector w, given by is a fractional transversal • Def: G is called monotone if for all x∊X and y∊Br(x) • Lemma: If G is monotone then the vector is a fractional transversal • Corollary: If G is monotone an upper bound on AG(n,2r+1) is called the monotonicity upper bound MB(G,r)
16
Cont. Ex: The Z Channel • x,y∊{0,1}n, if y∊BZ,r(x), wH(y)≤wH(x) and degr(y)≤degr(x) • The vector given by is a fractional transversal • The monotonicity upper bound for the Z channel: • For r=1: • The average sphere packing bound • Is it possible to do better…?
17
Can we find the optimal transversal? • w is a fractional transversal if w ≥ 0 and • For the Z channel: – 2n constraints: Ex, n=3: • • • •
111: w111+w110+w101+w011 ≥ 1 110: w110+w100+w010 ≥ 1, 101: w101+w100+w100 ≥ 1, 011: w011+w010+w001 ≥ 1 100: w100+w000 ≥ 1, 010: w010+w000 ≥ 1, 001: w001+w000 ≥ 1 000: w000 ≥ 1
– Probably vectors w/ the same weight will have the same value – If so, only n+1 constraints: • • • •
3: w3+3w2 ≥ 1 2: w2+2w1 ≥ 1 1: w1+w0 ≥ 1 0: w0 ≥ 1
• Is it still possible to find the value of
? 18
Automorphisms on Graphs • Given a graph G=(X,E), an automorphism is a permutation that preserves adjacency π:X->X s.t. (x,y)∊E iff (π(x),π(y))∊E • The automorphisms set Aut(G)={π∊Sn : π is an automorphism in G} is a subgroup of Sn under composition • Aut(G) induces an equivalence order R on X: (x,y)∊R iff there exists π∊ Aut(G) and π(x)=y and partitions X into n(G) equivalence classes • For a vector w and automorphism π, the vector wπ is (wπ)i = wπ(i) • Lemma: If w is a transversal and π an automorphism then wπ is a transversal as well 19
Automorphisms on Graphs • Given a graph G=(X,E), an automorphism is a permutation that preserves adjacency π:X->X s.t. (x,y)∊E iff (π(x),π(y))∊E Proof: • The automorphisms set Need to show: for 1 ≤ i ≤n Aut(G)={π∊Sn : π is an automorphism in G} y ∊ Br(xof ∊ Br(π(x i) iff i)) under composition is a subgroup Snπ(y) Therefore, • Aut(G) induces an equivalence order R on X: (x,y)∊R iff there exists π∊ Aut(G) and π(x)=y and partitions X into n(G) equivalence classes • For a vector w and automorphism π, the vector wπ is (wπ)i = wπ(i) • Lemma: If w is a transversal and π an automorphism then wπ is a transversal as well 20
Automorphisms on Graphs • Aut(G) = {π∊Sn : π is an automorphism in G} • An equivalence order R on X: (x,y)∊R iff there exists π∊ Aut(G) and π(x)=y • Lemma: If w is a transversal and π an automorphism then wπ is a transversal as well • Wc = {w : w is a transversal and Σwi=c} • Theorem: If Wc≠∅ then Wc contains a transversal which assigns the same weight to the equivalence classes of R • This result holds also for any subgroup of Aut(G)
21
Cont. Ex: The Z Channel • For every σ∊Sn define a permutation πσ:{0,1}n->{0,1}n for all x∊{0,1}n, (πσ(x))i=xσ(i) • The set K={πσ : σ∊Sn} is a subgroup of Aut(GZ) and partitions {0,1}n into n+1 equivalence classes XK(GZ) = {X0,X1,…,Xn}, Xi=all vectors of weight i • The generalized sphere packing bound now becomes:
22
Cont. Ex: The Z Channel • The generalized sphere packing bound now becomes:
• For r=1: where
,
and
– For example, n=3: w*3=0, w*2= (1-0)/3=1/3, w*1= (1-1/3)/2=1/3, w*0= 1
23
Best known upper bound by Weber, De Vroedt, and Boekee ‘88
24
Cont. Ex: The Z Channel • For arbitrary r, the optimal solution is given by
25
Comparison for r=2
Comparison for r=3
26
Limited Magnitude Channels • Asymmetric errors GA,q=(X,E): X=[q]n
• The graph is monotone • The monotonicity upper bound for r=1 is
• The average sphere packing bound 27
The Asymmetric Channel • The linear programming problem to find is given by
• By automorphisms as in the Z channel we can divide to the equivalence classes
28
The Asymmetric Channel • Theorem: the vector
given by
is a fractional transversal
29
Results for q=3
30
The Symmetric Channel • No longer monotonicity because the channel is symmetric • Can follow similar steps… Comparison for q=3
Comparison for q=4
31
Does the average sphere packing hold? • The average size of the ball is (1∙5+4∙1)/5=9/5 • The average sphere packing value 5/(9/5)=25/9 • But the min dist of the code {x2,x3,x4,x5} is infinity… • Another example
– n=k2 vertices into two groups k and n-k – Every vertex from the 1st group is connected to exactly k-1 vertices from the 2nd group – All n-k vertices in the second group are connected
– But there is a code with k vertices… 32
Back to the Deletion Channel… • GD=(XD,ED), XD={0,1}nU{0,1}n-1 and
• For x∊{0,1}n, • The hypergraph H(GD,1)=(XD,1,ED,1), XD,1={0,1}n-1, ED,1 = {BD,1(x) : x∊{0,1}n } • The generalized sphere packing bound
33
Back to the Deletion Channel… • For x∊{0,1}n, ρ(x)= the number of runs in x – x=001010010, ρ(x)=7
degD,1(x) = ρ(x) For every y∊BD,1(x), ρ(y)≤ρ(x)=degD,1(x) This graph satisfies a similar property to monotonicity The vector given by is a fractional transversal (KK’12) • The corresponding upper bound is • • • •
34
Is it possible to do better…? • A transversal is given by wx=1/ρ(x) • A vector x has ρ(x) neighbors – If they all have ρ(x) runs then we can’t do better • x=001100: BD(x)={01100(w=1/3), 00100(w=1/3), 00110(w=1/3)}
– But if many neighbors have less than ρ(x) runs…?
• x=000010: BD(x)={000010(w=1/3), 00000(w=1), 00001(w=1/2)}
• For x, μ(x) = # of middle runs of length 1 – x=001010010, μ(x)=4 – 0≤μ(x) ≤ρ(x)-2
• Nn(ρ, μ) = # of vectors with ρ runs and μ middle-1 runs – Nn(1,0)=2
35
Is it possible to do better…? • • • •
For x, μ(x) = # of middle runs of length 1 Nn(ρ, μ) = # of vectors with ρ runs and μ middle-1 runs Lemma: For 2≤ρ≤n and 0≤μ≤ρ-2, Theorem: The vector defined by
is a fractional transversal • Theorem: The value
satisfies
36
Comparison of the Different Bounds
37
Conclusion • • • •
A method to generalize the sphere packing bound The Deletion channel – Kulkarni & Kiyavash ’12 General Results Some specific channels – – – – –
The Z channel Asymmetric errors Deletion channel Grain errors Projective space
Thank You! 38
