PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS OF ...

arXiv:1507.07848v1 [cs.CR] 28 Jul 2015

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS OF DIAGONALIZABLE GROUPS ´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV Abstract. We develop a public key cryptosystem based on invariants of diagonalizable groups. Theoretical results about degrees of invariants, which are related to the security of such cryptosystem, are derived. Further, we derive results on invariants of superanalogs of tori.

Introduction A new idea for a public-key cryptosystem based on the invariant theory was proposed by Grigoriev in [5]. His original idea was later developed in the paper [6]. The last paragraph of the paper [5] reads as follows: ”The current state of the art in cryptography does not allow one to prove the security of cryptosystems; this is usually a question of belief in the difficulty of a revelant problem and a matter of experience (that is why it is not quite unusual to have a paper on cryptography without theorems, for example, this paper). Quite the opposite, one can expect a ”disappointing” breaking of a particular cryptosystem. This can happen for any of the afforementioned examples (without solving the graph isomorphism problem, see the discussion above). On the other hand, such breaking could lead to interesting algorithms in the theory of group representations. Thus one can treat the above examples (and the general construction as a whole) just as a suggestion to play with cryptosystems based on the invariant theory.” The purpose of our paper is twofold. The first goal is to develop and design a public-key cryptosystem based on invariants of diagonalizable groups. For this part, we go beyond the philosophy of the preceeding quote and design a concrete public-key cryptosystem, present an algorithm for its implementation and show how to break systems based on invariants of some groups. Our second goal is related to the security of the invariant-based cryptosystem. In this connection, we will investigate and prove results about related mathematical conpcepts like minimal degrees of invariants. 1. Invariants of finitely-generated linear groups In this paper, we will consider only finitely generated groups G acting faithfully on a finite-dimensional vector space V = F n over a field F of arbitrary characteristic. Therefore we can asume that G ⊂ GL(V ). From the very beginning, assume that the representation ρ : G → GL(V ) is fixed, and the group G is given by a finite set of generators. With respect to the standard basis of V , each element g Key words and phrases. cryptosystem, invariants, diagonalizable group. This publication was made possible by a NPRF award NPRP 6 - 1059 - 1 - 208 from the Qatar National Research Fund (a member of The Qatar Foundation). The statements made herein are solely the responsibility of the authors. 1

2

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

of G is therefore represented by an invertible matrix of size n × n, and g acts on vectors in V by matrix multiplication. Let F [V ] = F [x1 , . . . , xn ] be the algebra of polynomial functions on GL(V ). Then G acts on F [V ] via gf (v) = f (g −1 v), where g ∈ G, f ∈ F [V ] and v ∈ V . An invariant f of G is a polynomial f ∈ F [V ] which has a property that its values are the same on orbits of the group G. In other words, for every vector v ∈ V and for every element g ∈ G, we have f (gv) = f (v). We note that different representations of G lead to different invariants in general but this is not going to be a problem for us since our represantation of G is fixed. We will denote the algebra of invariants of G by F [V ]G . 2. Public key-cryptosystem based on invariants We start by recalling the original idea of the public-key cryptosystem based on invariants from the paper [5] and recalling its modification presented in [6]. 2.1. Design of cryptosystems based on invariants. To design a cryptosystem, Alice needs to choose a finitely generated subgroup G of GL(V ) for some vector space V = F n and a set {g1 , . . . , gs } of generators of G. Alice also needs to know an invariant f of this representation of G. Alice chooses two elements v0 and v1 from V and a ∈ GL(V ) such that av0 and av1 are separated by the invariant f , that is, f (av0 ) 6= f (av1 ). The matrix a will be part of her secret key, while v0 and v1 will be part of the public key. Alice also chooses a set of randomly generated elements g1 , . . . , gm of G (say, by multiplying some of the given generators of G), which generates a subgroup of G that will be denoted by Gs . Alice announces as a public key the elements v0 , v1 standing for plaintext symbols 0 and 1, respectively, and the group H = a−1 Gs a, conjugated to Gs , by announcing its generators hi = a−1 gi a for i = 1, . . . , m. In the first paper [5] its author assumes that the group G, its representation in GL(V ) and the invariant f are in the public key. We refer to this setup as variant one. However, the version in paper [6] assumes that G, its representation in GL(V ) and the invariant f are secret. We refer to this setup as variant two. We will comment on both variants later. For the encryption, every time Bob wants to transmit a symbol 0 or 1, he chooses a randomly generated element h of the group H (by multiplying some of the generators of H given as a public key), and computes u = hv0 if the symbol is 0 or u = hv1 if the symbol is 1. The vector u ∈ V is then transmitted to Alice. To decript the message, Alice first computes au and then applies the invariant f . If u = hvi , then f (au) = f (ahvi ) = f (aa−1 gavi ) = f (gavi ) = f (avi ). Since a was chosen so that f (av0 ) 6= f (av1 ), Alice can determine from the value of f (au) whether the symbol v0 or v1 was encrypted by Bob. 3. Security and possible attacks on invariant cryptosystems Let us note that it is important that during the encryption process by Bob he uses all generators hi for scrambling the message. If some generators are not involved, then to decode his message Charlie would succeed if he finds an invariant of a subgroup of H, which is an easier task.

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS

3

The attacks described below are mentioned in [5] and [6]. We are providing their description for the convenience of the reader and for further clarification. To break the encryption, it is enough for Charlie to find any invariant f ′ of the group H that separates v0 and v1 . Indeed if f ′ (v0 ) 6= f ′ (v1 ), then Charlie computes f ′ (u) = f ′ (hvi ) = f ′ (vi ) and then compares f ′ (u) with f ′ (vi ) to determine which symbol v0 or v1 was used by Bob. The security of the cryptosystem depends on the difficulty of finding an invariant f ′ of the group H. The condition that f ′ separates v0 and v1 might not be difficult to satisfy because the set of polynomials in F [V ], that take on different values when evaluated at v0 and v1 , is open in the Zariski topology. Therefore it is likely that a randomly chosen invariant f ′ of H will separate v0 and v1 . 3.1. Variant one. Consider variant one of the cryptosystem - that is, the group G, its representation in GL(V ) and an invariant f are known. We can assume that f is a homogeneous polynomial of degree d. In this case, it is known that there is a homogeneous invariant f ′ of H of degree d that is of the form f ′ (v) = f (bv) for some matrix b ∈ GL(V ). Then f ′ is an invariant of H if and only if f (bhi v) = f ′ (hi v) = f ′ (v) = f (bv) i
= 1, . . . , m of H.
for each generator hi , where n+d−1 Comparing coefficients at n+d−1 monomials we obtain m linear equations d d in n2 variables (entries of b). Any solution of this system produces an invariant of H. Another possible way to attack the system is to find a matrix b ∈ GL(V ) such that bHb−1 ⊂ G. This technique is related to the conjugacy problem for matrix groups and the graph isomorphism problem. 3.2. Variant two. In variant two of the cryptosystem, the group G, its representation in GL(V ) and the invariant f are secret. However, Charlie can attempt to find an invariant f ′ directly by choosing a possible degree d and solving linear systems derived from the equations f ′ (hi v) = f ′ (v) for each generator hi , where
i = 1. . . . , m. This produces a linear system consisting of m n+d−1 equations in d
′ the n+d−1 unknowns that are the coefficients at monomials in f . d Another approach is to find a matrix h ∈ H such that hu = v0 or hu = v1 (attempting to recover the encryption done by Bob). This problem is related to the vector transporter problem and the graph isomorphism problem. 3.3. Guarding against the linear algebra attack. Denote by MG,V , or simply by MG or M if we need not emphasise the group G or the vector space V it is acting on, the minimal positive degree of an invariant from F [V ]G . That is G MG,V = min{d > 0|F [V ]G d 6= 0}. If F [V ] = F , then we set MG,V = ∞. The notion of the minimal positive degree of an invariant and the value of M = MG,V are important for the security of the invariant-based cryptosystem (both variants one and two) we For example, if we know that
are considering. r MG is so small that m n+M−1 = O(n ) is polynomial in n, then Charlie can find M an invariant f ′ of G in polynomial time by solving consecutive linear systems


for n+M−1 n+d−1 n+d−1 d = 1, . . . , , each consisting of m equations in the variM d d ables described in the previous section. For a fixed d, this can be accomplished
4 in time O(m( n+d−1 ) ) and the total search will take no more than time O(n8r ). d
Therefore, for the security of the system it must be guaranteed that m n+M−1 is M not polynomial in n.

4

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

4. Finding a polynomial invariant of G Since the encryption is broken once Charlie finds an invariant f ′ of G, we will now discuss an algorithm that will enable us to find it. The algorithm works inductively, and as a special case, it works when G is a finite group and the characteristic of the ground field F is arbitrary. Assume that H is a subgroup of G of finite index in G. Assuming we know a nonzero invariant f of H, we will find a nonzero invariant of G. Lemma 4.1. Let H be a subgroup of G of finite index s in G such that f is an invariant of H of degree t. Then G has a nonzero invariant of degree not exceeding
s+1 sMH that can be found in time O(snt+2 n+t−1 ). t

Proof. Denote by g1 , . . . , gs , where s = [G : H], representatives of all coset classes of G/H. Let f be an invariant of H of degree MH . Denote xi = gi f for i = 1, . . . s, and denote by ps (x1 , . . . , xs ) = x1 . . . xs the s-th elementary symmetric function in x1 , . . . , xs . It is easy to see that ps (x1 , . . . , xs ) is invariant with respect to G, because each element g ∈ G permutes coset classes of G/H, hence it permutes the set of polynomials {x1 , . . . , xs }. Also, the polynomial ps (x1 , . . . , xs ) = x1 . . . xs is nonzero and xi
in time
thas the degree sMH . We can evaluate all polynomials n+t−1 s O(sn2 n+t−1 n ). The product of all x can be computed in time O( ).  i t t Corollary 4.2. If G is a group of finite order s, then the algorithm in the proof of the previous lemma (applied to H = 1) produces a nonzero invariant of G of order not exceeding s which can be computed in time O(sn3 ns+1 ).

Note that the time required to run the computation is exponential in the order of G if no invariant of a subgroup of G is known and when we attempt to find an invariant of G from H = 1. Nevertheless, there are cases when an invariant of H can be computed in polynomial time; see the next lemma. The following lemma is well-known, see [2]. Lemma 4.3. If G ⊂ GLn (R) and G is finite, then G has an invariant of degree two. Proof. Let g1 = 1, . . . , gs be all elements of G and R[V ] = R[t1 , . . . , tn ]. Denote by xi = gi (t21 + . . . + t2n ) for i = 1, . . . , s. Since values of each xi are non-negative when evaluated as polynomials in t1 , . . . , tn , the values of the invariant polynomial P s i=1 xi evaluated as polynomial in t1 , . . . , tn are non-negative and they can be equal Ps to zero only if each xi is zero. But x1 = 0 only if t1 = . . . = tn = 0. Therefore i=1 xi is a positive-definite quadratic form in t1 , . . . , tn , hence a non-zero invariant of G.  It follows from the previous section that a quadratic invariant of the group H, within a context of our public-key cryptosystem, can be found using linear algebra techniques in the polynomial time in n. Therefore, for the security of the cryptosystem, we need to make sure that if H is finite, then it is not represented by matrices with real coefficients. 5. Lower bounds for degrees of polynomial invariants The significance of understanding the minimal degree MG,V of invariants for the security of the invariant-based cryptosystem was established above. In particular,

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS

5

it is important to find a nontrivial lower bound for MG,V . Unfortunately, we are not aware of any articles establishing lower bounds for the minimal degree of invariants, except in very special circumstances, e.g. [7]. On the other hand, there are numerous upper bounds for the minimal degree β(G, V ) such that F [V ]G is generated as an algebra by all invariants in degrees not exceeding β(G, V ). For example, a classical result of Noether [12] states that if the characteristic of F is zero and G is finite of order |G|, then β(G, V ) ≤ |G|. There is an extensive discussion of Noether bound and results about β(G, V ) in section 3 of [16]. It was conjectured by Kemper that for G 6= 1, and arbitratry ground field F , the number β(G, V ) is at most dim V (|G| − 1). Recently, this conjecture was proved by Symonds in [17]. When one wants to find an invariant of G, it seems natural to consider an upper bound β(G, V ). However, if we wants to show that there are no invariants of small degrees (as is our case), then we need to find lower bounds for MG,V . Until now, there was no real impetus to consider such problem. Assume again that G is a (finitely generated) subgroup of GL(V ), and denote MG,V just by MG . Denote by G = G the Zariski closure of G. We will assume that G is a linearly reductive subgroup in GL(V ) (in particular, this assumption is satisfied if G is a finite group and charF6 | |G|). According to [8] (see also [4]), F [V ]G = F [V ]G is a Cohen-Macaulay algebra. Therefore F [V ]G is a free module over its subalgebra F [p1 , . . . , ps ], freely generated by the (homogeneous) parameters p1 , . . . , ps , which are called the first generators. In other words, F [V ]G = ⊕1≤i≤l F [p1 , . . . , ps ]hi , where h1 , . . . , hl are called the second generators. If F [V ]G 6= F , then MG = min{{deg hi > 0}, {deg pj }}. In what follows we will denote by ζk a primitive root of unity of order k. If the order k is clear from the context, we will denote it just by ζ. Additionally, every time ζk is mentioned, we assume that it is an element of the ground field F . If a matrix g ∈ GL(V ) has a finite order k, then all eigenvalues λ1 , . . . , λn of g are roots of unity. If we denote ζ = ζk , then there are integers ki such that λi = ζ ki , where 0 ≤ ki < k and gcd(k1 , . . . , kn , k) = 1. For g 6= 1 denote by kg the positive integer kg = min{

n X i=1

ai > 0|

n X i=1

ai ki ≡ 0 (mod k), where integers a1 , . . . , an ≥ 0}.

The following lemma describes invariant polynomials and Mhti for a diagonal matrix t of finite order. Lemma 5.1. Assume that t is a diagonal matrix of the finite order k with diagonal entries λ1 = ζkk1 , . . . , λn = ζkkn , where the exponents ki are as above. Then the a1 an hti a invariant Pn subalgebra of F [V ] is generated by monomials x = x1 . . . xn such that i=1 ai ki ≡ 0 (mod k). Additionally, if t 6= 1, then Mhti = kt .

Proof. The properties of numbers ki follow immediately. Since t acts on the corresponding coordinate function as txi = λ−1 i xi , we P obtain that a monomial xa = xa1 1 . . . xann is an invariant of F [V ] if and only if ni=1 ai ki ≡ 0 (mod k). Because every monomial xb is a semi-invariant of t, monomials xa as above generate F [V ]hti . The formula for Mhti is then clear. 

6

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

For the next lemma we apply standard results from algebraic group theory, that can be found, for example, in the book [9]. For an element g ∈ G let g = gs gu be its Jordan-Chevalley decomposition. Let Gs and Gu denote the sets of semisimple and unipotent components of all elements from G, respectively. Lemma 5.2. Assume that a group H is abelian. Then MH = M . Proof. Since the algebraic group H = H is abelian, it can be written as a product H = Hs × Hu of its closed subgroups Hs and Hu . The inclusions Hs ⊆ Hs and Hu ⊆ Hu imply that Hs = < Hs > and Hu = < Hu >. Furthermore, F [V ]H = F [V ]H = (F [V ] ) . Since the group < Hu > ) 6= 0. This means is unipotent, F [V ]d 6= 0 implies F [V ]H d = (F [V ]d  that MH = M = M . Since H ≤ G implies MH ≤ MG , we obtain immediately the following corollary. Corollary 5.3. If H is an abelian subgroup of G, then M ≤ MG . A subgroup G of GL(V ) is called small, if there is an abelian subgroup H of G such that MG = MH . Lemma 5.4. If g 6= 1 is of finite order, then M = kg . Proof. Lemma 5.2 implies M = M . With respect to a basis of V , consisting of eigenvectors of gs , gs is represented by a diagonal matrix. By Lemma 5.1 we  obtain M = kgs . Since k = k , the lemma follows. Corollary 5.5. If G is finite, then max{kg ; g ∈ G, g 6= 1} ≤ MG . Lemma 4.3 has the following interesting consequence. Corollary 5.6. Let g 6= 1 correspond to a matrix from GLn (R) of finite order. Then either one of the eigenvalues of g equals 1 or there are two eigenvalues λ and µ of g, both different from 1 such that λµ = 1. To illustrate the difficulty of finding a lower bound for MG,V , we will determine the value of MG,V explicitly for certain finite subgroups G of GL2 (C). The list of all finite subgroups of GL2 (C) is presented in [7]. Let G be a finite group from Lemma 2.1 of [7]. The group G has two generators  v   g  λ 0 λ1 0 A= , B = , 0 λjv2 0 λdg where λ is an e-th primitive root of unity, v1 , v2 > 1, v1 v2 |g, g|e, d|e, gcd(v1 , v2 ) = gcd(e, j) = gcd(v1 , d) = gcd(v2 , d) = 1. Additionally, the number d is square-free and each prime factor of e divides one of the numbers v1 , v2 or d. In particular, G ≃< A > × < B >= Ze × Z eg . To calculate MG , we need to consider the following system of congruencies: v1 a1 + jv2 a2 ≡ 0 (mod e), ga1 + dga2 ≡ 0 (mod e),

where a1 , a2 ≥ 0 are such that a1 + a2 > 0. The second congruence implies that a1 = et g − da2 , where t is a positive integer. Substituting the value of a1 into the first congruence we receive ev1 t = (dv1 − jv2 )a2 (mod e). g

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS

7

Since gcd(e, dv1 − jv2 ) = 1, we obtain that evg1 divides a2 , which implies that ev2 e g divides a1 . Since both a1 and a2 are multiples of g , the second congruence ga1 + dga2 ≡ 0 (mod e) can be eliminated from the system since it is automatically satisfied. Define a1 = evg2 a′1 , a2 = evg1 a′2 . Then a′1 + ja′2 = 0 (mod v1gv2 ), or equivalently, a′1 + ja′2 = v1gsv2 for some s > 0. This congruence has the solution a′1 =

gs(j + 1) gs − jt, a′2 = − + t. v1 v2 v1 v2

Since a′1 , a′2 ≥ 0, the parameter t satisfies gs gs gs ≤t≤ +[ ]. v1 v2 v1 v2 jv1 v2 Additionally, a1 =

es ev1 t es(j + 1) ev2 jt − and a2 = − + . v1 g v2 g

Thus

es(j + 1) es et − − (v2 j − v1 ). v1 v2 g ≤ t ≤ v1gsv2 + Finally, observe that for every s > 0 and for every t such that vgs 1 v2 ], the right-hand-side of the above formula for a1 + a2 is greater than zero. [ jvgs 1 v2 Now are are ready to determine the values of MG,V . a1 + a2 =

Proposition 5.7. Assume G is a finite group from Lemma 2.1 of [7], as above. Then the value of MG,v is given as follows. If jv2 < v1 , then MG = ve1 . If jv2 > v1 , 1) }, ve2 }. ] e(v2 j−v then MG = min{ min { vs1 − [ jvgs g 1 v2

0<s<j

Proof. If jv2 < v1 , and s is fixed, then the minimum of such a1 + a2 equals ves1 and is attained for t = v1gsv2 . Therefore MG = min{a1 + a2 } = ve1 . If jv2 > v1 , and s is fixed, then the minimum of such a1 + a2 equals ves1 − 1) and is attained for t = ] e(v2 j−v [ jvgs g 1 v2

gs gs v1 v2 + [ jv1 v2 ]. ] = v1glv2 then [ jvgs 1 v2

If s = jl + s′ , where 0 ≤ s′ < j, this into the above expression for a1 + a2 we obtain a1 + a2 =

′

+ [ jvgs1 v2 ]. After substituting

el s′ gs′ e(v2 j − v1 ) +( −[ ] ). v2 v1 jv1 v2 g

If s′ = 0, then the minimum for such a1 + a2 is attained for l = 1 and it equals to a1 + a2 = ve2 . If s′ > 0, then the minimum for such a1 + a2 is attained for l = 0 and it equals to s′ gs′ e(v2 j − v1 ) min { − [ ] }. 0<s′ <j v1 jv1 v2 g The statement follows by combination of the last two formulas.  Assume that a group G and matrix A are as above and the condition jv2 < v1 is satisfied. Then MG is equal to the degree of one of the first generators (parameters) of C[V ]G (see Theorem 3.1 of [7]). Therefore using Proposition 5.7 we compute MG = M = kA and obtain the equality of the two expressions in Corollary 5.5 showing that G is small.

8

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

Example 5.8. The following example shows that not all finite subgroups of GL(V ) are small. Let G be a subgroup of SL2 (C) generated by the matrices         1 −1 + i 1 − i 0 1 i 0 −1 0 , , . , −1 − i −1 − i −1 0 0 −i 0 −1 2

The group G is the group from Lemma 2.3 of [7] and V = C2 . If g ∈ G is not an identity matrix, then it has eigenvalues λ and λ−1 , where λ 6= 1 is a root of unity. If H is an abelian subgroup of G, then Hs can be conjugated with a subgroup H ′ of ′ the group of diagonal matrices. Thus x1 x2 ∈ C[V ]H , i.e. MH = MHs ≤ 2. On the other hand, Lemma 4.1 of [7] (see the first row in the table on page 327) implies MG = 6. Based on the above discussion, the following problem seems natural. Problem 5.9. Characterize the class of small finite subgroups G of GL(V ).

A more general problem is to estimate the value of MG for a given finite subgroup G ≤ GL(V ). There are no general results for the lower bound for MG but the following result of Thompson gives an upper bound for MG in general. Proposition 5.10. If G is a finite subgroup of GLn (C) and G has no non-trivial characters, then MG ≤ 4n2 . Proof. In the notation of the paper [18], the integer MG coincides with dG . The main theorem of [18] states that dG ≤ 4n2 .  6. Cryptosystem based on invariants of finite diagonalizable groups Let G ⊂ GL(V ) and elements g of G be written as n× n matrices with respect to a fixed basis of V . A group G ⊂ GL(V ) is called diagonalizable if there is a basis of V with respect to which all of its elements g are represented by diagonal matrices. Therefore G is diagonalizable if and only if there is an n × n invertible matrix P such that all elements of the conjugate group P −1 GP , denoted by T , are given by diagonal matrices. If a diagonalizable group G is finite, then the invariants and the minimal degree Mhti of elements t from the corresponidng T are described in Lemma 5.1. We will show that any cryptosystem designed on invariants of a finite diagonalizable group is not secure against polynomial break. Lemma 6.1. If there is an algorithm that runs in polynomial time in n, that finds a root of every polynomial p(x) with complex coefficients such that all of its roots are roots of unity, then the cryptosystem based on invariants of a finite diagonalizable group G can be broken in polynomial time in n. Proof. Denote the dual space V ∗ of V by W . We will show how to find an element of W that is a common eigenvector for all generators gi of G. In the first step, we compute the characteristic polynomial of g1 and find at least one eigenvalue of g1 , say λ1 . Then using linear algebra, we can compute the eigenspace {f ∈ W |g1 f = λ1 f }. Since the group G is abelian, the space W1 is a G-submodule of W . Then we find an eigenvalue of g2 |W1 , say λ2 and compute the eigenspace {f ∈ W1 |g2 f = λ2 f } which we denote by W2 . Proceeding this way, we wil find a semi-invariant f ∈ W , that is an element f such that gi f = λi f for 1 ≤ i ≤ s. Since all eigenvalues λi are roots of unity, there is an non-negative integer k such

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS

9

that λki = 1 for every i = 1, . . . , s, hence the polynomial f k is a G-invariant. All of the above computations can be done in time polynomial in n.  The reader is referred to the paper [13] for excellent survey on a variety of algorithms designed to approximate a root of polynomial with complex coefficients in polynomial time of its degree n. Methods based on Weyl algorithm and divide and conquer algorithm converge globally, whereas other methods (e.g. those based on the Newton method) are not convergent for some initial values used in the iterations. There are also very fast probabilistic algorithms, see for example [14] and [15]. 7. Cryptosystem based on invariants of infinite diagonalizable groups The next cryptosystem is using invariants of an infinite diagonalizable group. Let us fix a number field F = Q(θ) and the subring Z = Z[θ] of the ring of algebraic integers of Q(θ). Choose a finite set S of integers of cardinality q and a set Sm = {p1 , . . . , pm } of elements of Z. Denote by Pm the set of all products of elements from the set Sm . 7.1. Design of the cryptosystem. To start, Alice chooses sets S and Sm = {p1 , . . . , pm } as above. Afterwards, she chooses her secret key, which is the n-tuple of nonnegative integers (e1 , . . . , en ), where one component, say en equals 1. Then she will construct a set of generators t1 , . . . , ts of T in such a way that the monomial f = xe11 . . . xenn is invariant under the action of each ti , hence belongs to F [V ]T . At the i-th step of the process, Alice chooses the i-th generator ti of the group T as follows. (i) First, for every k = 1, . . . , m and 1 ≤ j ≤ n − 1 she chooses numbers bk,j (i)

(i)

from the set S. Then she computes the numbers a1 , . . . , an−1 from the set Pm as Qm b(i) (i) (i) aj = k=1 pkk,j where 1 ≤ j ≤ n − 1. Alice then computes an in such a way that (i)

(i)

the diagonal matrix gi = diag(a1 , . . . , an ) has f = xe11 . . . xenn as an invariant. (i) Since she has chosen en = 1, it is easy to see that the appropriate value of an is Qn−1 (i) −ej (i) an = j=1 (aj ) . Once all generators ti of the group T are constructed, Alice chooses an invertible n × n matrix P as a part of her secret key and computes conjugates gi = P ti P −1 . Alice then announces the diagonalizable group G given by its generators gi for i = 1, . . . , s. When she receives the encrypted message, she can use her secret key P to switch from G to T and apply her previously chosen invariant f = xe11 . . . xenn of T to decrypt the message, as explained in section 2. She knows that f is an invariant of T because T was constructed to satisfy that condition. 7.2. How to break the cryptosystem in partial cases. We will explain how the above cryptosystem could be broken in the polynomial time for some rings Z. Lemma 7.1. Assume that a ring Z is such that the group of units of Z is finite, Z is an Euclidean domain, and the Euclidean algorithm over Z runs in polynomial time in its input. If a vector, encrypted by the above cryptosystem, has no zero components, then it can be decrypted in polynomial time.

10

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

Proof. If the group of units of Z is finite, then it consists of roots of unity. Assume that it is generated by ζE . At first, we compute the characteristic polynomials of all matrices gi and find all of their eigenvalues. This can be done in polynomial time - see above. Then, we follow the algorithm explained in the proof of Proposition 15.4 of [9] and simultaneously diagonalize all matrices gi and obtain generators t′i of the conjugate group T ′ consisting of diagonal matrices. For simplicity of notation, we can assume that (i) (i) T ′ = T . Actually, what is important for us are only the eigenvalues a1 , . . . , an and their order with respect to fixed order of the eigenvectors. We will not work with the actual eigenvectors of V . Since all eigenvalues are ratios of elements from Z, we can consider the set X of integers that appear in the numerators or denominators of any eigenvalue of any matrix ti . Using Euclidean algorithm we can compute the set Y of all greatest common divisors of all pairs of elements from X. Then we can write a partial factorization of all elements of X in the form where x = yz and y is a product of elements from Y and Z is not divisible by any element from Y . Afterward we replace X by a new set X ′ consisting of all elements in Y and of elements z from the above factorization. In the next step we replace the set Y by the set Y ′ consisting of all greatest common divisors of all pairs of elements from X ′ . We continue in the same fashion and after finitely many steps this process will stabilize. Then we arrive at a set Y (d) of numbers that are pairwise coprime divisors of integers from X. Since the Euclidean algorithm in Z runs in polynomial time in n and there are no more than qm steps of the above process, we find the atoms in polynomial time in n. Let us call elements of Y (d) atoms of X and denote them by {a1 , . . . am′ }. For every atom a, every element x of X is either coprime to a, or is written as x = al b, where Q ′ b′(i) e (i) (i) k,j b is coprime to a. Every aj has the atom factorization aj = ζEi,j m k=1 ak , where 0 ≤ ei,j < E. We can find an invariant of T from the structure of these diagonal matrices by solving, in nonnegative integers, the system of s equations Qm (i) yj = 1 in n variables. Since Charlie has the atom factorization of each j=1 (aj ) (i)

element aj , he can compare the exponents in the atom factorization and obtain P m′ P ′ ′(i) a system of s(m′ + 1) linear equations m j=1 ei,j yj = 0 with j=1 bk,j yj = 0 and ′(i)

bounded coefficients bk,j and ei,j in n variables yj . An integer solution of this system can be found in polynomial time in n - see subsection 1.5.2 of [3]. The task to find a nonnegative integer solution of the linear system with integer coefficients is an NP-complete problem. If a solution of our system that has all nonnegative components is found, then it corresponds to a polynomial invariant of G. However, every integral solution corresponds to a rational invariant, that is a rational function that is invariant under the G-action. If the intercepted encoded vector has no zero coordinates, then Charlie can use his rational invariants to decode the message. 

Let us remark that the assumption of the above lemma are satisfied for integers Z = Z or Gaussian integers Z = Z[i]. It is well known that the Euclidean algorithm runs in polynomial time over Z and Z[i]. For a survey of algorithmic results see Section 3 of [1]. Also, the above results can be extended further if we replace the assumption that Z is Euclidean domain by the assumption that Z is complex

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS

11

quadratic unique factorization domain. According to [10] there is an algorithm, running in polynomial time, that computes gcd in such rings Z. Analogous atom or prime factorisation is not available in suitable form for number fields in general. For simplicity assume that Z = Z[θ] coincides with the ring of algebraic integers of the field Q(θ). If Z is a principal ideal domain but not a Euclidean domain, then we have a factorization of every element of Z into a product of primitive elements and units of Z. However, without the Euclidean algorithm, it is not clear if we can produce prime factorization of principal ideal in polynomial time. If Z is not a principal ideal domain, then instead of with primitive elements we need to work with divisors. Namely, for each x ∈ Z there is the prime ideal decomposition (x) = p1 . . . pl , where pi are (not necessarily principal) ideals in Z. An ideal generated by each prime number p splits up to a product of many prime ideals (their number does not exceed the degree of the extension [Q(θ) : Q], and this number is attained for totally ramified primes p). The problem of finding the prime ideal factorization in Z is very difficult. Its special case for F = Q is the prime factorization problem in Z. The difficulty of factoring of a product of two large primes is the basis of the RSA public-key cryptosystem. √ There are rings Z that are not a unique factorization domain, for example Z = Z[ −5]. If the class number of the ring Z[θ] is bigger than one, then Z is not a unique factorization domain. If Z is not a unique factorization domain, then it is not a principal ideal domain and is not a Euclidean domain. Even if we assume that the prime factorization of principal ideals generated by (i) aj is known, by itself it would not be enough to break the above system. The additional difficulty lies in the structure of the group of units of Z. For example, if we choose all elements of Sm to be units of Z, then the whole idea of atom or prime decomposition is utterly useless. In order to facilitate the conversion into a system of linear equations, we would need to determine a factorization of each appearing unit into a product of roots or unity and fundamental units of the ring Z. Finding a set of fundamental units of the ring Z and decomposition of units of Z into products of root of unity and fundamental units is by itself a very difficult problem and we are not aware of any algorithm solving these problems in polynomial time. Therefore the break described in Lemma 7.1 cannot be duplicated for rings Z that are not unique factorization domains or those containing units of infinite orders. We remark that there is a plethora of examples of such rings Z appearing in the algebraic number theory. A combination of obstacles related to factorization of principal ideals and factorization of units of Z as a product of fundamental units is the reason why we propose the above cryptosystem based on approprately selected Z. We are unable to find a polynomial algorithm for finding an invariant of the corresponding diagonalizable group G. 7.3. Security issues. We will talk about the possible choices Alice can make and how they affect the security of the system. a) The choice of the ring Z This is perhaps the most critical choice because the security of the cryptosystem depends on the arithmetic of the ring Z. The ring has to be chosen so that Z is not a unique factorization domain and preferably such that the class number of the ring Z is high. There are numerous examples of rings of integers of number fields that satisfy this condition. Secondly, we should choose Z so that the rank of its group

12

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

of units is high. Since the Dirichlet theorem describes the rank as r + s − 1, where r is the number of real embeddings and s is the number of complex embeddings of the field Q(θ), the second condition is easy to satisfy. b) The choice of the set Sm We could choose elements pi from the set Sm in such a way that some of them are primitive. Also, we should choose them in such a way their norms will have many common prime factors p. If we chose them randomly, then there is a great probability that the prime ideals dividing p in the prime decomposition of different pi are actually different. Also, we could choose some elements of Sm to be units of Z. c) Choice of the set S A choice of a finite set S does not seem to be important hence we can take it to be small, for example S = {−1, 0, 1}. d) The choice of the secret key (e1 , . . . , en ). In order to prevent linear algebra attacks P described in Section 3.3, the secret key (e1 , . . . , en ) must be chosen so that E = ni=1 Pei is at least of the order of n. For example she can choose ei ∈ {0, 1} such that ni=1 ei = [ n2 ]. See also 7.4 a) below. (i) e) Choice of the exponents bk,j We would like to make sure that the minimal degree MT is close to E, which is the degree of f , or at least of the order n. However, if the number s of generators (i) t is high and all exponents bk,j are chosen randomly, we expect that MT is going (i)

to be of order n. It is an interesting problem to investigate how to choose bk,j to guarantee that MT is sufficiently large, say bigger than E/2. If we cannot gurantee that MT is of order n, then we can add another generator diag(ζE , . . . , ζE ) to T . That would require replacing the field Q(θ) by Q(θ, ζE ) and chainging the ring Z. This would give away to Charlie the degree of our invariant f but it would also make sure that MT = E. Since E is of order n, this prevents the linear algebra break discussed in subsection 3.3. f ) The choice of the transition matrix P . The idea of using conjugate group G instead of T is to make matrices representing elements g ∈ G as far away from the diagonal matrices as possible. Therefore the matrix P should be complicated, and with many nonzero entries, in order to accomplish this. 7.4. Possible attacks. We will now describe possible attacks on the above cryptosystem. a) Linear algebra attack Charlie might attempt to find an invariant of G directly using the linear algebra attack described in Section 3.3. The complexity of this approach is exponential in n if MG is of the order of n, which is likely going to be the case due to (random) choices (i) of aj and which can be guaranteed by adding another generator diag(ζn , . . . , ζn ) to T . Therefore this linear algebra attack is ineffective. b) Finding the conjugate group T Charlie might attempt to find a conjugate group T ′ of G, consisting of diagonal matrices. In order to diagonalize G, he would find all eigenvalues of elements gj by computing their characteristic polynomials, which he can do in polynomial time in n. We have remarked earlier that he can factor out all characteristic polynomials

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS

13

in polynomial time in n. Once the eigenvalues of matrices corresponding to every gi are computed, he can simultaneously diagonalize all matrices gi (see the proof of Proposition 15.4 of [9]) and obtain the generators of a group T ′ , in polynomial time in n. c) Finding an invariant using ideal and units factorisation For rings Z, that are non-Euclidean or have infinite group of units, the attack described in Lemma 7.1 is not viable. 7.5. Possible modification of the system. We have seen before that switching P Q (i) (i) yj = 1 to the linear system m from the system of equations m j=1 bk,j yj = 0 j=1 (aj ) is important for possible breaking of the system. This can be accomplished by prime ideal factorization - see 7.4 c) above. One possibility to prevent this method is to (i) choose the numbers aj to be arbitrary and random complex numbers. Then the Pm (i) corresponding linear system would consists of equations j=1 log(aj )yj = 0. It appears to be difficult to find a solution of such general system in integers. On the other hand for computatational purposes we need to approximate the (i) numbers aj by complex numbers with finite decimal expansions. This would create difficulty estimating errors of the encryption process. For such system it would be necessary to estimate possible error of encryption and also it would be necessary that the vectors v0 and v1 used in the encryption process could be distinguishable within the errors of such computations. 7.6. More general systems. The main reason we were able to design a system for diagonalizable groups was that we were able to easily construct matrices that have a given monomial as its invariants. In the case of finite diagonalizable G we have a reasonable description of the invariants for diagonal matrices given by Lemma 5.1. For infinite diagonalizable G the situation is similar but we have equations instead of congruences. One could hope that designing a system based on nonabelian G would be more secure than that based on a diagonalizable group T because it is more complicated to find invariants of such G than those of T . A system based on invariants of nonabelian group G would have an advantage that simultaneous diagonalization as described in 7.4 b) is not possible. Therefore the conjugation problem is more difficult to solve for nonabelian G. Also, we need to take into account that the minimal degree of G must be at least of order n to prevent linear algebra attacks. In the paper [6] the authors have proposed a process of generating a more complicated (nonabelian) group G, its representation and a corresponding invariant starting from simpler groups using four types of operations. Their main idea was that it would be more difficult to find an invariant of G than that of the simpler groups. We will investigate how this construction affects the minimal degrees of invariants since they are important in regard to the possible linear algebra attack on the corresponding cryptosystem described in subsection 3.3. For the first operation, assume that G ≤ GL(V ), where V ≃ Rn is a free module over a ring R of rank n; and a ring homomorphism π : R → R′ , replacing R with a new ring R′ , are given. If R′ is a direct summand of R and π is a projection onto R′ (in which case R′ is called smaller), then every invariant of R[V ]G remains an invariant of R′ [π(V )]G , hence this operation does not increase the minimal degree MG,V . If R is embedded into R′ , then R[V ]G ⊆ R′ [R′ ⊗R V ]G , hence MG,R′ ⊗R V ≤ MG,V . The authors of [6] do not specify what they mean when

14

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

R′ is larger, and we were unable to follow their arguments. However, if the kernel of the map π is nontrivial, then some of the invariants can be annihilated using this process and the minimal degree can potentially increase. The second operation replaces G by a conjugated subgroup H = h−1 Gh for some h ∈ GL(V ). Since the algebras/rings R[V ]G and R[V ]H are isomorphic, we have the equality of the minimal degrees MG,V = MH,V . The third operation requires two groups G1 ≤ GL(V1 ) and G2 ≤ GL(V2 ) and replaces them by their direct product G1 × G2 embedded in a natural way into GL(V1 ⊕ V2 ). In this case the isomorhism R[V1 ⊕ V2 ]G1 ×G2 ≃ R[V1 ]G1 ⊗ R[V2 ]G2 implies MG1 ×G2 ,V1 ⊕V2 = min{MG1 ,V1 , MG2 ,V2 }, thus the minimal degree will not increase. Finally, the fourth operation replaces G by the wreath product L = G ≀ H, where H is a subgroup of the symmetric group Sm . The group L can be identified with the set of all m + 1-tuples (g1 , . . . , gm , σ), where g1 , . . . , gm ∈ G and σ ∈ H. The above element of L acts on V ⊕m by the rule (g1 , . . . , gm , σ)(v1 , . . . , vm ) = (g1 vσ(1) , . . . , gm vσ(m) ). The subgroup consisting of all elements with σ = 1 is normal and it is isomorphic to the direct product Gm and L is isomorphic to the semi-direct product H⋉Gm . Then m R[V ⊕m ]L = (R[V ⊕m ]G )H = ((R[V ]G )⊗m )H and, for any invariants f1 , . . . , fm from R[V ]G , the element X 1 ⊗ ...⊗ fi ⊗...⊗ 1 |{z} i−th place

is L-invariant. Therefore ML,V ⊕m ≤ MG,V . Summing up, all four operations as presented in [6] do not increase the minimal degrees of invariants of given representations of the initial groups (possibly with the exception of the first operation with non-injective map π). Therefore, regardless of how complicated the resulting group G and its representation is, it is no more secured against the linear algebra attack described in subsection 3.3 and great care needs to be taken that the initial minimal degrees of the starting groups are large enough, say of the order n. On the other hand, if the minimal degrees of the starting group is sufficiently large, then from the point of view of such linear algebra attack it is not necessary to construct a more (structurally) complicated group or representation. 8. Invariants of supergroups

8.1. Definitions and actions. Let V be a superspace, that is a Z2 -graded space with even and odd components V0 and V1 respectively. If v ∈ Vi , then i is said to be a parity of v and it is denoted by |v|. In what follows, morphisms between two superspaces V and W are assumed to be graded. The tensor product V ⊗ W has L Vk ⊗ Wl . the natural structure of a superspace given by (V ⊗ W )i = k+l=i,k,l∈Z2

A Z2 -graded associative algebra A is called a superalgebra. The superalgebra A is said to be supercommutative if it satisfies ab = (−1)|a||b| ba for all homogeneous elements a and b. For example, any algebra A has the trivial superalgebra structure defined by A0 = A, A1 = 0. The tensor product A ⊗ B of two superalgebras A and

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS

15

B has the superalgebra structure defined by (a ⊗ b)(c ⊗ d) = (−1)|b||c| ac ⊗ bd for a, c ∈ A and b, d ∈ B. The category of all supercommutative superalgebras with graded morphisms is denoted by SAlgF . A superalgebra A is called a superbialgebra if it is a coalgebra with the coproduct ∆ : A → A ⊗ A and counit ǫ : A → F such that both ∆ and ǫ are P superalgebra homomorphisms. In what follows we use Sweedler’s notation ∆(a) = a1 ⊗ a2 for a ∈ A. A superspace V is called a left/right A-supercomodule if V is a left/right Acomodule and the corresponding comodule map τ : V → V ⊗ A is a morphism of superspaces. A superalgebra A is called a Hopf P superalgebra P if there is a superalgebra endomorphism s : A → A such that a1 s(a2 ) = s(a1 )a2 = ǫ(a) for a ∈ A. Additionally, we assume that s is bijective and it satisfies the condition ∆s = t(s ⊗ s)∆, where t : A ⊗ A → A ⊗ A is a (supersymmetry) homomorphism defined by ′ a ⊗ a′ 7→ (−1)|a|||a a′ ⊗ a for a, a′ ∈ A. Let A be a supercommutative superalgebra. Then the functor SSp A : SAlgF → Sets, defined by SSp A(C) = HomSAlgF (A, C) for C ∈ SAlgF , is called an affine superscheme. If X = SSp A is an affine superscheme, then A is denoted by F [X] and it is called the coordinate superalgebra of X. If A is a Hopf superalgebra, then G = SSp A is a group functor that is called an affine group superscheme, orPshortly, an affine supergroup. The group structure of G(C) is given by g1 g2 (a) = g1 (a1 )g2 (a2 ), g −1 = gs and 1G(C) = ǫ for g1 , g2 , g ∈ G(C) and a ∈ A. The category of affine supergroups is dual to the category of supercommutative Hopf superalgebras. If F [G] is finitely generated, then G is called an algebraic supergroup. If F [G] is finite-dimensional, then G is called a finite supergroup. A (closed) subsupergroup H of G is uniquely defined by the Hopf ideal IH of F [G] such that for every C ∈ SAlgF an element g ∈ G(C) belongs to H(C) if and only if g(IH ) = 0. For example, the largest even subsupergroup Gev of G is defined by the ideal F [G]F [G]1 . The category of left finite-dimensional G-supermodules coincides with the category of right F [G]-supercomodules. In fact, if V is a right F [G]-supercomodule, P then G(C) acts on V ⊗ v1 ⊗ g(a2 ) for PC by C-linear transformation g(v ⊗ 1) = g ∈ G(C) and τ (v) = v1 ⊗ a2 . Let V be a superspace such that dim V0 = m and dim V1 = n. The superspace V corresponds to an affine superscheme Am|n , called the affine superspace of (super)dimension m|n, such that Am|n (C) = C0m ⊕ C1n for every C ∈ SAlgF . The affine superscheme Am|n can be identified with the functor (V ⊗?)0 . In fact, choose a homogeneous basis consisting of elements vi such that |vi | = 0 for 1 ≤ i ≤ m and |vi | =P1 for m + 1 ≤ i ≤ m + n. Then every element w of (V ⊗ C)0 has the form w = 1≤i≤m+n vi ⊗ ci , where |ci | = |vi |. The coordinate superalgebra of Am|n is isomorphic to the polynomial superalgebra freely generated by the dual basis xi of V ∗ such that xP i (vj ) = δij for 1 ≤ i, j ≤ m + n. In other words, w(xi ) = xi (w) = ci for every w = 1≤i≤m+n vi ⊗ ci ∈ (V ⊗ C)0 and C ∈ SAlgF . In order to make the notation consistent, we will also denote F [Am|n ] by F [V ].

16

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

Every g ∈ G(C) induces an even operator on the F -superspace V ⊗ C. Thus (V ⊗ C)0 is a G(C)-submodule of V ⊗ C. Since this action is functorial, it gives the left G-action on the affine superscheme Am|n . The composition of this action with the inverse morphism g 7→ g −1 defines the right action of G on Am|n , which is equivalent to the right coaction of F [G] on F [V ]. Since the comodule map F [V ] → F [V ] ⊗ F [G] is a superalgebra homomorphism, the F [G]-supercomodule structure of F [VP ] is defined by F [G]-supercomodule strucP ture of V ∗ = 1≤i≤m+n F xi . If τ (vi ) = 1≤k≤t vk ⊗ aki for 1 ≤ i ≤ m + n, then X τ (xi ) = xk ⊗ (−1)|vk |(|vi |+|vk |) s(aik ). 1≤k≤t

There is a natural pairing (F [V ] ⊗ C) × (V ⊗ C) → C given by

(f ⊗ a)(v ⊗ b) = (−1)|a||v| f (v)ab = (−1)|a||v| v(f )ab

for a, b ∈ C and C ∈ SAlgF such that the above coaction is equivalent to the standard action (g(f ⊗ a))(v ⊗ b) = (f ⊗ a)(g −1 (v ⊗ b)) for g ∈ G(C). 8.2. Cryptology application. The invariants of supergroups have two possible applications in the design of public-key cryptosystem. The first option is to work with relative invariants from the C-superalgebra C[V ]G(C) = (F [V ] ⊗ C)G(C) for some superalgebra C ∈ A ∈ SAlgF . The second option is to work with absolute invariants from the superalgebra F [V ]G , consisting of all f ∈ F [V ] such that τ (f ) = f ⊗ 1, or equivalently, g(f ⊗ 1) = f ⊗ 1 for every g ∈ G(C) and C ∈ SAlgF . We will leave a consideration of these options for the future. 9. Invariants of certain supergroups Recall that every diagonalizable algebraic group is isomorphic to a finite product of copies of the one-dimensional torus Gm and groups µn , where µn is the n-th roots of unity and n > 1. Here µn (C) = {c ∈ C × |cn = 1} for every commutative algebra C (see Theorem 2.2 of [19]). Let D be a diagonalizable algebraic group and X = X(D) be the character group of D. Then F [D] = F X is a group algebra of X. The Lie algebra Lie(D) can be identified with the subspace of F [D]∗ = (F X)∗ consisting of all linear maps y : F X → F such that y(g1 g2 ) = y(g1 ) + y(g2 ) for every g1 , g2 ∈ X. Fix a pair (g, x), where g ∈ X and x ∈ Lie(D) such that if x 6= 0 then g 2 = 1. The following supergroup Dg,x was first introduced in [11]. The coordinate algebra K[Dg,x ] is isomorphic to KX ⊗ K[z] = KX ⊕ (KX)z, where z is odd and z 2 = 0. The Hopf superalgebra structure on K[Dg,x ] is defined as ∆(h) = h ⊗ h + x(h)hz ⊗ hgz, ∆(z) = 1 ⊗ z + z ⊗ g, ǫ(z) = 0, ǫ(h) = 1,

s(h) = h−1 for h ∈ X and s(z) = −g −1 z. Denote F h ⊕ F hz by L(h). Then every L(h) is an indecomposable injective Dg,x -supersubmodule of F [Dg,x ] and F [Dg,x ] = ⊕h∈X L(h). Let Y denote {h ∈ X|x(h) = 0}. The supermodule L(h) is irreducible if and only if h 6∈ Y . If L(h) is not irreducible, then it has the socle S(h) = F h and L(h)/S(h) ≃ ΠS(gh). If we denote the basis elements h and hz of L(h) by f0 and f1 respectively, then τ (v0 ) = f0 ⊗ h + x(h)f1 ⊗ hgz and τ (v1 ) = f0 ⊗ hz + f1 ⊗ hg.

Also, L(h)∗ ≃ ΠL(g −1 h−1 ) and S(h)∗ ≃ S(h−1 ).

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS

17

Proposition 9.1. (Proposition 5.1 of [11]) Every irreducible Dg,x -supermodule is isomorphic either to L(h) for h 6∈ Y or to S(h) for h ∈ Y . Moreover, every finite-dimensional Dg,x -supermodule is isomorphic to a direct sum of (not necessary irreducible) supermodules Πa L(h) and Πb S(h′ ) for h ∈ X, h′ ∈ Y and a, b = 0, 1.

Consider a (finite-dimensional) Dg,x -supermodule V such that V ∗ ≃ V (h1 ) ⊕ . . . ⊕ V (hs ). The superalgebra F [V ] is generated by the elements fj,0 and fj,1 , for 1 ≤ j ≤ s, such that |fj,0 | = 0, |fj,1 | = 1 and τ (fj,0 ) = fj,0 ⊗ hj + x(hj )fj,1 ⊗ hj gz and τ (fj,1 ) = fj,0 ⊗ hj z + fj,1 ⊗ hj g.

Let l = (l1 , . . . , ls ) be a vector with non-negative integer coordinates and let Q Q lj , f1J = j∈J fj,1 , J be a subset of s = {1, 2, . . . , s}. Denote f0l = 1≤j≤s fj,0 Q Q l hl = 1≤j≤s hjj and hJ = j∈J hj . For 1 ≤ j ≤ s let ǫj denote the vector that has the j-th coordinate equal to 1 and all remaining coordinates equal to zero. For a basis monomial f0l f1J we have X l−ǫ τ (f0l f1J ) =(f0l ⊗ hl + lj x(hj )f0 j fj,1 ⊗ hl gz)× 1≤j≤s

(f1J

J |J|

⊗h g

+

X

J\j

(−1)kj,J fj,0 f1

j∈J

=f0l f1J ⊗ hl hJ g |J| +

X

⊗ hJ g |J|−1 z) l−ǫj

(−1)kj,J∪j lj x(hj )f0

j6∈J

f1J∪j ⊗ hl hJ g |J|+1 z

X l+ǫ J\j + (−1)kj,J f0 j f1 ⊗ hl hJ g |J|−1 z, j∈J

where kj,J is the number of elements j ′ ∈ J such that j ′ > j. Since g |J|+1 = g |J|−1 , this implies the following proposition. P Proposition 9.2. A (super)polynomial f = l,J al,J f0l f1J belongs to F [V ]Dg,x if and only if the following conditions are satisfied. (1) If al,J 6= 0, then hl hJ g |J| = 1, (2) The polynomial X X X l−ǫ l+ǫ J\j al,J ( (−1)kj,J∪j lj x(hj )f0 j f1J∪j + (−1)kj,J f0 j f1 ) j6∈J

l,J

j∈J

vanishes. We can rewrite the polynomial X X X l−ǫ l+ǫ J\j al,J ( (−1)kj,J∪j lj x(hj )f0 j f1J∪j + (−1)kj,J f0 j f1 ) l,J

j6∈J

j∈J

from the second condition of the above proposition as X X X f0l f1J ( (−1)kj,J (lj + 1)x(hj )al+ǫj ,J\j + (−1)kj,J∪j al−ǫj ,J∪j ), l,J

j∈J

j6∈J

where lj = 0 implies al−ǫj ,J∪j = 0. P Corollary 9.3. A polynomial f = l,J al,J f0l f1J belongs to F [V ]Dg,x if and only if its coefficients al,J , for all pairs (l, J), satisfy the following equations. (1) If hl hJ g |J| 6= 1, then al,J = 0,

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

18

(2)

P

kj,J (lj j∈J (−1)

+ 1)x(hj )al+ǫj ,J\j +

P

kj,J∪j al−ǫj ,J∪j j6∈J (−1)

= 0.

If s = 1, then F [V ]Dg,x = F . Therefore, from now on we will assume that s > 1. Define the partial operator Pj acting on the set of all pairs (l, J) by Pj (l, J) = (l + ǫj , J \ j) in the case when j ∈ J, and Pj (l, J) is undefined if j ∈ / J. Also define the partial operator Qj acting on the set of all pairs (l, J) by Qj (l, J) = (l−ǫj , J ∪j) in the case j 6∈ J and lj > 0, and Qj (l, J) is undefined if j ∈ J or lj = 0. Lemma 9.4. The operators Pj and Qj satisfy the following conditions. (1) If Pj is defined on (l, J), then Qj Pj (l, J) = (l, J). Also, if Qj is defined on (l, J), then Pj Qj (l, J) = (l, J), (2) If j 6= j ′ and Pj Qj ′ is defined on (l, J), then Pj Qj ′ (l, J) = Qj ′ Pj (l, J). Alo, if j 6= j ′ and Qj ′ Pj is defined on (l, J), then Qj ′ Pj (l, J) = Pj Qj ′ (l, J).

Two pairs (l, J) and (l′ , J ′ ) are called equivalent if there is a chain (l, J) = (l0 , J0 ), . . . , (lk , Jk ) = (l′ , J ′ ) such that (li+1 , Ji+1 ) = Si (li , Ji ) for 0 ≤ i ≤ k − 1 and each Si is an operator of type P or Q. Lemma 9.4 implies that this relation is an equivalence and the set of equations from Corollary 9.3 is a disjoint union of subsets corresponding to these equivalence classes. Moreover, each such equivalence class has a unique representative of the form (l, s) or (0, J), where the cardinality of J is maximal over this class. In the first case, all pairs from the equivalence class of (l, s) can be obtained from this representative by appplying operators of type Q only. In the second case, all pairs from the equivalence class of (0, J) can be obtained from (0, J) by applying operators of type P only. Example 9.5. Let D = Gm . Since X(D) ≃ Z, we can fix a generator h of X = X(D). Then x ∈ Lie(D) is determined by the value x(h) = α ∈ F . We will describe invariants of D1,x correposponding to the partial case when s = 2. Denote h1 = hk1 , h2 = hk2 . The subset of equations in Corollary 9.3 corresponding to the pair (0, {1}) is given as αk1 a(1,0),∅ = 0 = a(0,0),{1}

and the subset corresponding to the pair (0, {2}) is given as αk2 a(0,1),∅ = 0 = a(0,0),{2} .

The subset of equations, which corresponds to the pair ((l1 , l2 ), {1, 2}), consists of the equations α(−(l1 + 1)k1 a(l1 +1,l2 ),{2} + (l2 + 1)k2 a(l1 ,l2 +1),{1} ) = 0, α(l2 + 1)k2 a(l1 +1,l2 +1),∅ − a(l1 ,l2 ),{1,2} = 0, α(l1 + 1)k1 a(l1 +1,l2 +1),∅ + a(l1 ,l2 ),{1,2} = 0 and a(l1 +1,l2 ),{2} + a(l1 ,l2 +1),{1} = 0. If α = 0 and k1 , k2 6= 0, then the superspace F [V ]D1,x is generated by the elements {2} {1} and f0l+ǫ1 f1 − f0l+ǫ2 f1 such that (l1 + 1)k1 + (l2 + 1)k2 = 0. If α 6= 0 and k1 , k2 6= 0, then the superspace F [V ]D1,x is generated by the elements {1,2} {1,2} {2} l+ǫ1 +ǫ2 f0 − α(l1 + 1)k1 f0l f1 = f0l+ǫ1 +ǫ2 + α(l2 + 1)k2 f0l f1 and f0l+ǫ1 f1 − {1} f0l+ǫ2 f1 such that (l1 + 1)k1 + (l2 + 1)k2 = 0. The remaining cases, when k1 = 0 or k2 = 0, are left for the reader. f0l+ǫ1 +ǫ2

PUBLIC-KEY CRYPTOSYSTEM BASED ON INVARIANTS

19

We have outlined the procedure to determine vector-space generators of the space of invariants of the supergroup Dg,x and completed this goal in some particular cases above. It would be desirable not only to find the vector-space generators in general but also to describe algebra generators of invariants of the supergroup Dg,x . This task and the design of the cryptosystem based on invariants of supergroups will be a subject of our future work. References [1] Agarwal, S. and Frandsen, G.S.: Binary GCD like algorithms for some complex quadratic rings, Algorithmic number theory, 57–71, Lecture Notes in Comput. Sci., 3076, Springer, Berlin, 2004. [2] Burnside, W., On groups of linear substitutions of finite order which possess quadratic invariants, Proc. London Math. Soc. S2-12 no. 1, 89–93. [3] Conforti, M., Cornu´ ejols, G. and Zambelli, G.: Integer programming. Graduate Texts in Mathematics, 271. Springer, Cham, 2014. [4] Derksen, Harm; Kraft, Hanspeter, Constructive invariant theory, Algebre non commutative, groupes quantiques et invariants (Reims, 1995), 221–244, Semin. Congr., 2, Soc. Math. France, Paris, 1997. [5] D. Grigoriev, Public-key cryptography and invariant theory, Journal of Mathematical Sciences 126 (2005), no.3, 1152–1157, translated from Zapiski Nauchnych Seminarov POMI, 293 (2002), 26–38. [6] D. Grigoriev, A. Kojevnikov and S.J. Nikolenko, Algebraic Cryptography: New constructions and their security against provable break, St. Peterburg Math. J. 20 (2009), no.6, 937–953, translated from Algebra i Analysis 20 (2008), no.6. [7] Huffman, W. Cary, Polynomial invariants of finite linear groups of degree two. Canad. J. Math. 32 (1980), no. 2, 317–330. [8] Hochster, M., Roberts, J., Rings of invariants of reductive groups acting on regular rings are Cohen-Macaulay, Adv. Math. 13 (1974), 115–175. [9] Humphreys, James E. Linear algebraic groups. Graduate Texts in Mathematics, No. 21. Springer-Verlag, New York-Heidelberg, 1975. xiv+247 pp. [10] Kaltofen, E. and Rolletschek, H.: Computing greatest common divisors and factorizations in quadratic number fields, Math. Comp., 53(188):697720, 1989. [11] Akira Masuoka and Alexandr N. Zubkov, Solvability and nilpotency for algebraic supergroups, submitted to Pacific.J.Math., see also arXiv: 1502.07021v1. [12] Noether, E., Der Endlichkeitssatz der Invarianten endlicher Gruppen, Math. Ann. 77 (1916), 89–92. [13] Pan, Victor Y.: Solving a polynomial equation: some history and recent progress. SIAM Rev. 39 (1997), no. 2, 187–220. [14] Shub, M. and Smale, S.: Computational complexity. On the geometry of polynomials and a theory of cost. I. Ann. Sci. Ecole Norm. Sup. (4) 18 (1985), no. 1, 107–142. [15] Shub, M. and Smale, S.: Computational complexity: on the geometry of polynomials and a theory of cost. II. SIAM J. Comput. 15 (1986), no. 1, 145–161. [16] Smith, Larry, Polynomial invariants of finite groups - a survey of recent results, Bull. Amer. Math. Soc. 34 (1997), no. 3, 211–250. [17] Symonds, Peter, On the Castelnuovo-Mumford regularity of rings of polynomial invariants, Ann. of Math. (2) 174 (2011), no. 1, 499–517. [18] J.G.Thompson, Invariants of finite groups, J.Algebra, 69 (1981), 143–145. [19] W.C. Waterhouse, Introduction to affine group schemes, Springer-Verlag, New York, 1979. E-mail address: [email protected] Qatar University, Department of Mathematics and Physics, College of Arts and Sciences, P. O. Box 2713, Doha, Qatar E-mail address: [email protected]

20

´ S, ˇ FRANTISEK ˇ MARTIN JURA MARKO, AND ALEXANDR N. ZUBKOV

Penn State Hazleton, 76 University Drive, Hazleton, PA 18202, USA E-mail address: [email protected] Omsk State Polytechnic University, Mira 11, 644050, Russia