Investigation of metabelian platform groups for protocols based on (simultaneous) conjugacy search problem

Delaram Kahrobaei; Carmine Monetta; Ludovic Perret; Maria Tota; Martina Vigorito

doi:10.1515/jmc-2023-0036

Article Open Access

Investigation of metabelian platform groups for protocols based on (simultaneous) conjugacy search problem

Delaram Kahrobaei , Carmine Monetta , Ludovic Perret , Maria Tota and Martina Vigorito

Published/Copyright: April 14, 2025

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information

From the journal Journal of Mathematical Cryptology Volume 19 Issue 1

Abstract

There are many group-based cryptosystems in which the security is related to the conjugacy search problem or the simultaneous conjugacy search problem in their underlying platform groups. In this article, we show that some metabelian groups do not provide strong security for these cryptosystems and so they cannot be chosen as platform groups.

Keywords: metabelian groups; conjugacy search problem; non-commutative public key cryptography

MSC 2010: 20F14; 94A60

1 Introduction

The field of group-based cryptography began with the seminal work of Anshel et al., in 1999, when they proposed the commutator key-exchange protocol whose security is related to the simultaneous conjugacy search problem (SCSP) [1]. The search for a platform group for this protocol has been an active area including several cryptanalysis. In general, to be a suitable platform group, there are certain properties that the group must satisfy, for instance, we need fast computation of the products and fast comparison of the elements. Indeed, to allow fast computation, the group has to be finitely presented. Also, the problem on which the protocol is based should be hard in the underlying platform. Then, a good candidate could be a polycyclic group. In fact, polycyclic groups are natural generalizations of cyclic groups and cyclic groups have been used in the classical cryptosystems such as Rivest Shamir Adleman [2]. Moreover, polycyclic groups own more complex algorithmic problems and a finite presentation, which provide suitable platforms for cryptography. Actually, Eick and Kahrobaei [3] introduced a new line of investigation for cryptography which has been called polycyclic group-based cryptography. More precisely, they proposed such groups as platform groups for the Commutator Key-Exchange protocol, also known as the Anshel–Anshel–Goldfeld (AAG) Key-Exchange protocol, as well as for the non-commutative Diffie–Hellman (a.k.a. Ko-Lee) Key-Exchange protocol [4]. The security of these protocols is related to SCSP and the conjugacy search problem (CSP) in the platform group G . This means that solving the SCSP in G implies breaking the AAG protocol and that solving the CSP in G implies breaking the Ko-Lee protocol in G . The argument used in the study of Eick and Kahrobaei [3] is based on experimental results for the CSP in certain metabelian polycyclic groups arising from field extensions. These groups are not virtually nilpotent; hence, the CSP cannot be solved using the analysis provided in the study of Monetta and Tortora [5]. Nevertheless, some of these groups must be avoided as platform since, Kotov and Ushakov [6] cryptanalyzed the AAG Key-Exchange protocol for some metabelian groups of this type, using the so-called Field-Based-Attack (FBA). Then, Gryak et al. [7] investigated a particular class of metabelian groups, i.e., the split metabelian groups of finite Prüfer rank. Indeed, they obtained a complexity result concerning the CSP, which is proved to be at most exponential for the class of groups under investigation. The methods used to test the CSP include experiments conducted with machine learning algorithms, as done by Gryak et al. [8], but also Length-Based-Attack (LBA) [9].

In this context, we got inspired by previous studies [6,7] and, in this study, we extend the arguments in the study of Kotov and Ushakov [6] to a more general class of metabelian groups, including non-polycyclic ones. The same class of groups as the one considered in the study of Gryak et al. [7] and we use a different approach to study the CSP in these groups. Actually, we address the AAG protocol and the Ko-Lee protocol and investigate the possibility of using some particular metabelian groups as platform groups for these protocols. It turns out that the groups under consideration are not suitable because do not provide security for the corresponding Key-Exchange protocols. In fact, we prove the following theorems.

Theorem 4.1: Let G = M ⋉ N , where M ≅ Z n and N = Z [ m 1 ± 1 , … , m n ± 1 ] (as additive groups), with m 1 , … , m n positive integers. Then, there exists a polynomial-time algorithm that solves the SCSP, which implies that it is possible to break the Commutator Key-Exchange protocol for such a group G .

Theorem 4.2: Let G = M ⋉ N , where M ≅ Z n and N = Z [ m 1 ± 1 , … , m n ± 1 ] (as additive groups), with m 1 , … , m n positive integers. Then, there exists a polynomial-time algorithm that solves the CSP, which implies that it is possible to break the non-commutative Diffie-Hellmann Key-Exchange protocol for such a group G .

This article is structured as follows: in Section 2, we provide the reader with some needed definitions from group theory, and we recall the definitions of some algorithmic problems. Then, we describe the Key-Exchange protocols we are interested in, i.e., the non-commutative Diffie–Hellman and the AAG. Section 3 introduces the family of metabelian groups of interest, along with some examples. In Section 4, we prove Theorems 4.1 and 4.2. The conclusions of our work are in Section 5.

Related works: For an overview of group-based cryptography in the quantum era, refer to [10,11]. Garberet al. [9] demonstrated that the LBA is inefficient for certain classes of metabelian polycyclic groups. However, Kotov and Ushakov [6] explored the security properties of polycyclic groups used as the platform for the Commutator Key-Exchange protocol and showed that, despite the limited success of the LBA, the protocol can still be broken by a deterministic polynomial-time algorithm. This approach, called the FBA, was implemented in groups, algorithms and programming to compare the performance of LBA and FBA.

2 Background

2.1 Algorithmic problems

We start clarifying the difference between a decision problem and a search problem, as this distinction will form the basis for the next definitions. In general, decision problems are problems of the following nature: given a property P and an object O, find out whether or not the object O has the property P. On the other hand, search problems are of the following nature: given a property P and an object O with the property P, find a witness for the property P; for example, given two conjugate elements, find a conjugator. Therefore, in the next definitions, we will assume that the related decision problems are solvable; otherwise, if you cannot even determine whether a solution exists, then it does not make sense to search for that solution.

Here and in the following, if x and g are group elements, the conjugate of g by x is denoted by g x and it is the element x − 1 g x .

CSP: Let G be a finitely presented group such that the conjugacy decision problem is solvable. Given g , h ∈ G such that h = g x for some x ∈ G , the CSP asks to find such an element x ∈ G .

SCSP: Given a finitely presented group G and g 1 , … , g n , h 1 , … , h n elements of G such that h i = g i x , for all i ∈ { 1 , … , n } and some x ∈ G , the SCSP asks to recover such a conjugator x ∈ G .

Note that the CSP and the SCSP are always solvable since we assume that the corresponding decision problems are solvable. Also, a solution of g x = h is not unique. In fact, given a solution x , the set of solutions is { a x : a ∈ C G ( g ) } , being C G ( g ) = { a : g a = g } the centralizer in G of g .

2.2 Polycyclic and metabelian groups

As mentioned in Section 1, the search for the platform group for non-commutative cryptographic protocols is a very active area. In particular, in this context, polycyclic groups show up. Here, we recall the definition.

Definition 2.1

A group G is said to be polycyclic if it has a descending chain of subgroups

G = G 1 ≥ G 2 ≥ … ≥ G n + 1 = 1 ,

in which each G i + 1 is a normal subgroups of G i and the quotient G i ⁄ G i + 1 is non-trivial cyclic for i ∈ { 1 , … , n } .

Related definitions are the following.

Definition 2.2

A group G is called nilpotent if it has a central series, i.e., a normal series

G = G 1 ≥ G 2 ≥ … ≥ G n + 1 = 1

such that G i ⁄ G i + 1 is contained in the center of G ⁄ G i + 1 for all i ∈ { 1 , … , n } .

For example, nilpotent groups have been suggested for multilinear cryptography in previous studies [12,13].

Definition 2.3

A group G is said to be virtually nilpotent if it has a non-trivial normal subgroup H such that H is nilpotent and G ⁄ H is finite.

Definition 2.4

A group G is said to be metabelian if it has a non-trivial normal subgroup H such that both H and G ⁄ H are abelian.

In the case of finitely generated metabelian groups, the conjugacy decision problem is solvable [14].

2.3 Non-commutative Diffie-Hellman (a.k.a. Ko-Lee) key-exchange protocol

Originally proposed by Ko et al. in [4] using braid groups, the non-commutative Diffie-Hellman Key-Exchange protocol can be generalized to work over other platform groups. Let G be a finitely presented group, with A , B ≤ G such that all elements of A and B commute.

An element g ∈ G is chosen, and g , G , A , and B are made public. A shared secret can then be constructed as follows:

Alice chooses a random element a ∈ A and sends g a to Bob.
Bob chooses a random element b ∈ B and sends g b to Alice.
The shared key K is then g a b , as Alice computes ( g b ) a , which is equal to Bob’s computation of ( g a ) b as a and b commute.

Hence, in order to find K , an attacker needs to obtain a or b , which are private, from g , g a , and g b , which are public. In fact, the security of this protocol is related to the CSP.

2.4 AAG Key-Exchange protocol

The AAG Key-Exchange protocol [1] is a two-party protocol performed as follows:

Fix a finitely presented group G , called the platform group, a set of generators g 1 , … , g k for G and some positive integers n 1 , n 2 , l , m . All this information is made public.
Alice prepares a tuple of elements a ¯ = ( a 1 , … , a n 1 ) called Alice’s public tuple. Each a i is generated randomly as a product of g i ’s and their inverses.
Bob prepares a tuple of elements b ¯ = ( b 1 , … , b n 2 ) called Bob’s public tuple. Each b i is generated randomly as a product of g i ’s and their inverses.
Alice generates a random element A as a product a s 1 ε 1 … a s l ε l , where a s j ∈ { a 1 , … , a n 1 } and ε j ∈ { + 1 , − 1 } , for all j ∈ { 1 , … , l } . The element A (or more precisely its factorization) is called the Alice’s private element.
Bob generates a random element B as a product b t 1 δ 1 … b t m δ m , where b t j ∈ { b 1 , … , b n 2 } and δ j ∈ { + 1 , − 1 } , for all j ∈ { 1 , … , m } . The element B (or more precisely its factorization) is called the Bob’s private element.
Alice publishes the tuple of conjugates b ¯ A = ( A − 1 b 1 A , … , A − 1 b n 2 A ) .
Bob publishes the tuple of conjugates a ¯ B = ( B − 1 a 1 B , … , B − 1 a n 1 B ) .
Finally, Alice computes the element K A as a product:
A − 1 ( B − 1 a s 1 ε 1 B … B − 1 a s l ε l B ) = A − 1 B − 1 A B = [ A , B ]
using the elements of Bob’s conjugate tuple a ¯ B .
Similarly, Bob computes the element K B as a product:
( A − 1 b t 1 δ 1 A … A − 1 b t m δ m A ) − 1 B = A − 1 B − 1 A B = [ A , B ]
using the elements of Alice’s conjugate tuple b ¯ A .
It is easy to check that K = K A = K B = [ A , B ] in G . The obtained commutator is the shared key.

The security of this protocol is based on the computational hardness of computing the commutator [ A , B ] based on the intercepted public information, the tuples a ¯ and b ¯ and their conjugates b ¯ A and a ¯ B . In practice, it is often achieved by solving systems of conjugacy equations for A or B , i.e., finding X and Y satisfying the following systems:

(1) X − 1 b 1 X = b 1 ′ , … X − 1 b n 2 X = b n 2 ′ ,

(2) Y − 1 a 1 Y = a 1 ′ , … Y − 1 a n 1 Y = a n 1 ′ .

This means that the security of this protocol is related to the SCSP.

3 Examples of metabelian groups

Here, we describe the families of groups we are interested in. To be more precise, we consider groups G of the form G = M ⋉ N , with both groups M and N abelian. This implies that N is a normal abelian subgroup of G with abelian quotient, i.e., G is metabelian. We use multiplicative notation for the whole group G but additive notation for N . So if s ∈ M and c ∈ N , the action of the element s maps c to

c ⋅ s with additive notation or, c s = s − 1 c s with multiplicative notation.

These kinds of groups arise quite naturally in linear algebra and ring theory, as we will show in more details in the following examples.

Example 3.1

The first example we want to mention is the one introduced by Kotov and Ushakov [6]. Given an irreducible monic polynomial f ( x ) ∈ Z [ x ] , they put:

(3) F = Q [ x ] ⁄ ( f ) .

Then, they considered the group G = F * ⋉ F , where F * is the multiplicative group of F . If a ∈ F * and b ∈ F , the action of a maps b to b ⋅ a . This is a metabelian group, which is also polycyclic, and Kotov and Ushakov [6] showed that in such a group, it is possible to reduce systems (1) and (2) to two systems of linear equations over the field F .

More generally, we can start with a vector space and we obtain the following example.

Example 3.2

Let V ( + , ⋅ ) be a vector space over a field F , and consider the group G = F * ⋉ V , where F * is the multiplicative group of F . If λ ∈ F * and v ∈ V , the action of λ maps v to v ⋅ λ . Hence, G has the same structure of the general group we introduced at the beginning of the section. Note that, for V = F , if F is of the form described in (3), then we obtain Example 3.1. Similarly, we could start with a module over a commutative unitary ring.

Such examples are interesting from a mathematical point of view but more examples, as they have been described in [7, Section 2], follow.

Example 3.3

Split metabelian groups of finite Prüfer rank.

Let m i j be integers, and consider the following group presentation:

G = ⟨ q 1 , … , q n , b 1 , … , b s ∣ [ q i , q j ] = 1 , [ b l , b t ] = 1 , b j q l = b 1 m 1 j b 2 m 2 j … b s m s j ⟩ .

Let M the subgroup of G generated by q 1 , … , q n and N the normal subgroup of G generated by b 1 , … , b s . Then, G = M ⋉ N is a split metabelian group of finite Prüfer rank (meaning that the number of generators needed to generate any finitely generated subgroup is bounded). For such a group, it is possible to see that there exists an embedding N → Q s . Observe that the action of M on N can be described using integer matrices: the action of q l is encoded by the ( s × s ) -matrix M l with columns m 1 j , … , m s j . Moreover, G is polycyclic if and only if the matrices M l can be taken to be integer matrices with integral inverses [15].

One of the main advantages of these groups is that they admit the following fairly simple set of normal forms:

q 1 α 1 … q n α n b 1 β 1 … b s β s q 1 γ 1 … q n γ n ,

with γ 1 , … , γ n > 0 . Moreover, there is an efficient algorithm (collection) to transform any word in the generators to the corresponding normal form: given an arbitrary word in the generating system, move all of the instances of q i with negative exponent to the left and all the instances of q i with positive exponents to the right.

Example 3.4

Generalized metabelian Baumslag–Solitar groups.

Let m 1 , … , m n be the positive integers. We call the group given by the following presentation a generalized metabelian Baumslag–Solitar group:

G = ⟨ q 1 , … , q n , b ∣ [ q i , q j ] = 1 , b q i = b m i , i , j = 1 , … , n ⟩ .

It is a split metabelian group of finite Prüfer rank and G ≅ M ⋉ N with M = ⟨ q 1 , … , q n ⟩ ≅ Z n and N = Z [ m 1 ± 1 , … , m n ± 1 ] (as additive groups).

This group represents a natural platform for group-based protocols. Indeed, it possesses good algorithmic properties, as it admits a finite presentation that allows us to work within several proposed cryptosystems.

4 Investigation of platform groups for the commutator and non-commutative Diffie–Hellman key-exchange protocols

We begin studying the CSP and the SCSP in a metabelian group of the form G = M ⋉ N , as described in Section 3. Following [7, Section 3], assume that we have conjugated elements g , h ∈ G and we want to solve the CSP for g , h , i. e., we want to find x ∈ G such that

g x = h .

We put g = s c , h = s ′ c ′ , and x = t d , where s , s ′ , t ∈ M and c , c ′ , d ∈ N . Then,

g x = x − 1 g x = d − 1 t − 1 s c t d = d − 1 s t − 1 c t d = s ( d − 1 ) s c t d .

Now, g x = h implies s ′ = s and c ′ = ( d − 1 ) s c t d . Since the element ( d − 1 ) s c t d belongs to N , we can write it additively as

− d ⋅ s + c ⋅ t + d = d ⋅ ( 1 − s ) + c ⋅ t .

This means that the CSP above is equivalent to the problem of finding t ∈ M and d ∈ N such that

(4) d ⋅ ( 1 − s ) + c ⋅ t = c ′ ,

where s ∈ M and c , c ′ ∈ N are given.

Similarly, (see [6, Section 3]) assume that we have conjugated elements b i X = b i ′ in G and we want to solve the SCSP, i.e., we want to find such a conjugator X ∈ G . If we put b i = s i c i , b i ′ = s i ′ c i ′ with s i , s i ′ ∈ M and c i , c i ′ ∈ N , for all i ∈ { 1 , … , n 2 } , and X = t d with t ∈ M , d ∈ N we can apply the reduction process described above and we end up with the following system of equations:

(5) d ⋅ ( 1 − s 1 ) + c 1 ⋅ t = c 1 ′ , … d ⋅ ( 1 − s n 2 ) + c n 2 ⋅ t = c n 2 ′ ,

where s i ∈ M and c i , c i ′ ∈ N are given and we need to find t ∈ M and d ∈ N .

Then, we apply the discussion in [6, Section 3] to the groups described in Example 3.4. In particular, we will show that the generalized metabelian Baumslag–Solitar group is not a suitable platform group for the AAG protocol.

In fact, we have the following theorem.

Theorem 4.1

Let G = M ⋉ N , where M ≅ Z n and N = Z [ m 1 ± 1 , … , m n ± 1 ] (as additive groups), with m 1 , … , m n positive integers. Then, there exists a polynomial-time algorithm that solves SCSP, which implies that it is possible to break the Commutator Key-Exchange protocol for such a group G.

Proof

In the AAG protocol, the attacker knows b 1 X , b 2 X , … , b n 2 X for some b 1 , … , b n 2 (which are public) and n 2 > 1 . To find X = t d , with t ∈ M and d ∈ N , the attacker has to solve a system of equations such as (5). Let us consider two equations from the system

d ⋅ ( 1 − s ) + c ⋅ t = c ′ , d ⋅ ( 1 − s ˜ ) + c ˜ ⋅ t = c ˜ ′ ,

were s , s ˜ , c , c ˜ , c ′ , and c ˜ ′ are known and the attacker has to find t and d . Recall that c ′ , c ˜ ′ , c , c ˜ , and d lie in N which is a subring of Q . If we identify s and t with the integer they act by, then they also lie in N . So the above can be seen as a system of two equations in N ; moreover, we know a priori that the system has a solution. This means that unless the second equation is a multiple of the first one, this solution is unique and the standard procedure to solve the system yields then the suitable value of t and d in polynomial time.□

The argument in the previous proof applies also when G is as described in Example 3.2, choosing V = F n with n ∈ N . Moreover, if F is as defined in (3), it gives back the FBA described in [6].

Next, we move to the non-commutative Diffie–Hellmann Key-Exchange protocol.

Theorem 4.2

Let G = M ⋉ N , where M ≅ Z n and N = Z [ m 1 ± 1 , … , m n ± 1 ] (as additive groups), with m 1 , … , m n positive integers, then there exists a polynomial-time algorithm that solves CSP, which implies that it is possible to break the Diffie–Hellmann Key-Exchange protocol for such a group G.

Proof

In the Ko-Lee protocol, the main problem is that Alice and Bob must agree on a set Ω of pairwise commuting elements and then choose their conjugators a and b from that set. Let a = t d , where t ∈ M and d ∈ N . As M is abelian, a possible choice would be Ω = M , and if a lies in M , then the attacker can find a from g a in polynomial time. Another possibility would be to choose a ∈ N . But then a = d and the equation (4) for g a is

d ⋅ ( 1 − s ) + c = c ′ ,

and the only unknown is d , which can be easily found in polynomial time, since we can see it as a linear equation over Q . In the case when a is an arbitrary element lying neither in M nor in N , Ω must be a subset of the centralizer C G ( a ) of a in G .

Things are particularly easy when the element a belongs to M r for some r ∈ N , which happens if and only if

a = t d = t 1 r = r − 1 t 1 r = t 1 t 1 − 1 r − 1 t 1 r = t 1 r − t 1 r ,

for some t , t 1 ∈ M and d ∈ N . Additively, this is equivalent to

d = r − r ⋅ t = r ⋅ ( 1 − t ) .

It is a standard fact that M r = { x δ ( x ) ∣ x ∈ M } , where δ is the inner derivation given by δ ( x ) = r ⋅ ( 1 − x ) . In this case, it is easy to check that

Ω ⊆ C G ( a ) = M r .

If the attacker has the extra information that a belongs to M r for some r , then the equation that he has to solve is

r ⋅ ( 1 − t ) ( 1 − s ) + c ⋅ t = c ′ .

Equivalently,

( c − r + r ⋅ s ) ⋅ t = c ′ − r + s ⋅ r .

This can be seen as an equation in Q and only requires to perform the quotient of c ′ − r + r ⋅ s by c − r + r ⋅ s thus can be solved in polynomial time.

Moreover, we are going to see now that by embedding our group G in a bigger group, we may always assume that a lies in some conjugated subgroup of M . Let G ˜ = M ⋉ N ˜ where N ˜ = N ⊗ Z Q = Q . Then, a = t d lies in M r for some r ∈ N ˜ if and only if d = r ⋅ ( 1 − t ) . This can always be solved in Q ; in other words, we can always find a suitable r ∈ Q . Then, one proceeds as we did before with this r . The fact that r might not belong to N does not create any troubles: recall that we are dealing not with the conjugacy problem but with the conjugacy search problem, meaning that we know a priory that our equations have a solution so the aforementioned procedure yields the right values of t , d even if r does not belong to N .

Observe that behind what we said above is the fact that for the group G ˜ , the first cohomology group H 1 ( M , N ˜ ) is zero; thus, all the complements of N ˜ in G ˜ are conjugated.□

Note that exactly the same argument in the previous proof applies for any group G = M ⋉ N with N ⊆ Q n for some n , so it can be extended to the groups described in Example 3.3.

5 Conclusion

We have shown that the generalized metabelian Baumslag–Solitar groups are not suitable platforms neither for the AAG nor for the Ko-Lee protocol.

Actually, our results could also be generalized to all protocols whose security is related to the CSP and the SCSP. For example, the Kahrobaei–Koupparis Digital Signature Scheme [16] and the Khan–Kahrobaei non-commutative ElGamal Key-Exchange [17] (see also [18]).

Nevertheless, we want to point out that our results do not exclude the possibility for a metabelian group to be a suitable platform group for the protocols under consideration.

Acknowledgements

DK thanks the University of Salerno (Italy), where most of this article was discussed and written. We thank Professor Conchita Martinez-Perez for fruitful discussions. MV thanks Initiative for the Theoretical Sciences at CUNY GC, which hosted her Fall 2022. CM, MT, and MV are members of the “National Group for Algebraic and Geometric Structures, and their Applications” (GNSAGA - INdAM). These results have been presented in September 2024 at CIFRIS24, the Italian congress of De Cifris (www.decifris.it/cifris24). The authors acknowledge support of the Institut Henri Poincaré (UAR 839 CNRS-Sorbonne Université), and LabEx CARMIN (ANR-10-LABX-59-01).

Funding information: This research was funded by GNSAGA - INdAM and by the European Union - Next Generation EU, Missione 4 Componente 1 CUP B53D23009410006, PRIN 2022- 2022PSTWLB - Group Theory and Applications. DK has conducted this work partially with the support of ONR Grant 62909-24-1-2002.
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Conflict of interest: Dr. Delaram Kahrobaei is a member of the Editorial Board of the Journal of Mathematical Cryptology but was not involved in the review process of this article.
Ethical approval: The conducted research is not related to either human or animal use.
Data availability statement: Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

References

[1] Anshel I, Anshel M, Goldfeld D. An algebraic method for public-key cryptography. Math Res Let. 1999;6:287–91. 10.4310/MRL.1999.v6.n3.a3Search in Google Scholar

[2] Rivest R, Shamir A, Adleman L. A method for obtaining digital signatures and public-key cryptosystems. Commun ACM. 1978;2:463–74. 10.7551/mitpress/12274.003.0047Search in Google Scholar

[3] Eick B, Kahrobaei D. Polycyclic groups: A new platform for cryptology?. 2004. arXiv:math/0411077v1 [math.GR]. Search in Google Scholar

[4] Ko KH, Lee SJ, Cheon JH, Han JW, Kang J, Park C. New public-key cryptosystem using braid groups. Adv Cryptol CRYPTO 2000;1880:166–83. 10.1007/3-540-44598-6_10Search in Google Scholar

[5] Monetta C, Tortora A. The multiple conjugacy search problem in virtually nilpotent polycyclic groups. Adv Group Theory Appl. 2022;13:61–70. Search in Google Scholar

[6] Kotov M, Ushakov A. Analysis of a certain polycyclic-group-based cryptosystem. J Math Cryptol. 2015;9:161–7. 10.1515/jmc-2015-0013Search in Google Scholar

[7] Gryak J, Kahrobaei D, Martinez-Perez C. On the conjugacy problem in certain metabelian groups. Glasgow Math J. 2019;61(2):251–69. 10.1017/S0017089518000198Search in Google Scholar

[8] Gryak J, Haralick R, Kahrobaei D. Solving the Conjugacy Decision Problem via Machine Learning. Experiment Math. 2020;29:66–78. 10.1080/10586458.2018.1434704Search in Google Scholar

[9] Garber D, Kahrobaei D, Lam HT. Length based attack for polycyclic groups. J Math Cryptol. 2015;9:33–44. 10.1515/jmc-2014-0003Search in Google Scholar

[10] Kahrobaei D, Flores R, Noce M Group-based cryptography in the quantum era. Notices of the American Mathematical Society. 2023. pp. 752–63. 10.1090/noti2684Search in Google Scholar

[11] Kahrobaei D, Flores R, Noce M, Habeeb M, Battarbee C. Applications of group theory in cryptography. in: Mathematical Surveys and Monographs. Vol. 278. Providence, Rhode Island: American Mathematical Society; 2024. 10.1090/surv/278Search in Google Scholar

[12] Kahrobaei D, Tortora A, Tota M. Multilinear cryptography using nilpotent groups. Elementary theory of groups and group rings and related topics. Proceedings of the Conference Held on 1-2 November 2018 at Fairfield University and Graduate Center, CUNY. De Gruyter Proceedings in Mathematics, Boston, 2018. p. 127–34. 10.1515/9783110638387-013Search in Google Scholar

[13] Kahrobaei D, Tortora A, Tota M. A closer look at the multilinear cryptography using nilpotent groups. Int J Comput Math Comput Syst Theory. 2022;7(1):63–7. 10.1080/23799927.2021.2006314Search in Google Scholar

[14] Noskov GA. Conjugacy problem in Metabelian groups. Math Notes Acad Sci USSR. 1982;31:252–8. 10.1007/BF01138933Search in Google Scholar

[15] Auslander L. On a problem of Philip Hall. Ann Math. 1967;86(1):112–6.10.2307/1970362Search in Google Scholar

[16] Kahrobaei D, Koupparis C. Non-commutative digital signatures using non-commutative groups. Groups Complexity Cryptol. 2012;4:377–84. 10.1515/gcc-2012-0019Search in Google Scholar

[17] Kahrobaei D, Khan B. Nis05-6: A non-commutative generalization of ElGamal key exchange using polycyclic groups. IEEE Global Telecommunications Conference (GLOBECOM ’06). New York: IEEE Press; 2006. 10.1109/GLOCOM.2006.290Search in Google Scholar

[18] Gryak J, Kahrobaei D. The status of the polycyclic group-based cryptography: A survey and open problems. Groups Complexity Cryptol. 2016;8(2):171–86. 10.1515/gcc-2016-0013Search in Google Scholar

Received: 2023-10-11

Revised: 2024-09-30

Accepted: 2024-10-15

Published Online: 2025-04-14

This work is licensed under the Creative Commons Attribution 4.0 International License.

Articles in the same Issue

https://doi.org/10.1515/jmc-2023-0036

Keywords for this article

metabelian groups; conjugacy search problem; non-commutative public key cryptography

Creative Commons

BY 4.0