Comparison of various in-order iterator implementations in C++

2.1 Tree

A tree is a special type of graph. Thus, the set of all trees is a subset of the set of all graphs. The literature provides multiple equivalent definitions of the tree. In this article, we use the definition that says that a graph is a tree if it is acyclic and connected – each edge is a bridge (removing the edge would cause the graph to split into two disconnected subgraphs) [10]. We use the term node as a synonym of the vertex since it is a more common name used in the context of trees.

A tree is a hierarchical structure. The edges denote the parent–son relationship. A node can have multiple sons but at most one parent. There is at most one node that has no parent, which we call the root of the tree. A node that has no sons is called leaf. All other nodes are internal nodes. The number of sons of a node is also known as its degree. Another important property of a node is its level, defined as the distance from the root node, i.e., how many times we need to access the parent to reach the root. The root node has a level of zero. A tree is considered balanced if the level of all leaves is roughly the same. In Figure 1, we can see an example of a balanced tree with nodes labeled using letters A–F. Node A is the root node. Nodes C, D, E, and F are leaves, and B is an internal node. Node A has degree three, node B has degree two, and the leaves have degree zero. Node A has a level of zero; nodes B, C, and D have a level of one; and nodes E and F have a level of two. Depth of a tree is defined as the maximum of levels of its nodes, i.e., a tree consisting of a single node has depth zero.

Figure 1

Example of a tree with seven nodes labeled using letters A–F.

The tree is a recursive structure – each son of a node and, consequently, each node of a tree – can be considered as the root node of a standalone tree, which we sometimes refer to as subtree. This property is important in the design of recursive algorithms that operate on the tree.

The literature classifies trees in several ways using different criteria. One such criterion is the limit in the degree of nodes. Using this criterion, we can classify trees as:

1. multi-way – there is no limit to the degree of nodes,

2. K-way – each node has degree at most K ∈ N ,

3. binary – a special case of K -way tree for K = 2 .

Though the binary tree is just a special case of the more general K -way tree, it is listed as a separate category. The reason is that it has numerous applications and, therefore, has been studied extensively by computer scientists. In addition to the separate category, we also use specific names when we work with binary trees. Namely, we refer to the two sons of a node as left son and right son. Furthermore, for a balanced binary tree of size n , we can estimate the depth as log 2 ( n + 1 ) − 1 .

2.2 BST

One of the numerous applications of binary trees is the implementation of BSTs. BST [11,12] is an implementation of the Abstract Data Type (ADT) table, which is also known as map [13], dictionary [14,15], or as an associative array ^[1] [16]. The ADT table restricts the domain of the elements by the condition that each element needs to be associated with a unique key. Consequently, its implementations typically store pairs of the form (key, value). The three main operations defined by the ADT table are summarized in Table 1.

The table implementations aim to provide an efficient implementation of the above operations. To do so, BST imposes another restriction on the domain – a strict weak ordering on the keys must exist, i.e., the keys must be comparable using the operator < . BST stores the (key, value) pairs in the nodes and forces a notoriously known restriction on the structure. Let us denote the key stored in node A as Key(A), the left son of node A as LeftSon(A), and the right son as RightSon(A). Then, we denote the restriction as follows:

• Key(LeftSon(A)) < Key(A),

• Key(A) < Key(RightSon(A)).

• if Key(A) = key, return Value(A);

• if key < Key(A), repeat the procedure in LeftSon(A);

• if Key(A) < key, repeat the procedure in RightSon(A);

• if the node does not exist, return failure.

Figure 2 shows a simple BST with integer keys. The shape of the tree in the figure is just one of numerous possible shapes. Unfortunately, the shape of the tree influences the complexity of the operations. In the best case – when the tree is balanced – the time complexity of the operations is O ( log n ) , where n is the number of nodes. However, the time complexity can be as bad as O ( n ) in the worst case. This happens when the tree degenerates to a linked list (which also is a valid binary tree), for example, as a result of inserting keys in ascending or descending order.

Figure 2

Example of a simple BST with integer keys depicted in the nodes.

This disadvantage is addressed by various extensions of the BST that implement a mechanism that automatically balances the tree after each operation. Examples of such extensions are the Treap [18], AVL tree [2], Splay tree [19], or Red-black tree [20], which are typically implemented in standard libraries of programming languages. Thus, the BSTs that programmers encounter in practice are practically guaranteed to be balanced, thanks to the mentioned extensions.

3.1 Iterator

The implementation of the traversals in a specific programming language may differ according to the conventions of the language and the programming paradigm used. For example, in functional programming, libraries typically define a function that takes two parameters, namely, a data structure and a function to apply to each element of the data structure [21]. Each type of traversal is implemented as a standalone function.

In object-oriented programming, libraries usually use the iterator design pattern [22]. An iterator is an object that allows efficient access to elements of a data structure without exposing internal details of the data structure. An iterator usually possesses knowledge of the internal organization of a data structure, which allows it to access its elements efficiently. The interface of the iterator differs between different programming languages. However, conceptually, it can be summarized by the operations (methods) presented in Table 3.

Many programming languages contain a so-called for-each loop, which conveniently iterates over all elements of a data structure. Such loops are usually just syntactic sugar for an explicit iteration of the structure via iterator [23–25]. Such languages usually define an interface that prescribes a method that returns an iterator of the structure (which is used by the for-each loop). The iterator often also implements an interface prescribing methods similar to those presented in Table 3. Just like with the functional approach, each type of traversal is implemented as a different iterator type. However, since the interface prescribes only a single method, each (iterable) data structure has only one primary type of iterator^[2].

As stated at the beginning of this section, BSTs and their extensions work conveniently with the in-order traversal. Consequently, implementations of such trees use an in-order iterator as the primary iterator type. In practice, this means that when we use a for-each loop over, e.g., std::map (C++), TreeMap (Java), or SortedDictionary (.NET), we access the keys in ascending order^[3]. The mentioned data structures are part of the standard libraries of the respective languages. Thus, the in-order traversal is arguably the most frequently used traversal. Therefore, we focus solely on this type of traversal in the article. We start by describing different approaches to the implementation of in-order iterators.

3.2 Queued iterator

The first approach that we describe aims at simplicity at the cost of higher memory complexity. It builds on the fact that the functional approach can also be used in imperative languages. In Algorithm 1, we can see a simple recursive implementation of the in-order traversal. The algorithm has two parameters, node – the root of the tree to iterate over, and f – a function to apply to each element of the tree.

It might seem that the presented algorithm is sufficient and that there is no need for an actual iterator. Indeed, such a simple approach is sufficient in many situations. Especially since most imperative programming languages support lambda functions, which can be conveniently used as the parameter f. Containers often provide a method that accepts a function to be applied to all elements, for example, the forEach method in Java [26]. However, an iterator has multiple advantages. For example, it allows multiple simultaneous traversals, it can traverse only part of the structure, and, most importantly, its advancement can be programmatically controlled.

The implementation of the iterator is based on the usage of a First-In-First-Out (FIFO) queue. The queue is filled during the creation of the iterator using Algorithm 1 with a simple lambda function that pushes the element into the queue. The other operations of the iterator are then defined as simple operations on the queue. Consequently, the implementation is really simple – it fits into a single table (Table 4). In the table, we can see that the Init operation is the most expensive. On the contrary, Advance and Access operations do not require any navigation in the tree, so the traversal itself should be among the fastest. The actual performance also depends on the data structure used to implement the queue. Standard implementations usually use a linked list, a dequeue, or a circular buffer (array). In our implementation, we used std::vector with pre-allocated capacity. Finally, the last implementation remark concerns C++ and, possibly, other languages that support value semantics. C++ uses iterators extensively in its standard library and passes them around by value. Therefore, using an iterator with a state of considerable size (the queue member variable^[4]) would result in many expensive copies. In the worst case, the size of the queue is the same as the size of the entire tree. Therefore, such an iterator is practically useless outside of a for-each loop.

3.3 Explicit iterator

As we mentioned, one of the disadvantages of the functional approach is that the execution of the ProcessInOrder function cannot be programmatically controlled – the function cannot be “paused” and “resumed” later –, thus, all the elements are visited within a single call of the function. Therefore, the second approach we present is based on the idea of the ProcessInOrder and allows the user to control the next recursive call by calling the Advance operation. Algorithm 1 is short and simple because, in the background, it implicitly uses the process call stack to store current and past positions in the iteration.

The explicit implementation is based on an explicitly maintained stack of positions (member variable of the iterator). The position is represented by a structure that matches the call stack frame of the ProcessInOrder function. Table 5 contains fields of the structure, their types, and get and set functions used in pseudocode. Algorithm 2 contains pseudocode describing the implementation of the operation Advance. The code uses two auxiliary operations TryGoLeft and TryGoRight, which are also presented in the pseudocode. Furthermore, the code also uses operations PushPosition and PopPosition to maintain the stack of positions.

In our implementation, we use an intrusive linked list to implement the stack. Our structure described in Table 5 has one additional field – a pointer to an instance of the previous position. This is represented using the stack member variable in the pseudocode. Another option is to use a stack implementation from the standard library, which typically uses a linked list, an array list, or a deque. Just like with the queued iterator, in C++, we also need to consider the cost of copying the stack when we pass the iterator by value. Fortunately, the size of the stack is considerably limited – it is constrained by the depth of the tree, which, assuming that the tree is balanced, can be calculated as ⌈ log 2 ( n + 1 ) ⌉ .

3.4 Stateless iterator

The last approach that we present does not require any auxiliary data structure. It maintains the current position in the form of a single pointer to the current tree node. Hence, we refer to it as stateless. The main logic of the iteration is implemented in the Advance operation, and it requires a simple initialization procedure in the Init operation. The pseudocode of these two operations can be found in Algorithms 4 and 3, respectively. Let us note that this exact algorithm is used in the implementation of the Red-black tree in libc++ standard library [27], MS STL standard library [28], and a similar algorithm is used in the libstdc++ standard library [29] of the C++ language. However, there is one a notable difference between our implementation of the Init operations and analogous implementations from standard libraries – our Init starts at the root node and searches the leftmost node (with O ( log n ) complexity) while the standard implementations track the leftmost node explicitly and, thus, create the iterator with O ( 1 ) complexity.

4.1 Query

When writing the query for the chatbots, we avoided specifying iterator properties beyond the fact that we wanted an in-order iterator. This is because we did not want to affect the form of the generated iterator. The goal was to write the query similarly to how a student would write it. Our query for chatbots, therefore, looked like the following:

Provide an implementation of an in-order iterator in C++ for a BST, which has the following node structure:

and was followed by the structure of our node, which can be seen in Figure 3. We chose to query four popular chatbots, namely, ChatGPT, Gemini, Copilot, and DeepSeek [30–33].

Figure 3

C++ structure representing node of BST.

4.2 Iterators in C++

Before we proceed with the description of the iterators generated by chatbost, let us briefly describe how iterators are implemented in C++ according to its standards to make the following section more clear. C++ does not have an interface (in the sense of virtual methods) that must be implemented by an iterator. Instead, any class that overloads a prescribed set of operators with certain behavior and provides prescribed member types can be used as an iterator. We can see a list of selected operators and the corresponding iterator operation in Table 6.

4.3 AI iterator

All chatbots responded to us with a fundamentally identical implementation of the in-order iterator. They differed only in details, which we describe later. Henceforth, in the article we will describe only one implementation of the iterator, which we will refer to as the AI iterator ^[5]. Concerning the implementation, the AI iterator comes closest to the implementation of the explicit iterator we described in Section 3.3. The implementation of the key operations Init and Advance of the AI iterator can be seen in Algorithms 5 and 6, respectively.

The implementation is built on a stack of nodes that is used to store the current position and previous positions. All generated implementations used std::stack from the standard library, which internally uses a deque to store the elements. Similar to the explicit iterator, when working with the AI iterator, we need to consider the impact of stack copying when we pass the iterator by value. Here, too, however, the stack size is considerably limited.

4.4 Summary

Each chatbot generated code that was functional and able to compile. However, each had minor C++ language-specific flaws and, in some cases, minor logic errors in some operations.

ChatGPT generated a valid C++ iterator. However, it introduced a bug in the operator!=. The issue was that it only checked if at most one of the stacks (of two iterators compared) was empty. Such implementation works correctly when we just use the iterators to traverse the entire tree (for example, by using the for-each loop), but would fail in a general comparison of iterators. After pinpointing the issue, ChatGPT provided the correct implementation.

Gemini provided a more extensive implementation of an in-order iterator. In addition to the operator++, it also provided (without explicitly asking for it to be generated) an implementation of operator–. Furthermore, it also generated the correct member types required by the C++ standards by an iterator. Also, in contrast with ChatGPT, it provided the correct implementation of the operator!=.

Copilot provided an interesting answer. The code it generated was almost identical to the code generated by ChatGPT. However, surprisingly, it used different names for the operations. Instead of overloding operator!= and operator++, it used operations named hasNext() and next(), which are used in the Java programming language. However, at least it generated the hasNext() (equivalent of the operator!=) correctly. After pointing out this issue, the Copilot provided the correct answer.

Deepseek was the last chatbot that we tried. The code provided was almost identical to the one provided by ChatGPT. Like ChatGPT, it also generated operator!= that would not work correctly in general. Once again, after pointing out the issue, the correct implementation was provided.

In summary, each chatbot provided a working and (almost) correct in-order iterator implementation. However, only Gemini generated a class that fulfilled all the C++ iterator requirements. Naturally, we assume that the other chatbots would also be capable of generating a complete iterator implementation (from the C++ language point of view) after explicitly asking for it. However, this requires a deeper knowledge of the chatbot from the user. Unfortunately, students or programmers coming from another programming language may not possess the required knowledge, and consequently, they may not know what to ask for. Also, the bug introduced to operator– by two chatbots is a bit concerning because it does not show up in normal use but may show up later in specific use of the iterator. Finally, an interesting question is how the AI-generated implementation will perform compared to the ones written by humans. We provide the results of our comparison in the following section.

5.1 Experiment setup

In practice, programmers use one of the mentioned extensions of the BST, which means that the trees they work with are well-balanced. Therefore, in the experiment, we focused only on the traversal of balanced trees. For simplicity, we implemented a plain BST with the node type shown in Figure 3. To construct a balanced tree with n nodes, we first generate a sequence of the form [ 0 , 1 , … , n − 1 ] . Then, we inserted the median of the sequence, split the sequence in half, and repeated the procedure on each half until the halves consisted of a single number. Using this procedure, we constructed a tree for different values of n , specifically, for n = 2 k − 1 for k = 1 , 2 , … , 25 . The given values gave us a perfectly balanced tree (also known as a perfect binary tree) with depth k − 1 . We believe that a BST generated in this way closely resembles, for example, a Red-black tree (which guarantees logarithmic complexities) and that it is a good-enough approximation of a tree structures that emerge in practice.

The C++ language is implemented in multiple compilers. The three most notable ones are GCC, clang, and MSVC. In addition, each of those compilers comes with its own implementation of the standard library. Therefore, we decided to run the experiments with each compiler and its standard library. The summary of the compilers and standard libraries used is given in Table 8. The experiments were performed on a PC with an Intel i9-10900KF processor with 128 GB of DDR4 RAM running the Void Linux operating system (for gcc and clang) and Windows Education (for MSVC). For each value of n , we measured the average time required to iterate over all tree elements (including time to initialize the iterator with the Init operation) using different iterators. The averages were obtained by 100 replications for better accuracy. Algorithm 7 presents a summary of the code for single replication. Furthermore, we wanted to assess the quality of our implementation. Thus, in addition to our BST, we conducted the same experiment using std::map (which is implemented as a Red-black tree) of the same size, containing the same elements. Naturally, the exact shape of the Red-black tree may differ from our balanced BST. However, since the Red-black tree also guarantees the balance of the tree, we may consider them isomorphic (approximately).

5.2 Results and discussion

We present the results of the comparison in the form of line charts. In the charts, the x -axis displays the depth of the perfectly balanced tree. Therefore, it is scaled linearly in the depth and logarithmically in the number of nodes n . The y -axis displays nanosecond duration and is scaled logarithmically. The duration presented in each chart is an average value obtained from 100 replications.

Figures 4, 5, and 6 show the average total time required to create an iterator and iterate over all elements of the tree using different compilers. The results are quite similar for the GCC and clang compilers. Up until depth 16, all the iterators perform roughly the same, except the explicit iterator, which is consistently slower. From this point on, the performance becomes more similar, while the AI and stateless iterators are slightly faster than the others. However, let us remind the logarithmic scale of the y -axis, which means that the real difference in the performance is more significant. The MSVC compiler shows slightly different results, mainly in the performance of the explicit iterator, which is consistently the slowest for all depths of the tree. However, the performance of the other iterators is almost identical, especially for higher depths of the tree.

Figure 4

Average total duration in nanoseconds required to create and iterate over all elements of a tree using different iterators and the clang compiler.

Figure 5

Average total duration in nanoseconds required to create and iterate over all elements of a tree using different iterators and gcc compiler.

Figure 6

Average total duration in nanoseconds required to create and iterate over all elements of a tree using different iterators and MSVC compiler.

Figures 7, 8, and 9 present the average time required to Init an iterator. These results are more influenced by the specifics of the particular implementation. First, we can see that the time grows linearly (considering the logarithmic scale of the y -axis, the real dependency on the depth is exponential) with the depth for the queued iterator. This agrees with the expectation since this iterator needs to traverse the entire tree in the Init operation. It is interesting to observe that the complexity of the Init operation of the std::map iterator is constant. However, this also agrees with the expectation since all three tested standard libraries (Table 8) use an optimization, which remembers the leftmost node and, thus, all three implementations are able to initialize the iterator in constant time. The observed growth of the duration for the other iterators should be logarithmic (the real dependency is linear in the depth) since they need to visit all levels of the tree in their construction. This agrees with the results, especially for GCC and clang.

Figure 7

Average total duration in nanoseconds required to create different iterators when using clang compiler.

Figure 8

Average total duration in nanoseconds required to create different iterators when using gcc compiler.

Figure 9

Average total duration in nanoseconds required to create different iterators when using MSVC compiler.

Figures 10, 11, and 12 show the average time required to traverse the tree. These figures are similar to the figures in Figures 4–6. This once again agrees with the expectation since the time for the initialization of the iterators becomes negligible for bigger trees. There is one exception, though, and it is the queued iterator, which does the majority of the work in the Init operation. In the subsequent iteration, it should be the fastest one, and this is exactly confirmed by the results.

Figure 10

Average total duration in nanoseconds required to iterate over all elements of a tree using different iterators and clang compiler.

Figure 11

Average total duration in nanoseconds required to iterate over all elements of a tree using different iterators and gcc compiler.

Figure 12

Average total duration in nanoseconds required to iterate over all elements of a tree using different iterators and MSVC compiler.

Finally, Figure 13 shows the average total time required to iterate over all elements of the tree. It presents the same data that was already in Figures 4–6, but this time, it compares the performance of std::map iterators from different standard libraries in a single chart. The results clearly show that the performance is practically identical for bigger trees.

Figure 13

Average total duration in nanoseconds required to iterate over all elements of a tree using different iterators and MSVC compiler.