Quantum image processing on real superconducting and trapped-ion based quantum computers

1 Introduction

This contribution focuses on quantum image processing, not quantum imaging. Throughout, we assume the image data to be given as a classical gray-value image which should then be processed on a quantum computer. Therefore, we first have to encode the gray-value information into quantum states. There are basically three concepts for this encoding, namely basis encoding, phase encoding, and amplitude encoding. These three encoding strategies have been implemented in various algorithms [1]. Here, we concentrate on the phase encoding method Flexible Representation of Quantum Images (FRQI) [2], for which we introduced an improved implementation in [3].

After the encoding, the quantum states representing the image information can be processed. Initially, quantum algorithms were only formulated in theory or executed on simulators of quantum computers. Only since 2016, it has also been possible to execute algorithms on real quantum computers. Here, we aim at algorithms that run on the actual quantum hardware. A short overview of currently available algorithms is given in [4]. However, only images of size 2 × 2 can currently be encoded and decoded correctly [3], while for larger images the influence of quantum computing errors is still too large. Therefore, in a subsequent study, we developed a quantum edge detection algorithm that works with 2 × 2 image patches [5].

This paper reviews the current possibilities for image processing using FRQI quantum image encoding on quantum computers. In order to be accessible to non-quantum-expert readers, it also includes some basics on quantum computing. We summarize our paper [3] where an improved implementation of FRQI was proposed and limitations on superconducting quantum computers were investigated. Taking these limitations into account, a hybrid processing approach was proposed for the exemplary use-case of quantum edge detection [5] on IBM’s superconducting quantum computers [6]. As an extension to [3, 5] and as an alternative to the superconducting hardware, we test the image encoding additionally on a simulator of a trapped-ion based quantum computer [7].

The paper is organized as follows. Section 2 provides some basics of quantum computing. In Section 3, we describe the experimental setup including the quantum computers, the software, and the classical computers used. We explain our improved version of the FRQI image encoding in Section 4 and show results both for superconducting and trapped-ion based quantum computers. In Section 5, we present the idea of hybrid quantum image filtering and illustrate the performance for the case of detecting edges in images with a quantum computer. Two variants of the quantum edge detector with 2D and 1D masks are detailed. Section 6 concludes the paper.

2 Quantum computing basics

Before diving into quantum image processing, we summarize some basic concepts of quantum computing [8]. Classical computing and quantum computing follow completely different paradigms, starting with the basic elements. Classically, everything builds on bits, that can attain either state 0 or 1. The quantum analogue are the quantum bits (qubits) – two-state quantum systems that allow for more flexibility. Analogous to 0 and 1, there are two basis states of a qubit: |0⟩ = (1,0) ^T or |1⟩ = (0,1) ^T . However, any linear combination (superposition)

of the basis states with α , β ∈ C and |α|² + |β|² = 1 defines a possible state, too. The overall phase of a quantum state is unobservable [8]. That is, |ψ⟩ and e^iξ |ψ⟩ for ξ ∈ [0, 2π] define the same state. Hence, it is sufficient to consider α ∈ R .

As a consequence, the state of a single qubit can be visualized as a point on the unit sphere in R 3 (Bloch sphere) with spherical coordinates ϕ and θ, where α = cos(θ/2) and β = e^iϕ  sin(θ/2). All operations on a qubit must preserve the condition |α|² + |β|² = 1, and can thus be represented by 2 × 2 unitary matrices. Standard operations (so-called gates) acting on single qubits are

where the X-gate acts like a classical NOT operator and the Hadamard gate (H) superposes the basic states of a single qubit. A qubit in superposition can be thought of as having all possible states at the same time. The phase shift gate (P) rotates by θ about the z-axis of the Bloch sphere. Phase shift gates can be used to encode gray-values.

Additionally, we need operations that link two or more qubits. The most common operation in quantum computing is the controlled NOT-gate (CX-gate) taking two input qubits. The target qubit’s state is changed depending on the state of the control qubit:

That means, if the control qubit is in state |1⟩, then we apply an X-gate to the target qubit. Otherwise, we do nothing. For example, assume our two-qubit system has the state |10⟩ = |1⟩ ⊗|0⟩, where the first qubit is the control, the second the target qubit, and ⊗ is the tensor product. Then, the application of the CX-gate results in the state

So basically, the application of quantum gates can be formulated in terms of linear algebra.

In general, we can apply any unitary operation to the target qubit. For example, a controlled phase shift gate applies a P-gate to the target qubit if and only if the control qubit is in state |1⟩. We can also increase the number of control qubits even further. The operation with two control qubits and an X-gate applied to the target qubit is called the Toffoli gate.

Applying such controlled operations to two or more qubits with the control qubits in superposition results in the entanglement of the qubits involved. In terms of linear algebra, an entangled state of several qubits cannot be written as a tensor product of states of the individual qubits. Entanglement is exactly where we benefit from the quantum computing properties. Together with superposition, entanglement allows using a logarithmically lower number of qubits compared to the number of classical bits.

While all bits are connected to each other in classical computers, in IBM’s quantum computer the qubits are arranged in a special, so-called heavy-hexagonal scheme (see the honeycomb structure in Figure 1). That is, each qubit is directly connected to at most three other qubits. To apply two-qubit gates to unconnected qubits, the information has to be swapped to neighboring qubits by the application of additional CX-gates. Each CX-gate, however, increases the overall error considerably such that an algorithm should employ as few CX-gates as possible.

Figure 1:

Coupling map of the backends used in this paper. Every circle represents a qubit, and lines represent connections between the qubits. Colors code the readout errors (circles) and the CX-errors for the connections (lines). Dark blue indicates a small error, and purple a large one. Errors are shown for ‘ibmq_ehningen’. ‘ibmq_toronto’ has the same coupling map, but errors differ slightly (see Table 2).

Lastly, the readout is also completely different for classical and quantum computing. On classical computers, you can always read the current state of the bits, copy them, or just continue running an algorithm with the same state of the bits as before the readout. Unfortunately, this is not possible on quantum computers. First, according to the no-cloning theorem [8], a state cannot be copied. Second, when measuring (reading out the state of) a qubit, its state collapses to one of the basis states |0⟩ or |1⟩. Hence, it is impossible to continue the algorithm after reading out. Additionally, measuring a qubit does not immediately yield the values of α and β in Equation (1). Rather, the probability of collapsing to |0⟩ is given by |α|² while the state |1⟩ is obtained with probability |β|². Repeated measurements (shots) of the same state allow for an estimation of these probabilities and thus the values α and β. For further reading on quantum computing basics, we recommend [8].

3 Near-term quantum computers

In this section, we introduce very shortly the two quantum computer paradigms used here.

3.2 Trapped-ion based quantum computers

Some of the shortcomings of superconducting quantum computers, the restricted connectivity and the low temperatures for the qubits (15 mK), are avoided when using trapped-ion based quantum computers [7, 10, 11]. These systems use individual ions as qubits, which are trapped in electromagnetic fields and manipulated using laser pulses. One of the main advantages of trapped-ion based quantum computers is their precise control over the qubits. Individual ions can be manipulated with high accuracy, allowing for a wide range of quantum operations to be performed. For example, IONQ’s trapped-ion based quantum systems [11] can perform entangling qubit operations with a lower error (0.4 %) than IBM’s superconducting quantum computers ( ∼ 0.7 % ) [6].

Another advantage of trapped-ion based quantum computers is the all-to-all connectivity of the qubits. Instead of being restricted by a hardware induced connectivity as depicted by the coupling map in Figure 1, we are free to entangle any pair of qubits. Figure 2a shows exemplarily the connectivity map of a trapped-ion based quantum computer by Alpine Quantum Technologies (AQT).

Figure 2:

Visualization and features of AQT’s trapped-ion based quantum computers. (a) Fully connected coupling map of AQT’s trapped-ion based quantum computer [7]. Every circle represents a qubit, and lines represent connections between qubits. (b) Selectable AQT devices with free ticket January 2023.

This full connectivity can reduce the number of required operations a lot because we do not have to insert SWAP-gates or other operations to bridge the missing connections of the qubits.

However, trapped-ion based quantum computers are still in the early stages of development. Currently, their use is hampered by three main challenges – scalability of the system, time to solution, and accessibility: First, trapped-ion based quantum computers can operate only on a small number of qubits, typically less than 50, much less than e. g. IBM’s with up to 433 qubits. Thus, only basic quantum algorithms can be executed and only a very small number of qubits can be used for error correction. Second, the times needed for initialization and calibration as well as the times for applying the individual gates are several times longer. This easily sums up to a 100 times longer run time. On the other hand, the decoherence times T1 and T2 are about 1 s and therefore also longer than those of IBM’s backends ( ∼ 150 μ s ) . Finally, trapped-ion based quantum computers are not as freely accessible via the cloud as, for example, IBM’s superconducting quantum computers. In our tests, we therefore only use AQT’s simulator to investigate the difference between the representation on superconducting and trapped-ion based quantum computers.

AQT also supports Qiskit. That means, we can use the same code as for the superconducting quantum computers when replacing the qasm_simulator or IBM’s real backends by AQT devices. Figure 2b shows the current freely available AQT devices with the number of qubits, number of shots, and basis gate set. The number of shots (200) for these devices is low compared to the 100,000 of IBM’s qasm_simulator. This restriction can however be circumvented by executing the code repeatedly and combining the results. This yields not identical but similar results. We also refer to [12] for more information about the basis gate set and the definitions of the gates. The gate set can change in the future due to ion-trapped quantum computers still being under development as mentioned above.

Regardless of the used quantum device, we need a classical computer for preparing data and generating and storing the circuits before sending them to the quantum computer. We use a classical computer with an Intel Xeon E5-2670 processor running at 2.60 GHz, a total RAM of 64 GB, and Red Hat Enterprise Linux 7.9.

4 Quantum image encoding

There are many methods for encoding images in quantum computers. FRQI, originally introduced in [2], is one of the most frequently mentioned ones. Assume that we want to encode a 2 ⁿ × 2 ⁿ pixel gray-value image. Instead of representing this image with 2²ⁿ ⋅ 8 classical bits, we only need an exponentially lower amount of qubits, 2n + 1. We split the required qubits into two parts – 2n qubits for the pixel positions and one qubit for the gray-value information. Practically, FRQI can be implemented on superconducting quantum computers by using entanglement between the position qubits and the gray-value qubit. We take a closer look at the heart of the FRQI algorithm, the multi-controlled y-rotation gate (MCRY). It applies a rotation around the y-axis corresponding to the gray-value only if all position (aka control) qubits are in state |1⟩. Subsequently, we change the state to which the actual phase should be applied by X-gates. Thus, in total, we need one MCRY gate for each gray-value in the classical image. As discussed above, on a real backend, complex operations like MCRY have to be constructed by concatenating available basis gates.

Inspired by [13], we replace MCRY with what we call multi-adapted-controlled y-rotation gates (MARY). For a detailed description of the method, its implementation, and the associated circuits, we refer to our previous paper [3]. Here, we will only summarize the outcomes. Our MARY gates need fewer basis gates, especially less of the particularly error-prone CX-gates [3]. Thus, the replacement reduces the overall error significantly. Moreover, fewer gates and lower circuit depth speed up calculations (Figure 3).

Figure 3:

Circuit depth for varying image sizes and MCRY-/MARY-implementation on backend ibmq_toronto. Mean values of 10 observations in logarithmic scale.

The impact of replacing MCRY by MARY increases with image size. In MCRY, all qubits would ideally have to be connected with each other. Hence, missing connections on IBM’s superconducting quantum computers have to be circumvented by swapping with CX-gates. In contrast, MARY requires much less connectivity of the qubits.

Figure 4 shows the performance on a 2 × 2 sample image.

Figure 4:

Results for 2 × 2 gray-value images using the mean of the executions. In the last column, measurement error mitigation has been applied [14].

The hardware-induced error is clearly visible in the results achieved on the real backend. There, we can only retrieve the image with an error less than 5 % when applying measurement error mitigation [14]. That is, from observations on exactly this backend, the distribution of the error is estimated. Inversion of the error model then improves the results. Nevertheless, to our knowledge, FRQI encoded images larger than 2 × 2 can currently not be retrieved from real backends, see also [14–16].

Table 3 shows our findings for the maximum executable and usable image sizes for the MCRY- and MARY-implementations. Executable here means, it is possible to run the algorithm at all without focusing on the outcomes. Usable implies that the relative difference between the input image and the reconstructed image is less than 5 %. We see a benefit of the MARY-implementation in terms of maximum executable image size. However, due to the high noise level of the backends, we could not increase the maximum usable image size.

For comparison, we show the results for AQT’s simulator and 200 shots. Images larger than 4 × 4 pixels cause an error message that the request entity is too large. This happens in spite of both circuit depth and number of RXX-gates being significantly lower than circuit depth and number of CX-gates on IBM’s simulator. An example is shown in Figure 5 for the MARY-implementations for an 8 × 8 grey-value image on qasm_simulator and aqt_qasm_simulator. Because of these limitations on the current usability of trapped-ion-based quantum computers, we restrict to IBM’s superconducting quantum computers in the following section.

Figure 5:

Circuit depth and number of basis gates for MARY-implementation for an 8 × 8 gray-value image. (a) qasm_simulator. (b) aqt_qasm_simulator.

5 Quantum image filtering

We still aim at image processing algorithms executable on the real backends in the current noisy intermediate-scale (NISQ) era. That means, our algorithms have to cope with the observed tight restriction in image size and be robust to the hardware noise. These requirements are met by hybrid approaches where quantum parts are restricted to image patches of size 2 × 2.

In this section, we summarize our design of hybrid algorithms combining classical filtering with quantum computing on 2 × 2 pixel patches from [5]. As an example, we combine classical edge detection with a quantum artificial neuron [17] as sketched in Figure 6. We calculate the inner product of the input image patch and the filter mask not only on a simulator but also on real quantum computers [5]. Being restricted to a 2 × 2 mask, we can either apply that directly or split our task into one-dimensional filtering steps. The latter requires only a very small number of gates and only one qubit per direction and pixel. It is therefore more robust concerning noise [5]. As a consequence, a very small number of shots (measurements) suffices for identifying the edges in the image. The lower number of shots in turn reduces the execution time significantly. The quantum circuits of the two implementations are shown in Figure 7.

Figure 6:

Scheme from [5] for edge detection in a 30 × 30 sample image by using 2 × 2 filter masks, qasm_simulator and backend ibmq_ehningen (executed on October, 15 2021) with 8.192 shots, and ToolIP [18] for post-processing. U _I encodes the input image patch and U _W the filter mask patch either in horizontal or vertical direction.

Figure 7:

Hybrid quantum edge detection. U _I encodes the input image patch and U _W the filter mask. The gray value information is encoded in the P-gates. In the 1D case, the additional diagonal direction is required for detecting corners, too. For a detailed description and the mathematical formulation of the circuits, we refer to our previous paper [5].

Figure 8 shows the results of our hybrid 2D edge detection for a typical toy example image [19].

Figure 8:

Results for the 256 × 256 House image [19]. The qasm_simulator and backend ibmq_ehningen results differ only slightly.

In [5], we process 256 × 256 pixel gray-value images. A further extension to larger images increases the number of circuits, only, but does not decrease the robustness of our algorithm. Nevertheless, in the end, we create one circuit for each combination of the input image patch and the filter mask. Hence, the overall execution time will increase linearly in the number of pixels. In classical computing, this can be compensated by parallelization. This is also an option in quantum computing. We can use several qubits in parallel and process multiple image patches at the same time. By that, we decrease the number of needed circuits and also the execution time. Mid-circuit measurement [6] allows to measure a qubit at any step of the algorithm and use the same qubit again for further calculations.

6 Conclusions

Quantum computing is potentially very useful in image processing. With image encoding schemes like the here used FRQI it promises exponentially lower number of qubits compared to classical bits and also faster calculations. Theoretically deduced results have been proven to be implementable. However, the currently available noisy intermediate-scale quantum computers are still quite error-prone and hardware improvement is subject of vividly ongoing research. At the moment, image retrieval is only possible for images up to a size of 2 × 2 pixels. Besides error mitigation techniques, also quantum error correction methods like repetition codes, Shor, or surface codes could be used to avoid errors [8]. However, this was not in the scope of this paper.

Another strategy to deal with the limitations of the current NISQ era is to combine quantum and classical algorithms. In such hybrid solutions, the quantum computing part is much smaller than the classical part. We use only a small number of gates, and avoid or decrease the number of particularly error-prone types. The quantum computing share can be extended gradually along with the hardware progress. Instead of trying to implement all image processing functionality on quantum computers, we should rather identify, for which problems and which steps in complex algorithms quantum computing can be helpful or eventually even beat classical machines. Regarding speed-ups, we are skeptical that they can be achieved in practice with the current NISQ hardware, although being theoretically verifiable.