Quantum Theory and Quantum Chemistry: Past, Present and Future

Past

Quantum theory is completely different from classical physics. First, in 1900, Max Planck made a stunning hypothesis that the blackbody emission spectrum can be explained by energy quanta, E=nhv where n=1, 2, 3, … and v is the frequency of light and most importantly h is a constant named after him, h=6.63×10^-34 J×s as carved in his gravestone. Such hypothesis was soon employed by Albert Einstein to explain the low temperature behavior of the heat capacity which strongly deviate the Dulong-Petit law from classical statistical thermodynamics. In the same year, Einstein employed Planck’s concept of light quanta to explain the photoelectric effect, first discovered by Heinrich Hertz. Einstein’s photoelectric equation is simply expressed as: 12meve2=hν-Φ where the left side is the kinetic energy of the emitted electron and Φ on the right-hand side is the work-function of metal. If these are regarded as problems in physics, then the difficulty in understanding the atomic structure and spectroscopy could be considered as chemistry. Namely, the emission spectra of a hydrogen atom were found to be discrete lines such as Balmer series, while classical theory would predict continuous curves. From Rutherford’s α-particle scattering experiment (Nobel Prize in chemistry 2010), there must exist a positively charged nucleus with a radius about five orders of magnitude less than that of atom. This looks like a planet model of classical physics orbiting the sun. A milestone in the development of quantum theory is the Congrès Solvay taking place in the Hotel Metropole in Brussels in 1911. Chaired by the great physicist Hendrik Lorentz, a group of the 18 greatest minds at that time discussed “Radiation and the Quanta.” The second youngest participant Albert Einstein presented his work on “The Problems of Specific Heat.”

In 1913, Niels Bohr suggested his atomic model assuming that (i) electrons orbit nucleus and can only stay in stable orbitals associated with definite energy; (ii) and electron can only jump between these orbitals and the difference of energy equals to hv. Most importantly, using classical physics and imposing his famous correspondence principle, he successfully derived an expression for hydrogen atom with quantized energy level: En=-me4(4πε0)2⁢2ℏ2n2. This great formula expressed the mysterious optical spectrum of hydrogen atom with fundamental physical constant such as electron mass and charge as well as Planck’s constant. In addition, he and Arnold Sommerfeld derived that the angular momentum of electron in atom is quantized: L = nħ, n = 1, 2, 3, … with a fancy name called “space quantization.” Unfortunately, Bohr’s theory failed for hydrogen molecule or helium atom. In classical physics, angular momentum is a continuous vector. Bohr’s theory achieved great success and had aroused strong interest in the science community to design experiment for verification. In 1919, Otto Stern and Walter Gerlach had designed an apparatus to measure the magnetic moment of (silver) atomic beam under inhomogeneous magnetic field. They were so excited to demonstrate “space quantization” to show only two spots appeared in the screen, instead of uniformly distributed line predicted from classical physics, as the orientation of atomic magnetic moment is random. It turned out the two spots demonstrated the existence of electron spin, an intrinsic quantum number for any microscopic particle. Spin provided a natural explanation of the anomalous Zeeman effect puzzling many physicists including Wolfgang Pauli, the latter suggested exclusion principle, essential for understanding the structure of periodic table of chemical elements. This also helped Enrico Fermi and Paul Dirac to establish quantum statistics for particles with spin half integer (Fermions), along with Satyendra Bose and Albert Einstein for particles with spin integer (Boson). The concept of identical particle is essential for quantum world. The Stern-Gerlach experiment is not only the greatest atomic physics experiment, but also opens the door of preparation of quantum state, a precursor to quantum information science and technology to influence our future. We will come back to this point later.

Figure 1. Scheme of Stern-Gerlach experiment [1]

The secretary for the first Congrès Solvay was Maurice de Broglie from Paris, who inspired his younger brother Prince Louis de Broglie with the fascinating while mysterious progresses in quantum physics. In 1924, Louis suggested particle-wave duality, namely, a particle with momentum p=mv be associated with a wave with wavelength λ=hp. de Broglie’s formula stands on a par with Planck’s formula E=hν and Einstein’s E = mc², i.e., three great formulae in physics. Then, Schrödinger wrote down an equation to describe the matter wave, known as Schrödinger Equation in 1926 under the enlightenment of de Broglie’s formula and Planck’s formula, which reads for one-dimension:

In fact, for the case of free particle (V=0) and plane wave Ψ(x,t)=ei(kx-ωt) (where k=2πλ, ω=2πν), thenleft hand side equals to hνΨ(x,t)=EΨ(x,t) and the right-hand side equals to (hλ)22mΨ(x,t)=p22mΨ(x,t)=EΨ(x,t). The former corresponds to Planck’s formula and the latter corresponds to free-particle energy with de Broglie’s formula.

Schrödinger solved his equation for a hydrogen atom (V=-e24πε0r), and he reproduced the famousBohr’s expression for energy level. And applying the same linear second order differential equation for harmonic oscillator, he could reproduce Heisenberg’s result from matrix mechanics. This was the birth of wave mechanics, which is mathematically much easier to handle and to extend to many different problems. It was soon proved by Dirac to be equivalent to Heisenberg’s matrix formulation of quantum mechanics developed in 1925 [2], for which the General Assembly of United Nation proclaims the year of 2025 as The International Year of Quantum Science and Technology to celebrate 100 years of the establishment of quantum mechanics. Heisenberg later established the “uncertainty principle” to shock the world and laid the foundation of Copenhagen’s orthodox of quantum mechanics along with Max Born’s statistical interpretation of wavefunction [4] which has caused a century-long debate on the nature of quantum world.

Present

Most working quantum scientists did not care about the century debate between Einstein and Bohr on the understanding of quantum mechanics. They just used it and made calculations. Such “shut up and calculate” strategy led Walter Heitler and Fritz London to postulate the variational wavefunction for a hydrogen molecule in 1928. This is the foundation of modern quantum chemistry. To calculate the electronic structure of many-electron atom and molecules, Douglas Hartree and Vladimir Fock suggested a mean-field approximation method in a self-consistent way already in 1929 with Slater’s anti-symmetrized multi-electron wavefunction ansatz where electron correlation is overlooked. Linus Pauling laid the foundation of chemical bonding theory in 1930’s with valence bond theory and orbital hybridization theory. Friderich Hund made three rules under his name to explain the electronic structure of many-electron atoms. Erich Hückel postulated the very first semi-empirical quantum chemistry Hamiltonian to describe hydrocarbon conjugated molecules in the 1930’s, followed by a series of improved parameterized models widely used even today, such CNDO, INDO, MNDO, AM1, PM-series, etc. In the 1950s, Clemens Roothaan was tired of fitting parameters for semiempirical quantum chemistry and decided to change the trend by developing the now known Hartree-Fock-Roothaan (HFR) equation, the starting point of “ab initio” quantum chemistry. The author happened to take a picture of a medieval stele in the Charles Bridge in Prague. It seems that this was the earliest appearance of “ab initio.” The latin word “ab initio” means “from the beginning”, namely, with the knowledge of chemical elements, one could obtain the molecular geometry and electronic structure as well as all the physical and chemical property. This is a certainly a formidable task, not only challenging for the past but also for the future.

The HFR equation became the main race horse for computational chemistry and the computational scaling with number of orbitals is N⁴. Subsequent developments include configuration interaction and its multi-reference self-consistent field formulation, perturbation expansion, coupled cluster expansion (now regarded as golden standard of quantum chemistry). From 1960 to 1990, John Pople had played a leading role in pushing the field forward by developing a series of algorithms to solve the molecular Schrödinger equation and establishing the most influential quantum chemistry computational package Gaussian which earned him a Nobel Prize in chemistry in 1998 [4]. Noting the essential role of quantum mechanics in solving chemical problem, under the inspiration and support from Louis de Broglie, five quantum chemists (Raymond Daudel, Per-Olov Löwdin, Robert Parr, John Pople and Bernard Pullman) founded the International Academy of Quantum Molecular Science (IAQMS) in 1967 headquartered in Menton, the favorite Mediterranean coast city for de Broglie, to promote the application of quantum theory for molecular science. Through selecting members and awarding an annual medal, IAQMS has provided a forum for international contact and collaboration as well as periodic evaluation of the main developments, advances and promising directions of research in the broad field of quantum molecular science [5].

A wavefunction for N-electron system can be written as Ψ(r→1,r→2,…,r→N) with 3N variables. Due to the quantum superposition principle, the complexity in describing quantum system increases exponentially with the number of particles in the exact full configuration interaction method ~ O(d^N)  . The success of quantum chemistry lies in the HFR’s mean-field approximation with scaling of O(N⁴) as a reasonable starting point plus successive corrections for electron correlation treatments from perturbation to coupled cluster scaling of O(N^5-8). A revolutionary development is density functional theory (DFT) by Walter Kohn, Pierre Hohenberg, and Lu Sham in the 1960’s, in which electron density ρ(r→) with only three variables instead of wavefunction Ψ(r→1,r→2,…,r→N) with 3N variables becomes the center of interest. Hohenberg and Kohn had shown that ρ(r→) only is enough to determine the ground state of quantum system. The practical way to solve ρ(r→) is through the Kohn-Sham equation, very similar to HFR equation with exchange-correlation term in place of exchange term only. Theoretical physicists started the field of DFT beginning with local density approximation (LDA) with uniform electron density distribution. A milestone for chemistry-DFT is the publication of a book by Robert G. Parr and Waitao Yang “Density-Functional Theory of Atoms and Molecules” in 1994. We make a plot for the yearly publications containing keyword “DFT” or “density functional theory.”

Figure 2. On a medieval stele in Prague, there appeared phrase “AB INITIO.”

Such an exponential increase is very remarkable and can be attributed to the efforts towards achieving chemical precision by building more and more sophisticated exchange-correlation functionals along the so-called “Jacob’s ladder” starting from the bottom LDA, all the way through GGA with functionals depends not only local density, but also its first-order gradient, meta-GGA with second-order gradient (kinetic energy term) dependence in addition, hybrid functionals to add Hartree-Fock exchange energy as a portion in total energy with the prominent B3LYP as an excellent representative which is still the most popular functional in practical DFT calculation, double hybrid functionals with both occupied and virtual orbitals corrections, and eventually to the heaven with chemical accuracy for all the properties. It is no doubt that DFT is the most practical and the most popular computational tool for chemistry and materials science with quantum theory. Especially, with the rapid advancements in machine learning and artificial intelligence, both the accuracy and efficiency have been improved.

Figure 3. Yearly publications containing keyword DFT or density functional theory

Another practical way to avoid the exponential difficulty of quantum theory is to combine with classical mechanics. From the latter perspective, a molecule could be viewed as balls linked with sticks, the motion of which follows Newton’s mechanics with parameterized force field. This immediately reduces the computational cost to O(N²). In fact, in the 1940s, such classical mechanics approach was developed to approach molecular problems, namely, molecular mechanics (MM). And in the ‘70s, Martin Karplus, Michael Levitt, Arieh Warshel (Chemistry Nobel Prize Laureates in 2013) developed a quantum mechanics molecular mechanics hybrid approach, QM/MM to tackle biomolecules and conjugated molecules and beyond, where active electrons such as π-electrons or chemical reaction centers are treated by QM but σ-bonds or surrounding parts are treated by MM. QM/MM coupled with molecular dynamics became the mainstream computational tool for modeling biomolecules, catalytic process, and molecular design [6].

Quantum theory not only explains the nature of chemical bonding, but also reveals the nature of subtle intermolecular forces, such as exchange force, dispersion and induction effects, key components of van der Waals interaction. These interactions along with the classical electro-static and/or hydrogen force determine the complicated molecular self-assembly behavior, a key for molecular nanotechnology and biomolecules.

Future

In the past 100 years, quantum mechanics has revolutionized our society by developing semiconducting technology, atomic energy, laser, nuclear magnetic resonance, superconducting and global positioning satellite technologies, etc. Humankind is continuing to enjoy the outcomes from the “first quantum revolution.” Even though, we still do not understand quantum mechanics, as claimed by Richard Feynman that “I think I can safely say that nobody understands quantum mechanics.” Taking the double-slit thinking experiment as example, only when a single electron passes through both slits simultaneously can there appear interference fringe due to superposition principle satisfied by Schrödinger equation. The mysterious Schrödinger- cat illustrates a paradox of quantum superposition. In fact, in 1935, three famous papers were published which had led the century long debate over quantum mechanics [7]. It started with Albert Einstein, Boris Podolsky and Natan Rosen (EPR) raised a serious question: “Can quantum-mechanical description of physical reality be considered complete?” published in Physical Review. EPR was challenging the Copenhagen philosophical interpretation of our world, namely, before a measurement, (i) nothing can be said about the value of a given physical quantity unless the wavefunction represents an eigenfunction of the operator of this physical quantity and (ii) except this case, the system does not have any fixed value of physical quantity at all. It was hard for EPR to accept such probabilistic view of reality, namely, Heisenberg’s uncertainty principle. In EPR’s thought experiment, there are two particles in the system with position and momentum x₁, p₁, x₂, p₂, respectively. With Heisenberg uncertainty principle, one has: [x1,p1]=iℏ, and [x2,p2]=iℏ. EPR defined the relative position X as X=x1-x2 and total momentum P=p1+p2. It is easy to show that:

Namely, X and P commute. According to Heisenberg, X and P can be determined and measured simultaneously. EPR further argued that the two particles are now separated very far without any interaction. One can measure p₁ and to obtain p₂through P-p₁. And one can measure x₁to obtain x₂ = x-X. EPR concluded that x₂ and p₂ can be determined simultaneously. Thus the Heisenberg uncertainty principle is violated. This is termed as EPR paradox, namely, using Heisenberg’s principle to deduce something contradictive. Immediately, Niels Bohr sent a reply, also published in Physical Review with the same title as EPR but pointed out that due to quantum superposition, measuring particle 1 would lead to collapse of the total wavefunction thus altering particle 2, even though the separation of two particles can be very far due to the “entanglement,” a term that first appeared in Schrödinger’s 1935 paper on his poor cat. Such debate had continued decades and become philosophical instead of scientific debate until John Bell suggested a famous inequality in 1964. The 2022 Nobel Prize in physics was awarded to Alain Aspect, John Clauser and Anton Zeilinger for their experiments with entangled photons, establishing the violation of Bell inequality. This closes the century long debate and opens a door for quantum information science. The third milestone paper in 1935 was the famous Schrödinger’s cat, which is continuing to haunt us even today. For the first time, the term “entanglement” was introduced in quantum world to describe the superposition of microstates. Quantum entanglement is the reason behind the exponential wall catastrophe for classical computers.

Figure 4. Schematic illustration of Schrödinger’s cat state: a superposition of dead and alive cat, which contrasts our common sense [7c].

The best way to illustrate Bell’s inequalities is the Stern-Gerlach experiment for two spins, see below. Imagine now the central part is a generator of entangled spin pair with opposite spin and momentum: |S⟩=12(|↑↓⟩-|↓↑⟩) with spin 1 to the left and spin 2 to the right screen. Thus, if the right screen records a down spin, then the left must be up, or vice versa. This is due to quantum entanglement between the two spins regardless of how far the two screens separate.

Leonard Susskind of Stanford University has presented an easy-to-understand version of Bell’s inequality which can distinguish the difference between quantum entanglement and classical probability such as left/right hand correlation. According to classical probability, the three sets A, B, C (see Fig. 6) should satisfy:

In fact, S(A,¬B)is simply part 1 plus part 4; S(B,¬C) is part 2 plus 3. And S(A,¬C) is just part 1 plus 2. It is so natural for the inequality to be true for classical probability.

But for the two-spin Stern-Gerlach apparatus, from quantum mechanics, the probability to find particle 1 spin up in the direction of e₁ and particle 2 spin up in the direction of e₂ is P(e1↑,e2↑)=|⟨e1↑e2↑|S⟩|2, and for simplicity, we can choose e→1=(0,0,1) along the z-direction and e→2=(sinθ,0,cosθ) in the x-z plane with an angle. Then,it is easily to obtain P(e1↑,e2↑)=12sin2θ2 as the quan-tum probability. Now, lets define property A be spin 1 up along n₁=(0, 0, 1) (so that spin 2 down in the same direction n₁) and property B be spin 1 up along n2=(32,0,12) (namely, ¬B is spin 1 down in n₂ and spin 2 must be up in n₂ due to quantum entanglement) and property C be spin 1 up along n2=(32,0,-12). Similarly, ¬C means spin 2 up along n₃. Namely, we can write down the following expressions:

Figure 5. Scheme of two-spin Stern-Gerlach experiment for the entangled singlet state [8]

Figure 6. Three sets A (blue), B(red) and C(yellow) with overlaps: part 2 overlaps A and B; 4 overlaps A and C; 5 overlaps A, B and C; 6 overlaps B and C [9]

Apparently, Bell’s inequality is violated in this case:

That is, one may find some special case that the inequality could fail. Alain Aspect et al. had first demonstrated such violation in the early ‘80s with entangled photons, which opened the door of quantum information technology [10].

The so-called “second quantum revolution” emerged from the dispute over understanding the quantum mechanics. As mentioned before, The 2022 Nobel Prize in physics was awarded for pioneering quantum information science, and the Breakthrough Prize in physics 2022 was awarded to Hidetoshi Katori and Jun Ye for quantum measurement and 2023 awarded to David Deutsch for Quantum Turing Machine, to Peter Shor for quantum algorithm, and to Charles Bennett and Gilles Brassard for quantum communication (BB protocol). These indicate the emerging importance and expectation of second quantum revolution, within which, the center is quantum computing. The conventional Turing Machines or processing times must grow exponentially with the length of digital numbers. But for quantum Turing Machines, one only needs to add qubits to handle longer numbers, namely, the scaling is linear. In fact, Shor’s algorithm for factorization changed our perception of quantum computing. For example, to factorize a 500-digit number, a classical computer need 10⁸ more time than to factorize 250-digit number (the present limit). This guarantees our information safety to use long digit-numbers. However, according to Shor’s algorithm, a quantum computer needs only 8 times more for the task. It certainly caused a global panic in information security. Namely, once there will be a quantum computer, our information encryption will be broken in any minute. Quantum encryption is the only way to avoid such catastrophe.

Richard Feynman famously said “Nature isn’t classical, dammit, and if you want to make a simulation of nature, you’d better make it quantum mechanical, and by golly, it’s a wonderful problem, because it doesn’t look so easy.” A classical computer uses 0 or 1 to carry out computational operation or data storage. For example, to store a state vector of N-spin state, one needs 2^N float number multiplying 64 bits in a classical computer. But for a quantum computer, one just needs N qubits to represent such a vector. Here is the general procedure for a quantum computer: (i) first, mapping the problem to qubit model; (ii) then preparing an initial state; (iii) and applying a sequential of unitary 1-qubit or 2-qubit gate operations to the state; (iv) and finally measuring the state to obtain the results. A quantum computer can naturally capture the problem of electron correlation, a key challenge for quantum chemistry, through capturing the entangled nature of quantum gates consisting of quantum bits, or qubits. There are two types of quantum computing approaches: analog simulation and digital computing. The former relies on build-up of a controllable quantum system (like cold-atom or even cold-molecule) whose effective dynamics is similar to the one of the desired models. The latter is more like the operation of conventional computers with gates. A number of algorithms have been demonstrated great speed-up compared to classical algorithm such as quantum Fourier transformation and Shor’s factorization. The typical 1-qubit gates include Pauli gates (X, Y, Z), and Hadamard (H) gate. An H-gate can produce an entangled state from a pure state such that H|0⟩=12(|0⟩+|1⟩)≡|+⟩. A typical 2-qubit gate is the popular control-NOT (CNOT), for example, CNOT |0+⟩=12(|00⟩+|11⟩). Quantum Fourier transformation (QFT) involves a series of Hadamard gates and controlled rotation operations. QFT becomes an important component for other quantum algorithm due to its quantum advantage. [10]

Quantum phase estimation (QPE) was first proposed to solve eigenvalue problems such as stationary Schrödinger equation. It involves a series of Hadamard, rotation and QFT. The problem of QPE is the depth of quantum gates, which renders the algorithm impractical in near term noisy quantum devices. We are now in the era of NISQ (noisy intermediate-scale quantum) since our hardware is limited by the number of qubits and the fidelity due to decoherence. VQA (variational quantum algorithm) seems to be the more appropriate choice for quantum chemistry, where quantum circuit is viewed as a black box which receives parameters for optimizations. The state-of-the-art results are from Google and University of Science and Technology of China with 12 qubits for calculating N₂ and F₂ [11]. A number of limitations for the NISQ quantum computing in addition to total number of coherent qubits: (i) low fidelity caused by noise and hardware errors, which increase with the quantum circuit depth; (ii) deeper circuit depth causes accumulation of errors; (iii) highly demanded quantum measurement, for example, in the VQE algorithm, large amounts of measurements should be performed for optimization of the quantum circuit; (iv) Capability of error correction. NISQ devices lack such capability to correct the error of quantum circuit execution. Thus, it is imperative to develop fault tolerant hardware device and algorithm in the near future.

Even though, a general-purpose quantum computer is not envisaged for next decades, quantum – classical hybrid computing seems to be a more practical way to profit quantum advantage in some special cases. In fact, the advantage of quantum computing lies in its ability to store and manipulate highly entangled quantum states. While quantum computers, in principle, can perform any task that a classical Turing machine can, these tasks are often better handled by classical computers due to their significantly lower cost and higher clock speed. In the near future, instead of serving as general-purpose machines, quantum computers will likely function as specialized components within traditional computing systems, similar to the relationship between GPUs and CPUs. As a result, quantum computers are increasingly referred to as quantum processing units (QPUs). Quantum chemists are actively developing algorithms to leverage QPUs, in combination with traditional CPUs and GPUs, to address complex chemical problems.

Perspective

Quantum mechanics revolutionalized modern chemistry by illustrating the nature of atomic structure and chemical bonding. And it further revolutionalized chemistry through developing computational methods and software to predict structure and reaction processes, in addition to design molecules with targeted properties. Using quantum mechanics, together with statistical mechanics and electrodynamics to solve complicated problems in chemistry, theoretical and computational chemistry become an indispensable branch. A centenary celebration for the establishment of quantum mechanics means a lot for chemistry. Quantum science and technology is going to conquer the exponential difficulty for molecular design. It is generally believed that chemistry is at the center to solve global challenging problems in energy and environment. We are entering into a 3E time, namely, energy, environment, entanglement. Quantum entanglement is an earthed resource in a classical world, awaiting us to explore for future applications in chemistry.