Volume 9, Issue 13 - Special Issue: Photonics for Computing and Computing for Photonics, Guest Editors: Daniel Brunner, Aydogan Ozcan, Alireza Marandi, Wim Bogaerts

Microelectronic computers have encountered challenges in meeting all of today’s demands for information processing. Meeting these demands will require the development of unconventional computers employing alternative processing models and new device physics. Neural network models have come to dominate modern machine learning algorithms, and specialized electronic hardware has been developed to implement them more efficiently. A silicon photonic integration industry promises to bring manufacturing ecosystems normally reserved for microelectronics to photonics. Photonic devices have already found simple analog signal processing niches where electronics cannot provide sufficient bandwidth and reconfigurability. In order to solve more complex information processing problems, they will have to adopt a processing model that generalizes and scales. Neuromorphic photonics aims to map physical models of optoelectronic systems to abstract models of neural networks. It represents a new opportunity for machine information processing on sub-nanosecond timescales, with application to mathematical programming, intelligent radio frequency signal processing, and real-time control. The strategy of neuromorphic engineering is to externalize the risk of developing computational theory alongside hardware. The strategy of remaining compatible with silicon photonics externalizes the risk of platform development. In this perspective article, we provide a rationale for a neuromorphic photonics processor, envisioning its architecture and a compiler. We also discuss how it can be interfaced with a general purpose computer, i.e. a CPU, as a coprocessor to target specific applications. This paper is intended for a wide audience and provides a roadmap for expanding research in the direction of transforming neuromorphic photonics into a viable and useful candidate for accelerating neuromorphic computing.

Rapidly growing demands for high-performance computing, powerful data processing, and big data necessitate the advent of novel optical devices to perform demanding computing processes effectively. Due to its unprecedented growth in the past two decades, the field of meta-optics offers a viable solution for spatially, spectrally, and/or even temporally sculpting amplitude, phase, polarization, and/or dispersion of optical wavefronts. In this review, we discuss state-of-the-art developments, as well as emerging trends, in computational metastructures as disruptive platforms for spatial optical analog computation. Two fundamental approaches based on general concepts of spatial Fourier transformation and Green’s function (GF) are discussed in detail. Moreover, numerical investigations and experimental demonstrations of computational optical surfaces and metastructures for solving a diverse set of mathematical problems (e.g., integrodifferentiation and convolution equations) necessary for on-demand information processing (e.g., edge detection) are reviewed. Finally, we explore the current challenges and the potential resolutions in computational meta-optics followed by our perspective on future research directions and possible developments in this promising area.

The technologically-relevant task of feature extraction from data performed in deep-learning systems is routinely accomplished as repeated fast Fourier transforms (FFT) electronically in prevalent domain-specific architectures such as in graphics processing units (GPU). However, electronics systems are limited with respect to power dissipation and delay, due to wire-charging challenges related to interconnect capacitance. Here we present a silicon photonics-based architecture for convolutional neural networks that harnesses the phase property of light to perform FFTs efficiently by executing the convolution as a multiplication in the Fourier-domain. The algorithmic executing time is determined by the time-of-flight of the signal through this photonic reconfigurable passive FFT ‘filter’ circuit and is on the order of 10’s of picosecond short. A sensitivity analysis shows that this optical processor must be thermally phase stabilized corresponding to a few degrees. Furthermore, we find that for a small sample number, the obtainable number of convolutions per {time, power, and chip area) outperforms GPUs by about two orders of magnitude. Lastly, we show that, conceptually, the optical FFT and convolution-processing performance is indeed directly linked to optoelectronic device-level, and improvements in plasmonics, metamaterials or nanophotonics are fueling next generation densely interconnected intelligent photonic circuits with relevance for edge-computing 5G networks by processing tensor operations optically.

Ising machines are novel computing devices for the energy minimization of Ising models. These combinatorial optimization problems are of paramount importance for science and technology, but remain difficult to tackle on large scale by conventional electronics. Recently, various photonics-based Ising machines demonstrated fast computing of a Ising ground state by data processing through multiple temporal or spatial optical channels. Experimental noise acts as a detrimental effect in many of these devices. On the contrary, here we demonstrate that an optimal noise level enhances the performance of spatial-photonic Ising machines on frustrated spin problems. By controlling the error rate at the detection, we introduce a noisy-feedback mechanism in an Ising machine based on spatial light modulation. We investigate the device performance on systems with hundreds of individually-addressable spins with all-to-all couplings and we found an increased success probability at a specific noise level. The optimal noise amplitude depends on graph properties and size, thus indicating an additional tunable parameter helpful in exploring complex energy landscapes and in avoiding getting stuck in local minima. Our experimental results identify noise as a potentially valuable resource for optical computing. This concept, which also holds in different nanophotonic neural networks, may be crucial in developing novel hardware with optics-enabled parallel architecture for large-scale optimizations.

Recently, there has been growing interest in the utilization of physical systems as heuristic optimizers for classical spin Hamiltonians. A prominent approach employs gain-dissipative optical oscillator networks for this purpose. Unfortunately, these systems inherently suffer from an inexact mapping between the oscillator network loss rate and the spin Hamiltonian due to additional degrees of freedom present in the system such as oscillation amplitude. In this work, we theoretically analyze and experimentally demonstrate a scheme for the alleviation of this difficulty. The scheme involves control over the laser oscillator amplitude through modification of individual laser oscillator loss. We demonstrate this approach in a laser network classical XY model simulator based on a digital degenerate cavity laser. We prove that for each XY model energy minimum there corresponds a unique set of laser loss values that leads to a network state with identical oscillation amplitudes and to phase values that coincide with the XY model minimum. We experimentally demonstrate an eight fold improvement in the deviation from the minimal XY energy by employing our proposed solution scheme.

Gain-dissipative systems of various physical origin have recently shown the ability to act as analogue minimisers of hard combinatorial optimisation problems. Whether or not these proposals will lead to any advantage in performance over the classical computations depends on the ability to establish controllable couplings for sufficiently dense short- and long-range interactions between the spins. Here, we propose a polaritonic XY-Ising machine based on a network of geometrically isolated polariton condensates capable of minimising discrete and continuous spin Hamiltonians. We elucidate the performance of the proposed computing platform for two types of couplings: relative and absolute. The interactions between the network nodes might be controlled by redirecting the emission between the condensates or by sending the phase information between nodes using resonant excitation. We discuss the conditions under which the proposed machine leads to a pure polariton simulator with pre-programmed couplings or results in a hybrid classical polariton simulator. We argue that the proposed architecture for the remote coupling control offers an improvement over geometrically coupled condensates in both accuracy and stability as well as increases versatility, range, and connectivity of spin Hamiltonians that can be simulated with polariton networks.

A high efficiency hardware integration of neural networks benefits from realizing nonlinearity, network connectivity and learning fully in a physical substrate. Multiple systems have recently implemented some or all of these operations, yet the focus was placed on addressing technological challenges. Fundamental questions regarding learning in hardware neural networks remain largely unexplored. Noise in particular is unavoidable in such architectures, and here we experimentally and theoretically investigate its interaction with a learning algorithm using an opto-electronic recurrent neural network. We find that noise strongly modifies the system’s path during convergence, and surprisingly fully decorrelates the final readout weight matrices. This highlights the importance of understanding architecture, noise and learning algorithm as interacting players, and therefore identifies the need for mathematical tools for noisy, analogue system optimization.

Event-activated biological-inspired subwavelength (sub- λ ) photonic neural networks are of key importance for future energy-efficient and high-bandwidth artificial intelligence systems. However, a miniaturized light-emitting nanosource for spike-based operation of interest for neuromorphic optical computing is still lacking. In this work, we propose and theoretically analyze a novel nanoscale nanophotonic neuron circuit. It is formed by a quantum resonant tunneling (QRT) nanostructure monolithic integrated into a sub- λ metal-cavity nanolight-emitting diode (nanoLED). The resulting optical nanosource displays a negative differential conductance which controls the all-or-nothing optical spiking response of the nanoLED. Here we demonstrate efficient activation of the spiking response via high-speed nonlinear electrical modulation of the nanoLED. A model that combines the dynamical equations of the circuit which considers the nonlinear voltage-controlled current characteristic, and rate equations that takes into account the Purcell enhancement of the spontaneous emission, is used to provide a theoretical framework to investigate the optical spiking dynamic properties of the neuromorphic nanoLED. We show inhibitory- and excitatory-like optical spikes at multi-gigahertz speeds can be achieved upon receiving exceptionally low (sub-10 mV) synaptic-like electrical activation signals, lower than biological voltages of 100 mV, and with remarkably low energy consumption, in the range of 10–100 fJ per emitted spike. Importantly, the energy per spike is roughly constant and almost independent of the incoming modulating frequency signal, which is markedly different from conventional current modulation schemes. This method of spike generation in neuromorphic nanoLED devices paves the way for sub- λ incoherent neural elements for fast and efficient asynchronous neural computation in photonic spiking neural networks.

Semiconductor lasers (SLs) that are subject to delayed optical feedback and external optical injection have been demonstrated to perform information processing using the photonic reservoir computing paradigm. Optical injection or optical feedback can under some conditions induce bandwidth-enhanced operation, expanding their modulation response up to several tens of GHz. However, these conditions may not always result in the best performance for computational tasks, since the dynamical and nonlinear properties of the reservoir might change as well. Here we show that by using strong optical injection we can obtain an increased frequency response and a significant acceleration in the information processing capability of this nonlinear system, without loss of performance. Specifically, we demonstrate numerically that the sampling time of the photonic reservoir can be as small as 12 ps while preserving the same computational performance when compared to a much slower sampling rate. We also show that strong optical injection expands the reservoir’s operating conditions for which we obtain improved task performance. The latter is validated experimentally for larger sampling times of 100 ps. The above attributes are demonstrated in a coherent optical communication decoding task.

Computer generated optical volume elements have been investigated for information storage, spectral filtering, and imaging applications. Advancements in additive manufacturing (3D printing) allow the fabrication of multilayered diffractive volume elements in the micro-scale. For a micro-scale multilayer design, an optimization scheme is needed to calculate the layers. The conventional way is to optimize a stack of 2D phase distributions and implement them by translating the phase into thickness variation. Optimizing directly in 3D can improve field reconstruction accuracy. Here we propose an optimization method by inverting the intended use of Learning Tomography, which is a method to reconstruct 3D phase objects from experimental recordings of 2D projections of the 3D object. The forward model in the optimization is the beam propagation method (BPM). The iterative error reduction scheme and the multilayer structure of the BPM are similar to neural networks. Therefore, this method is referred to as Learning Tomography. Here, instead of imaging an object, we reconstruct the 3D structure that performs the desired task as defined by its input-output functionality. We present the optimization methodology, the comparison by simulation work and the experimental verification of the approach. We demonstrate an optical volume element that performs angular multiplexing of two plane waves to yield two linearly polarized fiber modes in a total volume of 128 μm by 128 μm by 170 μm.

The prediction and design of photonic features have traditionally been guided by theory-driven computational methods, spanning a wide range of direct solvers and optimization techniques. Motivated by enormous advances in the field of machine learning, there has recently been a growing interest in developing complementary data-driven methods for photonics. Here, we demonstrate several predictive and generative data-driven approaches for the characterization and inverse design of photonic crystals. Concretely, we built a data set of 20,000 two-dimensional photonic crystal unit cells and their associated band structures, enabling the training of supervised learning models. Using these data set, we demonstrate a high-accuracy convolutional neural network for band structure prediction, with orders-of-magnitude speedup compared to conventional theory-driven solvers. Separately, we demonstrate an approach to high-throughput inverse design of photonic crystals via generative adversarial networks, with the design goal of substantial transverse-magnetic band gaps. Our work highlights photonic crystals as a natural application domain and test bed for the development of data-driven tools in photonics and the natural sciences.

Finding the solution to a large category of optimization problems, known as the NP-hard class, requires an exponentially increasing solution time using conventional computers. Lately, there has been intense efforts to develop alternative computational methods capable of addressing such tasks. In this regard, spin Hamiltonians, which originally arose in describing exchange interactions in magnetic materials, have recently been pursued as a powerful computational tool. Along these lines, it has been shown that solving NP-hard problems can be effectively mapped into finding the ground state of certain types of classical spin models. Here, we show that arrays of metallic nanolasers provide an ultra-compact, on-chip platform capable of implementing spin models, including the classical Ising and XY Hamiltonians. Various regimes of behavior including ferromagnetic, antiferromagnetic, as well as geometric frustration are observed in these structures. Our work paves the way towards nanoscale spin-emulators that enable efficient modeling of large-scale complex networks.

In recent years, there has been a growing interest in optical simulation of lattice spin models for applications in unconventional computing. Here, we propose optical implementation of a three-state Potts spin model by using networks of coupled parametric oscillators with phase tristability. We first show that the cubic nonlinear process of spontaneous three-photon down-conversion is accompanied by a tristability in the phase of the subharmonic signal between three states with 2 π /3 phase contrast. The phase of such a parametric oscillator behaves like a three-state spin system. Next, we show that a network of dissipatively coupled three-photon down-conversion oscillators emulates the three-state planar Potts model. We discuss potential applications of the proposed system for all-optical optimization of combinatorial problems such as graph 3-COL and MAX 3-CUT.

As an optical machine learning framework, Diffractive Deep Neural Networks (D 2 NN) take advantage of data-driven training methods used in deep learning to devise light–matter interaction in 3D for performing a desired statistical inference task. Multi-layer optical object recognition platforms designed with this diffractive framework have been shown to generalize to unseen image data achieving, e.g., >98% blind inference accuracy for hand-written digit classification. The multi-layer structure of diffractive networks offers significant advantages in terms of their diffraction efficiency, inference capability and optical signal contrast. However, the use of multiple diffractive layers also brings practical challenges for the fabrication and alignment of these diffractive systems for accurate optical inference. Here, we introduce and experimentally demonstrate a new training scheme that significantly increases the robustness of diffractive networks against 3D misalignments and fabrication tolerances in the physical implementation of a trained diffractive network. By modeling the undesired layer-to-layer misalignments in 3D as continuous random variables in the optical forward model, diffractive networks are trained to maintain their inference accuracy over a large range of misalignments; we term this diffractive network design as vaccinated D 2 NN (v-D 2 NN). We further extend this vaccination strategy to the training of diffractive networks that use differential detectors at the output plane as well as to jointly-trained hybrid (optical-electronic) networks to reveal that all of these diffractive designs improve their resilience to misalignments by taking into account possible 3D fabrication variations and displacements during their training phase.

Photonics offers exciting opportunities for neuromorphic computing. This paper specifically reviews the prospects of integrated optical solutions for accelerating inference and training of artificial neural networks. Calculating the synaptic function, thereof, is computationally very expensive and does not scale well on state-of-the-art computing platforms. Analog signal processing, using linear and nonlinear properties of integrated optical devices, offers a path toward substantially improving performance and power efficiency of these artificial intelligence workloads. The ability of integrated photonics to operate at very high speeds opens opportunities for time-critical real-time applications, while chip-level integration paves the way to cost-effective manufacturing and assembly.