Machine-learning-assisted photonic device development: a multiscale approach from theory to characterization

1.1 Background

Photonics is deterministic in the classical limit of Maxwell’s equations but designing, fabricating, and measuring real materials can be a noisy, probabilistic process because of large design spaces, fabrication errors, and noisy measurements. Modern ML is especially powerful at learning, optimizing, and manipulating these probabilistic processes, since this is what it was originally built for [19], [22]. ML starts by assuming that the data x ∼ p _data(x) are drawn from a data distribution p _data(x), e.g., measurement errors, material distributions, spectra, etc. Classically, generative models p _θ (x), e.g., unconditioned variational autoencoders (VAEs) [23], generative adversarial networks (GANs) [24], and diffusion models [25], aim to learn the joint distribution p _data(x) of all random variables in the data via maximizing the log-likelihood

with various methods, including direct analysis, expectation maximization, and gradient descent [19]. Generative models have found applications everywhere a random process needs to be modeled, e.g., simple design generators (Section 4), measurement noise (Section 6), and data augmentation (Section 6).

One limitation of generative models is that they are inherently restricted to learning the joint distribution of the data [22]. For example, data x and their labels y have a joint distribution p _data(x, y), such as measurements and designs [26], measurement errors and measurement parameters [27], fabrication artifacts and ideal designs [28]. However, often a goal in ML-PDD is to train a discriminative model p _θ (y|x) to learn the conditional distribution y ∼ p _data(y|x) for classification, regression, and conditional generative tasks such as classifying material properties [29], characterizing structural defects [30], and predicting the optical responses [31] of generated devices. Discriminative modeling is widely used when only certain aspects of a variable are of interest, such as a subset of devices x or performance metrics y. For example, if a generative model p _θ (x) generates designs in a subset, a discriminative model p _θ (y|x) can be trained to more accurately compute the labels y for designs sampled in that subset. Generally, discriminative models perform better in practice and much of modern ML, including large language models [32], diffusion models [25], [33], etc. involve some form of conditional distribution [22]. Another advantage of discriminative models is that they can encompass varying degrees of randomness because they are conditional. In other words, they are often treated as parameterized, deterministic functions p θ ( y | x ) = δ y ( y ̂ ϕ ( x ) ) , where y ̂ ϕ ( x ) is a multi-layer perceptron [34], transformer [35], or convolutional neural network [36]. Discriminative models have found uses all throughout ML-PDD, including efficient forward modeling of optical systems, providing real-time predictions of device behavior without requiring full numerical simulations (Section 3), and conditional generation of designs from additional design criteria (Section 4).

Many processes in PDD have distributions that belong to a known distribution family [19], e.g., Gaussian noise belongs to exponential families, etc., but the parameters of the distribution are unknown, such as Gaussian noise N ( μ , σ ) with unknown mean μ and variance σ ². For these simpler distributions, it is more effective to learn the parameters of the distribution using maximum likelihood estimator models, e.g., x ∼ N ( μ ̂ , σ ̂ ) where μ ̂ = ∑ i x ( i ) n and σ ̂ = ∑ i ( x ( i ) − μ ̂ ) 2 n , than it is to use a large machine learning model [19]. Maximum likelihood estimation is, in spirit, the driving force behind training both generative models and discriminative models. Each model has a set of parameters, e.g., p _θ has θ, that has to be optimized to sample new data from the distribution p _data. However, for many tasks with larger, unknown distribution families, such as in inverse design (Section 4), increasingly complex models are required to capture the nuances of the data. As the complexity of the model increases, estimating the optimal parameters becomes increasingly difficult, hence the explosion of large-scale, specialized hardware and distributed techniques for training these large models [32]. Modern generative ML models and training techniques exploit large compute [32], [37] and gradient-based optimization [38] to find the optimal parameters θ for optimizing Eq. (1) and its many variants [19], [22]. As such, ML-PDD takes advantage of these techniques to tackle difficult problems in PDD.

1.2 ML-assisted photonic device development (ML-PDD) framework

ML-PDD begins by deriving the optical properties of a photonic device from its parameters (see Section 2). Each device is represented with a parameter vector x ∈ X in a design space X that holds unique information about the design, for example, material topology height maps X ⊆ [ 0,1 ] n × n [39], [40], spectra X ⊆ R n [41], voltage biases X ⊆ [ 0,5 ] n , etc. Each device x has a corresponding optical response y(x) usually derived from Maxwell’s equations and any multiphysics coupling. Often, there is an ideal optical response y* which the design is optimized to realize, e.g., emission spectra y * ∈ R n or 2D phase image y* ∈ [0,2π) ^n×n . In the simulation step, as discussed in Section 3, the optical response y(x) is approximated with a numerical model y ̂ ( x ) derived from computational electromagnetic techniques [42]. A deterministic model y ̂ ϕ ( x ) with parameters ϕ known as a surrogate estimator model (Section 3) is trained to approximate the numerical model y ̂ ( x ) , speeding up the inference time by orders of magnitude at the cost of some accuracy. The surrogate estimator model is deterministic but it can be augmented to model noisy measurements, for example Gaussian noisy measurements y = y ̂ ϕ ( x ) + ϵ : ϵ ∼ N ( μ , σ ) . Taking the surrogate estimator model y ̂ ϕ ( x ) and the ideal optical response y*, the following design step (Section 4) constructs a figure of merit (FOM) score function f(y) to rank designs based on their optical response, e.g., f(y) = ||y* − y|| _l for some l-norm. Then, a generative model x ∼ p _θ (x) is trained, see Section 4, to sample designs that maximize the FOM in the expectation

Often, the design step requires several iterations t = 1, …, T of sampling designs x ( t ) ∼ p θ ( t ) ( x ) , evaluating their performance f ( y ̂ ϕ ( x ( t ) ) ) , and retraining the generative model θ ^(t) → θ ^(t+1) before choosing a sufficiently optimal set of designs X * = x ( t i ) i = 1 N for fabrication. Given these optimized designs X*, a probabilistic fabrication process r _η (χ|x), see Section 5, with fabrication parameters η, e.g., alignment, tolerances, etc., takes an intended design x ∈ X* and fabricates a device χ. Fabrication is a very difficult and time-consuming process that cannot fully realize a design x. Therefore, techniques such as reinforcement learning have been crucial in reducing randomness and improving the final quality of designs, which will be discussed in Section 5. Once the design is fabricated χ ∼ r _η (χ|x), the optical response of the fabricated design is measured via a noisy measurement device υ ∼ m _ρ (υ|χ) with noisy measurements υ and measurement parameters ρ, as discussed in Section 6. Naturally, to accommodate noisy measurements, the FOM is augmented f ̂ to use a finite sample of noisy measurements ϒ = { υ ( i ) ∼ m ρ ( υ | χ ) } i = 1 M . The overall objective of ML-PDD is to optimize the device performance in this noisy environment

To make the ML-PDD objective tractable, assumptions are made on the fidelity of measurements, the optimality of designs and fabrication processes. Steps are isolated and optimized. For example, in inverse design, often the numerical simulation FOM is treated as the ground-truth without considering fabrication processes r _η (χ|x) or noisy measurements m _ρ (υ|χ). As such, the remainder of this review will explore each step both in isolation and coupled to other steps, broken down into theory (Section 2), simulation (Section 3), design (Section 4), fabrication (Section 5) and characterization (Section 6).

2.1 Discovering governing laws

A central application of ML in EM theory is the discovery of governing equations and physical laws to model optical responses. Traditional approaches to understanding EM phenomena often involve laborious analytical derivations or heuristic methods informed by experimental observations [44]. ML models, particularly symbolic regression and neural networks, automate this process by identifying mathematical relationships within data [47]. A rapidly growing Python implementation known as PySR [48] combines symbolic regression with genetic programming to construct analytic expressions. Similar techniques have successfully rediscovered Maxwell’s equations by analyzing datasets of electric and magnetic field values in various scenarios [49]. Beyond rediscovering well-established laws, ML has been employed to hypothesize new governing equations in complex environments, such as through experimental physical discovery. For instance, ML has identified novel constitutive relationships that better describe the interaction of fields and materials in metamaterials and plasmas, where traditional models often struggle due to high degrees of complexity and anisotropy [50]. These breakthroughs enable more accurate predictions and a deeper understanding of exotic materials and devices. For example, neural networks trained on EM field distributions and boundary conditions can infer constitutive relations for materials [16], [51], such as the relationship between electric field intensity and polarization in nonlinear media of photonic devices [13].

2.2 Enhancing interpretability and physical insight

One of the key challenges in ML-PDD, in general, is integrating EM theory into training to ensure that models are interpretable and aligned with physical principles [52], [53]. Advances in ML techniques, such as attention mechanisms and feature importance analysis [54], have made it possible to reveal explanations for models’ decisions and enhance our understanding of the underlying physics. For example, Yeung et al., in Figure 2(a)–(c), used a three-step explainable ML approach to uncover the relationship of specific regions of a nanophotonic structure to the presence of an absorption peak, leading to a better understanding of the behavior of complex nanophotonic devices and enabling more efficient and targeted design improvements [52].

Furthermore, by highlighting the most critical parameters, ML models could also provide actionable insights that guide iterative design processes, estimating important parameters such as permittivity, permeability, or conductivity by analyzing data from spectroscopy, reflectometry, or scattering experiments [55], [56]. For instance, Yesilyurt et al. employed a discriminative ML model to extract material refractive indices and loss coefficients from the spectral transmission and reflection data for the realistic design and fabrication of single material variable-index multilayer films in Figure 2(d) and (e) [29]. The classification of modes and band structures is also essential for understanding and optimizing photonic device performance. ML models can efficiently identify guided modes, radiative modes, or photonic band gaps based on structural and material inputs to enhance our understanding of photonics devices [57]. For example, Martinez-Manuel et al. applied support vector machines (SVMs) (Figure 2(f)) for unambiguous refractive index measurement of the fiber fundamental mode in Figure 2(g) [58]. Similarly, Li et al. utilized deep learning approaches to analyze photonic band structures of phononic crystals, identifying band gaps and key dispersion features [59].

Figure 2:

Machine learning applications in photonic theory. Three steps of explainable ML elucidating the behavior of nanophotonic structures: (a) converting 3D metal–dielectric–metal metamaterials into 2D representations. (b) Training a convolutional neural network (CNN) to predict the electromagnetic response. (c) Elucidating the underlying physics learned by explaining the relationships between structural features and predicted parameters to construct new designs with new target properties. Subfigures a, b, and c are adapted with permission from ref. [52]. Copyright 2020 American Chemical Society. (d) Network architecture for fabrication-in-loop NN-based inverse design of single-material multilayer optical stacks with continuously changing refractive index in simulated fabrication case. (e) Measured and experimentally retrieved layer parameters are integrated back into the optimization cycle. Subfigures d and e are adapted with permission from ref. [29]. Copyright 2023 the author(s), published by De Gruyter, Berlin/Boston, licensed under the Creative Commons Attribution 4.0 International License. (f) Flowchart of the SVM model and (g) diagram of the classification process in unambiguous refractive index measurement. Subfigures f and g adapted with permission from ref. [58]. Copyright 2022, IEEE.

Latent space engineering in generative models, such as autoencoders, offers another avenue for interpretability (see Section 4). These models encode high-dimensional data, like photonic design geometries or spectral responses, into compact representations that reveal patterns and relationships. For instance, clustering designs in latent space based on efficiency metrics can help researchers identify common features of high-performance devices [60].

2.3 Summary and outlook

ML is transforming how we apply EM theory and materials science in PDD. As ML continues to evolve, its integration with foundational physics principles offers a promising avenue for breakthroughs in device performance, material characterization, and the exploration of new photonics device frontiers. However, achieving this vision requires addressing key research gaps, such as overcoming data scarcity through efficient learning methods, enhancing the robustness, interpretability, and real-time adaptability of ML models. By tackling these challenges, researchers can foster a synergistic ML-EM paradigm that drives innovation in photonics and unlocks unprecedented opportunities for scientific and technological advancement.

3.1 Surrogate forward simulator models

ML-based surrogate estimators are discriminative and data-driven, involving an inference task that approximates the optical response y ̂ ( x ) , of a device design, x. Therefore, discriminative models that learn the complex relationship between input and output based on a training dataset of labeled input-output pairs are well-suited for simulation problems. Since the models generate data-driven predictions of the output response from the input structure using ML, they offer orders-of-magnitude speed-up over numerical simulations, which directly solve simulation problems. In the context of ML applications for simulations, both the input and output are typically represented as vectors. For optical devices with a low degree of freedom, such as multilayer films and periodic structures containing simple geometric patterns, the structures can be fully represented by a vector x comprising a series of thicknesses or geometric parameters, as depicted in Figure 3(a) and (b). For high-degree-of-freedom structures, such as those containing freeform patterns, the structures can be represented using binary images, where materials are distinguished by ‘0’ and ‘1’, as shown in Figure 3(c) and (e). For the output, spectra, for example, can be sampled at a sufficiently fine rate to construct a vector representation of y ̂ ( x ) . A variety of simulation problems in optics have been demonstrated using discriminative neural networks.

Figure 3:

Optical simulations using deep learning methods. (a) A fully connected neural network predicts the scattering spectra of multilayered core–shell nanoparticles. Adapted with permission from ref. [31]. Copyright 2018 the authors, distributed under the Creative Commons Attribution-Non Commercial license. (b) A simple geometry of chiral metamaterials, consisting of two gold twisted split-ring resonators, is represented using a few geometric parameters. A neural network model predicts circular dichroism from the given geometry. Reprinted with permission from ref. [61]. Copyright 2018 American Chemical Society. (c) Freeform meta-atoms are represented as 64 × 64 binary images. The prediction of the real and imaginary components of the spectral responses is obtained through CNN. Adapted with permission from ref. [62]. Copyright 2020 Optical Society of America. (d) The architecture of the CcGAN, which generates synthetic spectra for data augmentation. The generated spectra were conditioned on the temperature label y. The performance of the ML temperature prediction model was significantly enhanced through data augmentation using the CcGAN. Adapted with permission from ref. [63]. Copyright 2023 Optica Publishing Group. (e) Self-supervised learning is applied in a VAE architecture. The encoder predicts reflection spectra as well as compresses the input structure into a latent vector, enabling the model to train with both labeled and unlabeled data. Adapted with permission from ref. [64]. Copyright 2019 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim.

The simulation of optical devices with a low degree of freedom, including layered structures [31], [65], [66] and patterns with primitive geometries [61], [67], can be conducted using the fundamental discriminative model, fully connected networks. Peurifoy et al. employed them to approximate light scattering from a dielectric spherical nanoparticle with alternating silica and titanium oxide shells [31]. The model achieved high precision in predicting scattering spectra and generalized well to the samples that were not seen during the training, enabling it to serve as a surrogate model for forward simulation of such structures. These forward simulation networks play a crucial role in inverse design by being integrated with inverse design networks that map the optical responses to design parameters. For layered structures, these networks have facilitated the inverse design of core–shell nanoparticles to achieve specific electric and magnetic dipole extinction spectra [66], as well as thin-film structures of alternating SiO₂ and Si₃N₄ for target transmission spectra [65]. Similarly, meta-atoms with primitive geometries, represented by vectors of a few geometric parameters, can also be simulated and designed using neural networks. For instance, deep learning-based simulations and inverse designs have been demonstrated for chiral metamaterials composed of two twisted gold split-ring resonators separated by dielectric spacers [61], as well as metasurfaces with H-shaped gold nanostructure on top of ITO-covered glass [67].

For the simulation of optical structures with a high degree of freedom, more advanced models are required to ensure accurate predictions. Convolutional neural networks (CNNs), which utilize convolution operations with kernels to extract local features and capture spatial hierarchies in input data, are well-suited for the simulation task of optical devices with complex geometries. Freeform patterns, represented as images, benefit from CNNs due to their ability to efficiently capture local correlations within the image, making them a desirable choice for such simulations. An et al. simulated the spectral responses of dielectric metasurfaces, where quasi-freeform meta-atoms were represented by 64 × 64 pixel images using a CNN [62]. The trained CNN simulator demonstrated high precision in predicting the responses of not only quasi-freeform patterns in the test dataset but also generalized well to circle- and ring-shaped meta-atoms, which were entirely different geometries from those in the training dataset. Wiecha et al. successfully simulated the internal fields of arbitrary 3D nanostructures using a CNN in a U-net-like architecture with residual connections [68]. This sophisticated model enabled accurate simulations that reproduced complex physical effects in both plasmonic and dielectric nanostructures. Furthermore, the integrated architecture of ResNet-CNN with residual connections – combined with a recurrent neural network (RNN) demonstrated a good match with numerical simulations of the absorption spectra of periodic structures consisting of silver on top of a glass substrate [69]. In this setup, the CNN extracted spatial features, while the RNN predicted the spectra based on the output from the preceding CNN.

Recently, instead of data-driven simulations that learn functions x ↦ y, which map input and output vectors, neural operators, which learn operators x( r ) ↦ y( r ) that map between functions, have gained significant attention. Neural operator architectures consist of multiple layers of linear integral operators followed by nonlinear activations [70]. Neural operators have been applied to learn solution operators for partial differential equations (PDEs), including Darcy flow, Burgers’ equation, and the Navier–Stokes equations, enabling significantly faster simulation than conventional numerical solvers [70], [71]. Moreover, neural operator models generalize well across different levels of discretization, as they learn mappings between continuous functions, whereas traditional vector-to-vector neural networks are heavily dependent on the discretization scheme [72]. The idea of using neural operators to solve PDEs has also been extended to Maxwell’s equations for electromagnetic simulations. Gu et al. proposed the NeurOLight framework, which combines a PDE encoder with a neural-operator-based backbone to build a surrogate forward solver for simulating multi-mode interference photonic devices [73]. The NeurOLight enables efficient simulation of a family of parametric photonic devices, rather than solving only a single instance or one conditioned on fixed parameters.

3.2 Data augmentation using generative models

While the forward simulation problem is inherently discriminative from a machine learning perspective, generative models can play a role in enhancing the data efficiency of simulator network training. Since the training of simulator networks is data-intensive, achieving highly accurate simulator networks requires a large amount of labeled data – input structure and output optical response pairs – obtained through computationally expensive numerical simulations. Reducing dependency on data can be achieved by incorporating known optical knowledge [74], [75], [76] and employing machine learning techniques such as transfer learning [77], [78], [79] and generative models [63], [64], [80], [81]. Physical knowledge serves as a form of regularization during the training of simulator networks, helping the network find valid solutions with less data. Transfer learning is a technique where a model trained on a source task with a fairly large dataset is used to initialize the trainable parameters for a target task. This approach enables faster convergence and improved performance on the target task, even when the target dataset is relatively small, by leveraging the knowledge learned from the source task. Furthermore, generative models alleviate the burden of data collection by augmenting datasets through their generative capabilities and enabling self-supervised learning within encoder-decoder architectures.

Generative models enable efficient data-driven simulation by generating synthetic pseudo-labeled samples, which support semi-supervised learning for simulator networks by leveraging the generated data. Kim et al. employed a denoising diffusion probabilistic model (DDPM) to enhance the performance of a simulator network designed to predict the transmission spectra of photonic crystal waveguide unit cells [80]. The DDPM was trained on images of photonic crystal patterns, allowing it to generate unlabeled new patterns later labeled by a pretrained CNN simulator network. Since these newly generated patterns from the DDPM were produced by learning the probabilistic distribution of the training data, the pseudo-labeling process achieved a high-fidelity approximation using the pretrained simulator. Additionally, Zhu et al. introduced a continuous conditional GAN (CcGAN) to generate synthetic spectra of a self-interference microring resonator sensor at different operating temperatures [63]. As illustrated in Figure 3(d), the CcGAN was conditioned on the temperature, and the corresponding generated spectra were combined with the original small training dataset for data augmentation. By leveraging the augmented data with synthetic pseudo-labeled samples for training ML forward prediction networks, the prediction accuracy improved, and the error distribution shifted toward lower losses.

Another approach to achieving efficient data-driven optical simulation using generative models is the application of self-supervised learning in encoder-decoder architectures [64], [81]. Ma et al. utilized a variational autoencoder (VAE), where the encoder not only compresses the input nanophotonic structure into a latent vector but also predicts the corresponding spectral response. This setup enables the self-supervised learning strategy to improve the performance of nanophotonic structure characterization [81]. As illustrated in Figure 3(e), in this VAE architecture, an additional loss term for prediction accuracy was introduced, along with the reconstruction error and KL divergence. During training, both structures from readily available labeled data obtained from numerical simulations and dynamically generated structures were fed online into the encoder. The ground truth spectra were used to calculate the loss for labeled samples, while the generated structures were pseudo-labeled by the encoder itself. This approach allowed the training process to be self-supervised by the encoder. The total loss, which included the prediction error, was backpropagated through the entire encoder-decoder structure, leading to a further reduction in prediction loss compared to a fully supervised counterpart, thereby improving the simulation capability of the model.

3.3 Physics-inspired constraints in model training

A key advancement in ML modeling for photonics is the incorporation of physics theory-based constraints into model training. For instance, by embedding Maxwell’s equations as regularization terms or constraints, ML models can ensure that their outputs adhere to fundamental physical laws [82]. It not only reduces the reliance on large datasets, but also simultaneously improves the interpretability and reliability of the ML models for both simulation and design of photonic devices. Especially, surrogate simulator models y ̂ ϕ ( x ) are trained by minimizing a loss function dependent on the parameters ϕ. By incorporating physical constraints into the loss function, the surrogate estimator y ̂ ϕ ( x ) is more interpretable and physics informed, thereby better able to approximate the optical response y ̂ ( x ) .

Physical constraints play a pivotal role in reducing the effort required for data collection in training models. In PDD, assuming symmetry, e.g., rotational or axis mirror, simplifies the design space and computational requirements. For instance, simulations can be restricted to a quarter or half of the unit cell by applying symmetric boundary conditions along the axes, depending on the type of symmetry involved. Symmetry also ensures continuity and connectivity at the boundaries between neighboring unit cells, which is critical in the design of gratings. Additionally, symmetry is often leveraged to augment training datasets and regularize machine learning models, ensuring improved performance and data efficiency. For example, in periodic structures based on optical principles, invariances in the spectrum under transformations such as translation, flipping, and 180° rotation can be exploited [12], [83]. Furthermore, 90° or 270° rotations of the pattern induce cross-polarization, effectively swapping the x and y polarization components while maintaining the overall spectral properties. These symmetries enable data augmentation for simulator networks that predict optical characteristics by including structure-characterisitics pairs induced by physics constraints [84], [85]. This approach is not limited to discriminative simulator models but also applies to generative models for design, significantly reducing the reliance on computationally expensive numerical simulations or topology optimizations [12], [83].

Incorporating physical knowledge about resonances [74], [75] and electrical field distributions [76], [86] as constraints into the model training process serves as a form of regularization, further enhancing data efficiency. In optics, integrating Maxwell’s equations as a loss function has proven particularly effective for solving inverse problems. Applications include tasks such as permittivity retrieval [87], [88], designing invisible cloaking devices [87], and optimizing meta-lenses [89], [90], all achieved with significantly reduced data requirements. These methods highlight the importance of embedding physical principles to improve the robustness and efficiency of machine learning models in photonics.

In addition, fabricability imposes significant physical constraints, requiring the elimination of intricate design features that are infeasible to fabricate and ensuring robustness against fabrication errors [12], [83], [91], [92], as we will detailedly discuss in Section 5.

3.4 Summary and outlook

In this section, we explored how ML simulator models can perform simulations much faster than numerical approaches by predicting the optical response of photonic devices in a data-driven manner, rather than directly solving equations. However, these models require training on a large amount of labeled data, which must inevitably be obtained from time-consuming numerical simulations to achieve sufficiently accurate results. Generative ML models mitigate the burden of data collection through self-supervised learning, leveraging generative techniques to synthesize labeled data, while physical constraints contribute through data augmentation and regularization during model training. Therefore, well-trained simulator models, combined with effective data generation strategies, can expedite the iterative performance evaluation of intermediate designs, enabling computationally efficient PDD. With fast performance evaluation of devices in place, how to effectively explore the large design space and identify optimal designs will be discussed in detail in Section 4.

4.1 Constructing the figure of merit (FOM)

A figure of merit (FOM) f ( y ) ∈ R is a scalar measure of device performance which scores high-dimensional optical response labels y into a single metric to facilitate optimal device design. A higher FOM value for an optical response y _i compared to another response y _j (f(y _i ) > f(y _j )), indicates that y _i is “closer” to the optimal response y*.^[1]

A common approach to defining FOMs in deterministic optical responses is using a normalized L-norm loss against the optimal response y*, i.e., ||y − y*|| _l [104]. In simpler cases, when the response has to be maximized to some finite value-such as reflection, transmission coefficients or normalized power efficiency-the FOM can be directly defined as f(y) ≡ y. Deterministic formulations are the most common, since any finite sampling can be represented with a sufficiently larger vector y.

Conversely, when the responses exhibit inherent variability due to noise, probabilistic metrics such as the KL divergence [85], Reny divergence [105], or Jenson–Shannon divergence [106] provide more robust alternatives by quantifying the statistical distance between observed and ideal optical response distributions, thereby capturing deviations that would otherwise not be apparent in deterministic methods. Furthermore, neural networks have recently demonstrated alternative loss functions to better guide the optimization process, such as KID, FID, and perceptual loss [33], [104]. While underutilized in PDD, these methods can effectively serve as learned FOMs, quantifying how closely a given response aligns with an optimal target. These networks are trained on large classification datasets and the loss is computed as an L-norm distance between activations in the deeper layers of the network, which encode important structural similarities. Neural network-based losses align well with human intuition for differences in data, however they can be much slower than the aforementioned losses.

4.2 Latent optimization

The design space for PDD problems X is frequently exponentially large with a sparse distribution of useful designs; for example, the space of useful material height maps X ⊂ [ 0,1 ] n × n usually has rounded features, axial symmetries and minimum feature widths, all of which are sparsely distributed around [0,1] ^n×n . In which case, directly optimizing the designs through adjoint methods is too costly and slow, as most variations are suboptimal with small, but computationally expensive step sizes. Instead, designs can be compressed into lower-dimensional latent feature vectors z ∈ Z using an encoder q _θ (z|x) such that searching the associated latent space is more tractable using adjoint, gradient and global optimization techniques. To use this more efficient design space, we train a decoder p _θ (x|z) and modify our design objective in Eq. (2) to employ latent optimization methods [12], [104] to generate the optimal design

Here, the choice of optimizer q _θ (z) and decoder p _θ (x|z) are especially important, as they direct the performance. While the design space X is typically sparse, the latent space Z is very dense, making it more efficient to explore. For example, introducing continuous variations on the optical response y ̂ ϕ ( x ( 0 ) + δ x ( t ) ) by perturbing latent vectors z ^(t) ← z ⁽⁰⁾ + δ _z (t) allows the exploration of the design space x ^(t) ∼ p _θ (x ^(t)|z ^(t)). By adding subtle but impactful perturbations to latent representations of complex structures through global optimization [39], evolutionary and genetic algorithms [85], differential evolution [12], and even ML-assisted optimization [104] can efficiently explore the latent space and offer alternative designs outside of human intuition [103], [104], [107]. Typically, the optimizer q _θ (z) is chosen such that it is efficient to sample and is coupled to the FOM by a latent surrogate model E _θ (z) that is trained to predict the FOM f ( y ̂ ϕ ( x ) ) : x ∼ p θ ( x | z ) . Recent work by the authors has demonstrated that simple L-norm energy-matching losses, e.g., | | E θ ( z ) − f ( y ̂ ϕ ( x ) ) | | l , are difficult to train and too restrictive for optimization. Naturally, the neighboring correlations between FOMs are more important to capture in the surrogate model than exactly matching the FOM. Therefore, by using a Pearson correlation loss instead of an L-norm, the optimizer can more efficiently sample E _θ (z). The optimizer itself is often inspired by combinatorial optimization optimizers, such as, simulated annealing [39], global optimizers [12], recurrent neural networks [104] and even quantum samplers (Section 4.4) [107].

4.2.1 Variational autoencoders (VAEs)

Variational autoencoders (VAEs) are a type of unsupervised generative models that compress high-dimensional designs x ∈ R n into low-dimensional latent vectors z ∈ Z ⊆ R d : d < n [23]. VAEs are often applied to denoising, compression, and optimization applications, where the feature rich latent vectors z represent designs x without its redundancies, noise, or symmetries which cannot be removed efficiently otherwise [108]. The basic VAE workflow in ML-PDD begins with a set of optimized designs that is generated by classical methods, e.g., topology-optimized material distributions. Then, each design is compressed via an encoder q _θ (z|x) and eventually decompressed into the original design via a decoder x ∼ p _θ (x|z). The loss function is a two-term loss derived from the variational evidence lower bound

(5) D KL ( q θ ( z | x ) | | q ( z ) ) − E z ∼ q θ ( z | x ) [ log p θ ( x | z ) ] ,

where q(z) is a prior over z, q _θ (z|x) is the encoder, and p _θ (x|z) is the decoder. The first term D KL is the Kullback–Leibler (KL) divergence which regularizes the distribution of the encoder with the prior q(z) and the second term is the reconstruction loss that biases the generated design to be similar to the data design [23].

A significant advantage of using VAEs over diffusion models is their structured latent space Z given by optimizing the evidence lower bound in Eq. (5). When a VAE is properly trained [109], the decoder p _θ (x|z) efficiently constructs designs x given latent vectors z that are distributed according to the prior q(z). This dense distribution of latent vectors around the prior is what enables latent optimization to be so successful because small perturbations around the prior have a large impact on the design.

However, the trade-off between training the reconstruction loss and KL divergence terms often results in suboptimal generative performance, particularly when dealing with fine structural features. Moreover, poor prior model assumptions q(z), such as a single Gaussian distribution [23], can lead to poor regularization [104], blurry reconstruction [107] and difficulties in capturing multi-modal distributions for multi-objective problems. For more advanced photonic design problems, such as nonlinear devices [110], plasmonic devices [111], or multi-objective devices [112], these problems become more exaggerated and difficult to overcome. Several advanced strategies have been used to address these deficiencies in design.

First, conditional VAEs (cVAEs) incorporate conditional labels into the latent space as a latent feature, guiding the decoder to generate more specific properties on-demand, such as optical, structural, and material requirements, which must be incorporated into the inverse design process. cVAEs are effective because the latent conditional label is also feature rich label and contains a lot of information for the decoder. Secondly, cVAEs are well-suited for managing one-to-many mapping challenges by conditioning the model on desired outputs, enabling it to produce diverse solutions that meet the same optical response criteria. This flexibility is valuable in exploring designs that meet multiple requirements, as cVAEs generate solutions within specific constraints. Thirdly, the introduction of β-VAEs weighs the KL divergence with a scalar β to balance the influence of the prior on training.

For example, in Figure 4(a), Kumar et al. implemented a constrained β-VAE model to optimize dielectric multilayer structures using genetic algorithms, allowing for multi-solution inverse design [101]. In another study by Lin et al., the authors examined eigenmodes for the inverse design of 2D BIC structures by using a β-VAE for encoding and decoding geometries, and two CNNs for forward simulation and inverse design (Figure 4(b)) [102]. By exploring the latent space of the β-VAE, the authors contributed to understanding protected/unprotected modes in complex geometries, enhancing future band engineering capabilities in photonic structures.

Figure 4:

Variational autoencoder (VAE) and generative adversarial network (GAN) models in photonic design applications. (a) Architecture of GA-β-VAE for dielectric multilayer structures inverse design. Adapted with permission from ref. [101]. Copyright 2024 Optica Publishing Group. (b) A DNN fusion model comprised of β-VAE and CNN1-z-CNN2 for inverse design and forward prediction, respectively, for bound states in the continuum (BICs) in freeform structures. Adapted with permission from ref. [102]. Copyright 2021 Chinese Laser Press. (c) Schematic of the conditional global topology optimization networks for metagrating generation. Adapted with permission from ref. [91]. Copyright 2019 American Chemical Society. (d) The workflow of the cGAN-based methodology for the inverse design of multifunctional microwave metasurfaces. Adapted with permission from ref. [113]. Copyright 2022 the authors. Advanced Photonics Research published by Wiley-VCH GmbH.

Moreover, using attention mechanisms allows the model to focus on small details over multiple passes [35], [40]. For instance, transformer-based architectures [35] leverage self-attention mechanisms to model long-range dependencies between data points, which is crucial for capturing complex interactions in photonic structures [114].

Lastly, rather than relying on the KL divergence for regularizing the encoder, introducing an adversarial discriminator mechanism forces the latent space to better align with the prior through a discriminator model, thus improving the generative quality [12], [83]. Despite these improvements to VAEs, outside of latent optimization, they have largely been replaced by a more general framework which uses the aforementioned adversarial mechanism known as generative adversarial networks or GANs [115].

4.2.3 Diffusion models

While GANs are able to generate high-quality results, they still suffer from training instability due to their sensitivity towards hyperparameters and the balance of the relative power of the generator and discriminator. Furthermore, VAEs are known to have issues balancing terms in the evidence lower bounds and their generative performance is lacking. A class of models which recently gained popularity are denoising diffusion models. Diffusion models are far more stable to train and robust towards hyperparameter changes as compared to GANs, and they have been shown to possess great generative capabilities and superior distribution coverage [130], outperforming GANs as evaluated by generative metrics such as Fréchet inception distance (FID). Diffusion models have found applications in various domains, from computer vision to bioinformatics [131], [132]. However, diffusion models are often more computationally expensive during training and sampling as compared to GANs and VAEs, requiring many more training steps to generate an image [130].

A diffusion model uses a Markov chain q(x ^(t+1)|x ^(t)) to gradually add noise, e.g., Gaussian noise x ( t ) + ϵ ( t ) : ϵ ( t ) ∼ N ( μ t , σ t ) , to the training data x ⁽⁰⁾ ∼ p _data(x) [133] so that the designs converge to a noisy Gaussian over time [25], [134]. A neural network predicts previous image q _θ (x ^(t−1)|x ^(t)) to reverse the noising process, which during training iteratively predicts the previous image given the current image over all time steps. To sample from the diffusion model, isotropic Gaussian noise is generated at the final time step T and the neural network iteratively generates the previous image for T − 1 time steps until a novel image is produced [25], [135]. To improve on the iterative denoising process, Ho et al. [25] introduced the denoising diffusion probabilistic model (DDPM), where they parameterize the neural network to predict the original source of noise ϵ _θ (t) instead of the previous image, which is shown in Figure 5(a). Typically for image applications, DDPMs use U-Nets which are deep convolutional networks with a spatial dimension bottleneck and skip connections to preserve information [25], [136], [137]. Recently, latent diffusion models have been introduced, which perform the diffusion process within the latent space Z of a pre-trained autoencoder, reducing the training and sampling computational requirements as the expensive diffusion process occurs on a lower dimensional representation [33].

Figure 5:

Diffusion model applications in photonic design. (a) Illustration of reverse process, where the neural network learns how to denoise across the Markov chain to generate novel images. Adapted with permission from ref. [25]. Copyright 2020 Neural Information Processing Systems Foundation, Inc. (b) Training architecture of conditional diffusion model, where a conditional and unconditional U-Net are mixed to produce new inverse designs through the denoising process. Adapted from ref. [138]. Copyright 2023 the author(s), published by De Gruyter, Berlin/Boston, licensed under the Creative Commons Attribution 4.0 International License. (c) Illustration of semi-supervised training process. Adapted from ref. [80]. Copyright 2024 the author(s). Laser & Photonics Reviews published by Wiley-VCH GmbH, licensed under a Creative Commons Attribution-Non Commercial License.

Diffusion models have been applied to a variety of tasks in nanophotonics, such as metasurface design, classification, and image deconvolution [80], [138], [139], [140], [141]. Zhang et al. [138] focused on conditionally generating dielectric metasurfaces using a conditional diffusion model on an all-dieletric freeform unit cell dataset to generate high-quality designs, guided by S-parameters. The researchers use a neural network to predict the spectral response and find that their diffusion model outperforms SLMGAN, WGAN-GP, and cVAE on frequency response accuracy, achieving a 43 %, 48 %, and 54 % improvement respectively. Their framework is shown in Figure 5(b). Kim et al. [80] sought to improve upon a classification task through implementing a semi-supervised learning strategy, facilitated by a diffusion model, that takes advantage of a large amount of unlabeled data with a small amount of labeled data, comparing their approach to a supervised learning scheme on a photonic crystal waveguide dataset. Their strategy, as shown in Figure 5(c), first consists of training a “teacher model” to predict the spectral response of the generated dataset, and then training a DDPM to expand the original dataset, to which the “teacher model” affixes labels. Finally, a “student model” with the same architecture as the teacher model is trained on this expanded dataset. The study finds that this semi-supervised learning strategy can enhance average training losses of the student classification model by up to 102.8 % as compared to the teacher model. Naturally, DDPMs are built for denoising. So, using them outside of design is also worthwhile. Chakravarthula et al. [139] used a diffusion model to improve the optical quality of images captured from an on-sensor metalens array, a challenging problem prevalent at smaller wavelengths. The authors introduce a flat array of metalenses to increase the field of view and implement a diffusion model to recover an image from the on-sensor metalens array. The image recovery process takes a captured image and alternates between inverse filtering, diffusion sampling steps, and merging steps. Their study finds that their probabilistic deconvolution method outperforms existing traditional and more recent machine learning deconvolution methods in various metrics.

Diffusion models also provide an alternative approach to tackle one-to-many mappings in transforming random noise into structured designs. Zhu et al. demonstrated this method for meta-atom design [140], using gradual noise-based generation specific to diffusion models to create multiple designs meeting the same specifications by varying the initial noise conditions. This allows the model to produce diverse solutions for the same performance criteria, which is ideal for cases where similar spectral responses arise from different structures.

In the most recent studies, diffusion model-based multi-modal machine learning frameworks bridge the fields of natural language processing and computer vision by combining models that deal with these two domains. Sun et al. created a multi-modal machine learning framework that incorporated the contrastive language-image pre-training (CLIP) and stable diffusion (SD) models to predict the photonic modes in one certain structure from structural information text [141]. The CLIP model is used to evaluate how similar a word is to an image, and the SD model performs the same diffusion process, but in the latent space of pre-trained autoencoders. The authors report that their method displays signs of stabilization across aHash, FID, and CLIP score metrics after only 6 training epochs. Furthermore, their method achieves an approximately 2-fold speedup in efficiency over conventional techniques that utilize Maxwell’s equations.

4.3 Reinforcement learning

Reinforcement learning (RL) algorithms excel when design choices build on one another, making them well-suited for photonic design problems where each step can, in turn, drastically alter the optical response. They recently gained popularity for their ability to beat humans at chess, Go, and other popular games [142]. RL stands apart from other ML techniques because it uses a goal-driven, trial-and-error approach, where the RL agent learns by interacting with an environment and receiving feedback (rewards or penalties) based on its actions [143]. RL agents generate their datasets in real time instead of relying on large datasets at the outset of the training process. This reduces the number of simulations needed to meet specific performance benchmarks. Their autonomy and adaptability make RL uniquely suited to streamline photonic device design while minimizing the computational overhead required for training datasets.

4.3.1 Reinforcement learning applied to PDD

RL enables inverse design by forcing the agent to design a device environment x (comprised of different materials and geometries) that optimizes the optical response, for example, the transmission spectrum of a metasurface [144], [145]. Figure 6(a) shows an example in which the device parameters x = {NT, L, AT, D} are the dimensions of the Si unit cells on a Si₃N₄ substrate [69]. The agent iteratively adjusts these parameters to achieve desired optical response using feedback from a simulation r ̂ ( y | x ) to generate positive and negative rewards. The agents’ actions adjust the geometry or materials in the design environment x and, over time, the RL model converges on optimal configurations. We simplify the RL training process to the following steps:

1. Initialize environment: Create the input parameters x ⁽⁰⁾ for the policy function π(x ⁽⁰⁾). For example, in Figure 6(a), this means initially assigning values to the dimensions in the unit cell. The agent could also select from a list of materials although it is not included in this example.

2. Action Selection: The agent uses its policy π(x ^(t)) to determine the next action a _t+1 ∼ π(x ^(t)), choosing from a range of probabilistically-generated actions based on the current state x ^(t). The agent balances exploration (trying new actions) and exploitation (repeating effective actions) as it refines the design.

3. Simulate the environment: Each action’s outcome x ^(t+1) ∼ a _t+1(x ^(t)) is evaluated by simulating the optical properties y ̂ ϕ ( x ( t + 1 ) ) , see Section 3. This simulation provides feedback on how well each action advances the design toward the target outcome.

4. Process Rewards: The RL model estimates rewards (or penalties) for each action based on how closely the resulting design meets the target outcome. Then, these rewards (or penalties) are used to update the agent’s policy, in preparation for the next step.

5. Repeat: Let the FOM f(y ^(t)) rank each design x ^(t) by its optical response y ( t ) = y ̂ ϕ ( x ( t ) ) from each generated design x ^(t), for example using a loss function designed to minimize deviation from a target spectrum [146]. Iterate until the FOM f(y ^(t)) reaches a target threshold that signifies the otimization is complete.

Figure 6:

Reinforcement learning for photonic design. (a) Example of reinforcement learning (RL) agent applied to the optimization of color generation using dielectric metasurfaces. Adapted with permission from ref. [69]. Copyright 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement. (b) Training process for RL algorithm for the design of 1-dimensional photonic crystals. The agent uses the transfer matrix method (TMM) to simulate the spectrum of the design created by the sequence generator. Adapted with permission from ref. [146]. Copyright 2021 the author(s). Published by IOP Publishing Ltd. (c) Result of the RL algorithm implemented in (b). The system outperforms the state-of-the-art memetic algorithm for the emissive power of light bulbs in the visible regime. Adapted with permission from ref. [146]. Copyright 2021 the author(s). Published by IOP Publishing Ltd. (d) Silicon on Insulator (SOI) diffraction grating structure designed with RL algorithm. The RL model (PHORCED) outperforms GLOnet a compatible discriminative neural network trained with supervised learning. Both graphs in this subfigure adapted with permission from ref. [147]. Copyright 2021 the author(s). Published by De Gruyter, Berlin/Boston.

The previous architectures discussed in this review update their parameters based on some form of gradient descent incorporated with the results of a loss function, such as Eq. (5). Similarly, RL techniques must specify a method for updating their agent’s policy; however, unlike the previously discussed architectures, this update is not necessarily derived from a loss function. Instead, RL techniques rely on feedback in the form of rewards (or penalties) to adjust the policy, based on the agent’s progress toward achieving a specified goal. This distinction has a significant implication: RL does not require the reward calculation process to be differentiable which means they can incorporate external simulations, such as 3D field solvers, into the optimization process. This flexibility expands the range of problems that RL can address and allows researchers to integrate diverse computational tools into the training process

4.4 Quantum generative models

Quantum information science is a rapidly growing field, with recent breakthrough demonstrations accelerating enthusiasm for its potential [153], [154]. Quantum-enhanced sampling and optimization algorithms are speculated to yield at least quadratic speed-up [155] for combinatorial optimization; thus, there is increasing interest to apply quantum algorithms for optimizing designs in PDD. The core idea is that designs x are embedded in quantum states |x⟩ and sampled using a quantum device q _θ (x) = |⟨x|U _θ |ψ ₀⟩|² where U _θ is a unitary acting on the initial state ψ ₀. Quantum sampling devices can be accessed commercially through D-wave [156], IBM [157], quantinuum [158], to name a few. To sample designs using a quantum device, the pure states |ψ⟩ of the native Hamiltonian H ̂ of the device must encode the design x. Likewise, the FOM is often encoded onto the low-energy states of a surrogate Hamiltonian H ̂ such that the energy ⟨ x | H ̂ | x ⟩ is anti-correlated with the FOM f ( y ̂ ϕ ( x ) ) . Then, the quantum device implements an algorithm, such as quantum-enhanced MCMC [107], [159], quantum annealing [156], or QAOA [155], to sample low-energy states of H ̂ to produce designs that optimize the FOM [39], [107] (Figure 7(a)). Initial work using this scheme in PDD problems was based on problems whose designs could be naturally encoded on pure states of the surrogate Hamiltonian. For example, in Figure 7(c) Inoue et al. optimized photonic-crystal surface emitting lasers [160]. The authors observe that non-uniform spatial distributions would be useful in accommodating for varying lattice constants or hole shapes in photonic crystal devices [161]. These parameters are formulated into a Hamiltonian sampling problem using a factorization machine and sampled by D-wave’s quantum solvers. Similarly, Kitai et al. [162] explored the design of metamaterials for radiative cooling applications using quantum sampling and a factorization machine. Properties such as compositional inhomogeneity can be formulated into minimization problem for the quantum annealer’s Hamiltonian, which identified candidates with high FOMs. The resulting structures were evaluated through rigorous coupled-wave analysis for their radiative properties, and the results were iteratively used to refine the factorization machine model. Direct mapping of large problems increases the complexity of the system Hamiltonian, making it susceptible to noise and errors, while also exacerbating local minima trapping and significantly lengthening sampling times. In an effort to reduce this complexity, Ye et al. explored a hybrid quantum-classical strategy using mixed-integer linear programming (MILP), which optimizes material layouts within a continuous domain using linear equations with continuous and discrete variables based on generalized benders’ decomposition (GBD) [163], [164]. This approach optimizes goals such as structural compliance and heat transfer efficiency while managing constraints such as local displacement [165]. GBD decomposes the original PDD problem into a sequence of MILPs, and uses material layouts from previous iterations to determine each new iteration, leading to faster convergence compared to classic direct optimization methods. The material is represented as a binary variable, and the formulations are tested on D-wave’s systems, and the implementations using GBD (GBD splitting-direct and GBD splitting-CQM) predictably took less time to converge, taking 234.12 and 111.97 s respectively [166]. Linear programming reduces iterations to reach optimal solutions, yielding sharper designs with fewer grey areas. Iterative refinements provide direct mappings, individual optimal solutions, faster convergence, and simpler implementation on quantum computers, extending to other continuous optimization problems. Despite the efficiency gains from MILP, the need for robust feature extraction and dimensional reduction persists.

For other PDD problems, such as freeform material distribution optimization, the designs cannot be embedded directly into pure states via direct mapping because there are too few qubits, the design space is too sparse or unknown FOM mappings onto the native Hamiltonian. Sparse design spaces in PDD especially complicate the optimization landscape, leading to inefficient use of quantum resources as many sampled quantum states may correspond to sub-optimal solutions. This further motivates the use of autoencoders: more particularly, binary variational autoencoders are useful in representing continuous optimization problems by traversing a latent space [167]. To address these challenges, discriminative models based on encoders compress designs onto pure states |z⟩ that can be natively sampled [107]. These frameworks typically begin with a classical pre-processing step, where the PDD designs x are encoded q _θ (z|x) onto a “native” representation z, sampled via the quantum device q _θ (z) = |⟨z|U _θ |ψ ₀⟩|², and then decoded x ∼ p _θ (x|z) [107], [168]. The bVAE-QUBO framework proposed by Wilson et al. aims to optimize dielectric, free-form diffractive meta-grating for beam steering [39]. As shown in Figure 7(b), the framework begins by compressing input designs into a binary latent space with a corresponding polynomial (QUBO) formalism. Quantum annealing models optical behaviors as state transitions, solving them as combinatorial problems where optimal configurations represent solutions. Statistical mechanics models enhance this approach by encoding metamaterial features like refractive indices and conductivity levels. Using D-Wave’s leap hybrid models and simulated annealing, bVAE-QUBO achieves 96.5 % and 96.7 % respective efficiencies. The framework was also applied to optimize the efficiency of a thermal emitter for TPV cells, maximizing power generation and minimizing radiation losses. The bVAE-QUBO framework uses a reduced dimension space for faster processing and traversal, ensuring feature-rich encoding through one-to-one mapping and generating a probabilistic distribution of all possible topologies. Notably, these techniques have been applied to freeform metamaterial unit-cell design for radiative cooling applications, thermophotovoltaic energy recapture [39], diffractive meta-gratings [39], and even improving future quantum devices [107]. Generally, these techniques have shown marked improvements beneficial in fields like thermophotovoltaics, incandescent light sources, biosensing, microbolometers, and drying furnaces [162].

Figure 7:

Quantum generative models in photonic design applications. (a) Simulated annealing versus quantum annealing. Simulated annealing aims to traverse a large solution space and converge at the global minimum as the solution. Quantum annealing works in a similar manner; it encodes the problem into a quantum system where the system’s qubits represent binary variables. By evolving the system’s Hamiltonian, all possible states of the qubits are explored simultaneously. Quantum annealing image adapted with permission from ref. [169]. Copyright D-wave systems. (b) bVAE-QUBO framework. Adapted with permission from ref. [39]. Copyright 2021 author(s). Published under an exclusive license by AIP Publishing. (c) Photon-crystal surface emitting lasers (PCSELs) using 3D RCWA, QUBO, and FMs. Adapted with permission from ref. [160]. Copyright 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement.

4.5 Summary and outlook

ML is transforming photonic design by enabling efficient exploration of complex, high-dimensional spaces to optimize device performance. Advanced generative models like VAEs, GANs, diffusion models, and RL streamline the design process by balancing innovation, precision, and practical constraints. These tools enable breakthroughs in photonic systems by facilitating rapid iteration, addressing multi-modal challenges, and uncovering new physical phenomena. Emerging quantum/hybrid quantum-classical approaches further extend these capabilities, driving innovation in advanced photonic applications with unprecedented efficiency and scalability.

5.1 Improving reliability with stochastic processes

Simple yet effective stochastic methods, such as smoothing, significantly enhance fabrication reliability. Smoothing typically employs Gaussian blurring ( x → x ̃ ) on optimized structures generated by methods like topology optimization (Figure 8(a)) [173], [174]. This pre-fabrication step mitigates vulnerabilities to small-scale defects by reducing the complexity of fragile or unintuitive geometries frequently generated during optimization [175]. Figure 8(c) and (d) demonstrates the effect of smoothing on a cylindrical metalens designed via topological optimization, illustrating improved manufacturability and consistency between simulated and fabricated results. Smoothing techniques have been successfully applied to various applications, such as improving waveguide mode matching and enhancing robustness against defects through structural optimization [176].

Figure 8:

Improving the fabrication of photonic devices with machine learning. (a) Inclusion of fabrication defects into the inverse design process. (b) Reinforcement learning (RL) uses a fab-in-the-loop process to optimize nanophotonic design over multiple iterations of the algorithm. (c) Design of cylindrical silicon metalens using topological optimization. This example shows the result of the topological optimization without taking into account the fabrication process. (d) The same design as (c) except that the topological optimization accounts for the fabrication process available to the designers of the system. Subfigures c and d are adapted with permission from ref. [177]. Copyright 2021 Optical Society of America. (e) Result of the fab-in-the-loop process applied to a grating coupler design. (f) Comparison between a traditional grating coupler and a grating coupler designed with the fab-in-the-loop RL algorithm. Subfigures a, b, e, and f are adapted with permission from ref. [151]. Copyright 2023 the authors, distributed under the Creative Commons License, AIP Publishing.

Vaccination is a similar technique that shifts the focus from the probabilistic incorporation of fabrication errors to adjusting misalignments common during experimental demonstration of free space optical designs, i.e., refining the fabrication parameters η to reduce error. Mengu et al. demonstrated this technique on their deep diffractive neural network (D2NN) platform [178]. Their inverse design process created a series of metasurfaces in which each unit cell was trained like a neuron in an artificial neural network. Each neuron’s precise control and individual freedom left the platform vulnerable to misalignments on the order of a single wavelength. With the introduction of vaccination, the researchers introduced random misalignments in the training process, resulting in a robust experimental demonstration at up to four wavelengths. This technique is another promising example of how careful incorporation of noise during the training process can be generally helpful in the realization of nanophotonic devices [179].

5.2 Correcting variations with discriminative models

While these simple stochastic processes are effective, researchers have sought more advanced techniques to protect against fabrication variations by adjusting the fabrication parameters η. As we have demonstrated throughout this review, a natural next step to the modeling of complex stochastic processes is to use highly effective discriminative models which can model complex, nonlinear fabrication processes. A common way to improve fabrication via discriminative modeling is to predict the optimal parameters η*(x) to improve the fabrication process r η * ( χ | x ) for any design x. For example, Gostimirovic et al. presented a feed-forward CNN to automatically correct layout distortions, which ensured that the resulting experimental demonstrations were robust against fabrication defects [180]. They trained their model on a small set of scanning electron microscopy images, enabling it to predict and then fix local variations such as corner rounding, bridging of narrow features, and over-etching in convex corners. Their approach requires no modifications to the existing fabrication process or proprietary foundry data, making it well-suited to a wide variety of photonic applications.

Alternatively, by predicting the error ϵ(χ|η, x) in the fabricated designs χ, adjustments can be made to the fabrication parameters η to improve robustness. In later work, the same group leveraged a deep CNN to predict nanoscale deviations from the original layout, revealing how proximity effects and other lithographic or etch-related artifacts degrade fidelity [181]. In a similar approach, Liu et al. use a tandem architecture of deep neural networks to address disparities between their design simulations and optical response [65]. They address inverse design for problems like thin-film filters and multiwavelength phase modulators. They combine the simulation, design, and fabrication steps of the PDD into a single feedback loop using multiple machine learning approaches. First, a discriminative model is trained to map a device layout to its optical behavior, then they combined the input design with a model that generates candidate layouts for a specified target performance. Although they set out to optimize their inverse design pipeline, their strategy naturally enhances robustness against manufacturing defects because the forward model is trained on fabrication data and the inverse network consistently generates layouts less prone to fabrication-induced errors. The resulting layouts avoid, for example, unrealistically thin layers that could cause over-etching or dimension loss.

5.3 Reinforcement learning in fabrication

RL is a strong candidate to improve fabrication errors because of its naturally iterative and adaptive nature which real-time feedback from the fabrication process. This makes it well-suited to handle the complexities and uncertainties inherent in manufacturing at the nanoscale. One of the critical advantages of RL is its ability to model the fabrication process as part of the environment in which the learning agent operates. Witt et al. demonstrated a fab-in-the-loop RL algorithm that automates adjustments to the design based on measured results (Figure 8(b)) [151]. The algorithm iteratively adjusted the design based on measured performance after each fabrication cycle. Specifically, the RL agent received feedback from insertion loss measurements and constructed rewards based on the results. After only six iterations of this process, they reduced insertion loss in a crystal grating coupler from 8.8 to 3.24 dB. In an alternative approach, Park et al. created a physics-informed RL model to design a reward system to encourage the feasibility of their fabricated devices [152]. Figure 8(e) and (f) highlight the potential of RL for scalable, robust, and innovative solutions in the general design of nanophotonic systems. RL models provide a very general framework for machine learning, so introducing something like fabrication variations into the reward process can be as simple or complicated as the experimental demonstration may require.

5.4 Summary and outlook

These fabrication-oriented methods showcase the growing capability of machine learning to address the randomness and complexity of real-world manufacturing conditions. Basic smoothing strategies provide fast, practical ways to refine the complex designs that sometimes result from inverse design techniques. Approaches like vaccination can ensure robustness against stochastic processes such as misalignment that might be present in experimental demonstration. Discriminative networks are an even more effective approach to protect against fabrication variations but they require datasets that can be time consuming to generate. Finally, RL provides a closed loop approach that can include fabrication results within the design process. Taken altogether, these techniques highlight a compelling trend. The inverse design of photonic structures push for tighter tolerances and higher complexity but machine learning techniques also offer a protective framework to ensure the manufacturability of those designs.

6.1 Physics-informed generative data augmentation for pre-training

Two powerful techniques in machine learning for increasing training data are data augmentation [209] and generative modeling [19]. Typically, data augmentation increases the number of training samples by applying random transformations to the training data, e.g., additive Gaussian noise, image manipulation, etc. If the data augmentation is performed in a physics-informed manner, e.g., by applying generative modeling to learn the hyperparameters of noise sources during characterization, the generative models can be manually applied to pretrain a model on simulation data. For example, in a recent work by Wilson et al. [40] as shown in Figure 9(b), a set of 40 dark field microscopy images were augmented to generate a 10,000 image dataset by applying a pre-trained segmentation model to extract segmented images of gold nanoparticles spread uniformly on silicon. This model, along with a labeled clustering algorithm, are used to calculate the distance matrix and nanoparticle radii for adversarial tampering and natural degradation sample types. The discriminator network is then trained by randomly selecting a synthetic tampering type based on the tampering Bernoulli distribution. The distribution of the nanoparticles was shown to be uniform, and the background Gaussian noise was measured. Taking the segmented nanoparticle images and measured background noise distributions, an arbitrary number of synthetic samples could mimic the characterization measurements by reproducing a uniform distribution of nanoparticles, each of which is randomly augmented with stretching, rotation, etc., and adding background noise. These methods work well when the physical model is well known, but the hyperparameters need to be learned. However, when the physical model is unknown and several candidates are available, we turn to alternative Bayesian methods.

6.2 Experimental physical discovery

The aim of automatic physical discovery [210] is to predict an analytical model for a set of measurements [197], [199], [200] through Gaussian processes, active learning, and symbolic regression [30], [48], [195], [211], [212], [213], [214]. Oftentimes, these models are employed when measurement data are sparse and time-consuming to collect and the physical model isn’t known a priori. For example, in hypothesis learning, a variant of active learning which uses reinforcement learning and Bayesian optimization, several candidate models are chosen with randomly initialized hyperparameters. These models use a set of prior physical models y ̂ ( i ) such as Gaussian priors and candidate models with preconfigured hyperparameters. Then, each of these models is evaluated using Bayesian inference based on reducing uncertainty. Using a greedy policy, the model which has the minimal uncertainty is used to predict the next sample point ϕ ^(t+1) and all the models are updated with the new characterization measurement. Active learning enables algorithms to prioritize and identify critical structural or design features by iteratively refining their understanding based on user-defined parameters or feedback, such as key microstructural elements based on operator-defined signal aspects. Further, hypothesis learning optimizes functions using physical or competing models, such as optimizing microscope resolution or exploring ferroelectric domain growth [197] (Figure 9(a)). While hypothesis learning and other active learning methods have been demonstrated on small examples, as automated physical discovery matures, methods will need to be augmented and improved to address larger fabrication priors. Along the same lines, active learning methods have few examples of integrating modern symbolic regression libraries for purely analytic modeling.

6.3 Summary and outlook

Treating characterization as an inference task where the objective is to construct a symbolic model or approximate the FOM allows us to employ new techniques like symbolic regression and automated physical discovery. To ensure that ML models are ready in low-shot environments like characterization, where sometimes a relatively low number of uniquely correlated samples are available, data augmentation techniques like generative modeling and random transformations can pre-train approximate FOM models. The introduction of symbolic regression and automated physical discovery enable more efficient modeling of physical phenomena. The biggest future opportunities are in the introduction of community-wide nanophotonic databases that would give more characterization data for training these models.

7.1 Conclusions

This work comprehensively reviews how machine learning-assisted photonic device development (ML-PDD) transforms the traditional, iterative photonic device development (PDD) approach. We broke down the PDD process into 5 steps: theory, simulation, design, fabrication, and characterization and showcase how machine learning (ML) algorithms improve each step of the process. This framework is presented in the context of discriminative models, which excel at representing sophisticated functions, and generative models, which excel at exploring large design spaces. Combining these techniques creates a data-driven approach that automates device exploration, speeds up the PDD process, and provides increasingly reliable real-world solutions.

The theory portion (Section 2) discusses how machine learning synergizes with the fundamental physics of electromagnetic theory to enrich the fidelity of photonic research. ML techniques like symbolic regression and discriminative models uncover and refine the underlying physical laws by proposing new, high-fidelity constitutive relationships in exotic media. Generative models provide insights into dominant mechanisms that aid performance through latent space engineering. Altogether, these developments pave the way for more comprehensive, accurate, and rapid electromagnetic modeling, linking photonic design directly to fundamental principles in ways that conventional analysis alone cannot achieve.

The simulation section (Section 3) highlights how ML approaches can assist or replace traditional simulation techniques like FDTD or RCWA. Discriminative models, which map design parameters to optical responses, act as fast, approximate simulators for specific problem areas. These techniques require extensive training data collected from numerical solvers or lab measurements. However, once trained, they can predict device behavior at a fraction of the time required by traditional techniques. Fortunately, research has shown that generative models can assist in data collection, offering data augmentation through synthetic examples that mimic actual or simulated measurements. This synergy helps to address the data scarcity problem that plagues many photonics workflows. Ultimately, the combination of these techniques is invaluable to the iterative optimization problems in photonic design, enabling the designer to search much larger design spaces than was previously possible while maintaining fidelity to established electromagnetic theory.

The design section (Section 4) showcases the latest techniques for the inverse design of photonic structures using machine learning. We highlight the latest techniques for VAEs, GANs, diffusion models, reinforcement learning, and quantum-hybrid solvers. The generative models encode geometric features and material information into latent variables. Researchers can efficiently explore these expansive design spaces to refine design features and inversely optimize the solution. Reinforcement learning techniques provide a precise solution while minimizing the training data required to navigate the design space. Meanwhile, quantum-inspired and hybrid quantum-classical frameworks bring new frontiers to photonic optimization by leveraging quantum annealers or factorization machines for combinatorial challenges. All these ML-driven advances in design reflect a growing shift in the research community. Instead of manually guessing or tuning geometry, scientists are using neural networks to discover innovative device configurations that often outperform traditional techniques.

Next, we discuss how common imperfections in the fabrication process (Section 5), such as defects, misalignments, or tolerances, can be mitigated with modern ML techniques. We introduce straightforward solutions like smoothing techniques and “vaccination” against alignment errors. Then, we highlighted discriminative models capable of predicting lithographic distortions or etching errors, thereby proactively correcting design layouts for real-world production steps. Lastly, we introduce reinforcement learning techniques that include the fabrication process directly into the algorithm’s training environment, training the RL agent to propose robust designs against the fabrication process. We showcase algorithms at several levels of complexity, highlighting how the ML toolkit can be adapted to the fidelity required by the experimental demonstration.

The characterization section (Section 6) emphasizes how ML can streamline data-heavy or noise-limited tasks. Discriminative models trained on partial information can quickly infer relevant properties when measuring the performance of complex photonic chips or metasurfaces. These models drastically reduce the need for exhaustive measurement protocols. These methods are critical for characterization, such as single-photon emitters, detecting subtle surface defects, or reconstructing near-field images. We then showcase how realistic noise models can ensure even more robust data augmentation techniques. Ultimately, ML-driven characterization enables more rapid insights into a device’s optical behavior, surpassing purely manual or physics-based interpolation.

Altogether, these new ML techniques demonstrate performance improvements across every step of the PDD. Discriminative models provide rapid forward mappings critical for speedy simulations, online process monitoring, and efficient characterization. Generative models offer unconstrained exploration of device topologies and parameter sets, which result in exotic yet robust design solutions. Combining both approaches in collaboration with physical knowledge and real fabrication constraints automates and accelerates the entire PDD process.

7.2 Future outlook

In the short term, we expect that ML techniques will continue to improve the efficiency of photonics designs in terms of accuracy and cost. By incorporating physics-based models and fabrication constraints into the training process, they will ensure that designs are both physically sound and practically realizable. Hybrid modeling techniques that combine quantum annealers, latent space engineering, and high-fidelity surrogate models promise unprecedented efficiency in exploring complex design landscapes over today’s solutions.

With regard to hybrid techniques that incorporate quantum annealers, however, it is evident that these tools are still in their incipient stage, and significant progress is required in both quantum hardware development and algorithmic design before their capabilities can be reliably leveraged in device design. In the foreseeable future, we anticipate the most viable role for quantum annealing to be a heuristic subroutine embedded within classical machine learning frameworks, offering possible advantages in constrained combinatorial tasks like QUBO, graph partitioning, or feature selection. Nevertheless, as Aaronson has emphasized, there are substantial hurdles like noise, limited qubit connectivity, and embedding inefficiencies that must be overcome to demonstrate the true utility of quantum optimization-assisted machine learning [215]. Even so, incremental advantages in quantum materials, co-design, and surrogate modelling may gradually enable quantum optimization to become a more practical tool within machine learning.

As these approaches mature, we anticipate that the leading models will become more robust and expand to larger design spaces. Along this process, they will become increasingly accessible, making photonics designers less reliant on extensive machine learning backgrounds for selecting, training, and deploying models. Eventually, we expect these models to integrate directly with commercially available simulation software, similar to how topological optimization is already available in tools like Lumerical or COMSOL. We anticipate that generative inverse design frameworks will follow a similar trajectory toward maturity and eventually become a familiar and user-friendly tool employed across the majority of photonics designs.

Finally, we anticipate the creation and sharing of large datasets of photonic structures will standardize and expedite progress. We imagine a comprehensive, community-driven database similar the Materials project [216] and Atlas [217], which are already useful resources for the reference of optical data in homogeneous materials. Still, for the development of metamaterials [218] and similar complex photonic structures, there is a lack of a similar effort to construct a unified dataset of characterization measurements and computational models for training machine learning models. With leadership, the photonics community could compile the datasets created throughout their training processes. Then, as the field evolves and machine learning-assisted design becomes more prevalent, a central design database could vastly improve the PDD design cycle. Over time, ML will automate more and more of the PDD, improving technologies from integrated photonic circuits and topological photonics to advanced metasurfaces and quantum information systems.