A newcomer’s guide to deep learning for inverse design in nano-photonics

2.1.1 Artificial neurons and neural networks

The basic building block of an artificial neural network (ANN) is the artificial neuron. An artificial neuron is simply a mathematical function taking several input values. A weight parameter is associated with each of the inputs. As depicted in Figure 1a, first the sum of the weighted input values is calculated, and then the value of an additional bias parameter is added. Subsequently a so-called activation function f is applied to the resulting number. The result of the activation function is the neuron’s output value, also simply called its activation. A neural network is nothing else than several of these artificial neurons connected to each other in some way, for instance by feeding the output of one neuron into other neurons’ input layers (c.f. Figure 1b). Please note that some network architectures also implement other types of connections, for example of neurons within a layer or to preceding layers, etc.

Figure 1:

Basic concepts of artificial neural networks and their training. (a) A single neuron where x is the input vector, w and b are, respectively, the weights and biases and f is the activation function. The inset shows the output of the sigmoid activation function. (b) A set of neurons arranged in several, consecutively connected layers, forming a neural network. (c) Learning rate size effect on the loss function optimization. (d) A typical training loss or validation error plot. When the loss decrease starts to stagnates, reducing the learning rate leads to further convergence. Adapted from Ref. [22].

2.1.2 Deep networks with nonlinear activations

A key hypothesis underlying deep learning (“deep” means that a network has many layers) is that the ANN is learning a hierarchy of features from the input data, where each layer is extracting a deeper level of characteristics. In an image for instance, the first layer could be recognizing lines and edges, the second layer may “understand” how they form specific shapes like eyes or ears and a subsequent layer may then analyze the relative positions, orientations and sizes in the ensemble of these features, to identify complex objects like animals or human faces. In consequence, using many layers is essential to ensure that a network has a high abstraction capacity.

Now, it is technically possible to use linear activation functions throughout a network. However, it is trivial to show, that any neural network with multiple layers of linear activation can be identically represented by a single linear layer, as the chain of two linear functions is still linear. Hence, using nonlinear activation functions is crucial in any deep ANN in order to perform hierarchical feature extraction. Various activation functions can be used. The activation which is closest to a biological neuron’s response is probably the sigmoid neuron, which implements a logistic activation function, depicted in the inset of Figure 1a. The sigmoid is however suffering from the fact that for inputs far away from the bias, the gradient is very small. For such inputs learning is very slow. Therefore, other activations such as rectified linear units (ReLU) expressed as max(0, x) often preferred in the hidden layers of an ANN, because of its constant gradient and cheap computation cost. In the “body” of the network, ReLU is usually a very good first choice and leads to robust networks.

The activation of the very last layer of a network, its output layer, needs to be chosen depending on the task to solve, as well as on the numerical range of the output data values. A softmax activation layer is a form of a generalized logistic function, where the sum of the neurons’ activations of the entire layer is normalized. Hence it is adequate for outputs that correspond to a probability distribution like in classification tasks. In regression tasks – inverse design typically falls in this category – a linear output activation function is the simplest first choice, since it can take arbitrary values. Be aware that in case of a linear output activation the ANN needs to learn the data range along with the actual problem to solve. Therefore one should consider normalizing also the output part of the dataset (see also below). In this case Sigmoid (data range [0, 1]) or tanh activation (data range [−1, 1]) can help the network to focus on the essential learning task (c.f. “inductive bias”).

2.1.3 Training

Deep learning is a statistical method; the goal is to adapt the network parameters (weights and biases of the artificial neurons) such that the ANN learns to solve a task that is implicitly defined by a large dataset. The trick to learn from such data is to define a loss function that quantifies the network’s prediction error. More precisely, the loss expresses how much the neural network’s predictions of a set of samples are different from the expected network outputs. The notion of “set” is important, because the loss is defined in a stochastic approximation for batches of samples [53]. The expected outputs obviously need to be known and thus are also part of the training data.

Using an optimization algorithm, the loss function is minimized by modifying the weight and bias parameters of the network. During this optimization, the model repeatedly computes a batch of training samples in forward propagation to calculate the loss. Subsequently the gradients of the loss function with respect to all network weights are calculated in a backpropagation step, using automatic differentiation (autodiff) [54]. The network parameters are finally adapted towards the negative gradient direction, in order to minimize the loss function. Autodiff is the core of deep learning and hence all DL libraries are essentially autodiff libraries with tools for neural network optimization.

In practice, the model parameters are randomly initialized, hence DL is typically not deterministic. Restarting a training process will not give the exact same network. During training, the size of the parameter modification steps is a crucial parameter. It is typically called “learning rate” and takes generally values smaller than 1 (10⁻³ or 10⁻⁴ are common start values). For a given network parameter w _i , the update is expressed as w _i = w _i − LR × ∂loss/∂w _i . If the learning rate (LR) is very small, the algorithm requires many iterations to reach the loss function minimum, and if it is too large, it can miss the optimal solution which often is a steep minimum. Thus it is important to reduce the learning rate during training, as depicted in Figure 1c and d.

2.1.4 Which optimizer

The most popular optimization algorithm in deep learning is stochastic gradient descent (SGD) [53, 55], which performs the weight updates as described above. A popular alternative, that often offers a faster convergence are the SGD variants “adam” [56] or “adamW” [57]. It is also more robust with respect to its hyperparameter configuration, and therefore an excellent first choice.

2.1.5 Practical implementation of neural networks

The de facto standard programming language in the deep learning community is Python, but frameworks exist for virtually all programming languages. The most popular libraries to build and train neural networks are “PyTorch” [58], “TensorFlow” with its high-level API “Keras” [59, 60], “Flax”/“JAX” [61], or “MXNet” [62], among others.

2.2.1 When to use deep learning

2.2.1.1 Intuition

Deep learning is a data-driven approach: during training, the neural network is trying to figure out correlations in the dataset that allow to link the input to the output values, eventually converging to an empirical model describing the implicit rules behind the dataset (in our case the implicit physics). In that sense, neural network training can be somehow compared to human learning: By mere observations, humans figure out causal correlations in nature. In nano-photonics for instance, a person who is studying plasmonic nanostructures will sooner or later develop an intuition for the expected red-shift of a particle’s localized surface plasmon resonance with increasing particle size.

Therefore, a good question to ask is: With the given dataset, does it seem easily possible to develop an intuition of the physical response? If this is the case, training a deep learning model on a large enough dataset promises to be successful. On the other hand, if a problem or its description is highly entangled and we can only hardly imagine that an intuitive understanding can be learned without further guidance, then an artificial neural network is likely going to have a hard time understanding the correlations in the dataset.

An example is shown in Figure 2. A Bragg mirror design problem is parameterized by its geometry of N dielectric layers with arbitrary material and thickness, and the target physical observable is the reflectivity at a fixed wavelength. The learning problem is a mapping of 2N values to a single reflectivity value. Providing a human with a large set of such data samples would most likely result in confusion rather than in an intuitive understanding of the correlation between geometry and physical property. The problem becomes easier to grasp intuitively if the layer-order is given to the network so it does not need to learn that the order is important. In deep learning this can be done using a convolutional layer, which keeps the input structure and searches correlations only between “neighbor” values. We exploit the convolution’s inductive bias (see also below). We know that periods of the same pair of layers form the ideal Bragg mirror. With that prior knowledge we could further simplify the intuitive accessibility and parameterize the geometry with only two thicknesses and two materials values, this layer-pair being repeated for N/2 times. Finally, instead of predicting only a single reflectivity at one wavelength, we could train the network on predicting reflection spectra in a large spectral window. Now, the geometry can be understood easier and the impact of changes in the layer pair geometry is easier to interpret. A wavelength shift of the stop-band for instance can be easily quantified if a spectrum is given instead of just one reflectivity value. In the same way as the latter representation is easier to interpret for a human, a deep learning model will be capable to develop an empirical model much easier. Other examples where such additional tasks, also called auxiliary tasks, are useful for improving the performance on the machine learning principle task, have been investigated in reference [65], and more recently in the case of the game “Go” [66]. In conclusion, richer information about the problem is often useful for making training faster and less data intensive.

Figure 2:

Comparison of two different parameterizations of the same problem. (a) Using a dense neural network, the order of the thicknesses (blue values) and refractive indices (green values) is lost, because every input value is fed into every neuron in the first layer. The network needs to learn during training that the order matters. Furthermore, the correlation between the many input values and the single output reflectivity are difficult to understand. (b) Using a CNN, the layer order can be conserved and using two input channels, even the association of thickness and refractive index of a single layer can be directly passed to the network, hence during training these correlations do not need to be learned. Furthermore, predicting a whole reflectivity spectrum facilitates to identify correlations between changes of the geometry and for example resonance peak shifts. Returning this again via a convolutional layer conserves the order of the spectrum. On small to medium size datasets, exploiting these inductive biases of a CNN can significantly improve performance.

2.2.2 When to NOT use deep learning

Writing a good data generation routine to create a useful training set is at least as challenging, but often even more challenging than writing a good fitness function for a conventional optimization technique. Additionally, it has been demonstrated on several occasions, that simple conventional methods often outperform heavy GPU-based black-box optimization methods [75–78]. Before rushing into data generation, we therefore urge the reader to consider whether conventional techniques may not be a sufficient alternative to deep learning on their specific problem.

2.2.2.2 Single design tasks and simple problems

We argue that the main argument against deep learning is that it is inappropriately expensive in many situations. If the geometric model used in the design problem is described by only a few free parameters (like 3 or 4), a systematic analysis is probably very promising and maybe even cheaper than generation of a large dataset. In many cases this may furthermore be done around an intuitive, first guess. Similarly, if the geometry is more or less known and only small variations are to be optimized, conventional optimization or again a systematic exploration of all parameters are more appropriate. Such situations are depicted in Figure 3. Even if a systematic analysis cannot be performed with sufficiently dense parameter steps, it may be worth considering conventional interpolation approaches such as Chebyshev expansion or also Bayesian optimization [79–82]. Finally, if the problem consists in solving a single design target, even for complex scenarios a conventional global optimization run is probably the more adequate approach [15].

Figure 3:

Examples of problem configurations, for which deep learning is probably not adequate. (a) Problems with few parameters like the here depicted plasmonic bow-tie antenna design (3 free parameters) can easier be solved by conventional approaches, intuition or may even be systematically explored. (b) In problems with a single design target, the computational overhead of deep learning is not paying back. Conventional global optimization is the method of choice.

In conclusion, finding problems that really benefit from deep learning based inverse design is not as obvious as it is often suggested in literature. As an example to support this claim we recall that to date all production-scale metasurface design is being done with conventional lookup tables, while deep learning is still only used on toy-problems for testing.

2.3.1 Overview of the major model architectures

Neurons can be connected in different ways to form an ANN. The most adequate neural network type will also depend on the dataset to be processed. Multilayer perceptrons (MLP), convolutional neural networks (CNN) and recurrent neural networks (RNNs) are the most widely used general architectures. Recently so-called “attention”-layers have gained significant popularity as well. In the following we will discuss the advantages and drawbacks of the different network architectures and give suggestions concerning their applications.

2.3.2 Which network type to use

Comparing the arguments in favor and against the different network architecture that are presented in the preceding section, we come to the conclusion that, whenever applicable, CNNs are generally the best first choice for approaching an inverse design (or other) problem with deep learning. They are simple to design, robust in training and have excellent scaling behavior. If the data is not in an adequate format, often it is even advantageous to reformat the data in order to render it compatible with a CNN. There is no “one” recipe to do so, but if a meaningful structure exists in the data values, it should be conserved. In case of a sequence of thin film layers for instance, it makes sense to concatenate layer thicknesses and layer materials in two lists of values that can be fed to a one-dimensional CNN (c.f. Figure 2b).

Besides the first technical tests, in general one should not use simple sequences of convolutions, typically called “visual geometry groups” (VGG). It is long known that such VGG-type CNNs have severe limitations and do not scale well with model size [22]. Instead, convolutional layers should be organized in residual blocks [93] or, even better and only slightly more complex, in residual “ResNeXt” blocks with inverted bottleneck layers and grouped convolutions [116, 118]. We refer the reader to the supplementary Python notebook tutorials for a detailed technical description and example implementation [52].

2.3.3 Choosing the inverse design method

2.3.4 The dataset – questions to ask about the data

The data are the most important resource in a deep learning model. It is therefore essential to meticulously carry out data generation or, if data already exists, to understand the dataset. In both cases it is often helpful to perform appropriate preprocessing. As a rough guide, we provide a few questions that one should ask about the data:

1. How much data is necessary?

   Most deep learning architectures require at the very least a few thousand samples, likely more. There are cases where less data can suffice, for instance in transfer learning/fine-tuning of an existing model. In some cases it’s possible to learn on slices of large but few samples, for instance in segmentation tasks [128, 129].

2. Is random data generation possible?

   The design targets may be sparse in the parameter space, in which case training using a random dataset may not work. This is often the case in free-form geometries, where the number of design parameters is high. Using a weak optimization method for data generation can help in such cases, but special care must be taken to avoid bias of the dataset towards a specific type of pre-optimized solution, so enough randomness is important [40]. Also systematic data sampling approaches (e.g. Sobol sampling) may be useful [130–132]. The potential difference between random and optimized data is illustrated in Figure 5.

3. Are there obvious biases in the data?

   Biased data is one of the most important challenges in deep learning. In short: “what you put in is what you get out”. For instance if a dataset is built with resonant nanostructures, a network trained on this data may not correctly identify non-resonant cases.

4. Is the data meaningful?

   Samples carrying the essential information may be sparse in the dataset or the important features may be hidden in a large number of irrelevant attributes. A careful pre-processing of the dataset may help in such cases.

5. Can I shuffle the data?

   Are subsets of the data partly redundant (e.g. after sequential data generation)? If yes, a splitting method is required that guarantees that training, validation, and test subsets are independent.

6. Are the subsets (training data, validation data) representative for the problem?

   In order for the training loss and benchmark metrics to be meaningful, each subset needs to fully cover the entire problem to be solved. If we apply the network on significantly different samples than it was trained on, we probe the extrapolation regime, where performance is generally weak. A non-representative validation or test dataset will therefore provide a bad benchmark [18]. An illustration depicting non-representative subsets is shown in Figure 4c.

7. How frequent are outlier

   If necessary, can the impact of exceptions or anomalies in the data be reduced, e.g. by selective removal or by pre-processing with a non-linear scaling function? [133, 134].

8. How should the data be normalized?

   Input values should in general be standardized: Subtract the mean and divide by the standard deviation. Figure 6 depicts why this is essential. The choice of the normalization for the network output is closely linked to the choice of the loss function, but also depends on the data representation and on the significance of outliers, among other factors. If the data has multiple output values or channels, are those of similar numerical magnitude? If not, should they be normalized with the same scaling process or each channel individually? Appropriate normalization is highly important for an accurate model.

9. Can we exploit inductive biases?

   The term inductive bias denotes any assumption about how to solve a task that is implicitly included in the model. In a classification task for instance, the output layer is chosen as a probability distribution. In a regression task, the output layer is a continuous activation function. Such inductive biases improve the network training. In their absence, a network needs to learn them from the data in addition to the actual task. For example a classifier would need to learn that the output is a binary yes/no decision, and parts of the network would be used for this aspect. Often it is possible to include some properties of the problem implicitly in the network architecture. CNNs typically imply translation invariance. Likewise, symmetries may be included, or the valid data range, by choice of an adequate output activation. In physics, a possibility to exploit inductive bias is causality, which may be enforced through Kramers–Kronig relations or, more simply, using a Lorentzian output layer [18, 135].

Figure 5:

Histograms of two illustrative datasets of the reflectivity (at λ = 500 nm) of 10,000 dielectric thin-film sequences. (a) Randomly generated thin film sequences. Their reflectivity is in general low. A network for the design of high reflectivity solutions, that is trained on this dataset, will very likely fail. (b) Weakly optimized data generation, starting from random samples. The dataset offers a large portion of medium to high reflectivity samples. Care should be taken that still enough “randomness’ is present to avoid biases towards high R solutions. A network trained on the optimized set will perform better at the task of reflectivity maximization.

Figure 6:

Normalizing the data is important. (a) Large numerical values do not exploit the non-linearity of typical neuron activation functions. In case of specific activations like Sigmoid or tanh, furthermore the gradients of the neuron output are very small. Both situations are unfavorable for network training. (b) The numerical values of normalized data cover the full range of an artificial neuron’s non-linearity. Also the neuron gradients are large for all typically used activation functions. This significantly helps the learning process. On non-normalized data, during training first the neuron biases need to be optimized which consumes unnecessary computation. The same argument explains why batch normalization is so effective.

2.3.5 Practical network implementation and training

Once the data is prepared, the full set needs to be split into training, validation and test subsets. With these subsets we are ready for the actual training of a network. The next step is to code and train the model. This section is a little technical, we ask the reader to look at the Python notebook tutorials that demonstrate explicit implementations of networks following the below guidelines [52].

Since a sophisticated model may train very slowly, it is important to do a first technical test with a very simple network model. Even though you shouldn’t use it as a final layout, the initial test network can be a sequence of a few stacked convolutional layers (“Visual Geometry Group”, VGG). With this we test the implementation of the input and output layer dimensions, if the data scales are correct, and if the network output format is adequate. Finally we get a first idea about the typical learning loss. As for the specific configuration of the convolutions, small 3 × 3 kernels, strides 1, and “ReLU” activations can be used as a relatively foolproof rule of thumb [118]. Kernels with larger receptive field are occasionally used in literature [116], but increasing the convolutional kernel size increases quadratically (in 2D) the computational cost as well as the number of network parameters. It can be easily shown that stacking N 3 × 3 convolutions covers a receptive field equivalent to a single layer with kernel size (2N + 1) × (2N + 1), but with significantly less trainable parameters, reduced computational effort and potentially the additional benefit of hierarchical feature extraction capabilities [136].

Once this technical test is passed, it is time to increase the model complexity. Simple convolutions are converted into residual blocks [93] or, even better, “ResNeXt” blocks [116, 118], and more and more layers are added to the network, while paying attention to overfitting. Once overfitting occurs, the network complexity needs to be reduced again, or if the accuracy is not yet sufficient, other regularization strategies can be considered, such as data augmentation (for example, shifting or rotating the input data, if possible). Following dense layers, dropout can be applied, which randomly deactivates some of the neurons during training, leading to a model less prone to overfitting [137].

2.3.6 Training loop good habits

A lot of frustration can be avoided if some good habits are respected concerning the training loop itself.

1. Learning rate: The learning rate is an important training hyperparameter. If it is too small, training gets stuck in a local minimum. If it is too large, the network parameters may “overshoot”, resulting in a diverging training. The choice of the initial learning rate (LR) is strongly dependent on the problem and on the network architecture. Typically it needs to be tested which LR works well. Once a good starting value is found, the LR should not be left constant through the entire training loop. Training rate schedules of varying complexity exist, but a good starting procedure is usually to gradually reduce the LR (see also Figure 1c and d). A simple learning rate decay schedule such as dividing the LR by a factor 10 every time the validation loss stagnates for a few epochs, is often already very efficient.

2. Batch size: Evaluating the training loss on small batches of random samples is one of the key mechanisms in deep learning. It is the pivotal procedure that impedes the optimizer to get stuck in local minima. As a result the batch size (BS) is also a crucial hyperparameter that has a tremendous impact on the training convergence and model performance. While it depends also on each specific problem, in general the BS should be small in early training. 16 or 32 are good initial values. Too large starting batch sizes usually converge to local minima and thus lead to less accurate models [152, 153]. However, once the loss stagnates, it is good practice to increase the batch size since it renders the gradients smoother, and helps further decreasing the loss function, similar to the reduction of the learning rate [154]. Please note that, as in any global optimization scheme, it is impossible to evaluate whether the actual global minimum was reached, because in general training will always end in a local minimum.

A few additional, rather technical tips are given in a dedicated Section 4.

3.1.1 Avoiding the extrapolation zone

The most imminent danger when using iterative optimization with deep learning surrogates is that the optimizer pushes the design parameters outside of the model’s validity range. Most neural networks are strong at interpolation but bad at extrapolation tasks. This is illustrated in Figure 8, where a neural network has been trained on predicting the reflectivity of a dielectric thin film layer stack. This network totally fails in doing so outside of the range of the training data. For input values that are 10 % larger than the largest training data, the average error is already in the order of 20 % (Figure 8b). This illustrates that it is important to constrain the allowed designs to the parameter range that has been used for training, hence to the interpolation regime of the forward model. Please note that even if the model is efficiently constrained, a non-negligible risk of converging to network singularities remains in any case, since optimization algorithms seek extrema of the target function [83, 120]. We insist therefore that careful verification of results obtained from deep learning is crucial in any case. A classical approach is also to retrain the surrogate model with the new data, after output data has been verified by a classical physical model.

Figure 8:

Illustration of failed extrapolation. (a) A neural network was trained on predicting the reflectivity spectrum of a thin film layer stack made of ideal dielectrics with constant permittivity ɛ. (b) Average forward network error as function of the excess of the input parameters outside of the training data range. (c) Failed prediction example outside of the training data range (smaller permittivities and thinner layers). (d) Example inside the range of the training-set design parameters (interpolation). The network predictions are accurate. (e) Same as (b) but for larger permittivities and thicker layers than used in training. Orange lines: PyMoosh simulated reflectivity spectra. Blue lines: neural network predicted spectra. Insets represent the used layer stack. Bar heights correspond to the layer thickness according to the scale bar, the color code indicates the layer dielectric’s permittivity.

In order to constrain the designs, in specific cases dependent on the design parameterization, it is possible to formulate a penalty loss. For instance when the design is described by size values for which the limits are known [121]. The penalty loss is added to the optimization fitness function, which then increases, if the optimizer leaves the allowed parameter range. A drawback of this method can be the increasing complexity of the fitness function, and the requirement of problem-specific weight tuning to balance the contributions to the total fitness function.

In many situations this tuning can turn out to be complicated or the parameterization of the design is difficult to constrain, for example in free-form optimization, where the design may be given as a 2D image. In such cases, a separate deep learning model can provide an elegant solution. By training in a first step a generative model on the designs alone (without any physics knowledge), a design parameterization can be learned from the dataset [71, 73, 163]. Adequate models are for instance variational autoencoders (VAEs) or generative adversarial networks (GANs). These two approaches are depicted in Figure 9a, respectively 9b. For technical details and an example implementation, we refer the reader to the accompanying tutorial notebooks [52].

Figure 9:

Avoid the extrapolation regime of the forward model with learned design parameterization. (a) Sketch of a variational autoencoder (VAE) trained to reconstruct the design. In a VAE, the encoder is trained to return the mean value (µ _z ) and standard deviation (σ _z ) of the latent variable z. A randomized, normal distributed latent vector is passed to the decoder for the reconstruction task (via random number generator “RNG”). By further constraining σ _z with a KL loss (c.f. text), one obtains a compact and smooth latent space that is normally distributed. (b) Sketch of a generative adversarial network (GAN). As in the VAE, by using a normal distributed random number generator during training for the latent space input, the generator develops a smooth and compact latent space, essentially representing the interpolation regime of the dataset.

The latent space of such a VAE or GAN represents a learned parameterization of the designs. Thanks to the properties of the regularized latent space of VAEs or GANs, the parameterization is compact and continuous, even if the original design description was not (e.g. if a list of discrete materials is used). Please note that some techniques like variational regularization are not differentiable. Since backpropagation necessarily requires deterministic gradients, reparametrization tricks need to be applied to the backward path in such cases [164]. Provided the VAE or GAN training did converge, this means that every point in the latent space yields a physically meaningful solution. Most importantly, the latent space of a well-trained VAE or GAN is regularized to a normal distribution with unitary standard deviation.

This means that in an iterative optimization loop, we can now constrain the latent variable that describes the designs to the numerical range of a normal distribution (e.g. to a 2σ confidence interval). This is then equivalent to constraining the entire problem to the interpolation regime of the dataset. Practically we replace the original design parameters by the latent input variable of the trained geometry generator, as depicted in Figure 10. In the subsequent optimization, the latent input can be conveniently constrained to a normal distribution e.g. by simply penalizing large values as in inspirational generation [165], or by using a Kullback–Leibler divergence loss (KL loss), as used for variational autoencoders [69, 166].

Figure 10:

Re-parameterized the forward model using a learned latent representation as design input: The trained geometry generator (e.g. from a VAE or a GAN) is simply plugged before the input of the forward network. It converts a latent vector z to a physical design x ̂ . If the latent space was properly regularized, sampling from within the range of a normal distribution with unitary variance will generate geometries in the interpolation regime of the training data, where the forward network works accurately. Instead of optimizing the physical design parameters, we can now run the optimization on the latent variable of the geometry generator. Constraining the numerical range of the optimization parameters accordingly, renders iterative optimization robust.

3.1.2 Heuristics: forward network with global optimization

A robust way to overcome local extrema and assure convergence to the global optimum is using gradient-free heuristics such as evolutionary optimization or genetic algorithms [15, 167–171].

Accelerating global optimization based inverse design with deep learning is in principle straightforward. Instead of using numerical simulations, the evaluation step in the loop of the optimizer (e.g. evolutionary optimization, particle swarm, genetic algorithm …) is done with a deep learning surrogate model. This is depicted in Figure 11a. As discussed in the previous section, it is recommended to constrain the designs to the interpolation regime, for example using a generative model that precedes the physics predictor, as illustrated in Figure 10. The actual optimization can be done with any algorithm, since the operation of the deep learning model is restricted to the calculation of the fitness function.

Figure 11:

Inverse design via iterative optimization using a forward model as fast physics solver surrogate. (a) Use forward model to accelerate a global optimization loop. (b) Neural adjoint method: neural networks are differentiable and allow gradient based optimization. Reduce risk of local minima by operating on a large set of random initial designs. The optimization tries to minimize the error between the predicted optical property (solid red line) and the design target (dashed black line).

One of the major advantages of deep learning surrogates is their differentiability. We therefore encourage to not use gradient-free global optimization alone, but combine it with gradient-based optimization for faster convergence. This will be discussed in the following.

3.1.3 Gradient descent – neural adjoint method

While global optimization is robust and generally converges well towards the overall optimum, such methods are also inherently slow since they do not take advantage of gradients. This is unfortunate because gradients are available “for free” in deep learning surrogate models. On the other hand, gradient based approaches tend to get stuck in local minima. This can be accounted for to a certain extent, but it usually depends strongly on the individual design problem if a gradient based method will work.

The idea of gradient based optimization is the same as in the Newton–Raphson method. A fitness function is defined such that it is a measure of the error of a solution compared to the design target. Then, the derivatives of the fitness function with respect to the design parameters of a test solution are calculated and used to modify the test-design towards the negative gradient. By minimizing the fitness function in this way, the solution iteratively gets closer and closer to the ideal design target until a minimum is reached.

Typical numerical simulation methods are not differentiable, and hence gradient based methods cannot be applied directly. While gradients can be calculated using adjoint methods [14], these still require multiple calls of the, generally, slow simulation, and hence are usually computationally expensive. Both problems can be solved to some extent by forward neural network models. A key advantage is, besides the evaluation speed, that gradients can be calculated “for free”, because the network is an analytical mathematical function. For the same reason, the gradients of the surrogate model are also continuous, since this is a key requirement for the network training algorithms. As stated in the beginning, the training procedure of a neural network is in fact a gradient based optimization by itself, therefore the main functionality of all deep learning toolkits is automatic differentiation. A forward neural network model can thus always be used for gradient based inverse design, which consists of two steps that are illustrated in Figure 11b. In a first step, a set of test-designs (typically random initial values) is evaluated with the forward model. Their predicted physical behavior is compared to the design target, for which a fitness function evaluates the error between target and prediction. Now the deep learning toolkit is used to calculate the gradients of this fitness with respect to the input design parameters via backpropagation and the chain rule. Finally, the designs are modified by a small step towards the negative gradients. Repeating this procedure minimizes the fitness [29, 36, 51, 121, 172, 173].

Note that, if the gradients of the underlying physics data source are known, they can be added in the training step. This often leads to better convergence of the model, since the training can use deeper correlations to build its model [174].

As mentioned before, the main difficulty in this approach is to avoid getting stuck in local minima of the fitness function. To a certain extent this can be accounted for by optimizing a large number of random designs in parallel (see Figure 12a). While such strategy would be prohibitively expensive using numerical simulations, with a machine learning surrogate model it is typically possible to optimize several hundreds or even thousands of designs in parallel. However, depending on the problem, the number of local extrema may be too large for successful convergence (see Figure 12b). This can be tested by running the optimization several times. If multiple runs do not converge to a similar solution, the parameter landscape of the problem is probably too “bumpy” for gradient based inverse design.

Figure 12:

Schematic fitness landscapes of (a) a friendly problem with relatively few local extrema. (b) A complicate problem with many local fitness minima. Using gradient based methods with a large number of initial test-sets, problem (a) will likely converge to the global optimum. In problem (b) on the contrary, the chance is high that none of the initial designs is close enough to the global optimum, and optimization will converge to a local solution. The paths taken by a gradient-based method path are indicated by white arrows.

As explained above, it is crucial also in gradient based optimization to remain in the forward model’s interpolation regime since extrapolation bears a high risk of converging towards non-physical minima of the deep learning model [121]. Also, if the dimensionality of a problem is high, the risk of strongly varying gradients further increases and optimization may always converge to unsatisfying local minima. As discussed above, in such cases it is helpful to train a separate generator network that maps the design parameters onto a regularized latent space (e.g. VAEs or GANs, c.f. also Figure 9). Instead of optimizing the physical design parameters, the optimizer then acts on this design latent space. Because the latent space is regularized, it is possible to constrain the designs to the neural network’s interpolation regime e.g. by using a KL loss term in the fitness function.

Finally we want to recall once again, that unlike conventional numerical simulation methods, deep learning surrogates can possess singularities, also called failure modes or adversarial examples [120]. Gradient based optimization, especially, comes with the risk of converging to such singularities of the surrogate network. In fact, the neural adjoint method is very similar to the “fast gradient signed method” that is specifically used to find network failure modes [83]. To avoid convergence to a network singularity, it has been proposed to alternate the evaluation in the optimizer loop between the surrogate network model and exact numerical simulations. Such occasional verification of the optimized solutions effectively eliminates non-physical designs [119].

3.1.4 Hybrid approach: global optimization followed by neural adjoint

As mentioned above, inverse design tasks often possess of a large number of local extrema. As illustrated in Figure 12b, in such case gradient based algorithms may get stuck in those local extrema, even if a large number of designs is optimized in parallel. A possible solution can be a combination of iterative global optimization and local gradient based neural adjoint. In such a scenario, a global optimizer runs for a few iterations with a rather large population of solutions. During the first generations global solvers usually converge the most rapidly towards the global optimum. However, they can be expensive in the final convergence towards the exact extremum. Using the population obtained from a few iterations of a global optimization run can be very helpful as the initial set of designs for the neural adjoint method. Those designs are then relatively close to the global optimum and the chance that at least a few manage to avoid local minima is considerably increased. The approach is depicted schematically in Figure 13c.

Figure 13:

Global pre-optimization. If too many local minima exist, a promising solution is to start by pre-optimizing a set of random designs with global optimization. The positions of the random initial samples in an illustrative fitness landscape are depicted in the bottom left. After a few iterations of a global optimizer, the solutions are closer to the global optimum, as illustrated in the bottom right. Using this set as initial population for gradient based neural adjoint is likely to converge.

3.2.1 Tandem network

One of the most simple configurations for a one-shot inverse design network is the so-called “tandem network” [33]. The tandem network is a variation on autoencoder acting on the physics domain. It takes as input the desired physical property and returns a reconstruction of these physics (for instance a target reflectivity spectrum and its reconstruction). The difference to a conventional autoencoder is that the decoder is trained in a first step on predicting the physical properties using the design parameters as input. This means, the decoder is simply a “forward” physics predictor, solving the direct problem (“fwd” in Figure 14a). Subsequently, a second training step is performed, in which the forward model weights are fixed, and the encoder, which is actually trained on generating the designs, is added to the model (generator “G” in Figure 14a). In this second step, the full model is trained, but now only the physical responses from the training set are used. The physical property (e.g. reflectivity spectrum, etc …) is fed into the encoder, which predicts a design. However, instead of comparing this design to the known one from the dataset, the generated design is fed into the forward model, that predicts the physical property of the suggestion. This predicted response is finally compared with the input response, the error between both being minimized as training loss. This means, that even if multiple possible design solutions exist, the training remains unambiguous since only the physical response of the design is evaluated, regardless of how it is achieved. The full model is then essentially an autoencoder of which the latent space is being forced to correspond to the design parameters by using the fixed, pre-trained forward network as decoder.

Figure 14:

Direct inverse design models. (a) Tandem model. The training is divided in two steps. At first a forward predictor is trained on the direct problem. Subsequently, the forward network is fixed and used to train the generator, (b) the conditional variational autoencoder (cVAE) is trained end-to-end in a single run. A latent space z is used to provide additional degrees of freedom to handle ambiguities in the design problem. (c) Inverse problems typically can be solved by multiple solutions. A tandem model will learn only one of possibly multiple solutions, the other remain inaccessible. The cVAE on the other hand typically learns the set of possible solutions which can be retrieved via the latent vector z.

A practical advantage of the tandem is that the inverse problem is split in two sub-problems that are individually easier to fit, compared to end-to-end training of the full inverse problem. In a first step the forward problem is learned, which is usually a relatively straightforward task. This physics knowledge is then used in the second step to guide training of the generator network.

3.2.2 Conditional variational autoencoder (cVAE)

A drawback of the tandem network is that only a single solution is learned, even if multiple designs are possible to reach the design target. Several network architectures have been developed to learn mappings to the set of multiple solutions in ambiguous inverse problems. We discuss in the following a very efficient and robust model, the conditional variational autoencoder (cVAE).

The reason why a “vanilla” variational autoencoder (VAE) cannot be directly trained on an inverse design task is the correlation between geometry and physics domain. The latent space of the autoencoder forms during training and represents the most efficient and compact, reduced representation of the inputs. This is generally not the design parametrization. In consequence it is necessary to force the latent space to correspond to the design space, which is achieved in the “tandem” architecture via a two-step training procedure (see above). The tandem is hence an autoencoder with design-regularized latent space.

By conditioning the designs on their physical properties (here: optical), a modified variant of a VAE, a so-called conditional variational autoencoder (cVAE), can however be trained as an inverse design network. To this end, the classical VAE, that reproduces a design through encoder–decoder architecture, is extended by an additional input, the design condition. Here this is a physical property (e.g. a reflectivity spectrum) – the design target. As depicted in Figure 14b, this additional condition is added as input to both, the encoder (blue) as well as the decoder (green). During training, multiple possible solutions are associated with different values of the latent vector z, i.e. can be treated without training ambiguities, as illustrated in Figure 14c. After successful network training, only the decoder is used for the inverse design. The possibility to identify multiple possible solutions with a cVAE is illustrated in Figure 15 by the example of multi-layer designs for a fixed reflectivity target.

Figure 15:

Dielectric layer stacks implementing an arbitrary reflectivity design spectrum. Inverse designed by a cVAE. By sweeping through the latent space of the cVAE generator with fixed target spectrum, multiple possible design solutions can be identified. Note that the cVAE discovered that mirrored structures yield the same reflectivity spectra (c.f. for example latents z = 0 and z = 0.8). A systematic latent inspection can also be done for further optimizing the solution, for example by a search for the best possible spectral match, or by identification of the most robust design, etc. The colors in the layerstack illustrations (bottom) correspond to the permittivity (c.f. color legend in Figure 8).

An advantage of the cVAE is its generally robust training. It often also works well with low-dimensional latent spaces, so that the latent space can be explored systematically, to identify different possible solutions [35]. A recent comparison indicates, that cVAEs are among the most effective methods for direct inverse design tasks [175].

3.3.1 Global optimization by deep reinforcement learning

Some key successes of recent deep learning are based on combining forward modeling and direct policy learning (such as Alpha-Zero [177]). In this regard, conventional global optimization can be replaced by deep reinforcement learning (RL) [178], or [39, 179] a RL policy can be applied directly for choosing the design parameters. However, comparison with classical evolutionary methods is in most cases not clearly in favor of deep reinforcement learning, so we believe it is questionable whether the extra effort of using a less studied method is advisable at the moment [180, 181]. Please note that there are debates whether RL can indeed outperform conventional global optimization in a general picture [182].

3.3.2 Conditional generative adversarial network (cGAN)

Generative adversarial networks (GANs) can be seen as a variation of VAEs. The position of the decoder and the encoder are interchanged, and rather than compressing information, the encoder acts as a dynamically trained loss-function, which has as objective to distinguish generated samples from real data. Analogously to the case of conditional VAEs, in order to solve an ill posed inverse design problem, the GAN inputs need to be conditioned on the physics design. The GAN becomes a conditional GAN (cGAN).

Note that, the original GAN uses a min-max loss function between generator and discriminator, which is in practice very difficult to handle and often suffers from severe convergence problems [183]. The convergence of the shortly later proposed Wasserstein-distance loss for GAN training (WGAN) is significantly more robust [184]. Typically it is combined with gradient penalty regularization (WGAN-GP), which leads to even better robustness, and is today used in the majority of large GAN models [185].

Note also that very well performing inverse design GANs have been proposed that take inspiration from NVIDIA’s “style GAN”. By progressively growing the network model as well as the design resolution during training, very good design results and high accuracy can be obtained, however at the cost of a highly increased computational training budget [67, 127, 186].

3.3.3 Diffusion models

Recently, stochastic generative modeling was shown to be capable to solve inverse problems. These so-called “diffusion models” were described by Y. Song et al. by: “Creating noise from data is easy; creating data from noise is generative modeling” [187]. The key idea is inspired by thermodynamic processes and is based on performing a sequence of denoising steps by a deep learning model. The model is trained on removing a small amount of noise from input data, which was perturbed by different amounts of noise [188]. After several denoising steps, a large amount of initial noise can be entirely removed. Starting this iterative process on random noise, and combining it with some guidance using latent information about the original content, a denoising diffusion model can generate samples that match the target latent description. Such networks are very popular in computer vision and image generation (“text to image”) [68, 189, 190]. Very recently, a first application of a diffusion model for metasurface inverse design has been demonstrated [191].

3.3.4 Invertible neural networks

A possibility to solve the inverse problem in a one-shot manner is invertible neural networks. Such networks are constructed using exclusively mathematical operations that guarantee that the full network remains an invertible mathematical expression, so that it is bijective. Every point in the target space matches exactly one point in the input space [192]. Obviously, this approach runs into troubles with the ill-posed character of typical inverse design problems that prohibits finding a bijective projection between physics and design space. To allow the network to learn such a bijective projection, latent dimensions need to be added to the design space, which during training are fitted to distinguish between multiple solutions or to identify cases with no solution at all [193].

3.3.5 Physics informed neural networks

As discussed above, a very interesting aspect of deep learning models is that they are differentiable. Furthermore, the basis of every deep learning application is the universal approximator theorem, stating that any function can be approximated with arbitrary accuracy with a sufficiently large neural network [28]. The idea behind physics informed neural networks (PINNs) [194, 195] is to learn an approximation to a solution for a partial differential equation (PDE), by testing arbitrary values and minimizing a physics-based loss that tests the validity of the predicted solution in the PDE. PINNs are known for their extremely high accuracy and have been used for inverse design as well, where design parameters are typically included in the model and iteratively fitted during the physics-based training [196–198].

In the context of PINNs, we want to note that periodic activation functions can be very powerful in the context of differential equation solving, since their derivatives naturally represent many typical solutions [174, 199].