Nanophotonic device design based on large language models: multilayer and metasurface examples

3.1 Forward modeling of multilayer films

In the forward modeling task, we utilized LLM’s capabilities to generate codes to calculate the transmittance and reflectance of multilayer structures. Benchmarks were conducted on five configurations: one, two, three, five, and finally 10 layers. Four distinct prompting techniques were tested, recording the number of successful calculations in 50 trials for each technique. A calculation was considered successful if all steps and output values were correct, and these results were used to evaluate the success rate of each approach. The benchmark results are summarized in Figure 3, and full details of the experiments can be found in the Supplementary materials.

Figure 3:

Benchmark results for numerical simulation of multilayer films using code generated by ChatGPT. We report the average success rates obtained across 50 trials for each of the four different prompt engineering techniques. The error bars represent the 95 % confidence intervals. Experiment conducted on January 15, 2025.

We initially tested zero-shot prompting without providing the lecture note. We found that the LLM failed to generate the correct code for the spectral responses in all five cases. This failure was mainly due to the generated code containing equations that were inconsistent with the principles of the TMM calculations (see Supplementary materials Figure S4). Although the LLM-produced code could execute and generate results, the outputs deviated significantly from the expected values.

Building on this initial attempt, we next tested a zero-shot CoT approach. As shown in Figure 2b, this method added the phrase “Let’s think step by step” to encourage the LLM to reason through the problem systematically [35]. While the prompting technique proved effective for other problems, it did not result in significant improvements for multilayer thin-film problems. The LLM frequently produced incorrect formulations when attempting to cascade multiple layers (see Supplementary materials Figure S5), highlighting the limitations of this simple method for addressing multilayer scenarios.

To address these challenges, we adopted one-shot CoT prompting by including a detailed process within the prompt. Specifically, a concise lecture note on TMM was provided, offering step-by-step explanations of the underlying physical theory and problem-solving methodology. This document addressed key concepts, focusing on the construction of matrices to analyze optical systems, including the representation of boundary transitions and propagation within layers in matrix form and the assembly of the overall transfer matrix to determine optical responses. The contents of the lecture note are available in the Supplementary materials. Importantly, the document was a general reference on TMM and was not specifically tailored for LLMs, with similar materials readily available in various academic courses on optics. Using this document, the LLM significantly improved its problem-solving capabilities, achieving over 70 % accuracy even for complex multilayer structures up to 10 layers. Compared to the zero-shot results, common errors were alleviated, particularly those related to the additional information provided in the lecture notes, such as incorrectly defining coefficients, reversing the order of matrix multiplication, and omitting denominator terms. However, some errors still occurred, such as applying incorrect phase terms during the calculations (see Supplementary materials Figure S6).

Lastly, we employed few-shot prompting techniques, supplementing the prompts with a more comprehensive lecture note that provided two specific TMM solutions (Figure 2c). This lecture note not only outlined the general problem-solving steps for TMM but also detailed the procedures for single-layer and triple-layer cases. As a result, the LLM further reduced errors and demonstrated improved generalization when addressing multilayer structures of varying complexity (see Supplementary materials Figure S9). These findings highlight the effectiveness of providing more information in the prompt, leading to more accurate outcomes from the LLM.

The benchmark results show that the performance of LLMs in calculating nanophotonic device properties is highly dependent on the provided prompts. While neither the user nor the LLM has working knowledge of TMM a priori, providing the LLM a lecture note with a few illustrative examples was enough to guide the LLM in creating an accurate numerical simulator and generate the correct answer. This trend of results remains consistent when a different prompt style is used by the user (e.g., a more natural-language-like style) as shown in Supplementary mateials Table S3. This highlights the model’s ability to understand domain-specific knowledge and generate code for numerical computation through in-context learning.

3.2 Conversational design of multilayer films

We now give the LLM a more difficult challenge of designing the multilayer films, where the goal is to find the refractive indices and thicknesses that are optimal to achieve a target spectral response. To demonstrate the design capabilities, we conducted experiments on optimization of optical multilayer devices by an interactive conversation between the user and the LLM. As a proof-of-concept, we designed a bandpass filter that passes light within the 500–600 nm range while blocking light in the rest of the visible spectrum. Again, we assumed that the user does not have expert knowledge in optics.

Figure 4a illustrates the flowchart of the conversational design process. Building on the forward model generated in the previous section, we ask LLM to define the objective function of the optimization problem. We incorporated prompts such as “I aim to develop optimization code for designing multilayer films to create a bandpass filter operating within the 500–600 nm range of the visible spectrum.” Subsequently, the model formulated objective function as the mean squared error (MSE) between the desired and calculated spectral values:

where T _i and T ̂ i represent current and target transmission, at wavelength point i.

Figure 4:

Inverse design of multilayer films through in-context learning. (a) Flowchart showing the conversational design workflow between a user and an LLM. (b) Configuration of the optimized multilayer film, consisting of 10 layers with a total thickness of 1,126 nm. (c) Spectral response of the optimized bandpass filter.

After defining the objective function through prompting, the initial structural configuration was established based on the constraints provided in the prompt and the generated material database. For instance, the design process began with settings such as a maximum number of layers of 5 and a total thickness of 2 μm. For material selection, the LLM recommended dielectric materials, including silicon dioxide (SiO₂), silicon nitride (Si₃N₄), and titanium dioxide (TiO₂). For the optimization, a gradient-free algorithm such as differential evolution was chosen for its effectiveness in exploring high-dimensional parameter spaces.

The design parameters and constraints were updated based on user feedback regarding the optimization results. The code is iteratively generated with adjusted material properties and structural configurations. Detailed prompts and code examples can be found in Supplementary materials. This approach resulted in an optimized 10-layer structure specifying the material and thickness of each layer, as shown in Figure 4b. The structure demonstrated an average transmittance exceeding 90 % within the specified wavelength range and dropping below 10 % outside that range, as illustrated in Figure 4c.

Our results indicate that interactive collaboration between users and AI allows design optimization without the need for in-depth knowledge of optical theory or numerical optimization. Through iterative feedback prompts, the code undergoes continuous refinement, progressively adapting to the specific requirements of the task. The final version of the code and its performance appear to closely match the quality typically achieved by domain experts.

4.1 Prediction of spectral responses of all-dielectric metasurfaces

Figure 5a depicts the unit cell of the metasurface, consisting of a TiO₂ cuboid on a SiO₂ substrate. The metasurface has a period of 300 nm, with its cuboid scatterers characterized by three design parameters: width (w), depth (d), and height (h). The metasurface is illuminated by an x-polarized plane wave at normal incidence. Depending on the size parameters, the metasurface may have one or more resonances in the visible wavelength range and the transmission would be reduced around those resonant wavelengths.

Figure 5:

Language-interfaced fine-tuning for predicting the properties of optical metasurfaces. (a) Fine-tuning process. Both metasurfaces and spectral response data are transformed into text format. The base LLM adjusts its parameters to establish the relationship between geometry and transmittance. (b) Inference phase. The fine-tuned LLM predicts the spectrum when provided with new input data.

We created a metasurface dataset using FDTD simulations with randomly chosen structural parameters as inputs. The spectral transmittance was recorded at 10 nm intervals between 400 nm and 700 nm after each simulation, resulting in an output vector with 31 points. The dataset comprises 2,000 training samples and 500 test samples.

To fine-tune the LLM for spectral transmittance prediction, we utilized the language-interfaced fine-tuning framework [27]. We converted a metasurface geometry into a textual format that the LLM can interpret directly. For example, a sample metasurface configuration is represented in a prompt as: “When we have w = 194 nm, d = 168 nm, and h = 109 nm, what is the transmission spectrum of the metasurface?” The corresponding transmission value vector is likewise represented in text format: “The transmission values range from 400 nm to 700 nm in increments of 10 nm are as follows: [0.967, 0.954, 0.917 …, 0.965, 0.966, 0.966].”

We utilized Meta’s Llama 3.1-8B-Instruct model [39] as the base LLM. Both the input and output texts were tokenized using the same tokenizer utilized during Llama 3 pretraining. Fine-tuning was conducted using LoRA on a single RTX 4090 GPU with the Unsloth framework. The rank r and the scaling factor α were set to 16 and 32, respectively, with a batch size of 20. The training process was carried out over 20 epochs using the AdamW optimizer with a learning rate of 3.00e-4. The fine-tuning employed the cross-entropy loss function used in the original language tasks for next-token prediction.

After fine-tuning the LLM, we evaluated its performance using a test set. Figure 5b represents the inference phase, which involves three key steps for language-based optical property prediction. First, the metasurface geometry is converted into a textual representation. Next, the fine-tuned LLM generates predictions by responding to the input text query. Finally, numerical data are extracted by parsing the generated text. As shown in the resulting graph, the predicted spectrum aligns closely with the ground truth values. We observed a mean squared error (MSE) of 7.23e-3 across the test samples, demonstrating the high accuracy of the model in capturing and predicting complex spectral features within the given structural parameter range (c.f. Supplementary materials for extrapolation). The performance of the model improves as the amount of training data increases, with the MSE decreasing and showing a trend toward saturation when the dataset size exceeds 2000 (see Supplementary materials Figure S1). Furthermore, we conducted experiments using other open-weights LLMs, and models with more than three billion parameters demonstrated comparable prediction accuracy, emphasizing the general applicability of LLM-based learning (see Supplementary materials Table S1).

As a side note, one out of the 500 inferences produced invalid outputs. They generated an inconsistent number of predicted values. This issue may arise from the probabilistic nature of generative models, which operate simply by predicting each successive word. However, in our experiments, regenerating the output consistently resolved such invalid results.

We also compared our approach with traditional neural network models. Specifically, we built a multilayer perceptron (MLP) with an input layer consisting of three design parameters and an output layer comprising 31 spectral points. The MLP demonstrated lower prediction error (MSE of 4.39e-3) and superior computational efficiency (<1 s). While traditional neural networks exhibit faster computation and higher accuracy in this case, they typically require task-specific architectural adjustments and lack the flexibility to handle a wide range of problems with a unified approach. We emphasize that the primary goal of fine-tuning LLMs is not to surpass traditional methods in accuracy or efficiency but to demonstrate the concept of “no-code machine learning” made possible by language-interfaced fine-tuning, as it eliminates the need for modifications to the underlying architecture, loss function, and hyperparameter optimization.

4.2 Inverse design of all-dielectric metasurfaces

To demonstrate the capabilities for design problems, we fine-tuned a base LLM to produce the unit cell structure of an all-dielectric metasurface that possesses the target spectral transmittance. The essential idea is to reverse the input–output relationship, with the spectrum provided as a string input and the structural parameters generated as textual output. As illustrated in Figure 6a, the input defines the desired optical properties, for example: “The desired transmission spectrum of a metasurface range from 400 nm to 700 nm in increments of 10 nm are as follows: [0.968, 0.954, 0.917,…, 0.966, 0.966, 0.966].” Similarly, the corresponding output specifies the geometric parameters, such as “w = 194 nm, d = 168 nm, and h = 109 nm.” After fine-tuning the base LLM with the reversed dataset, the LLM produces text-based descriptions of the structural parameters required to achieve the target spectral response, which are subsequently used to determine specific configurations. The resulting structure is then simulated, and its spectral response is compared with the input target spectrum (Figure 6b).

Figure 6:

Language-interfaced fine-tuning for the inverse design of optical metasurfaces. (a) Workflow for fine-tuning process. The input text describes the desired properties, and the output provides the corresponding geometries. (b) Inference phase. The fine-tuned LLM takes new transmission as input and generates the corresponding metasurface structure. (c–e) Designs produced from independent runs, each with similar transmission but distinct parameter values. (f) The transmission of various structures generated by the LLM is compared against the target transmission.

A notable feature of LLMs is their ability to produce diverse outputs even for the same target objective. This flexibility is particularly useful in inverse design tasks, as it enables the discovery of new design possibilities and expands the range of potential solutions. Figure 6c–e showcases the results of multiple tests, highlighting the variety of solutions generated by the model.

The diversity in outputs can be primarily controlled and refined by adjusting two key hyperparameters: temperature and the top-p sampling threshold, which shape the probability distribution of the generated output tokens. The temperature governs the randomness of the responses to the model. Formally, the probability of a token being selected as the next token is defined as:

where z _i is the logit of token, p _i represents its selection probability, and τ is the temperature. Intuitively, increasing the temperature leads to more diverse and creative outputs, while decreasing it results in more deterministic generations. Top-p sampling, on the other hand, selects tokens from the smallest set whose cumulative probability exceeds a threshold p, ensuring that the generation focuses on the most likely tokens. We set the temperature to τ = 2.0 and p = 0.9 for the experiments. Metasurfaces were generated across four independent runs, with all resulting structures having distinct designs yet demonstrating similar spectral responses, as depicted in Figure 6b and f.

Our findings suggest that language-interfaced fine-tuning of LLMs is effective for both forward prediction and inverse design of metasurfaces. In both cases, only the number-to-text encoding and text-to-number decoding processes are required, with the core fine-tuning process remaining consistent and not demanding in-depth knowledge of AI models. Moreover, the model exhibits flexibility in handling diverse input representations, such as resonance wavelength and bandwidth (see Supplementary materials Figure S3). Consequently, this method lowers the barrier for researchers with limited expertise in AI or physics, enabling them to participate in data-driven nanophotonic design tasks with ease and efficiency.