Collision of high-resolution wide FOV metalens cameras and vision tasks

2.1 Design

In the early stages of metalens development, designs mainly primarily plasmonic or metallic materials. However, due to their significant energy loss, the focus gradually shifted towards all-dielectric metalens. Dielectric materials can achieve high transmission and low loss by overlapping electric and magnetic dipole resonances. By carefully selecting the unit cell structures to align their resonant modes, the Kerker effect can be achieved, enabling a full 2π phase coverage. Currently, commonly used materials in the visible spectrum include TiO₂, GaN, and Si₃N₄.

We have created a metalens that operates at 633 nm using silicon (Si) unit structures. These unit structures have an average transmittance of over 80 % and can achieve a phase coverage of 0-2π, allowing for precise control of wavefronts. In terms of material selection, we chose Si instead of the lower-loss TiO₂ for the following reason: we conducted a parameter scan on TiO₂ cylinders with periods ranging from 200 to 310 nm and heights from 200 nm to 500 nm but were unable to find a TiO₂ structure that could cover the 0-2π phase range. While increasing the height of TiO₂ might satisfy this phase condition, a higher aspect ratio would increase the fabrication difficulty with our current processing capabilities. We believe that the inability to achieve the 0-2π phase coverage is due to the relatively low refractive index of TiO₂ at the 633 nm wavelength. Therefore, we selected Si and successfully identified a suitable unit structure. As for the loss in Si, although Si has higher losses compared to TiO₂ at 633 nm, this loss is acceptable. Significant loss in Si typically occurs at wavelengths below 500 nm. The simulation data for the unit structures were obtained using the FDTD. The metalens were built with an eight-level phase design, where we selected unit structures that maintained high transmittance while covering the full 0-2π phase range. To avoid higher-order diffraction, the period of the unit structures was kept smaller than the wavelength, and to satisfy the sampling theorem, the period needed to meet the condition p < λ 2 NA . In practical fabrication, however, the period could not be too small due to manufacturing constraints. After considering these factors, we selected cylindrical unit structures with a period of 347 nm and a height of 378 nm, as shown in Figure 1c (transmittance and phase distribution of the selected eight unit structures are provided in Supplementary Material S1).

Figure 1:

Metalens camera structure and imaging experiments. (a) Schematic of the aperture-stop metalens. (The different colors of the light rays represent only the different incident angles and are unrelated to the wavelength. The corresponding incident angles are explicitly labeled in the figure.) (b) The fabricated metalens sample and its corresponding localized electron microscopy images. (c) Schematic diagram of the unit structure. (d) The imaging effect of metalenses on resolution charts (marked in red as the half field of view). (e) Resolution chart imaging experimental setup diagram.

In Figure 1b, you can see the metalens sample that has been fabricated, with a close-up SEM image of the metalens displayed on the right side. In the field of metalens design, quadratic metalens is a well-established approach for achieving wide FOV performance. However, this method often overlooks the Modulation Transfer Function (MTF), a critical metric in optical systems that gauges imaging quality. To address this issue, our metalens design not only focuses on wide FOV but also emphasizes achieving near-diffraction-limited MTF. Theoretically, this design can achieve near-diffraction-limited performance over a 100° FOV at a wavelength of 633 nm. The design process for the metalens primarily involves using ZEMAX to optimize the surface phase. The optimized phase distribution of the metalens is described by Equation (1), where a _i are the polynomial coefficients. The main objective of the optimization is to enhance the lens’s MTF.

The metalenes operate at a wavelength of 633 nm within the visible spectrum. Its structure consists of three key components: an aperture stop at the front, a glass substrate, and the metalens itself. The aperture stop allows for precise control of the metalens’s local regions in response to incident light at different angles, enabling control over light at varying incident angles. Significantly reducing the phase control burden compared to traditional single-layer metalens without an aperture stop. In optical systems, the effect of the aperture stop on image quality is complex. Simply reducing the aperture size does not necessarily enhance imaging quality; an excessively small aperture can increase diffraction effects, reduce light throughput, and block high-frequency information, ultimately degrading image quality. After careful optimization, an aperture stop size of 0.8 mm was selected for this work, with a lens radius of 2.25 mm and a focal length of 1.96 mm. The structural schematic is shown in Figure 1a.

The integrated metalens camera primarily consists of a metalens barrel and a sensor. The metalens barrel is manufactured using 3D printing technology, with a 0.8 mm aperture stop at the front. Figure 1d shows the imaging performance of the metalens camera on the resolution test chart. The dashed area in Figure 1e shows the detailed exploded view of the metalens camera. In the resolution test experiment, we used an LED with a central wavelength of 633 nm as the light source. To prevent overexposure, a diffuser was placed behind the resolution chart to act as a secondary light source. The integrated metalens camera was positioned 26 mm in front of the LBTEK RTS3AB-N resolution chart. Transmission test chart. The camera successfully resolved the Group 2, Element 5 stripes, which correspond to a line width of approximately 0.0787 mm, sufficient for most vision scenarios. It is worth noting that in the resolution tests, the effective half field of view of the metalens-integrated camera reached only 45°, slightly lower than the designed 50°. This discrepancy is primarily attributed to vignetting caused by the thickness of the aperture diaphragm. The 3D-printed diaphragm inherently has a thickness limitation, introducing an additional vignetting effect in the metalens system. To mitigate this issue, the thickness of the aperture diaphragm can be effectively reduced by coating a reflective film on the back surface of the metalens to serve as the diaphragm or by using a metal-based aperture diaphragm. Further details and data supporting this discussion can be found in Supplementary Material S5.

2.2 Strehl ratio and MTF

The Strehl ratio is commonly used to assess the optical imaging quality of a system. It is defined as the ratio of the peak intensity of the system’s point spread function (PSF) to the peak intensity of an ideal, aberration-free PSF. In imaging lens design, a Strehl ratio of one indicates perfect imaging. However, this ideal scenario is nearly impossible to achieve in practice. During the lens design process, an optical element is generally considered diffraction-limited if its Strehl ratio exceeds 0.8. In this study, the proposed near-diffraction-limited wide-field metalens features a theoretical half FOV of 50°, which corresponds to a full FOV of 100°. The optimized metalenses achieved an average Strehl ratio of 0.75 over an incidence angle range of 0–50°, just 0.05 below the diffraction-limited standard of 0.8, supporting its classification as a near-diffraction-limited optical system. Detailed data are presented in Figure 2(a). In contrast, conventional quadratic metalens have an average Strehl ratio of only approximately 0.35 over the same incidence angle range. The optimized metalens, not only correct off-axis coma but also minimizes on-axis spherical aberration, resulting in significantly improved imaging performance.

Figure 2:

Simulation and experimental results of metalens performance. (a–c) Simulated Strehl ratio and simulated MTF in the meridional and sagittal planes for the optimized metalens. (d) Experimentally measured MTF data in the sagittal plane for the optimized metalens. (e) Experimentally measured MTF data in the meridional plane for the optimized metalens.

The MTF is a crucial parameter for evaluating the imaging quality of an optical system because it quantifies the system’s ability to transmit different spatial frequencies. To validate the MTF data for the optimized metalens, we performed a discrete Fourier transform of the PSF from the metalens optical system during the experiment to obtain the MTF data. Figure 2d shows the experimentally measured MTF in the sagittal direction for the optimized metalens, while Figure 2e shows the MTF measured in the meridional direction. The experimental MTF closely aligned with the theoretically predicted MTF, with minor discrepancies likely attributable to several factors:

1. Light Source Bandwidth: Although the metalens system was designed to operate at a monochromatic of 633 nm, the laser used in the testing apparatus inherently has a certain bandwidth. This spectral width can introduce errors into the measurement system.

2. Focal Spot Magnification System: The focal spot produced by the metalens optical system is comparable in size to the detector’s pixel size, typically on the order of micrometers. Without a magnification system in the focal plane of the metalens, observing the entire morphology of the focal spot using a detector. However, as the magnification system is not a perfect imaging system, some frequency components are lost when transmitting the PSF of the metalens system, leading to a degraded MTF measurement.

3. Electronic Noise: Noise in the electronic equipment introduces random intensity fluctuations in the captured image, which can obscure or distort the true signal.

These factors collectively contribute to the slight deviations observed between the measured and predicted MTFs.

2.3 Wavefront

We further examined the imaging performance of the optimized metalens from the perspective of the emergent wavefronts. The overall diameter of the metalens was 4.5 mm, with an aperture stop of 0.8 mm at the front. In a wide-field imaging system without an aperture stop, it is challenging to maintain consistency between the entire emergent wavefront and the ideal spherical wavefront. However, by introducing an aperture stop, it is possible to achieve near-ideal spherical wavefronts in localized regions.

In optical design, it is generally accepted that the closer the emergent wavefront is to an ideal spherical wavefront, the better is imaging quality of the lens. Based on this concept, we defined a wavefront error function (see Supplementary Material S2) and simulated the near-field wavefront and intensity distribution of the metalens when the aperture stop is placed in close contact with it, as shown in Figure 3a.

Figure 3:

Metalens wavefront calculation. (a) Two-dimensional wavefront and intensity distribution in the near-field region. (b) One-dimensional wavefront phase and wavefront error along the y-axis. The solid blue line represents the emergent wavefront of the optimized metalens, while the short blue dashed line represents the ideal wavefront. The solid red line shows the wavefront error of the optimized metalens relative to the ideal wavefront. (c) Solid lines represent the average wavefront error in the x or y direction for different incident angles, while dashed lines represent the overall average wavefront error in the x or y direction across all angles. For instance, the overall average value of the green solid line corresponds to the value of the green dashed line.

Figure 3a shows the wavefront phase and intensity along the y-axis for incident angles ranging from 0° to 45° (as the angle increased along the y-axis, as indicated in Figure 1a). Figure 3b presents a 2D view of the wavefront phase and intensity. As the incident angle increases, the primary energy distribution in the near-field region shifts upward along the y-axis. However, the primary energy of the emergent wavefront remains concentrated within the 0.8 mm region, corresponding to the size of the aperture stop. Thus, within the main energy distribution area, the emergent wavefront of the optimized phase metalens is largely consistent with the ideal spherical wave, demonstrating the excellent imaging performance of the optimized metalens.

Subsequently, we calculated the wavefront error relative to the ideal wavefront for the optimized metalens at incident angles ranging from 0° to 45°(Figure 3c).

2.4 PSF

The imaging process of an optical system can be considered as the integration of an ideal, aberration-free image with the optical system’s PSF. Therefore, the PSF directly reflects the imaging performance of the system. As a comprehensive evaluation metric, As a comprehensive evaluation metric, observing the PSF shape provides a basic assessment of the optical system’s quality. For a wide-field optical system that is free of off-axis aberrations, the PSF should ideally maintain a consistent circular shape across different field angles. Figure 4b, shows the simulated PSFs for the optimized phase designs, and the experimentally measured PSF for the optimized metalens, all under incident angles ranging from 0° to 45°. To enhance clarity when plotting the simulated PSFs, we excluded regions with energy intensity below 0.1 times the peak value. The complete PSF measured in the experiment can be found in Supplementary Material S7. The optimized phase design exhibits almost no noticeable sidelobes. In most large FOV metalenses, strong sidelobes often occur across the field due to on-axis spherical aberrations introduced while compensating for off-axis aberrations. This arises when the lens edges fail to provide a sufficient phase gradient, causing light from various regions to converge at different focal planes. By introducing an aperture stop, we can effectively truncate marginal rays carrying significant aberrations from the lens edges. This approach reduces the on-axis spherical aberration and, improves overall optical performance.

Figure 4:

PSF simulation and testingpsfsimula. (a) Schematic of the setup for measuring the PSF of the metalens. (b) Simulated and experimentally measured PSFs for the optimized metalens under incident angles ranging from 0° to 50°.

In subsequent experiments, we measured the PSF of the optimized metalens, and the results largely aligned with the theoretical predictions. During the measurement process, accurately aligning the aperture stop with the metalens proved challenging. To address this, we used a 0.8 mm aperture stop in the light source system to produce a parallel beam with a 0.8 mm diameter, simulating the ideal aperture stop condition. This method eliminated alignment errors but introduced a diffraction distance between the aperture stop and the metalens. At larger incident angles, this diffraction distance caused the beam width to expand, therefore, the light spot on the rear surface of the metalens changed from circular to elliptical (dashed areas in Figure 4a), which slightly degraded the metalens’ measured data. The experimental results showed that this beam expansion had minimal impact on the metalens’s performance. This indicates that our optimized metalens is tolerant to errors in the aperture size., and this will not affect the imaging quality of the integrated metalens camera. In such a camera, the aperture stop is housed within the lens barrel, with virtually no distance between the aperture stop and metalens.

The experimental setup for measuring the optimized metalens (Figure 4a), primarily consisted of a 633 nm laser, beam expander system, metalens, magnification system, 0.8 mm aperture stop, and camera. The laser, beam expander system, and 0.8 mm aperture stop were mounted on a rail, allowing the entire assembly to move along the track to measure the metalens data at different incident angles (Supplementary Material S3 for details of the actual experimental equipment). The dashed circles in Figure 4a highlight the beam broadening effect caused by the distance between the aperture stop and metalens.

2.5 Vision experiments

In the vision experiments, the captured images still exhibited some blurring at the edges, mainly due to the misalignment of the aperture stop and the vignetting caused by the thickness of the aperture stop. Since a mechanical aperture was used, placing the metalens substrate into the 3D-printed housing may result in a misalignment of the aperture stop with the metalens’ center, caused by unevenness in the housing or deviations in the printing process. This misalignment can cause the main energy region of the outgoing wavefront to fall on the unoptimized areas of the metalens, leading to significant aberrations (a discussion on aperture misalignment can be found in Supplementary Material S4, and a discussion on vignetting caused by the thickness of the aperture stop is included in Supplementary Material S5). This issue can be effectively resolved by coating the opposite side of the metalens substrate with a reflective film to replace the mechanical aperture stop. Reflective films typically have negligible thickness compared to mechanical aperture stops, which helps reduce vignetting effects, and aligning the coating area with the metalens during fabrication is easier, ensuring clearer imaging performance at the edges. Overall, due to assembly limitations, the metalenses camera was unable to fully showcase the optimized metalens’ complete imaging performance.

First, we tested the scanning capabilities of the integrated metalens camera by printing QR codes of different sizes, as shown in Figure 5a–d. We encoded and “National University of Defense Technology” into barcodes and QR codes, placing the QR codes 12 cm away from the integrated metalens camera for imaging experiments. To address the image distortion, we calibrated the metalens using a checkerboard pattern to obtain the intrinsic parameters and distortion coefficients of the integrated metalens camera, which were then used for distortion correction (details of the calibration process can be found in Supplementary Material S6). The experimental results demonstrated that the integrated metalens camera successfully decoded both barcodes and QR codes of various sizes.

Figure 5:

Target recognition and detection. (a–d) QR code and barcode scanning experiments were conducted using the integrated metalens camera, with barcode sizes indicated in black text. The barcode areas identified by the program are highlighted in green boxes, and the decoded content is displayed in red text below. (e–h) Pose recognition experiment. Human body poses are represented using colored line diagrams. (i–l) Object recognition experiment. Different objects are enclosed in boxes of various colors, with the object names labeled above each box.

Subsequently, we conducted pose recognition experiments using the metalens camera. Pose estimation is a task that involves identifying the locations of specific points in an image, which are commonly referred to as key points. These key points represent various parts of an object. We performed pose recognition using the YOLO11 model, which was specifically trained on the COCO8-pose dataset to satisfy the demands of pose estimation tasks. We stood at a distance of over 3 m from the metalens camera, posed it in different positions, and took pictures with the metalens camera. These images were then fed into a deep-learning model for pose recognition. Figure 5e–h display the results of the pose recognition, which indicate that the metalens pose recognition system effectively identified the key points on the human body and accurately constructed the human pose from these key points.

Finally, we demonstrated the metalens classification recognition system, which also utilized the YOLO11 framework and was trained on the COCO dataset to address classification tasks. We positioned the metalens classification recognition system approximately 1 m away from the scene to capture images of objects on a tabletop. The imaging results of the metalens system are shown in Figure 5i and k, with the corresponding recognition outcomes displayed in Figure 5j and l. When the lighting conditions are poor, resulting in shadows on the objects, the deep learning model exhibits some hesitation during recognition and classification, as seen with the cup and mouse in Figure 5i. However, adjusting the light source to minimize shadows significantly improves the model’s discrimination accuracy. This experiment highlights a key area for optimization in current deep learning frameworks: enhancing the model’s adaptability to variations in lighting, such as exposure changes or shadows, in visual tasks.

In summary, obtaining high-quality images of a scene is a fundamental prerequisite for performing visual tasks. Emerging metalenses can effectively provide high-quality images for computational vision frameworks. This study connects the metalens camera system with computer vision, showcasing its potential as a next-generation imaging technology while also driving computer vision toward broader applications. These two systems complement each other synergistically. The YOLO framework is a classic AI architecture that applies to general imaging systems. However, in certain specialized devices, the presence of metalenses becomes particularly important. For example, in miniature devices such as micro-robots, which are currently receiving significant attention, traditional imaging systems require bulky lens structures to achieve good imaging quality. In this study, the metalenses that actively modulated the optical wavefront had a volume of only 0.04 cm³ and a thickness of 0.25 cm. Traditional lenses of the same size are generally unable to achieve the imaging performances presented in this study. We demonstrated that such a thin and lightweight lens can provide high-quality images required for computational imaging. This study provides a promising solution for rapidly developing miniaturized intelligent imaging devices and counters some of the recent doubts surrounding metalenses.