Startseite High-resolution non-line-of-sight imaging based on liquid crystal planar optical elements
Artikel Open Access

High-resolution non-line-of-sight imaging based on liquid crystal planar optical elements

  • Zhibin Zhao , Qi Zhang , Xiaoyin Li , Yinghui Guo , Mingbo Pu , Fei Zhang , Hengshuo Guo , Zewei Wang , Yulong Fan , Mingfeng Xu und Xiangang Luo ORCID logo EMAIL logo
Veröffentlicht/Copyright: 10. Januar 2024
Artikel downloaden (PDF)
Veröffentlichen auch Sie bei De Gruyter Brill
Manuskript einreichen Informationen für Autor*innen
Nanophotonics
Aus der Zeitschrift Nanophotonics Band 13 Heft 12

Abstract

Non-line-of-sight (NLOS) imaging aims at recovering hidden objects located beyond the traditional line of sight, with potential applications in areas such as security monitoring, search and rescue, and autonomous driving. Conventionally, NLOS imaging requires raster scanning of laser pulses and collecting the reflected photons from a relay wall. High-time-resolution detectors obtain the flight time of photons undergoing multiple scattering for image reconstruction. Expanding the scanning area while maintaining the sampling rate is an effective method to enhance the resolution of NLOS imaging, where an angle magnification system is commonly adopted. Compared to traditional optical components, planar optical elements such as liquid crystal, offer the advantages of high efficiency, lightweight, low cost, and ease of processing. By introducing liquid crystal with angle magnification capabilities into the NLOS imaging system, we successfully designed a large field-of-view high-resolution system for a wide scanning area and high-quality image reconstruction. Furthermore, in order to reduce the long data acquisition time, a sparse scanning method capitalizing on the correlation between measurement data to reduce the number of sampling points is thus proposed. Both the simulation and experiment results demonstrate a >20 % reduction in data acquisition time while maintaining the exact resolution.

Keywords: metasurface; liquid crystal planar optical element; non-line-of-sight imaging

1 Introduction

Non-line-of-sight imaging (NLOS) aims to reconstruct three-dimensional (3D) objects obscured from the detector’s direct line of sight. This technique has attracted considerable attention due to its promising applications in diverse fields, including autonomous driving, robotic vision, rescue operations, remote sensing, and the medical sector. The current NLOS imaging methods mainly include photon time-of-flight imaging [1]–[12], coherent imaging [ , intensity imaging [19], [20], [21], speckle imaging [22], [23], wavefront shaping [17], [24], and deep learning [25], [26], [27]. Photon time-of-flight imaging is the most widely used approach, owing to its exceptional imaging quality and precision. However, its resolution is primarily limited by the scanning area on the relay wall rather than the numerical aperture of the collecting optical system. The scanning area on the relay wall can be considered the virtual aperture of the NLOS imaging system, which determines the amount of detectable photon time-of-flight data. Expanding the scanning area is a crucial way to improve the resolution of NLOS imaging. However, current scanning systems in NLOS imaging are typically limited by maximum scanning angle, preventing further expansion of the scanning area. Introducing a beam steering system composed of traditional optical elements in front of the scanning system is a common method to expand the scanning area, but the bulky size of traditional optical components restricts further applications of NLOS imaging.

Recently, metasurface has received extensive research interest due to its high integration, compact size, and multifunctionality [ . Metasurface can manipulate the amplitude, phase, and polarization of incident waves by tailoring each meta-atom’s morphology [ . Moreover, it has demonstrated its fast advancement in many fields, such as vortex recognition [42], optical encryption [38], and wavefront shaping [43]. In addition, metasurfaces also perform excellently in controlling diffraction [44] and enhancing photon upconversion [45]. Similar to metasurfaces, which impart abrupt phase shifts to incident waves at the interface, liquid crystal (LC) molecules with spatially varying orientations can also form desired phase profiles by exploiting the Pancharatnam–Berry (PB) geometric phase [ . Furthermore, LCs can be electrically controlled to achieve reconfigurability [56]. This characteristic makes LCs an ideal material for various applications across different domains, including display technology [57], optical modulation [58], and antenna design [59]. In contrast to the complex lithographic manufacturing process of metasurfaces, the fabrication of LC planar optical elements (POEs) only involves coating or filling liquid crystal materials onto a substrate for preparation [60]. Combining two LC-POEs to construct a miniature telescope, an efficient, wide-angle, and high-precision beam deflection has been achieved [61]. Inspired by these advancements, it is anticipated that integrating an electrically tunable LC beam steering device into NLOS imaging can result in a larger scanning area.

In this work, we have devised a large scanning range system consisting of a confocal NLOS imaging system and an angle magnification system. By constructing a telescope module with two cascaded LC POEs, we can achieve high optical efficiency, minimal wavefront distortion, and angle magnification independent of the incident spatial position. In the large scanning range system, the peak signal-to-noise ratio (PSNR) of the reconstructed results can be increased by about 5 dB, and the root mean square error (RMSE) can be reduced by approximately 15 %. However, achieving high-resolution reconstruction via the large scanning range system requires the collection of dense grid measurements, significantly elongating the data acquisition time. For further optimization, we proposed a novel sparse scanning method based on spatial correlation to reduce the redundant measurements. This distinctive scanning approach leverages the spatial correlation [62], [63] of measured data between adjacent scanning positions. By comparing the spatial correlation of measurement data from neighboring points across the entire scanning area, we found that the spatial correlation in the central region is higher than that in the edge region. Consequently, sparse scanning can be adopted for the center area of the relay wall. In simulation, our method achieves a 1.5-fold increase in reconstruction resolution while saving up to 26 % of the scanning time. In experiments, our method can increase the PSNR of the reconstructed image by approximately 5 dB and reduce the RMSE of the reconstructed image by 15 % while saving up to 23 % of the scanning time.

2 Results and discussion

2.1 Impact of scanning area and sampling rate on the lateral resolution

An angle-constrained NLOS imaging system, as shown in Figure 1a, records the time-of-flight of scattered photons containing information about hidden objects by scanning the relay wall in a dense grid pattern. Through reconstruction algorithms, the shape and position of the hidden objects can be recovered. In contrast to the angle-constrained NLOS system, the large scanning range system scans a broader area on the relay wall, as shown in Figure 1b, which allows the detector to capture a greater amount of scattered light, signifying a more extensive recording of photon time-of-flight containing valuable object information. Consequently, the reconstruction results obtained from the large scanning range system are smoother, with less background noise. Furthermore, in the large scanning range system, a broader scanning area supports the detector to resolve the time-of-flight of more photons, thus achieving higher reconstructed lateral resolution. Specifically, in a non-confocal system, the minimum resolvable spacing Δd is

(1) Δ d ≥ w 2 + z 2 c Δ t − δ x w

where 2w is the scanning width or height of the visible wall, c is the light speed, z is the distance between the hidden object and the relay wall, and δx is the path difference of the hidden object from the detection point on the relay wall (see Supplementary Material Note S1 for details). In a non-confocal system, the detector is typically aimed at the center position of the wall, so that δx takes a very small value. In contrast, in the confocal system, the minimum resolvable distance can be expressed as:

(2) Δ d ≥ c Δ t w 2 + z 2 2 w

Figure 1: Overview of large scanning range NLOS imaging scene. (a) Angle-constrained NLOS imaging scene. (b) Large scanning range NLOS imaging scene. (c) Sparse scanning NLOS imaging scene.
Figure 1:

Overview of large scanning range NLOS imaging scene. (a) Angle-constrained NLOS imaging scene. (b) Large scanning range NLOS imaging scene. (c) Sparse scanning NLOS imaging scene.

According to Equations (1) and (2), when the hidden object remains static, the minimum resolution of the reconstructed image is related to both the scanning area and the minimum resolution of the detection system. Therefore, increasing the scanning area is an important way to improve the resolution of the reconstructed image. However, while enlarging the scanning area, the scanning interval also increases. According to the Nyquist sampling theorem, the maximum scanning interval Δx is

(3) Δ x < Δ d 2

where Δd represents the minimum distance defined in Equations (1) and (2). Equation (3) indicates that the maximum scanning interval should be less than half of the theoretical minimum reconstruction resolution. In the Supplementary Material Note S2, we specifically discuss the impact on the reconstruction results under different undersampling conditions through simulations.

Equations (1)–(3) show that achieving high lateral resolution in NLOS imaging requires large and dense scanning of the relay wall surface. Each scanning position requires a certain exposure time for the detector to collect measurement data, which significantly reduces the speed of NLOS imaging. To alleviate this problem, we adopt sparse scanning into the large scanning range imaging system to accelerate the scanning speed with high spatial resolution due to the high spatial correlation among the measured positions, as revealed in Figure 1c. The lateral resolution of the reconstructed image from the sparse scanning is slightly lower as compared to the large scanning range system but far better than that of the angle-constrained system. Utilizing sparse scanning in the large scanning range system allows for a significant reduction in scanning time at the cost of sacrificing minimal reconstruction resolution.

2.2 Liquid crystal planar optical elements for angle magnification

In the large scanning range system, the POEs are responsible for achieving a larger scanning angle by steering the output beam through tailoring its geometric phase profile. We choose POEs based on LC materials, whose anisotropy allows to alter the alignment direction of LC molecules to create an LC geometric phase. Assume that the local optical axis of the LC molecule has an orientation angle of φ with respect to the x-axis. The Jones matrix can be represented as:

(4) J = M e − i Γ 2 0 0 e i Γ 2 M − 1 = cos ⁡ φ − sin ⁡ φ sin ⁡ φ cos ⁡ φ e − i Γ 2 0 0 e i Γ 2 cos ⁡ φ sin ⁡ φ − sin ⁡ φ cos ⁡ φ

where M is the rotation matrix, Γ is the LC phase retardation. For a circularly polarized incident light E i = 1 i σ , the output light can be calculated as:

(5) E o = J ⋅ E i = 1 2 1 i σ cos Γ 2 − i 2 1 − i σ sin Γ 2 e i 2 σ φ

where σ = ±1, corresponding to left-circularly polarized and right-circularly polarized light, respectively. The second term on the right side of Equation (5) indicates that the E i is converted to the orthogonal polarization carrying an additional geometric phase term e i2σφ . The geometric phase term indicates that a continuous phase variation from 0 to 2π can be achieved by rotating the LC molecules from 0 to π. Furthermore, the efficiency of the output light modulated by the geometric phase is sin2(Γ/2).

Here, we use these two POEs to develop a planar telescope, as schematically shown in Figure 2a. In the optical system, the POE I acts as an objective to converge the incident beam with a focal length of f 1. Then, the incident beam propagates through POE II, which functions as an eyepiece with a focal length of f 2 and ultimately delivers a collimated beam. When the incident beam with an oblique angle of θ 1 passes through the cascaded POE with an increased deflection angle of θ 2, the magnification factor can then be defined as:

(6) M = θ 2 θ 1 ≈ − f 1 f 2

Figure 2: The work principle and test results of the cascaded POEs. (a) Conceptual scheme of the cascaded POE for amplification of beam steering. (b) Phase profiles of the POE I and POE II. The upper image is POE I, and the lower image is POE II. (c) Samples and polarized optical microscopic images of POE I and POE II. The upper image is POE I, and the lower image is POE II. (d) Schematic of the measurement setup. (e) Performance of the deflection angle. (f) Performance of the angular efficiency.
Figure 2:

The work principle and test results of the cascaded POEs. (a) Conceptual scheme of the cascaded POE for amplification of beam steering. (b) Phase profiles of the POE I and POE II. The upper image is POE I, and the lower image is POE II. (c) Samples and polarized optical microscopic images of POE I and POE II. The upper image is POE I, and the lower image is POE II. (d) Schematic of the measurement setup. (e) Performance of the deflection angle. (f) Performance of the angular efficiency.

Therefore, the cascaded POE based on the Galileo telescope system is implemented for beam steering. Commercial software (Zemax OpticStudio) is used to optimize the phase profiles. The simulation is performed with a 532 nm monochromatic light source. The distance between the two POEs is 38 mm. The optimized design is obtained by minimizing the size of the output beam while maintaining the correct emission angle. The phase profiles of POEs are obtained as follows:

(7) ϕ r = α r R 2 + β r R 4 + γ r R 6

where r is the radial coordinate, R is the radius of POEs (2.895 mm for POE I and 4.90055 mm for POE II), and α, β, and γ are constant coefficients. The optimized phase profiles are shown in Figure 2b. The color bar represents phase values ϕ.

The fabricated POEs are shown in Figure 2c, 0.3 mm above the sample. The f-numbers of POE I and POE II are 13.1 and 3.9, respectively. The polarized optical microscopic images of the POEs are shown on the right side of each POE. To evaluate the performance of each POE, we construct a measurement setup, as shown in Figure 2d. A 532 nm linearly polarized laser beam is transformed into circular polarization using a quarter-wave plate. Then, it is incident onto the cascaded telescope composed of POEs through the reflective mirror. By rotating the reflecting mirror, the incident angle can be altered. Figure 2e and f illustrate the steering efficiency and the output angle at different incident angles, respectively. By fitting the output angle and incident angle relationship with the linear function, the line yields a slope of 1.95, which agrees well with the designed magnification factor of 2. Furthermore, the steering efficiency stays above 80 % at a wide angle.

2.3 Experimental results

The detection system is specifically depicted in Figure 3a, where the pulsed laser passes through a beam splitter and a galvanometer for scanning on the relay wall. A cascaded POE forms an angle magnification, enlarging the deflection angle of the beam from the galvanometer. Our system employing POEs is referred to as a large scanning range system. The system without POEs is referred to as an angle-constrained system. After several times of reflection, scattered light from the relay wall passes through the POEs, scanning galvanometer and beam splitter. Finally, they are collected by the single-photon avalanche diode (SPAD). Because the LC-POE elements operate in the geometric phase principle, a quarter-wave plate is added between the laser and the beam splitter to convert linearly polarized light into circularly polarized light.

Figure 3: Reconstruction results of the letter “E”. (a) Experimental setup of large scanning range system. SG: scanning galvanometer, BS: beam splitter, QWP: quarter-wave plate, SPAD: single-photon avalanche detector. (b–d) Experimental results (b) the PSNR (c) and (d) the RMSE of the angle-constrained system and the large scanning range system under different reconstruction algorithms.
Figure 3:

Reconstruction results of the letter “E”. (a) Experimental setup of large scanning range system. SG: scanning galvanometer, BS: beam splitter, QWP: quarter-wave plate, SPAD: single-photon avalanche detector. (b–d) Experimental results (b) the PSNR (c) and (d) the RMSE of the angle-constrained system and the large scanning range system under different reconstruction algorithms.

We demonstrate the effectiveness of the proposed method with experiments on the same object using the angle-constrained NLOS system and our large scanning range system, respectively. A 532 nm pulsed laser with a 24 MHz repetition rate and an average power of 60 mW is used as a light source. After multiple scatterings, the scattered light is recorded by a SPAD with a time jitter of 60 ps. The detection signal is subsequently recorded by a time-correlated single photon counting (TCSPC) module with a time resolution of 4 ps. It is worth noting that both the angle-constrained system and the large scanning range system utilize the same pulse laser, SPAD, and TCSPC. Afterward, we reconstruct the images from the recorded signals collected by the two systems using the light-cone transform (LCT) [1], filtered backprojection (FBP) [2], and phasor field (PF) [7] algorithms, respectively. We then compare the results obtained from each algorithm for both sets of data.

For confocal experiments, we choose the letter “E” to test our method. The measured data are captured at 101 × 101 focal points over a square region of 0.86 × 0.86 m2 in the large scanning range system. The hidden scenes are 0.5 m from the illumination wall. To establish a control group under the same experimental conditions, we also capture another set of measurement data using the angle-constrained NLOS system, where the scanning square region is 0.43 × 0.43 m2 due to the absence of POEs. Throughout all experiments, the exposure time is 17 min.

The reconstructed results of the letter “E” are displayed in Figure 3b. The maximum intensity projections along the depth direction are represented using the hot colormap. Due to the non-uniformity of the coordinate systems between the reconstructed images obtained from the two systems, we purposely magnify the results from the large scanning range system for easier comparison. In the angle-constrained NLOS imaging system, the upper part of the letter “E” fails to be reconstructed due to the lack of measurement data. In the large scanning range system, the letter “E” can be fully reconstructed, and the results show a significant improvement as compared to those from the angle-constrained system. Regarding the quantitative evaluation, the PSNR of our reconstructed results for the letter “E” are 15.6219 dB, 15.9858 dB, and 16.1009 dB under the LCT, FBP, and PF algorithms, respectively. These values are significantly higher than those obtained by the angle-constrained NLOS system using LCT (9.5535 dB), FBP (8.8779 dB), and PF (10.3791 dB) algorithms. The RMSE of our reconstruction results for the letter “E” are 01724, 0.1659, and 0.1634 under the LCT, FBP, and PF algorithms, respectively. These values are notably smaller than those obtained by the angle-constrained NLOS system using LCT (0.3466), FBP (0.3747), and PF (0.3152) algorithms. The evaluation metrics for the letter “E” are presented in the bar charts. From the bar charts of PSNR and RMSE evaluation metrics in Figure 3c and d, it is evident that the reconstruction results of our system are superior to those of the angle-constrained system. Furthermore, we performed NLOS imaging on more complex three-dimensional objects (see Supplementary Material, Figure S3).

2.4 Sparse scanning based on spatial correlation

To achieve high-resolution reconstruction, sufficient scanning points over an expanding area are necessary, which greatly impacts the data collection time for NLOS imaging. To overcome this problem, we employ the sparse scanning method illustrated in Figure 1c. This scanning method is based on the correlation of measured data between adjacent sampling points at the same interval on the relay wall. In the Cartesian coordinate system, the position of the laser scanning on the relay wall is given by (x, y, 0), and the coordinates of the hidden object are represented as (x 1, y 1, z 1). The measured signal is given by:

(8) φ x , y , t = ∭ Ω 1 r 4 g x 1 , y 1 , z 1 δ × 2 x − x 1 2 + y − y 1 2 + z 1 2 − t c × d x 1 d y 1 d z 1

where c is the speed of light, δ is the albedo of the hidden object at point (x 1, y 1, z 1) in the 3D half-space Ω, and r is the distance function r = ( x − x 1 ) 2 + ( y − y 1 ) 2 + z 1 2 . The obtained data from two adjacent points, as shown in Figure 4a, can be represented as φ(x, y, t) and φ(x + Δx, y, t). The Pearson correlation function d(x) representing the correlation between adjacent samples is

(9) d x = ∑ t φ x , y , t − φ x , y , t ̄ φ x + Δ x , y , t − φ x + Δ x , y , t ̄ ∑ t φ x , y , t − φ x , y , t ̄ 2 ∑ t φ x + Δ x , y , t − φ x + Δ x , y , t ̄ 2

where φ x , y , t ̄ = ∑ t φ x , y , t N is the average value, and N is the number of time bins. Due to the complexity of Equation (9), we simulate the results using MATLAB software. In the simulation, a letter “E” with dimensions of 0.3 m × 0.3 m is placed at a distance of 0.5 m in front of a 1.2 m × 1.2 m relay surface. The measured data are obtained using a 128 × 128 grid scanning on the relay surface in a confocal NLOS system. Figure 4b shows the correlation function values of the measurements for adjacent samples compared along the x-axis. Multiple curves represent measurements taken at different sampling intervals. As the sampling interval increases, the overall correlation of the measurements decreases. However, the spatial correlation coefficients in the middle portion of all curves are relatively high. It implies that reducing the sampling in the middle portion has the least impact on the result. We use the measurements scanned at the central position as a reference to discuss the impact of sampling intervals on the correlation coefficients of measurement data. We show the evolution of the correlation coefficient on the sampling intervals in Figure 4c. The scanning interval occupying ∼7 % of the overall scanning area’s side length can be used for sparse scanning while maintaining the spatial correlation of over 80 %.

Figure 4: Sparse scanning principle. (a) Measurement data for scanning adjacent points. (b) Correlation function curve at y = 0. The five curves represent different intervals. (c) Correlation value of different intervals. (d) PSNR result. (e) SSIM result. M represents the ratio of the sparse scanning area width to the overall scanning area width, and k represents the sparse scanning interval.
Figure 4:

Sparse scanning principle. (a) Measurement data for scanning adjacent points. (b) Correlation function curve at y = 0. The five curves represent different intervals. (c) Correlation value of different intervals. (d) PSNR result. (e) SSIM result. M represents the ratio of the sparse scanning area width to the overall scanning area width, and k represents the sparse scanning interval.

Currently, mainstream reconstruction algorithms cannot directly compute the data obtained from sparse scanning and further data processing is required. The primary approach for data processing involves transforming the sparse measurement data into dense grid measurement data using a method like bicubic interpolation. After interpolation, the measurement data from sparse scanning can be transformed into dense grid scanning measurement data. In addition, we also conduct a detailed discussion on the sparse region area and the sparse scanning interval. Figure 4d and e represent the PSNR and the Structure Similarity Index Measure (SSIM) of the sparse scanning results compared to the uniform grid scanning results. The image reconstruction quality decreases as the sparse scanning area and sparse scanning interval increase. However, we consider the sparse scanning to yield satisfactory results when the sparse scanning area is less than 25 % and the scanning interval is less than 6 pixels.

Then, we adopt the sparse scanning approach in our large scanning range system. In this scanning, the total scanning area is 0.86 × 0.86 m2 while 0.43 × 0.43 m2 in the center is scanned using larger sampling intervals. Suppose we define the ratio of the intermediate sampling interval to the edge sampling interval as a coefficient k. As k is related to the scanning time, in our experiment with an edge scanning sampling interval of 0.86 cm, the results of different values of k, i.e. 1, 2, and 4 are investigated.

Figure 5a–c depicts the reconstruction results under sparse scanning using the LCT algorithm for all outcomes. We also provide the reconstruction results for the FBP and PF algorithm in the Supplementary Material Note S4. Visually, as the central scanning interval increases, there is an increase in reconstruction noise. However, the letter “E” and letter “I” can still be fully reconstructed using the LCT. Consequently, we achieve significant savings in scanning time while maintaining the sampling rate by slightly compromising the image quality. When the scanning coefficient k is 2 and 4, it can save 185 s and 234.4 s, respectively. To eliminate background noise, we apply a block-matching and 3D filtering denoising process for the image reconstruction. In Figure 5a–c, it is evident that increasing the value of k leads to minor differences in specific details, while significantly reducing the scanning time. Figure 5c displays the results of experiments on the resolution chart. The results indicate a decrease in the reconstruction resolution with the increase of the value of k. We can easily find that when the coefficient k is 2, the best resolution of the reconstructed image is 3 cm, consistent with the uniform scanning. However, when the coefficient k is increased to 4, in the reconstruction results, it is barely possible to distinguish the resolution bars of 4 cm. Moreover, the PSNR and RMSE are used to quantitatively evaluate the reconstruction results under different sparse scanning intervals, as shown in Figure 5d and e. As the sampling interval increases, the evaluation metrics decrease but always remain within an acceptable range.

Figure 5: Reconstructed measurement data under sparse scanning. (a–c) Reconstruction results of the letter “E” (a), the letter I (b), and the resolution chart (c). (d–e) PSNR (d) and RMSE (e) for different values of k.
Figure 5:

Reconstructed measurement data under sparse scanning. (a–c) Reconstruction results of the letter “E” (a), the letter I (b), and the resolution chart (c). (d–e) PSNR (d) and RMSE (e) for different values of k.

3 Conclusions

In summary, we propose a method that combines sparse scanning with an expanding scanning area to achieve fast, high-resolution NLOS imaging. To this end, we introduce an efficient, compact angle magnification device made of LC-POEs into the NLOS imaging system. Experimental results demonstrate that our designed NLOS system can enhance the image quality and imaging resolution of NLOS imaging. Through sparse scanning, we effectively shorten the data acquisition time by reducing the sampling points in the area with high correlation. We believe that combining sparse scanning with an expanding scanning area can bring a novel perspective to the development of NLOS imaging, extending beyond the limitations of improving reconstruction algorithms and hardware performance. Finally, our method also introduces new insight into real-time NLOS imaging.

Corresponding author: Xiangang Luo, National Key Laboratory of Optical Filed Manipulation Science and Technology, Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China; State Key Laboratory of Optical Technologies on Nano-Fabrication and Micro-Engineering, Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China; and School of Optoelectronics, University of Chinese Academy of Sciences, Beijing 100049, China, E-mail:

Zhibin Zhao and Qi Zhang contributed equally to this work.

Funding source: National Natural Science Foundation of China

Award Identifier / Grant number: 62222513

Award Identifier / Grant number: U20A20217

Funding source: National Key Research and Development Program of China

Award Identifier / Grant number: 2021YFA1401003

  1. Research funding: This work was supported by the National Key Research and Development Program of China (2021YFA1401003); the National Natural Science Foundation of China (NSFC) (62222513, U20A20217).

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Conflict of interest: Authors state no conflicts of interest.

  4. Informed consent: Informed consent was obtained from all individuals included in this study.

  5. Ethical approval: The conducted research is not related to either human or animals use.

  6. Data availability: The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

[1] M. O’Toole, D. B. Lindell, and G. Wetzstein, “Confocal non-line-of-sight imaging based on the light-cone transform,” Nature, vol. 555, no. 7696, pp. 338–341, 2018. https://doi.org/10.1038/nature25489.Suche in Google Scholar PubMed

[2] M. Laurenzis and A. Velten, “Nonline-of-sight laser gated viewing of scattered photons,” Opt. Eng., vol. 53, no. 2, p. 023102, 2014. https://doi.org/10.1117/1.oe.53.2.023102.Suche in Google Scholar

[3] D. B. Lindell, G. Wetzstein, and M. O’Toole, “Wave-based non-line-of-sight imaging using fast fk migration,” ACM Trans. Graphics, vol. 38, no. 4, pp. 1–13, 2019. https://doi.org/10.1145/3306346.3322937.Suche in Google Scholar

[4] A. Kirmani, T. Hutchison, J. Davis, and R. Raskar, “Looking around the corner using transient imaging,” in 2009 IEEE 12th International Conference on Computer Vision, IEEE, 2009, pp. 159–166.10.1109/ICCV.2009.5459160Suche in Google Scholar

[5] C. Guo, et al.., “Imaging through scattering layers exceeding memory effect range by exploiting prior information,” Opt. Commun., vol. 434, pp. 203–208, 2019. https://doi.org/10.1016/j.optcom.2018.11.005.Suche in Google Scholar

[6] J. H. Nam, et al.., “Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second,” Nat. Commun., vol. 12, no. 1, p. 6526, 2021. https://doi.org/10.1038/s41467-021-26721-x.Suche in Google Scholar PubMed PubMed Central

[7] X. Liu, et al.., “Non-line-of-sight imaging using phasor-field virtual wave optics,” Nature, vol. 572, no. 7771, pp. 620–623, 2019. https://doi.org/10.1038/s41586-019-1461-3.Suche in Google Scholar PubMed

[8] X. Liu, S. Bauer, and A. Velten, “Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems,” Nat. Commun., vol. 11, no. 1, p. 1645, 2020. https://doi.org/10.1038/s41467-020-15157-4.Suche in Google Scholar PubMed PubMed Central

[9] S. Xin, S. Nousias, K. N. Kutulakos, A. C. Sankaranarayanan, S. G. Narasimhan, and I. Gkioulekas, “A theory of Fermat paths for non-line-of-sight shape reconstruction,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 6800–6809.10.1109/CVPR.2019.00696Suche in Google Scholar

[10] G. Gariepy, F. Tonolini, R. Henderson, J. Leach, and D. Faccio, “Detection and tracking of moving objects hidden from view,” Nat. Photonics, vol. 10, no. 1, pp. 23–26, 2016. https://doi.org/10.1038/nphoton.2015.234.Suche in Google Scholar

[11] X. Liu, J. Wang, Z. Li, Z. Shi, X. Fu, and L. Qiu, “Non-line-of-sight reconstruction with signal–object collaborative regularization,” Light Sci. Appl., vol. 10, no. 1, p. 198, 2021. https://doi.org/10.1038/s41377-021-00633-3.Suche in Google Scholar PubMed PubMed Central

[12] X. Liu, J. Wang, L. Xiao, Z. Shi, X. Fu, and L. Qiu, “Non-line-of-sight imaging with arbitrary illumination and detection pattern,” Nat. Commun., vol. 14, no. 1, p. 3230, 2023. https://doi.org/10.1038/s41467-023-38898-4.Suche in Google Scholar PubMed PubMed Central

[13] L. Xin, H. Liangyu, and T. Yixuan, “Direct object recognition without line-of-sight using optical coherence,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 11737–11746.Suche in Google Scholar

[14] F. Willomitzer, F. Li, P. Rangarajan, and O. Cossairt, “Non-line-of-sight imaging using superheterodyne interferometry,” in Computational Optical Sensing and Imaging, Orlando, Florida, USA, Optica Publishing Group, 2018, p. CM2E.1.10.1364/COSI.2018.CM2E.1Suche in Google Scholar

[15] A. Viswanath, P. Rangarajan, D. MacFarlane, and M. P. Christensen, “Indirect imaging using correlography,” in Computational Optical Sensing and Imaging, Orlando, Florida, USA, Optica Publishing Group, 2018, p. CM2E.3.10.1364/COSI.2018.CM2E.3Suche in Google Scholar

[16] B. M. Smith, M. O’Toole, and M. Gupta, “Tracking multiple objects outside the line of sight using speckle imaging,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6258–6266.10.1109/CVPR.2018.00655Suche in Google Scholar

[17] O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photon., vol. 6, no. 8, pp. 549–553, 2012. https://doi.org/10.1038/nphoton.2012.150.Suche in Google Scholar

[18] J. Boger-Lombard and O. Katz, “Passive optical time-of-flight for non line-of-sight localization,” Nat. Commun., vol. 10, no. 1, p. 3343, 2019. https://doi.org/10.1038/s41467-019-11279-6.Suche in Google Scholar PubMed PubMed Central

[19] A. Torralba and W. T. Freeman, “Accidental pinhole and pinspeck cameras: revealing the scene outside the picture,” Int. J. Comput. Vis., vol. 110, pp. 92–112, 2014. https://doi.org/10.1007/s11263-014-0697-5.Suche in Google Scholar

[20] K. L. Bouman, et al.., “Turning corners into cameras: principles and methods,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2270–2278.10.1109/ICCV.2017.249Suche in Google Scholar

[21] C. Saunders, J. Murray-Bruce, and V. K. Goyal, “Computational periscopy with an ordinary digital camera,” Nature, vol. 565, no. 7740, pp. 472–475, 2019. https://doi.org/10.1038/s41586-018-0868-6.Suche in Google Scholar PubMed

[22] O. Katz, P. Heidmann, M. Fink, and S. Gigan, “Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations,” Nat. Photon., vol. 8, no. 10, pp. 784–790, 2014. https://doi.org/10.1038/nphoton.2014.189.Suche in Google Scholar

[23] J. Bertolotti, E. G. Van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature, vol. 491, no. 7423, pp. 232–234, 2012. https://doi.org/10.1038/nature11578.Suche in Google Scholar PubMed

[24] R. Cao, F. de Goumoens, B. Blochet, J. Xu, and C. Yang, “High-resolution non-line-of-sight imaging employing active focusing,” Nat. Photonics, vol. 16, no. 6, pp. 462–468, 2022. https://doi.org/10.1038/s41566-022-01009-8.Suche in Google Scholar

[25] S. Zheng, M. Liao, F. Wang, W. He, X. Peng, and G. Situ, “Non-line-of-sight imaging under white-light illumination: a two-step deep learning approach,” Opt. Express, vol. 29, no. 24, pp. 40091–40105, 2021. https://doi.org/10.1364/oe.443127.Suche in Google Scholar

[26] J. He, S. Wu, R. Wei, and Y. Zhang, “Non-line-of-sight imaging and tracking of moving objects based on deep learning,” Opt. Express, vol. 30, no. 10, pp. 16758–16772, 2022. https://doi.org/10.1364/oe.455803.Suche in Google Scholar PubMed

[27] X. Feng and L. Gao, “Ultrafast light field tomography for snapshot transient and non-line-of-sight imaging,” Nat. Commun., vol. 12, no. 1, p. 2179, 2021. https://doi.org/10.1038/s41467-021-22461-0.Suche in Google Scholar PubMed PubMed Central

[28] C. Zhang, et al.., “Low-loss metasurface optics down to the deep ultraviolet region,” Light Sci., vol. 9, no. 1, p. 55, 2020. https://doi.org/10.1038/s41377-020-0287-y.Suche in Google Scholar PubMed PubMed Central

[29] Z.-L. Deng, et al.., “Diatomic metasurface for vectorial holography,” Nano Lett., vol. 18, no. 5, pp. 2885–2892, 2018. https://doi.org/10.1021/acs.nanolett.8b00047.Suche in Google Scholar PubMed

[30] D. Tang, et al.., “Ultrabroadband superoscillatory lens composed by plasmonic metasurfaces for subdiffraction light focusing,” Laser Photon. Rev., vol. 9, no. 6, pp. 713–719, 2015. https://doi.org/10.1002/lpor.201500182.Suche in Google Scholar

[31] F. Qin, et al.., “Hybrid bilayer plasmonic metasurface efficiently manipulates visible light,” Sci. Adv., vol. 2, no. 1, p. e1501168, 2016. https://doi.org/10.1126/sciadv.1501168.Suche in Google Scholar PubMed PubMed Central

[32] Y. Yang, I. I. Kravchenko, D. P. Briggs, and J. Valentine, “All-dielectric metasurface analogue of electromagnetically induced transparency,” Nat. Commun., vol. 5, no. 1, p. 5753, 2014. https://doi.org/10.1038/ncomms6753.Suche in Google Scholar PubMed

[33] G. Li, et al.., “Continuous control of the nonlinearity phase for harmonic generations,” Nat. Mater., vol. 14, no. 6, pp. 607–612, 2015. https://doi.org/10.1038/nmat4267.Suche in Google Scholar PubMed

[34] J. Li, et al.., “From lingering to rift: metasurface decoupling for near-and far-field functionalization,” Adv. Mater., vol. 33, no. 16, p. 2007507, 2021. https://doi.org/10.1002/adma.202007507.Suche in Google Scholar PubMed

[35] Q. Fan, et al.., “Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field,” Nat. Commun., vol. 13, no. 1, p. 2130, 2022. https://doi.org/10.1038/s41467-022-29568-y.Suche in Google Scholar PubMed PubMed Central

[36] Z. Fei, G. Yinghui, P. Mingbo, L. Xiong, M. Xiaoliang, and L. Xiangang, “Metasurfaces enabled by asymmetric photonic spin-orbit interactions,” Opto-Electron. Eng., vol. 47, no. 10, pp. 200366-1–200366-10, 2020.Suche in Google Scholar

[37] Z.-B. Fan, et al.., “Silicon nitride metalenses for close-to-one numerical aperture and wide-angle visible imaging,” Phys. Rev. Appl., vol. 10, no. 1, p. 014005, 2018. https://doi.org/10.1103/physrevapplied.10.014005.Suche in Google Scholar

[38] F. Zhang, et al.., “Meta-optics empowered vector visual cryptography for high security and rapid decryption,” Nat. Commun., vol. 14, no. 1, p. 1946, 2023. https://doi.org/10.1038/s41467-023-37510-z.Suche in Google Scholar PubMed PubMed Central

[39] R. Zhang, et al.., “Angular superoscillatory metalens empowers single-shot measurement of OAM modes with finer intervals,” Adv. Opt. Mater., no. 2300009, 2023. https://doi.org/10.1002/adom.202300009.Suche in Google Scholar

[40] X. Xie, et al.., “Generalized Pancharatnam–Berry phase in rotationally symmetric meta-atoms,” Phys. Rev. Lett., vol. 126, no. 18, p. 183902, 2021. https://doi.org/10.1103/physrevlett.126.183902.Suche in Google Scholar

[41] M. Pu, et al.., “Catenary optics for achromatic generation of perfect optical angular momentum,” Sci. Adv., vol. 1, no. 9, p. e1500396, 2015. https://doi.org/10.1126/sciadv.1500396.Suche in Google Scholar PubMed PubMed Central

[42] Y. Guo, et al.., “Spin-decoupled metasurface for simultaneous detection of spin and orbital angular momenta via momentum transformation,” Light Sci. Appl., vol. 10, no. 1, p. 63, 2021. https://doi.org/10.1038/s41377-021-00497-7.Suche in Google Scholar PubMed PubMed Central

[43] F. Zhang, et al.., “All‐dielectric metasurfaces for simultaneous giant circular asymmetric transmission and wavefront shaping based on asymmetric photonic spin–orbit interactions,” Adv. Funct. Mater., vol. 27, no. 47, p. 1704295, 2017. https://doi.org/10.1002/adfm.201704295.Suche in Google Scholar

[44] Z. L. Deng, F. J. Li, H. Li, X. Li, and A. Alù, “Extreme diffraction control in metagratings leveraging bound states in the continuum and exceptional points,” Laser Photon. Rev., vol. 16, no. 6, p. 2100617, 2022. https://doi.org/10.1002/lpor.202100617.Suche in Google Scholar

[45] Z. Feng, et al.., “Dual-band polarized upconversion photoluminescence enhanced by resonant dielectric metasurfaces,” arXiv preprint arXiv:2305.09113, 2023.10.1186/s43593-023-00054-2Suche in Google Scholar

[46] J. Kobashi, H. Yoshida, and M. Ozaki, “Planar optics with patterned chiral liquid crystals,” Nat. Photonics, vol. 10, no. 6, pp. 389–392, 2016. https://doi.org/10.1038/nphoton.2016.66.Suche in Google Scholar

[47] J. Chen, et al.., “Planar wide-angle-imaging camera enabled by metalens array,” Optica, vol. 9, no. 4, pp. 431–437, 2022. https://doi.org/10.1364/optica.446063.Suche in Google Scholar

[48] X. Ye, et al.., “Chip-scale metalens microscope for wide-field and depth-of-field imaging,” Adv. Photon., vol. 4, no. 4, p. 046006, 2022. https://doi.org/10.1117/1.ap.4.4.046006.Suche in Google Scholar

[49] Y. Lei, et al.., “Snapshot multi-dimensional computational imaging through a liquid crystal diffuser,” Photon. Res., vol. 11, no. 3, pp. B111–B124, 2023. https://doi.org/10.1364/prj.476317.Suche in Google Scholar

[50] D. Tang, et al.., “Simultaneous surface display and holography enabled by flat liquid crystal elements,” Laser Photon. Rev., vol. 16, no. 7, p. 2100491, 2022. https://doi.org/10.1002/lpor.202100491.Suche in Google Scholar

[51] Q. Xu, T. Sun, and C. Wang, “Coded liquid crystal metasurface for achromatic imaging in the broadband wavelength range,” ACS Appl. Nano Mater., vol. 4, no. 5, pp. 5068–5075, 2021. https://doi.org/10.1021/acsanm.1c00542.Suche in Google Scholar

[52] X. Lu, et al.., “Broadband high-efficiency polymerized liquid crystal metasurfaces with spin-multiplexed functionalities in the visible,” Photon. Res., vol. 10, no. 6, pp. 1380–1393, 2022. https://doi.org/10.1364/prj.452272.Suche in Google Scholar

[53] Y. Ji, et al.., “Active terahertz spin state and optical chirality in liquid crystal chiral metasurface,” Phys. Rev. Mater., vol. 5, no. 8, p. 085201, 2021. https://doi.org/10.1103/physrevmaterials.5.085201.Suche in Google Scholar

[54] S. Zhou, Z. Shen, X. Li, S. Ge, Y. Lu, and W. Hu, “Liquid crystal integrated metalens with dynamic focusing property,” Opt. Lett., vol. 45, no. 15, pp. 4324–4327, 2020. https://doi.org/10.1364/ol.398601.Suche in Google Scholar

[55] F. Zhang, et al.., “Extreme-angle silicon infrared optics enabled by streamlined surfaces,” Adv. Mater., vol. 33, no. 11, p. 2008157, 2021. https://doi.org/10.1002/adma.202008157.Suche in Google Scholar PubMed

[56] G. Si, Y. Zhao, E. S. P. Leong, and Y. J. Liu, “Liquid-crystal-enabled active plasmonics: a review,” Materials, vol. 7, no. 2, pp. 1296–1317, 2014. https://doi.org/10.3390/ma7021296.Suche in Google Scholar PubMed PubMed Central

[57] K. M. Lee, V. P. Tondiglia, and T. J. White, “Electrically reconfigurable liquid crystalline mirrors,” ACS Omega, vol. 3, no. 4, pp. 4453–4457, 2018. https://doi.org/10.1021/acsomega.8b00453.Suche in Google Scholar PubMed PubMed Central

[58] J. Albero, et al.., “Liquid crystal devices for the reconfigurable generation of optical vortices,” J. Lightwave Technol., vol. 30, no. 18, pp. 3055–3060, 2012. https://doi.org/10.1109/jlt.2012.2211567.Suche in Google Scholar

[59] D. Jiang, et al.., “Liquid crystal-based wideband reconfigurable leaky wave X-band antenna,” IEEE Access, vol. 7, pp. 127320–127326, 2019. https://doi.org/10.1109/access.2019.2939097.Suche in Google Scholar

[60] T. Zhan, et al.., “Pancharatnam–Berry optical elements for head-up and near-eye displays,” JOSA B, vol. 36, no. 5, pp. D52–D65, 2019. https://doi.org/10.1364/josab.36.000d52.Suche in Google Scholar

[61] Z. He, K. Yin, and S.-T. Wu, “Applications Miniature planar telescopes for efficient, wide-angle, high-precision beam steering,” Light Sci., vol. 10, no. 1, p. 134, 2021. https://doi.org/10.1038/s41377-021-00576-9.Suche in Google Scholar PubMed PubMed Central

[62] J. Tachella, et al.., “Bayesian 3D reconstruction of complex scenes from single-photon lidar data,” SIAM J. Imag. Sci., vol. 12, no. 1, pp. 521–550, 2019. https://doi.org/10.1137/18m1183972.Suche in Google Scholar

[63] A. Halimi, R. Tobin, A. McCarthy, J. Bioucas-Dias, S. McLaughlin, and G. S. Buller, “Robust restoration of sparse multidimensional single-photon LiDAR images,” IEEE Trans. Comput. Imag., vol. 6, pp. 138–152, 2019, https://doi.org/10.1109/tci.2019.2929918.Suche in Google Scholar

Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/nanoph-2023-0655).

Received: 2023-10-03
Accepted: 2023-12-10
Published Online: 2024-01-10

© 2023 the author(s), published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.

Artikel in diesem Heft

  1. Frontmatter
  2. Editorial
  3. Programmable nano-optics and photonics
  4. Reviews
  5. Towards large-scale programmable silicon photonic chip for signal processing
  6. Phase change material-based tunable Fano resonant optical coatings and their applications
  7. Perspectives
  8. Melting-free integrated photonic memory with layered polymorphs
  9. Chalcogenide phase-change material advances programmable terahertz metamaterials: a non-volatile perspective for reconfigurable intelligent surfaces
  10. Research Articles
  11. Fundamental limits to multi-functional and tunable nanophotonic response
  12. Spatio-spectral control of coherent nanophotonics
  13. Nanoantenna induced liquid crystal alignment for high performance tunable metasurface
  14. Programmable topological metasurface to modulate spatial and surface waves in real time
  15. Programmable flip-metasurface with dynamically tunable reflection and broadband undistorted transmission
  16. High-resolution non-line-of-sight imaging based on liquid crystal planar optical elements
  17. Ultrafast Q-boosting in semiconductor metasurfaces
  18. Inverse design of compact nonvolatile reconfigurable silicon photonic devices with phase-change materials
  19. Integrated multi-operand optical neurons for scalable and hardware-efficient deep learning
  20. Surface plasmon-cavity hybrid state and its graphene modulation at THz frequencies
  21. Language-controllable programmable metasurface empowered by large language models
  22. All optical tunable RF filter using elemental antimony
  23. Reconfigurable application-specific photonic integrated circuit for solving partial differential equations
  24. Optimization of a programmable λ/2-pitch optical phased array
  25. Closed-loop electron-beam-induced spectroscopy and nanofabrication around individual quantum emitters
  26. High-fidelity and polarization-insensitive universal photonic processors fabricated by femtosecond laser writing
  27. Direct electron beam patterning of electro-optically active PEDOT:PSS
https://doi.org/10.1515/nanoph-2023-0655
Schlagwörter für diesen Artikel
metasurface; liquid crystal planar optical element; non-line-of-sight imaging
Creative Commons
BY 4.0

Artikel in diesem Heft

  1. Frontmatter
  2. Editorial
  3. Programmable nano-optics and photonics
  4. Reviews
  5. Towards large-scale programmable silicon photonic chip for signal processing
  6. Phase change material-based tunable Fano resonant optical coatings and their applications
  7. Perspectives
  8. Melting-free integrated photonic memory with layered polymorphs
  9. Chalcogenide phase-change material advances programmable terahertz metamaterials: a non-volatile perspective for reconfigurable intelligent surfaces
  10. Research Articles
  11. Fundamental limits to multi-functional and tunable nanophotonic response
  12. Spatio-spectral control of coherent nanophotonics
  13. Nanoantenna induced liquid crystal alignment for high performance tunable metasurface
  14. Programmable topological metasurface to modulate spatial and surface waves in real time
  15. Programmable flip-metasurface with dynamically tunable reflection and broadband undistorted transmission
  16. High-resolution non-line-of-sight imaging based on liquid crystal planar optical elements
  17. Ultrafast Q-boosting in semiconductor metasurfaces
  18. Inverse design of compact nonvolatile reconfigurable silicon photonic devices with phase-change materials
  19. Integrated multi-operand optical neurons for scalable and hardware-efficient deep learning
  20. Surface plasmon-cavity hybrid state and its graphene modulation at THz frequencies
  21. Language-controllable programmable metasurface empowered by large language models
  22. All optical tunable RF filter using elemental antimony
  23. Reconfigurable application-specific photonic integrated circuit for solving partial differential equations
  24. Optimization of a programmable λ/2-pitch optical phased array
  25. Closed-loop electron-beam-induced spectroscopy and nanofabrication around individual quantum emitters
  26. High-fidelity and polarization-insensitive universal photonic processors fabricated by femtosecond laser writing
  27. Direct electron beam patterning of electro-optically active PEDOT:PSS
Jetzt anmelden und 20% sparen
Institutioneller Zugang
Wie funktioniert Authentifizierung?
Haben Sie eine Idee, wie wir unsere Website verbessern können?
Bitte schreiben Sie uns.
© 2025 De Gruyter Brill
Heruntergeladen am 22.9.2025 von https://www.degruyterbrill.com/document/doi/10.1515/nanoph-2023-0655/html
Button zum nach oben scrollen