Tunable holographic metasurfaces for augmented and virtual reality

1 Introduction

Augmented and virtual reality (AR/VR) devices are normally head-worn displays that project light directly into the eyes to simulate the appearance of 3D objects in front of the wearer. These devices will generally fall into one of two categories: AR headsets that use transparent waveguides to allow the wearer to directly see the physical world as well as simulated objects, and VR headsets that use opaque displays to simulate a full virtual world. Both devices require precise control of light close to the eyes, low weight, and low power consumption so that they can be worn on the head comfortably and without excess heating [1].

The current market for these devices is significant, with $7 billion in revenue globally in 2023 and an annual growth rate of up to 50 % [2]. While many applications have focused on gaming, the last decade has also shown a multitude of examples where AR/VR technology is used in manufacturing, education, and medicine, demonstrating a clear need for advanced 3D display technologies. For example, Lockheed Martin has used AR/VR headsets in the manufacturing of the Orion spacecraft. Using an AR/VR headset to show the exact position of fasteners on the spacecraft in AR resulted in a 90 % reduction in assembly time and a complete elimination of assembly errors [3]. AR/VR technology has also been shown to reduce the time it takes for medical students to learn human anatomy by a factor of two while maintaining the same level of performance on benchmark exams [4], see Figure 1. In the clinical world, these technologies can reduce the time to perform medical procedures by nearly a factor of two [5]. AR/VR technologies are even making an impact on our understanding of the structure of the brain by allowing experts to form a consensus around complicated 3D structures [6]. These are just a few examples that demonstrate the immense potential of AR/VR technologies to drive innovation across multiple fields. However, the future adoption and application of AR/VR devices hinges on improving wearability, accessibility, and usability, which is accomplished by decreasing hardware cost, weight, and power consumption.

Figure 1:

Example applications of current AR/VR devices in education and medicine. (a) Dr. T. Roma Jasinevicius from Case Western Reserve University teaches dental anatomy to first- and second-year students using AR technology. (b) Students using AR technology to visualize complex anatomical structures of the head and neck in real time. (Courtesy of Interactive Commons, Case Western Reserve University, Cleveland, Ohio.)

Holographic metamaterials merge holography with subwavelength engineering to dynamically control light, providing essential tools for AR/VR near-eye displays and enhancing the immersive quality of AR/VR by blending virtual content seamlessly with real-world perspectives [7]. Unlike traditional optics, which rely on bulky components, these materials use nanoscale fabricated “meta-atoms” to manipulate light’s phase, amplitude, and polarization on a compact surface. A high degree of control is possible, with achievable full 2π phase delay, allowing for lightweight designs that can reconstruct 3D images and perform complex optical functions [8], [9], [10], [11]. This control is particularly valuable in holography, where realistic 3D images are created by reconstructed light fields from a complicated phase surface, and highly local phase control is critical. In AR/VR applications, this approach allows devices to match virtual images with real-world views, providing immersive, adaptable experiences [12].

Scattering or diffractive metasurfaces will be important for next generation holographic displays, controlling light precisely through the interaction with nanostructured materials. The scale of the structuring means that the device thickness can be drastically decreased, reducing form factor. This ensures clarity in virtual images while reducing the weight and bulk associated with traditional optics – crucial for wearable AR/VR displays [13]. Advances by Capasso and colleagues in flat optics have established ultra-thin, high numerical aperture (NA) meta-lenses, feasible for integrating in wearable AR/VR displays for enhanced field-of-view (FOV) and reduced form factor [14], [15], [16]. In particular, transmissive dielectric metasurfaces are important for their high diffraction efficiency and low optical losses.

Beyond integrating holographic metasurfaces, AR/VR applications can be further advanced by tunable metasurfaces, enabling a dynamic response from a single optical device. To date, the most studied application has been dynamic focal adjustments in tunable metalenses [10], [17], [18]. These devices are well-poised for implementation in holographic displays, where depth variations enhance 3D viewing comfort. They can adjust to user gaze or environmental conditions, offering seamless focal transitions for natural AR/VR interactions [19]. As metasurface design has advanced, tunable holographic metamaterials have evolved to enable complex optical functions on a single surface, tailored to produce specific conditions such as different wavelengths responses or desired focal corrections [20]. This multifunctionality allows scalable, compact designs without added components.

One of the most critical aspects of metasurface implementation is the design and optimization of the subwavelength scale structure, to take advantage of the tremendous design space [8], [21], [22]. Metamaterials require strict attention to optimization and fabrication across a staggering range of length scales, from centimeters to nanometers, see Figure 2. As applications advance from relatively “simple” applications such as varifocal lenses where the phase conditions are broadly known to tunable holograms and free-form metasurfaces with hundreds of millions of geometrical degrees of freedom, the design complexity increases substantially. Optimization tools and precise manufacturing are key to meeting the demands for AR/VR technology, with modern approaches such as machine learning becoming increasingly important [8], [23], [24].

Figure 2:

Metasurfaces across scales. Metasurfaces require attention to design and fabrication across a large range of length scales from the production (∼10 cm), to the device (∼1 mm–1 cm), structural (∼10 μm), and meta-element (∼<100 nm) scales. The metasurface shown in this figure is described in detail elsewhere [25], and images are adapted with permission [26]. Copyright 2020, Proceedings of the National Academy of Sciences.

2 Challenges and opportunities

Metasurfaces, and especially tunable holographic metasurfaces, are well-positioned for large-scale integration into AR/VR devices, enabling next-generation systems. The challenges faced by current AR/VR approaches present opportunities for significant improvements with metasurface technologies. However, future implementations will require targeted advances in tunable metasurface technologies to overcome key technological limitations.

3 Key metasurface technologies

Tunable metasurfaces offer a highly versatile platform for manipulating light, enabling dynamic control over its propagation by the phase, amplitude, polarization, and spectral response, among others [34], [40], [41]. These devices are a central focus of current photonics research [8]. Dynamic tunability in metasurfaces is achieved either by designing nanostructures that respond to external stimuli, or incorporating active materials to dynamically control the local environment. Common stimuli include electric and light fields [42]; active materials include liquid crystals, phase-change materials (PCMs), and electronically tunable materials such as 2D transition metal dichalcogenides (TMDs). By leveraging these mechanisms, tunable metasurfaces can enable real-time control over light propagation, essential for varifocal lenses and dynamic holographic displays in AR/VR systems [43]. The most common categories of metasurface fabrication materials and strategies for incorporating tunability are listed in Table 1.

Focusing first on device fabrication, deep-UV lithography and nanoimprint lithography (NIL) are among the most promising commercially viable nanofabrication techniques [25], [44], [45], [46], [47]. In particular, NIL should be highlighted as a cost-effective method for creating nanoscale patterns in macroscale objects, covering the range of fabrication length scales and enabling scalable production of metasurfaces. The technique can serve as a replacement for traditional e-beam and photolithography processes, as illustrated in Figure 3(a). The patterned material is naturally polymer-based and can either be coated – using for instance an inverse masking and lift-off process – to create a metasurface in a different material or used directly as a dielectric polymer metasurface. Although most nanopatternable polymers such as PDMS [48] have a low refractive index, polymers can be desirable as metasurface materials in some applications due to their flexibility, mechanical properties, potentially high optical quality, transparency, and low-cost fabrication. Additionally, the growth of additive manufacturing techniques, such as two-photon polymerization lithography, could be useful for small-batch samples [49], [50]. Of depositable materials, dielectrics such as TiO₂ and Si₃N₄ are the most desirable for AR/VR devices due to their high refractive indices in the visible range (n ∼ 2) and low optical losses compared with plasmonic materials. These properties engender high clarity and energy efficiency in metasurface devices [28]. Ultimately, when choosing a material, durability and compatibility with the chosen tuning method are paramount.

Figure 3:

Strategies for fabrication and tuning of AR/VR metasurface devices. (a) Nanoimprint lithography (NIL) utilizes a single nanopatterned mold (master) to produce many replica nanoscale surfaces with an inverse structure, which can be used in inverse masking processes or directly as polymeric metasurfaces. Since the initial lithography is only performed once, the technique highly scalable and cost-effective. (b) Some of the most promising schemes for tuning metasurface devices are (top) electronic (gate voltage Vg) tuning of the light–matter interaction in 2D material coupled metasurface devices, including transition metal dichalcogenides (MoS₂ shown), and (bottom) liquid crystal devices (with applied electric field at voltage V) integrated with metasurfaces to control the ambient refractive index. The liquid crystal is shown to reorient in the applied electric field.

Although there are now a number of strategies for enabling tunability in holographic metasurfaces, we propose that integrating liquid crystals or 2D materials such as transition metal dichalcogenides (TMDs) are the most promising avenues for meeting the challenges of AR/VR applications. These are illustrated in Figure 3(b).

Liquid crystals exhibit a highly responsive reorientation effect in electric fields, among other tunable parameters, enabling a degree of local refractive index modulation within the metasurface [51]. A number of recent studies have utilized liquid crystals for metasurface applications such as varifocal lenses [17], [18], [52], and the physics and control technology are well established [53], [54]. Additionally, the tuning rate or switching speed far exceeds the required AR/VR refresh rate [55]. Despite their responsiveness and high degree of tunability, the drawbacks of utilizing liquid crystals as an active medium include higher power requirements for maintaining orientation, limitations on the local control due to long-range ordering, and thermal instabilities [56]. The issue of long interaction lengths becomes more significant as device dimensions are scaled down, and while it can be mitigated through adjustments to molecular chemistry, it cannot be fully eliminated. For these reasons, we propose that achieving further advancements in AR/VR devices will necessitate the development of more precise and tunable control mechanisms. Nevertheless, this technology warrants continued investigation due to its significant potential for near-term device improvements and ease of implementation.

Another class of materials, which have garnered much attention for optical tunability over the last few years, are 2D materials such as TMDs, with the most common being MoS₂ and WS₂. These materials offer unique excitonic properties, enabling optical tuning through applied electrical field gating [57], [58], as well as optical modulation [59]. In electronic gating schemes, interaction between the injected charge and excitonic modes can lead to local refractive index changes in the visible spectrum [60], [61], [62], [63], [64]. Integration with metasurfaces engender fast, solid-state tunable devices for total wavefront control, such as mirrors [65], lenses [66], and beam scattering control at high modulation frequencies [67]. Although these materials show promise for ultra-thin, highly tunable AR/VR holographic metasurfaces, research remains in the early stages. Significant additional effort is required to develop and understand systems at the level of AR/VR device integration. For fabrication, mechanical exfoliation is the most reproducible and widely available process. However, this is not currently a commercially scalable process, limiting the large-scale production of TMDs. Additionally, the long-term stability of these devices is not well understood, particularly when considering degradation mechanisms such as photo-induced spontaneous oxidation [68].

A key consideration when implementing a tunable mechanism is the external stimuli or stressors necessary to enable the dynamic response in the material, since this can have a large impact upon the device form factor, implementation, and usage [34], [69]. Electrical and optical stimuli are the most important for highly tunable, compact, and portable implementations, although other stimuli such as temperature, chemical, and mechanical exist [70]. Electrical tuning is preferred for its speed, reversibility, and integrability. Also, electronic control processes are well established and integrated with eminently electronic AR/VR devices. The main strategy for increasing the local control in liquid crystal integrated devices is interdigitation and miniaturization of contacts, similar to spatial light modulator technologies, which can be accomplished with traditional or CMOS techniques [71], [72]. In TMD materials, local control over the excitonic properties can be established through gating techniques [49], [57], [61], [73], where a blueprint for establishing fine-scale electronic control already exists in the semiconductor industry. In addition, all-optical control processes are interesting since in principle they do not require a direct contact method with the metasurface device and can achieve smaller form factor and exceptional switching speeds in 2D excitonic materials [74]. Although the tunable response expands the functional scope of metasurfaces, balancing speed, precision, and durability remains an area of ongoing research.

4 Discussion

There are two major pathways for tunable metasurface integration in AR/VR devices: tunable discrete optical elements and highly tunable holographic metasurfaces. We propose that these pathways both offer significant improvements for current AR/VR devices, meeting the challenges outlined above. In the former case, elements such as lenses and collimators are replaced with their metasurface equivalents, enabling greater precision in wavefront control. These are demonstrated in Figure 4(a) and can include, for instance, higher quality lensing through metalens technology [14]. Tunability in the metasurface can - furher enable highly flexible optical systems, such as varifocal lensing. This approach builds on traditional metasurface technologies to enhance AR/VR devices in their current form. Several studies have been explored in this direction, and with meta-optics rapidly approaching the stage of commercial viability, the technology is poised and ready for practical implementation [84]. Metalens integration has the potential to meet many of the challenges with AR/VR devices outlined above.

Figure 4:

Avenues for integrating tunable metasurfaces with AR/VR devices. (a) Replacing traditional refractive elements with metasurface devices, a simplified optical schematic is shown with a tunable meta-lens. (b) Schematic implementation of highly reconfigurable holographic metasurfaces with full phase and amplitude control to replace image generation engines. (c) Schematic implementation of tunable metasurface technology providing both hologram generation and tunable input and output coupling in optical waveguides.

Beyond drop-in replacement, tunable holographic metasurfaces have the potential to integrate the entire optical propagation pathway – including image generation, light propagation and steering, and projection – into a single device with small form-factor. As shown in Figure 4(b) and (c), the simplest approach uses a single surface to fully control both the propagation phase and amplitude. The optical pathway is simplified in this figure for clarity. A similar picture applies to waveguide AR/VR devices, where holographic metasurfaces can be integrated as direct surfaces or as tunable in- and out-coupling gratings for complete spatial light control. Metasurfaces have already demonstrated their effectiveness as replacements for diffractive elements, particularly in mitigating shadowing effects [85]. Although recent studies have made progress toward implementable devices, significant advancements are still required to improve both tunable metasurfaces and their application in AR/VR devices. However, the potential benefits of this approach are substantial, as these devices offer a pathway to achieving complete wavefront control, necessary for the next-generation of AR/VR devices.