Waveguide combiners for mixed reality headsets: a nanophotonics design perspective
Abstract
This paper is a review and analysis of the various implementation architectures of diffractive waveguide combiners for augmented reality (AR), mixed reality (MR) headsets, and smart glasses. Extended reality (XR) is another acronym frequently used to refer to all variants across the MR spectrum. Such devices have the potential to revolutionize how we work, communicate, travel, learn, teach, shop, and are entertained. Already, market analysts show very optimistic expectations on return on investment in MR, for both enterprise and consumer applications. Hardware architectures and technologies for AR and MR have made tremendous progress over the past five years, fueled by recent investment hype in start-ups and accelerated mergers and acquisitions by larger corporations. In order to meet such high market expectations, several challenges must be addressed: first, cementing primary use cases for each specific market segment and, second, achieving greater MR performance out of increasingly size-, weight-, cost- and power-constrained hardware. One such crucial component is the optical combiner. Combiners are often considered as critical optical elements inMR headsets, as they are the direct window to both the digital content and the real world for the user’s eyes. Two main pillars defining the MR experience are comfort and immersion. Comfort comes in various forms: - wearable comfort-reducing weight and size, pushing back the center of gravity, addressing thermal issues, and so on - visual comfort-providing accurate and natural 3-dimensional cues over a large field of view and a high angular resolution - vestibular comfort-providing stable and realistic virtual overlays that spatially agree with the user’s motion - social comfort-allowing for true eye contact, in a socially acceptable form factor. Immersion can be defined as the multisensory perceptual experience (including audio, display, gestures, haptics) that conveys to the user a sense of realism and envelopment. In order to effectively address both comfort and immersion challenges through improved hardware architectures and software developments, a deep understanding of the specific features and limitations of the human visual perception system is required. We emphasize the need for a humancentric optical design process, which would allow for the most comfortable headset design (wearable, visual, vestibular, and social comfort) without compromising the user’s sense of immersion (display, sensing, and interaction). Matching the specifics of the display architecture to the human visual perception system is key to bound the constraints of the hardware allowing for headset development and mass production at reasonable costs, while providing a delightful experience to the end user.
Abstract
This paper is a review and analysis of the various implementation architectures of diffractive waveguide combiners for augmented reality (AR), mixed reality (MR) headsets, and smart glasses. Extended reality (XR) is another acronym frequently used to refer to all variants across the MR spectrum. Such devices have the potential to revolutionize how we work, communicate, travel, learn, teach, shop, and are entertained. Already, market analysts show very optimistic expectations on return on investment in MR, for both enterprise and consumer applications. Hardware architectures and technologies for AR and MR have made tremendous progress over the past five years, fueled by recent investment hype in start-ups and accelerated mergers and acquisitions by larger corporations. In order to meet such high market expectations, several challenges must be addressed: first, cementing primary use cases for each specific market segment and, second, achieving greater MR performance out of increasingly size-, weight-, cost- and power-constrained hardware. One such crucial component is the optical combiner. Combiners are often considered as critical optical elements inMR headsets, as they are the direct window to both the digital content and the real world for the user’s eyes. Two main pillars defining the MR experience are comfort and immersion. Comfort comes in various forms: - wearable comfort-reducing weight and size, pushing back the center of gravity, addressing thermal issues, and so on - visual comfort-providing accurate and natural 3-dimensional cues over a large field of view and a high angular resolution - vestibular comfort-providing stable and realistic virtual overlays that spatially agree with the user’s motion - social comfort-allowing for true eye contact, in a socially acceptable form factor. Immersion can be defined as the multisensory perceptual experience (including audio, display, gestures, haptics) that conveys to the user a sense of realism and envelopment. In order to effectively address both comfort and immersion challenges through improved hardware architectures and software developments, a deep understanding of the specific features and limitations of the human visual perception system is required. We emphasize the need for a humancentric optical design process, which would allow for the most comfortable headset design (wearable, visual, vestibular, and social comfort) without compromising the user’s sense of immersion (display, sensing, and interaction). Matching the specifics of the display architecture to the human visual perception system is key to bound the constraints of the hardware allowing for headset development and mass production at reasonable costs, while providing a delightful experience to the end user.
Chapters in this book
- Frontmatter i
- Preface v
- Contents vii
-
Part I: Optoelectronics and Integrated Photonics
- Disorder effects in nitride semiconductors: impact on fundamental and device properties 3
- Ultralow threshold blue quantum dot lasers: what’s the true recipe for success? 23
- Waiting for Act 2: what lies beyond organic lightemitting diode (OLED) displays for organic electronics? 31
- Waveguide combiners for mixed reality headsets: a nanophotonics design perspective 41
- On-chip broadband nonreciprocal light storage 75
- High-Q nanophotonics: sculpting wavefronts with slow light 83
- Thermoelectric graphene photodetectors with sub-nanosecond response times at terahertz frequencies 89
- High-performance integrated graphene electro-optic modulator at cryogenic temperature 99
- Asymmetric photoelectric effect: Auger-assisted hot hole photocurrents in transition metal dichalcogenides 105
- Seeing the light in energy use 115
-
Part II: Lasers, Active optical devices and Spectroscopy
- A high-repetition rate attosecond light source for time-resolved coincidence spectroscopy 119
- Fast laser speckle suppression with an intracavity diffuser 131
- Active optics with silk 139
- Nanolaser arrays: toward application-driven dense integration 151
- Two-dimensional spectroscopy on a THz quantum cascade structure 173
- Homogeneous quantum cascade lasers operating as terahertz frequency combs over their entire operational regime 183
- Toward new frontiers for terahertz quantum cascade laser frequency combs 189
- Soliton dynamics of ring quantum cascade lasers with injected signal 197
-
Part III: Fiber Optics and Optical Communications
- Propagation stability in optical fibers: role of path memory and angular momentum 213
- Perspective on using multiple orbital-angularmomentum beams for enhanced capacity in freespace optical communication links 229
-
Part IV: Biomedical Photonics
- A fiber optic–nanophotonic approach to the detection of antibodies and viral particles of COVID-19 241
- Plasmonic control of drug release efficiency in agarose gel loaded with gold nanoparticle assemblies 253
- Metasurfaces for biomedical applications: imaging and sensing from a nanophotonics perspective 265
- Hyperbolic dispersion metasurfaces for molecular biosensing 301
-
Part V: Fundamentals of Optics
- A Tutorial on the Classical Theories of Electromagnetic Scattering and Diffraction 323
- Reflectionless excitation of arbitrary photonic structures: a general theory 351
-
Part VI: Optimization Methods
- Multiobjective and categorical global optimization of photonic structures based on ResNet generative neural networks 371
- Machine learning–assisted global optimization of photonic devices 381
- Artificial neural networks for inverse design of resonant nanophotonic components with oscillatory loss landscapes 395
- Adjoint-optimized nanoscale light extractor for nitrogen-vacancy centers in diamond 403
-
Part VII: Topological Photonics
- Non-Hermitian and topological photonics: optics at an exceptional point 415
- Topological photonics: Where do we go from here? 437
- Topological nanophotonics for photoluminescence control 447
- Anomalous Anderson localization behavior in gain-loss balanced non-Hermitian systems 455
-
Part VIII: Quantum Computing, Quantum Optics, and QED
- Quantum computing and simulation 467
- NIST-certified secure key generation via deep learning of physical unclonable functions in silica aerogels 471
- Thomas–Reiche–Kuhn (TRK) sum rule for interacting photons 479
- Macroscopic QED for quantum nanophotonics: emitter-centered modes as a minimal basis for multiemitter problems 491
- Generation and dynamics of entangled fermion–photon–phonon states in nanocavities 505
- Polaritonic Tamm states induced by cavity photons 527
- Recent progress in engineering the Casimir effect – applications to nanophotonics, nanomechanics, and chemistry 537
- Enhancement of rotational vacuum friction by surface photon tunneling 551
-
Part IX: Plasmonics and Polaritonics
- Shrinking the surface plasmon 561
- Polariton panorama 565
- Scattering of a single plasmon polariton by multiple atoms for in-plane control of light 595
- A metasurface-based diamond frequency converter using plasmonic nanogap resonators 605
- Selective excitation of individual nanoantennas by pure spectral phase control in the ultrafast coherent regime 613
- Semiconductor quantum plasmons for high frequency thermal emission 623
- Origin of dispersive line shapes in plasmon-enhanced stimulated Raman scattering microscopy 633
- Epitaxial aluminum plasmonics covering full visible spectrum 643
-
Part X: Metaoptics
- Metamaterials with high degrees of freedom: space, time, and more 657
- The road to atomically thin metasurface optics 661
- Active nonlocal metasurfaces 673
- Giant midinfrared nonlinearity based on multiple quantum well polaritonic metasurfaces 685
- Near-field plates and the near zone of metasurfaces 697
- High-efficiency metadevices for bifunctional generations of vectorial optical fields 703
- Printing polarization and phase at the optical diffraction limit: near- and far-field optical encryption 715
- Optical response of jammed rectangular nanostructures 723
- Dynamic phase-change metafilm absorber for strong designer modulation of visible light 731
- Arbitrary polarization conversion for pure vortex generation with a single metasurface 745
- Enhanced harmonic generation in gases using an all-dielectric metasurface 751
- Monolithic metasurface spatial differentiator enabled by asymmetric photonic spin-orbit interactions 759
Chapters in this book
- Frontmatter i
- Preface v
- Contents vii
-
Part I: Optoelectronics and Integrated Photonics
- Disorder effects in nitride semiconductors: impact on fundamental and device properties 3
- Ultralow threshold blue quantum dot lasers: what’s the true recipe for success? 23
- Waiting for Act 2: what lies beyond organic lightemitting diode (OLED) displays for organic electronics? 31
- Waveguide combiners for mixed reality headsets: a nanophotonics design perspective 41
- On-chip broadband nonreciprocal light storage 75
- High-Q nanophotonics: sculpting wavefronts with slow light 83
- Thermoelectric graphene photodetectors with sub-nanosecond response times at terahertz frequencies 89
- High-performance integrated graphene electro-optic modulator at cryogenic temperature 99
- Asymmetric photoelectric effect: Auger-assisted hot hole photocurrents in transition metal dichalcogenides 105
- Seeing the light in energy use 115
-
Part II: Lasers, Active optical devices and Spectroscopy
- A high-repetition rate attosecond light source for time-resolved coincidence spectroscopy 119
- Fast laser speckle suppression with an intracavity diffuser 131
- Active optics with silk 139
- Nanolaser arrays: toward application-driven dense integration 151
- Two-dimensional spectroscopy on a THz quantum cascade structure 173
- Homogeneous quantum cascade lasers operating as terahertz frequency combs over their entire operational regime 183
- Toward new frontiers for terahertz quantum cascade laser frequency combs 189
- Soliton dynamics of ring quantum cascade lasers with injected signal 197
-
Part III: Fiber Optics and Optical Communications
- Propagation stability in optical fibers: role of path memory and angular momentum 213
- Perspective on using multiple orbital-angularmomentum beams for enhanced capacity in freespace optical communication links 229
-
Part IV: Biomedical Photonics
- A fiber optic–nanophotonic approach to the detection of antibodies and viral particles of COVID-19 241
- Plasmonic control of drug release efficiency in agarose gel loaded with gold nanoparticle assemblies 253
- Metasurfaces for biomedical applications: imaging and sensing from a nanophotonics perspective 265
- Hyperbolic dispersion metasurfaces for molecular biosensing 301
-
Part V: Fundamentals of Optics
- A Tutorial on the Classical Theories of Electromagnetic Scattering and Diffraction 323
- Reflectionless excitation of arbitrary photonic structures: a general theory 351
-
Part VI: Optimization Methods
- Multiobjective and categorical global optimization of photonic structures based on ResNet generative neural networks 371
- Machine learning–assisted global optimization of photonic devices 381
- Artificial neural networks for inverse design of resonant nanophotonic components with oscillatory loss landscapes 395
- Adjoint-optimized nanoscale light extractor for nitrogen-vacancy centers in diamond 403
-
Part VII: Topological Photonics
- Non-Hermitian and topological photonics: optics at an exceptional point 415
- Topological photonics: Where do we go from here? 437
- Topological nanophotonics for photoluminescence control 447
- Anomalous Anderson localization behavior in gain-loss balanced non-Hermitian systems 455
-
Part VIII: Quantum Computing, Quantum Optics, and QED
- Quantum computing and simulation 467
- NIST-certified secure key generation via deep learning of physical unclonable functions in silica aerogels 471
- Thomas–Reiche–Kuhn (TRK) sum rule for interacting photons 479
- Macroscopic QED for quantum nanophotonics: emitter-centered modes as a minimal basis for multiemitter problems 491
- Generation and dynamics of entangled fermion–photon–phonon states in nanocavities 505
- Polaritonic Tamm states induced by cavity photons 527
- Recent progress in engineering the Casimir effect – applications to nanophotonics, nanomechanics, and chemistry 537
- Enhancement of rotational vacuum friction by surface photon tunneling 551
-
Part IX: Plasmonics and Polaritonics
- Shrinking the surface plasmon 561
- Polariton panorama 565
- Scattering of a single plasmon polariton by multiple atoms for in-plane control of light 595
- A metasurface-based diamond frequency converter using plasmonic nanogap resonators 605
- Selective excitation of individual nanoantennas by pure spectral phase control in the ultrafast coherent regime 613
- Semiconductor quantum plasmons for high frequency thermal emission 623
- Origin of dispersive line shapes in plasmon-enhanced stimulated Raman scattering microscopy 633
- Epitaxial aluminum plasmonics covering full visible spectrum 643
-
Part X: Metaoptics
- Metamaterials with high degrees of freedom: space, time, and more 657
- The road to atomically thin metasurface optics 661
- Active nonlocal metasurfaces 673
- Giant midinfrared nonlinearity based on multiple quantum well polaritonic metasurfaces 685
- Near-field plates and the near zone of metasurfaces 697
- High-efficiency metadevices for bifunctional generations of vectorial optical fields 703
- Printing polarization and phase at the optical diffraction limit: near- and far-field optical encryption 715
- Optical response of jammed rectangular nanostructures 723
- Dynamic phase-change metafilm absorber for strong designer modulation of visible light 731
- Arbitrary polarization conversion for pure vortex generation with a single metasurface 745
- Enhanced harmonic generation in gases using an all-dielectric metasurface 751
- Monolithic metasurface spatial differentiator enabled by asymmetric photonic spin-orbit interactions 759
