Augmented total theatre: shaping the future of immersive augmented reality representations

2.1 Total theatre

The Total Theatre concept embodies the notion that theatre is a convergence of arts, aiming to engage the audience fully by integrating various artistic and sensory elements. This form of theatre gained popularity around 1850, following the writings ¹ of Richard Wagner. He perceived theatre as a form of representation of reality. However, he found conventional theatre restrictive and lacking in audience engagement. As a result, he proposed an enhanced form of theatre: the Gesamtkunstwerk, or “total work of art”. This refers to an artwork in which several artistic disciplines are merged to make a unified, coherent whole, providing the audience with a meaningful and immersive experience.

When applied to theatre, TT implies a holistic approach: TT comprises elements such as music and sounds, lighting, movement, voices, painting, costumes, makeup, design, set design, architecture, literary expressions, and more. These components are composed into a unit greater than the sum of its parts. The essence of TT lies not in the accumulation of these components, but in the interaction and integration among them.

Since Wagner, TT has engaged artists, theorists, and artistic movements. While we won’t delve into the specifics of TT’s evolution here, it’s worth noting the authors who have played pivotal roles in shaping and enriching the concept. Following Wagner, contributors include Maeterlinck, Mallarmé, Appia, Claudel, Artaud, Graham, Grotowski, Cage, Barba, Brook, the Living Theatre, Wilson, and La Fura del Baus. These authors, along with others, have worked to broaden and occasionally deconstruct the boundaries of traditional theatre, with the aim of captivating the audience through innovative and multisensory experiences. ^{2 – 5}

Our focus here is the common thread through many theories and authors who have explored TT: traditional theatre has a restricted way of depicting reality; TT tries to go beyond this mode by incorporating more aspects of reality into the theatrical experience, aiming to engage the audience in a deeper and more comprehensive manner.

Additionally, we find another idea worth considering. It concerns a different view on the concept of Gesamtkunstwerk. This view stems from Wagner’s writings titled “The Artwork of the Future”, ¹ where Gesamtkunstwerk is also understood as the artwork of the future. From this perspective, TT, as a total work of art, can be seen as the future of theatre. As such, TT represents a never fully realized potential, so that its ultimate expression is constantly pursued but never fully achieved. ²

The 20th Century witnessed many realizations of TT thanks to technological evolution. Technological innovations have provided TT with new creative opportunities, enabling the integration of newer multisensory elements into performances. We can think of the use of computerized lighting systems, 3D sound spaces, interactive sets, or video projections on stage. These, and other technological components have enriched the sensory experience of spectators and expanded the limits of traditional performances.

However, there is a boundary that even recent representations of TT struggle to face: the physicality of the real world. In a form of theatre that is total, but still essentially physical and mechanical, even if contaminated by digital interventions, many objects on stage, the stage itself and the actors cannot evade the laws of physics and the materiality of the real world.^[1]

This boundary appears challenging to overcome, even with the most advanced technologies found in TT representations. That is, unless TT embraces a new technology: Augmented Reality. AR seems to be the only technology capable of overcoming the barrier that even the most recent forms of TT encounter.

This notion forms the core of our investigation into the relationship between AR and TT. We believe that the integration of AR with TT could result in a more evolved and enriched form of TT.

2.2 Augmented reality and augmented environments

Before discussing how AR can create an evolved form of TT, it is necessary to introduce some ideas about AR. We have examined AR features in depth in other works. ^{6 – 8} Here, we explore only AR features useful for our discourse. According to a classic definition, “AR allows digital content to be seamlessly overlapped and combined into our perceptions of the real world in real-time”. ⁹ This means that, through a technological device, the user’s view of the real world can be enriched in real-time with digital content to integrate seamlessly with real content and interact with it.

This definition, although correct, is too general for this work. It needs refinement, specifying that the AR we deal with is immersive, thus more complex than non-immersive created in mobile devices. Immersive AR uses wearable devices (Head-Mounted Displays (HMDs)), like Microsoft HoloLens 2, Meta Quest Pro, Magic Leap 2, and Apple Vision Pro) to create the augmented spaces the user is immersed in.

Later, we will make extensive use of augmented spaces, so from now on we defined them more clearly as Augmented Environments (AEs): complex spaces arising from merging real world elements with those created by AR technology.

We treat these AEs as cultural objects, adopting an approach to AR different from the more widespread one that sees AR simply as a technology. We think of AR as a medium, that is, a system of signification and communication.^[2] This perspective allows us to sees AEs as the products of that system of signification and communication, that is, as forms of communication that can be produced, studied, and used in various cultural contexts and knowledge domains like new media, digital arts, communication studies, semiotics, human-computer interaction, etc. ⁶

Next, we examine the most important characteristics of AEs, meant as forms of communication, refining characteristics of AR introduced in the 1990s. ^{9 , 10}

Augmented environments using AR technology are:

1. Augmented: This is a distinctive feature of AR and AEs. A layer of the virtual world is seamlessly overlaid in real-time onto the real world, creating a unified, complex and richer environment.

2. Three-Dimensional: All holograms (virtual objects) in the virtual world are 3D and remain realistically attached to real-world elements. They continually re-adapt their features to the real environment, which, from the user’s perspective, constantly changes in relation to their movements.

3. Interactive: This is not a unique feature of AR and AEs. Many interactive performances have been created in physical environments without AR. Usually, in such environments, users are free to interact with physical objects or even with virtual objects, mostly projected onto walls or onto real 3D objects existing in the environments (thus simulating a semblance of three-dimensionality). However, with AR this interactivity is enhanced. All virtual objects are 3D, can be anywhere within the AEs, and can interact with each other, with real-world objects, and with the user. They react in real-time to visual and auditory stimuli, voice commands or user’s gestures, or to commands given through input devices such as pads and controllers, or from scripts attached to the objects themselves. Most importantly, all virtual objects are programmable [ ¹¹ , p. 49]: thanks to the scripts, it is possible to define the rules for interaction, to modify the shape, color, size, and position, and, in general, the behavior of virtual objects in response to various inputs or conditions that arise during the user’s experience in the AEs.

4. Immersive: Immersivity is not a unique feature of AEs, nor is it new to theatre.^[3] Rather, it is a property that emerges from the other features examined so far. Given an AE which is 3D and interactive, and is also visitable and, in a sense, “inhabitable” by the user, it is reasonable to state that such an environment immerses the user within it.

5. Unconstrained by the physical laws: All holograms within an AEs are not constrained by the physical laws of the real world and can display features and behaviors that go beyond what is possible in the real world (see note 1).

6. Unbound by specific times: All AEs are generated by apps running on HMDs that can be launched at any time chosen by the user. This feature becomes relevant when, as in this work, we talk about particular AEs that are spaces for theatrical performances. Indeed, such AEs hosting theatrical performances free users from time constraint of traditional theatrical performances.

7. Portable: Portability of AEs is a consequence of AR real-time adaptability to various real environments, to its flexibility in interacting with the surrounding physical world. This enables users – regardless of their location – to instantly access any AE. Traditional installations and theatrical works, though portable during tours, require time-consuming setup in new locations. In contrast, AR’s adaptability to various environments is almost instantaneous, highlighting the superior portability of AEs.

8. Enabling Non-linear Storytelling: Any AE can host the representation of a story^[4] in which the user plays the dual role of spectator and main actor. At every stage of the story, the user can interact with the holograms existing within the AE, and their actions and decisions determine the story’s evolution. This allows for the construction of dynamic and non-linear narratives, offering the user a unique, personalized and interactive immersive experience.

2.3 Augmented total theatre enhances (and renews) total theatre

We have gathered the components necessary to define this innovative form of theatre, which we refer to as Augmented Total Theatre. ATT representations are crafted in AEs, harnessing the features of AR and AEs. At the same time, ATT also retains the tools and capabilities of TT, making it an enhanced version of TT. All the features of AEs are also the features of ATT representations. All the objects and actions that exist or are performed in AEs can similarly exist and be performed within ATT representations. Therefore, any author looking to create representations for ATT should take those features, objects, and actions into account, and then use them to give shape to new theatrical representations.

Comparing ATT with Wagner’s Total Theatre, the differences are clear. ATT is multisensory, immersive, interactive, unconstrained by physical laws, and portable. In ATT, the user becomes a performer, with the power to alter the narrative they are immersed in. ATT immerses users in captivating multisensory experiences, well beyond Wagner’s imagination.

Undoubtedly, AR’s role in shaping ATT representations makes ATT a unique form of theatre, clearly distinct from other forms. This is particularly noticeable in the different ways theatrical representations are experienced: in ATT, the experience is typically individual, and requires the user to wear some type of HMD. This perhaps is the most distinguishing feature of ATT, setting it apart from traditional theatrical forms, which value the collective enjoyment of an artistic event as a key aspect of the theatre-going experience.

Now, one might question the necessity of invoking theatre when discussing forms of communication produced with AR in contexts greatly different from theatrical ones, and involving technologies that have little to do with theatre. Why not simply discuss immersive AR experiences without the constant references to theatre?

We believe that there are undeniable benefits to preserving this connection between AR and theatre. In fact, the representations for ATT share many characteristics with the representations created for more traditional theatrical forms. Both at the level of the creative process that gives life to the representations, and in the expressive languages used and in the methods of constructing the narrative structures. This should not be disregarded. By maintaining the AR-theatre connection, we can immediately use established theories and methodologies for theatre and media to design, develop and analyze new ATT representations without the need to reinvent theoretical and practical tools from scratch.

We wish to conclude this section with some considerations on ATT that will prove important in the rest of this work. Our research on ATT originated with the goal of defining an innovative form of theatre capable of combining Wagnerian Total Theatre with the latest technologies for the augmentation of reality. However during the development of this idea, we realized that ATT has an even broader potential than we initially anticipated. Thanks to its characteristics, ATT radically extend the very object of theatrical representation. ATT allows us to “stage” a variety of real experiences and events^[5] that are normally considered thematically unsuitable, narratively uninteresting, or insufficiently structured to become representable events in a conventional theatrical representation.

Such events, although not conventionally theatrical, are all reproducible, or representable, as they are perceivable through one or more of our senses. And if they are representable, then it is possible to create their representation suitable for ATT. Thus, ATT becomes a powerful tool for the production of forms of communication in the arts and in entertainment, but also in education and in culture. And, most importantly, in many events of ordinary life.

2.4 Augmented total theatre in the real world

So far, our discussion on ATT has been primarily theoretical. We have explored the nature of ATT and also established that the use of AR to create representations of events, whether theatrical or not, implies the creation of AEs with all the characteristics already examined, also exploiting all the expressive potential of TT.

Now, the critical question is: are such representations for ATT practically achievable? In other words: once we have identified an event that can be represented, can we concretely create a representation of that event for ATT that can be experienced through a HMD?

The answer is affirmative, though with significant limitations.

An example of a perfectly working narrative AE is Microsoft’s Fragments, a crime thriller for the HoloLens, ¹² where the user must interact with the holograms of some detectives to help them solve a crime. Another example of a complex AE, in this case non-narrative, is the Learning Augmented Environment that we ourselves have developed to introduce the elderly to digital culture. ¹³ According to the perspective outlined so far on ATT, both AEs have all the characteristics to be considered in all respects as real representations for ATT, confirming the feasibility of our idea: ATT is a new theatrical form that can really be used to create immersive theatrical representations.

However, ATT, meant as a tool to guide the creation of representations, must confront reality, and the real process of creation of those representations. And it must also confront the inevitable limitations of this process.

The process of development of virtual and augmented environments largely follows that used to develop videogames, with five main phases: conception, pre-production, production, iteration and review, and finalization. ^{14 , 15} It involves generating creative ideas, detailed development of elements, coding and graphics to create prototypes, testing and refining, and resolving bugs for the final version.

This process is lengthy, complex, and costly, requiring the prolonged commitment of a team of professionals in various disciplines, including game artists, game designers and level designers, programmers, game developers, project managers, sound designers, audio engineers, cinematic and visual special effects artists. The time span from the inception to the launch of the finished product, whether it is a videogame or an AE, can stretch from months to years. And this situation remains true, although on a smaller scale, even for smaller and more manageable products.

This description of the process to create AEs – which, we recall, are representations for the ATT – helps us understand the meaning of our previous statement: we can create representations of many events, including non-theatrical ones, but with significant limitations. Limitations that arise precisely from the process necessary to create such representations. The process does work (otherwise, we would not be able to produce videogames and games in AR and VR). However, it requires a significant commitment of human and economic resources that inevitably require a return on investment. So, it is not realistic to think that such resources can be used to create non-business-oriented representations, or representations of events that are not particularly important, such as everyday events.

Currently, overcoming these limitations seems impossible. Eliminating them would need a radical change in the process of creating augmented and virtual environments, which is currently hardly achievable.

In conclusion, the idea of an ATT capable of representing many events in the world is possible and has great potential, but becomes difficult to realize when faced with reality. And as long as this situation remains, the ambition of the Augmented Total Theatre is significantly curtailed.

3.1 Technologies for the creation of multisensory virtual spaces

From a technological standpoint, the primary objective of technologies for immersive AR is to create devices that completely deceive our senses. When confronted with a person, or a fragrant flower, a monument, or a cup of tea on a table, if we can no longer discern its nature (is it real or virtual?), the goal of AR will have been achieved.

The current state of immersive technologies does not yet suffice to fully deceive human senses – a pivotal condition for creating augmented worlds that Augmented Total Theatre aims to realize. Current research developing these technologies primarily focuses on sight and hearing, as these are the senses we use most to construct the most significant parts of our knowledge of the world. Yet, touch, taste, and smell are also important for understanding the world, and AR cannot overlook their existence.

Thus, technologies for sensory deception aim to create distinct virtual spaces for each sense. An ideal device for experiencing fully multisensory immersive AR would integrate technologies that deceive all senses by constructing a virtual multisensory space that seamlessly overlaps with the real multisensory space.

Current technologies have not yet achieved this goal. To assess how close we are to such an ideal device, we will examine one by one the sensory spaces created by our senses, and also the technologies for creating virtual elements within those spaces.

3.2 Ten years in the future: the device for multisensory augmentation of reality

Now we combine the components examined in the previous sections to create an ideal Device for Multisensory Augmentation of Reality (DMAR), as shown in Figure 1 (with the mentioned components clearly visible).

Figure 1:

Four components of the DMAR, each designed to produce stimuli in the different sensory spaces: visual, auditory, olfactory, and gustatory (image by the author using DALL-E).

In ten years, the display will likely be the only visible component, albeit less so than in Figure 1. As for the other components, it is reasonable to predict that in the next decade they will become smaller, more fashionable, and less invasive.

Sensors, cameras, speakers, microphones, and modules for environment tracking and mapping, and for connectivity may integrate into the display, as it happens with current devices. However, more likely they will be concealed into smart clothing, gadgets, jewelry, or smart piercings (in the nose, tongue, ears) and smart tattoos. ^{30 , 31} And regardless of those components size and shape, they will be not far from the head, given that 4 of the 5 human sensory organs that need to be deceived receive their inputs from various areas of the head and face.

Micro-displays implanted in contact lenses could replace larger displays (see box in Figure 1), assuming that in a decade display technology achieves retinal resolution within a few square millimeters ^{32 , 33} and fully caters to all vision-related needs as discussed in Section 3.1.1. The adoption of contact lenses would eliminate the need for a visible display, thereby contributing to the creation of an invisible, comfortable Device for Multisensory Augmentation of Reality for everyday use.

To complete the design of our ideal DMAR, we must also consider the interfaces the user will wear to create a virtual tactile space: haptic gloves and suits (see Figure 2). As discussed in Section 3.1.4, existing interfaces are not perfect at creating believable virtual tactile spaces, but advancements are expected in ten years. Haptic interfaces will be lighter, more comfortable, and capable of simulating a broader range of tactile sensations. However, we don’t foresee any groundbreaking transformations in those interfaces. Indeed, we imagine that despite potential enhancements in comfort and performance, the prospect of donning gloves or a full haptic suit in everyday situations might not be appealing or particularly useful to the majority of people.

Figure 2:

The fifth component of the DMAR: haptic gloves integrated within a haptic suit, designed for the production of tactile stimuli (image by the author using DALL-E).

We believe that high-performance haptic interfaces will be used mostly in controlled environments (such as Labs or Research Centers, or Gaming Centers or Haptic Simulators) for specific experiences like games, rehabilitation, training, or other special activities requiring artificial tactile stimulations throughout the body. For everyday use, we instead foresee the development of more affordable, discreet solutions like haptic bands, patches, or clothes with integrated haptic actuators on body parts more exposed to contacts.

All other DMAR components, like CPUs, GPUs, RAM, power cells, and storage devices will be “everyware”, ³⁴ wirelessly connected to each other and the Cloud. They will be distributed across smart gadgets, ornaments, piercings, and clothing. Ultimately, the entire system will be powered by the wearer’s body movements and heat.

Wearable technologies are not a novelty. ^{35 , 36} However, in the next decade, technological advancements will further miniaturize these wearable components, enhancing their efficiency, reducing their energy consumption, making them more affordable, and more stylish. So much so that wearable computers will truly be everyware. And they will be invisible, unnoticeable, on our bodies and within our bodies. Which is precisely what we believe will happen with our DMAR.

3.3 Beyond hardware: the generator of representations for augmented total theatre

Finally, we can focus again on the Augmented Total Theatre, the central topic of this work, addressing the questions that concluded Section 2.4: What could be the future of ATT? Will future technology allow the creation of representations of events for ATT through a fast, efficient, economical process that does not require significant human resources? Will it be possible to integrate into ATT this new way of creating representations of events?

We will therefore describe an ATT from the near-future, ten years from now. And, obviously, such ATT will be equipped with the DMAR previously discussed.

We begin with some general observations. 2023 marked the rise of Artificial Intelligence (AI), specifically Generative AI systems, or Large-Language-Models (LLMs) like ChatGPT, Claude, Bard, Gemini, etc. These are forms of Weak Artificial Intelligence (WAI)^[7] that have rapidly gained popularity, impacting various production processes in a variety of fields. We believe these AI systems, in more advanced versions, will also play an essential role in the evolution of ATT.

Currently, Generative AI systems, using textual, vocal, and visual inputs, can produce high-quality images, videos, diagrams, music, voices that speak and sing, programming code, 3D models, novels, scripts, and a range of texts suitable for the most diverse personal and professional contexts. And almost every day we discover new ways to use Generative AI systems to create forms of multimedia communication that increasingly resemble works of human ingenuity.

Particularly, we think of the so called GPTs, or ChatBots, or AI Agents, that is, customized Generative AI systems created and trained by individual users or organizations. Currently, these AI agents can handle a variety of tasks efficiently and swiftly, using multiple knowledge bases that contain sets of specific information on an individual, or a topic, or a domain of knowledge. After training, AI Agents can act as digital alter-egos of the users who created them and fed them with personal information, or they can also take on specific professional roles, assisting flesh-and-blood professionals in their work.

A year ago, all of this was hardly imaginable. At the moment, these AI agents are imperfect: on various occasions they are unreliable, and produce disappointing results. But we are only at the beginning. Considering the speed at which they are evolving, it is easy to imagine that, regardless of their AI level, in ten years they will be able to take on the roles of real professionals across a variety of domains of knowledge. And they will most likely be economical, reliable, efficient, and fast.

We believe these Generative AI systems will be crucial for the development of a powerful ATT. This new ATT, far from being just a theoretical tool, will have the capability to create in real-time the representations of numerous events in the world, and will also facilitate their immediate use through the DMAR. We envision this new ATT as a concrete device that integrates the DMAR with a unique piece of software, which we refer to as the Generator of Representations for ATT (GR). The GR is essentially a management system for augmented multisensory immersive experiences. Its function is to ensure the smooth operation of the various hardware components previously examined (the DMAR), which are essential for deceiving human senses. More importantly, the GR is also responsible for the real-time creation and management of all elements of the virtual multisensory space that seamlessly blend with the real multisensory space to create immersive augmented experiences.

We can even try to imagine some details about the operation of this GR.

The GR will have at its disposal several custom knowledge bases, both private and public, stored on the Cloud or on local storage devices. These knowledge bases will contain information to create a wide range of immersive experiences. The GR will also coordinate a team of AI agents, each being the digital equivalent, in terms of knowledge, skills, and ability, of one of the professional figures (e.g., game artist, designer, developer, sound designer, visual effects artist, etc.) involved in the process of creation of representations of events examined in Section 2.4. These AI agents will all be connected to each other and to the Internet, constantly updating their knowledge and skills. The GR will manage and coordinate these AI agents, deciding how and when to use their specific skills during the process of creation of a particular representation of an event.

Building on this basic description of the GR, let’s try to think about its operation when a user, equipped with the ATT, is faced with an event (see footnote⁵).

The GR, using DMAR’s cameras and tracking systems, determines the user’s location and creates a 3-D map of the environment (or, to speed up the process, uses pre-existing maps, like Google Map Live View does). It also identifies the event type using a catalog of previously classified events in the knowledge bases.

If a user interacts with the GR, asking for example to augment the event they are facing with a story (maybe an adventure, mystery, fantasy, or romance story), the GR gathers information about the user (their age, interests, cultural background, education level, etc.), and about the environment (its history, geography, cultural and artistic significance, etc.) from the knowledge bases. Then, it instructs storytelling AI agents to craft a story fitting the environment and the user’s interests and request. The story will also be interactive, so that the user can change its evolution through real-time interaction with the characters and objects of the story. To concretely make that story alive in the multisensory virtual space the user will perceive, other AI agents create all the story elements (3D holograms of people, things, and environments, sounds and voices, gustatory, olfactory, and tactile stimuli) in the various sensory virtual spaces, and also define the behaviors of such elements. In almost no time, the story is ready for the user to experience.

3.4 Immersive encounters: joining Monet at the Jardin d’Eau through augmented total theatre

We would like to conclude this work with a real-world scenario that showcases the capabilities of the ATT device described in the previous sections. Here, we will see the ATT of the future at work. We will see how the ATT, through augmented reality, can profoundly enhance and enrich a user’s experience by merging history, fiction, art, and reality.

In this section, we explore the immersive journey of a user named Paola, as she uses her ATT to bring a literary world to life in contemporary Milan. Paola lives in Milan in the year 2035. She has just finished reading Michel Bussi’s Black Water Lilies, a compelling novel written back in 2011 and set around the 1960s in Giverny, Normandy, where the impressionist painter Monet worked until his death in 1926. Paola greatly enjoyed the book and now wishes to personally visit the locations described within. However, she realizes that a trip to Giverny would not suffice, as the Giverny of 2035 no longer resembles the town depicted in the narrative. Furthermore, Paola seeks to experience the impossible: not only does she want to meet Fanette, the young painter protagonist of the novel, but she also wishes to resurrect the Master painter, Monet, who died many years before the fictional Fanette was born, and to observe him as he works on one of his famous water lily paintings. Paola’s desire is complex: she wants to step into a chapter of Bussi’s novel that he never wrote.

She shares this wish with her ATT device that she is wearing while strolling toward Parco Sempione, a large park in the center of Milan. The ATT’s GR briefly engages in a dialogue with Paola to more precisely understand the shapes of her desires. Once this dialogue is over, the GR integrates this new information with existing data about Paola stored in one of its knowledge bases. The GR is now ready to create an augmented representation for ATT that can fulfill Paola’s requests. It immediately geolocates Paola: she is on Via Montello, a street not far from Parco Sempione. The GR accesses city maps provided by Google and prepares a reconstruction of the path that will lead Paola to the small lake in Parco Sempione. Meanwhile, the GR has relayed Paola’s requests to its team of AI agents. The storytelling experts, having already read the digital version of Black Water Lilies provided by Paola, possess a thorough understanding of the book and its characters, and have crafted a script for a new, phantom chapter – a brief narrative set in an imaginary Giverny that foresees a meeting between Paola and Fanette, followed by an encounter with Monet at work on a new painting. Other AI agents specialized in 3D modeling are still scouring the Internet for photos, possibly some video footage, maps, prints, and drawings of Giverny from 1920, to gather all elements necessary for creating a realistic interactive reproduction of the town, its people, their attire, automobiles, sounds, scents, and more. Additional AI agents are tasked with creating high-definition, life-size avatars for the two protagonists: Monet and Fanette, while other AI agents work on writing scripts that control the behavior of all interactive holograms in the augmented representation. Once they all have completed their tasks, Milan of 2035 is ready to merge with a Giverny of 1920 into a single, imaginary augmented city, within which Paola can live out her desires.

Paola is still on Via Montello, and the buildings she saw moments ago have disappeared: now, she sees only low houses typical of the rural dwellings of Giverny from 1920, with rough stone walls and peeling plaster. The windows boast flower boxes overflowing with petunias and geraniums. The slate roofs gently slope, the street is cobbled, and the men walking along the street wear suspenders and wide-brimmed hats, while the women are adorned in long skirts and colorful shawls. Suddenly, behind Paola, a black Citroën with huge, thin wheels, typical of the 1920s, noisily appears and slowly drives away.

Soon, Paola reaches Piazza della Lega Lombarda, which has now transformed into the main square of Giverny. She inhales the sweet scent of flower gardens surrounding the Moulin des Chennevieres, which is right in front of her. The sound of the mill’s water blends with the chirping of birds and the rustling of leaves.

Paola continues her walk in Parco Sempione, now transfigured into Giverny’s Jardin d’Eau. The paths around her are still those of Parco Sempione, which Paola knows well, but this time the flowers, flower beds, and plants mimic those from Monet’s iconic works. As she approaches the small Ponte delle Sirenette, she is thrilled to find that it closely resembles the well-known Japanese Bridge, complete with colorful water lilies floating on the water below. As Paola crosses the bridge, she glimpses the two figures she had wished for: Fanette, brought to life from the pages of a book, stands before her easel, painting. Nearby, at her side, is Monet, alive once more thanks to the almost-magical abilities of the ATT. With his black hat and beard, wearing a blue jacket and dark trousers, palette and brush in hand, he closely resembles the painter depicted in Renoir’s famous 1873 painting, Claude Monet Painting in His Garden at Argenteuil. However, this Monet looks very real, and much older, for it is now 1920. Paola approaches Master Monet; his canvas displays water lilies of various colors – dark blue, green, burgundy – alongside pale pink, intense pink, and light blue lilies. The water lilies seem to move in the clear water that reflects the white clouds in the sky. The scent of the water lilies intensifies. Fanette notices Paola approaching, turns around, and greets her, smiling just as she did in the book …

We leave Paola to continue her walk in the augmented environment that her ATT device has crafted for her. Later, she will have the opportunity to converse with Fanette, as outlined in the script created for the occasion by the AI agents. And she will also be able to get even closer to Monet and observe him at work, just as she requested, while remaining silent so as not to disturb him. Eventually, Fanette will accompany Paola out of Giverny’s Jardin d’Eau, towards her quaint brick house, marking the culmination of Paola’s journey into her desires …

In almost no time, the ATT of the future, through its Generator of Representations for ATT has transformed Paola’s fantasy into a vivid multisensory story to be experience through the Device for Multisensory Augmentation of Reality. Thanks to the efficiency of AR technologies of the future – which, we must remember, are also technologies for sensory deception – Paola will no longer be able to distinguish the familiar and real Milan she knows well from the multisensory virtual Giverny. She will simply have the perfect illusion of being immersed in the 1920 Giverny crafted by her ATT device: a unique form of reality that is rich, interesting, instructive, engaging, and stimulating.