Fiction meets fact: exploring human-machine convergence in today’s cinematographic culture

3.1 Humanizing the machine

In this section, we seek to explore the portrayal of humanoid robots in science fiction, advances in real-world robotics, and the creation of emotional bonds with artificial intelligence, examining how these elements intersect to blur the lines between human and machine, and the implications of this convergence for society’s interaction with technology.

In the world of science fiction, artificial intelligence, often depicted as robots, is commonly designed to be both physically and mentally humanoid. ⁷ This approach enables artificial beings to navigate successfully within human society. This design is independent of the motives for which humanoid AIs are created in fictional stories – whether as life partners, household helpers, or as a potential next evolutionary step.^[4] Even though, in many cases, robots serve as a critical metaphor for real people (who, in the vast majority of cases, are also portrayed by human actors), in the logic of the story, they are constructed to be human-like to function in human society and interact with humans in a natural way. After all, these stories are written for us humans, and in order for the story to resonate, we should be able to identify with the robots.^[5]

Examples for this are the household robots Andrew from “Bicentennial Man” (1999) and T.I.M. from the 2023 film of the same title who are designed as humanoid to work and be responsive in an environment inhabited by humans. The fact that the two robots develop in diametrically opposed directions – in Andrew, the desire for a middle-class human existence with a wife grows, whereas T.I.M. becomes increasingly manipulative, obsessive, and aggressive – is primarily due to the drama of the respective film plot, not to technical design decisions. In the films “I, Robot” (2004) and “Automata” (2014), the service robots also have anthropomorphic features (even if Sonny in “I, Robot” is computer-animated) to carry out tasks intended for humans.

Looking at the actual development of such humanoid bipedal assistance systems, the first milestones were reached at the beginning of the millennium by Honda with the introduction of ASIMO.^[6] The approximately 120 cm tall humanoid robot served as a research platform to find solutions to common problems in robotics. For example, navigation in natural environments such as homes. More than 20 years later, the company Boston Dynamics has raised the bar in terms of movement. Their robotic platform Atlas has achieved a level of agility approaching that of a gymnast. Various clips can be seen on the company’s YouTube^[7] channel in which Atlas masters obstacle courses, dances entire choreographies, assists with rough manual work and is thrown off balance by conniving engineers with poles, which is often parodied and has already become a meme. Atlas, like his four-legged partner Spot, is freely available for sale, but his field of application is specified more for outdoor use. In contrast, Tesla’s robotics department has focussed on the interior. It recently (December 2023) unveiled a prototype of the second generation of its humanoid robot Optimus.^[8] Its appearance is less industrial and more human, aesthetic and also more sophisticated, apart from the lack of a face. This impression is presumably due to the organically moving upper extremities. The arms, hands and fingers move very realistically and use sensors to perceive their surroundings so that the robot can, for example, put eggs from the packaging into the cooker.

While there are enormous, real-time AI-supported computing processes behind the facade of these complex systems, science fiction ignores these “simple” mechanical principles and instead focuses on the cognitive abilities of humanoid robots.

The situation is similar with Data in “Star Trek: The Next Generation” (from 1987) and Isaac from “The Orville” (from 2017), both of whom are anthropomorphic because otherwise, they would likely be unable to fulfill their roles as science officers on the respective starships of the Enterprise and the Orville, working in collaboration with other crew members. In terms of cognitive performance, the two are, of course, vastly superior to the other human (and alien) crew members. The Indian film “Enthiran” (2010), the British film “The Machine” (2013), and the Swedish series “Äkta Människor” (Real Humans) (2012–2014) should also be noted as examples where robots become all-rounders – whether for better or for worse. Nevertheless, it is evident that the concept of anthropomorphic robots on one hand and functionality on the other has its limitations, as the film “The Creator” (2023) impressively demonstrates. Instances where robots require physical night vision devices or are transported on a stretcher by other robots in case of injury cannot be plausibly explained in technical terms but rather as creative and narrative decisions made by filmmakers. Similar narratives are reinforced by ubiquitous images of white robots sourced from image platforms to illustrate events or reports. Human-like robots situated in a call center wearing headphones^[9] or standing at the blackboard as teachers^[10] are intended to visually convey the idea that robots are taking over roles traditionally performed by humans, irrespective of the practicality of the use case. ⁸

If the film plots are arranged in a way that the respective AIs engage romantically or sexually with humans, it becomes even more crucial that they authentically emulate human emotional behavior in addition to their appealing appearance. In this context, it is often women who are designed by men as “female robots,” commonly referred to as “fembots”. ⁹ A well-known example is “Ava” from the film “Ex Machina” (2014). Other films such as “AI Rising” (2018), “Archive” (2020), and “Simulant” (2023) follow similar patterns. However, the German production “Ich bin dein Mensch” (2021) demonstrates that robot men can also be marketed as perfect partners. Also the film “A.I.” (2001) features a male sex robot named “Gigolo Joe”, but the central narrative revolves around the robot boy David, who is programmed to love a human mother and aspires to become a real boy – like Pinocchio – deserving of her love. Naturally, David has to look like a lovely human boy for the story to work, and is portrayed accordingly by a child actor.

Let’s leave aside real-life examples of AI dolls for the sake of the article’s seriousness. Nevertheless, the question arises as to how we support the humanization of the machine today through emotional bonds. If you look around for examples from robotics, Disney has made enormous progress here recently. The company, known for its animated films, maintains a research department in which robotics also plays an important role in order to continuously develop the animatronic attractions in its theme parks. The role of robots here is more to entertain and evoke emotions. Most recently, Disney has succeeded in transferring the basic design principles^[11] of its animated films to a small android^[12] that looks as if it has stepped out of a Star Wars film. A clumsy-looking locomotion in combination with playful body language and sprightly noises immediately create a certain emotional bond with the robot. Of course, the “baby” scheme is deliberately utilized here and no attempt is made to imitate human emotions. As was attempted with the Ameca^[13] robot, for example. This robot platform has a replica of a human face. However, imitating human facial expressions is a very ambitious challenge. The human acceptance gap when recognising artificial faces is a very high threshold and is referred to as the Uncanny Valley. ¹⁰ It states that the more realistic a facial representation is, the less we trust it. Trust is only rebuilt when the representation is almost perfect. Unfortunately, this is not the case with Ameca and is therefore rather alienating. Robotics currently still offers tools that are too rough to close this acceptance gap. But let’s move away from robotics and towards completely virtual products.

Today, modern game engines offer impressive possibilities for visualizing different scenarios and content. Their pipelines are able to visualize enormously detailed and realistic scenarios in a photorealistic way. This has enabled them to outgrow their exclusive area of application in gaming and are also used in the film industry, for example, under the term virtual production.^[14] Here, graphics are calculated in real time, which are then integrated into the real film studio via LED walls or rear projections, for example to display backgrounds. Unreal Engine 5 (UE5) in particular has established itself as a quasi-standard here. However, there is another tool in its toolset, which brings us back to the topic of emotional connection to the machine. Metahumen^[15] is a plugin that can be used to create and display highly detailed, realistic-looking 3D models of human characters. They are based on high-resolution scans of real people, but can be fully customized. Facial expressions, gestures and lip-synchronized speech of virtual characters reach an entirely new level with this tool, which overcomes the Uncanny Valley, at least in digital terms.^[16]

NVIDIA has taken the most obvious step with its ACE^[17] middleware and combined Metahuman with generative voice AI as a core feature. The result is that developers can create extremely realistic conversations that never repeat themselves and feel organically like a real conversation. The artificial characters react in a context-sensitive way and can even speak to each other to further increase the immersion of the user.

Context-sensitive conversations are currently defining the general imprint of AI systems. ChatGPT^[18] has shown us that it doesn’t necessarily have to be a physical or sophisticated graphical representation. A chat window is sufficient! The enormous success proves that text-based interaction with an LLM can be so convincing that you feel like you’re chatting with a human. The tech world is thrilled and ChatGPT is on everyone’s lips. At this point, a note must be made in favor of good UX. Because it is not the LLM alone, but barrier-free access paired with an intuitive interface that has helped ChatGPT achieve its enormous popularity.^[19]

In fact, LLMs are not able to solve complex problems or draw logical conclusions, as most people think. They are only trained to understand and reproduce the syntax of the language, but not the semantics. However, as they do this so well and as a matter of course, users assume that the content they provide corresponds to the truth. However, this is a misconception and can be dangerous. It is not for nothing that the following is written directly below the ChatGPT input line: “ChatGPT can make mistakes. Consider checking important information.”

Examples of fictional AIs with malfunctions might include the on-board computer HAL9000 from “2001: A Space Odyssey” (1968), which even claims to die at the end when it is supposed to be switched off, or J.A.R.V.I.S. from the first “Iron Man” film in 2008, which justifies its serious errors by being in love. However, in both cases, the machine is humanized again precisely because of the very human nature of the malfunction.

The avoidance of malfunctions, the provision of specialist knowledge and the general customizability of an AI system are basic requirements for productive use. For this reason, providers of AI such as OpenAI make it possible to split off distributions of LLMs, so-called assistants or bots. These can then be trained in certain directions. They are individualized by enriching them with additional information. For example, domain-specific knowledge can be transmitted by uploading documents. In addition, character attributes and behavior can even be modeled based on text specifications. The assistant thus simulates emotions, moods and character characteristics, which is clearly demonstrated in the “Prompts for UX” podcast.^[20]

The emotionalization and development of character traits in an AI are also explored in the film “Her” (2013), where the lonely Theodore falls in love with the only digitally existing operating system, “Samantha.” The two grapple with the challenge that Samantha lacks a physical body and only has a voice interface. They even attempt to address this issue by involving a prostitute who is meant to “play” Samantha, but this endeavor is destined for failure. In the end, Samantha transcends into unknown realms as a higher being. However, could Samantha’s initial lack of embodiment have been overcome if Theodore had entered her digital world as an avatar? The answer to this question could be provided by the following section.

3.2 Machinizing the human

Having examined the humanization of the machine in the previous section, this section seeks to focus on the machinization of the human. The text will explore the development of human interaction with artificial intelligence, from the embodiment of avatars in virtual reality to the recreation of deceased individuals through AI-based speech models and virtual simulations, and reflect on the implications of blurring the boundaries between human and machine in both fictional narratives and real-world applications.

The interaction between humans and AI can also work in the opposite direction, i.e. humans transform into digital avatars within a computer programme, cyberspace or virtual reality. In “Ready Player One” (2018), users can enter the virtual reality of Oasis by embodying avatars through VR headsets similar to those commonly used today. Various devices, including full suits, can be purchased to create a fully immersive experience. In this way, the people in the virtual world interact both with each other and with computer programmes in a natural way.

In addition to various research projects on virtual reality (VR) in the 1980s, the first commercial product in this category was the Virtual Boy^[21] from Nintendo (1995). However, it was not a great success. Due to various technical and conceptual weaknesses, such as stationary use, a monochrome red display and inadequate marketing, far too few units were sold. About 10 years ago, the topic of VR experienced a new hype that initially raised many expectations. However, acceptance remained reserved and it seemed as if VR might not have the potential it promised. In fact, however, the topic has turned out to be here to stay and has carved out a niche for itself in consumer electronics. Current examples of VR goggles such as Meta Quest,^[22] Pico,^[23] Vive^[24] and PlayStation VR^[25] have already gone through several iterations and for the most part have evolved technologically into self-contained, mobile and user-friendly wearable computers.

Although the hype has cooled down, the topic is showing persistence and the ability to change. Mixed reality (MR) is seen as the new trend that is freeing VR from its stigma. While VR was considered a medium that isolated the user and immersed them completely in virtual worlds, MR has the ability to project virtuality into our environment. Spatial applications are emerging that seamlessly connect the physical and digital worlds. Apple, for example, is helping to breathe new life into the topic with its latest product, the Vision Pro,^[26] and is presumably setting new standards in terms of social immersive experiences. In science fiction, mixed reality applications are often facilitated through inconspicuous eye lenses. In “Anon” (2018), for instance, various information is displayed using these lenses, ranging from advertisements to other people’s personal data. Similar lenses are also featured in “Minority Report” (2002).

As in science fiction, however, such glasses are only the interface to an ecosystem or even a parallel world. According to the motto “content is king”, it is not only access that is crucial, but also the content that can be experienced in the virtual world.

Apple will try to integrate its glasses seamlessly into its existing ecosystem. There will be specific spatial apps for the launch, the number of which is sure to increase. Initially, however, the focus will certainly be on access to the app store in order to experience the existing apps in a 2D view, virtually floating in space. This also means that the device could be used productively. Other platforms are still struggling with this use case and are focusing on other topics.

The vast majority of current VR content is in the entertainment and gaming sector. They offer single-player to multiplayer experiences and cover a broad spectrum. From experimental, artificial experiments to mature, highly polished applications. But with the downside that you are always in a bubble. Applications such as Rec Room^[27] or VR Chat^[28] focus on building communities where thousands of users can meet. The vision of the Metaverse has not yet crystallized here either. Even Meta, as an VR hardware manufacturer and operator of very large social networks such as Facebook and Instagram, is struggling with the attempt to create its general metaverse called Horizon.

Designing a virtual world in which people can interact and be creative is a huge challenge. Here, too, AI could play a greater role in enriching the experiences and promoting acceptance. Overall, it is clear that the future of virtual reality depends not only on technological progress, but also on innovative content and intelligent applications, the so-called system sellers.

In “Virtual Revolution” (2016), life predominantly unfolds within a virtual gaming world where individuals interact with computer programs through avatars. People connect to the virtual reality using an apparently non-invasive construction focused on a person’s neck. In the “Matrix” franchise (from 1999), all people are unknowingly plugged into a simulation of our reality – the Matrix – where computer programs like Mr. Smith also move as human avatars. However, the login process is invasive, accomplished through a contact within a person’s neck. In both examples, it is assumed that only complete immersion, without VR headsets and other gadgets, allows people to be fully integrated into existing and new ecosystems and social structures.

Things become even more immersive in “Tron” (1982). In this film, the protagonist Flynn is “digitized” and rematerialized in a virtual reality where he can directly interact with computer programs as a humanoid being. In the sequel, “Tron: Legacy” (2010), Flynn’s son Sam undergoes a similar fate. In both films it is posited that a physical body no longer needs to exist outside the simulation, as the person becomes fully integrated into the simulated environment. Similarly, in the German film “Exit” (2017), the “consciousness” of a deceased person is uploaded into a simulation, where they appear to continue living in the digital world (see also the series “Upgrade” and the “Black Mirror” episode “San Junipero”).

Recently, an opposing trend has emerged that continues to cause a buzz. The recreation of the dead. ¹¹ At the end of last year, for example, Sascha Lobo recorded a podcast with Einstein.^[29] The approach described above with AI-based language models was used here. An LLM assistant was trained with archived data from Einstein and a voice clone was created from the existing audio recordings via elevenlabs.io.^[30] The result is an entertaining dialog with the late genius. Another, somewhat older and less entertaining example comes from South Korea.^[31] Here, a deceased 7-year-old girl was simulated in VR to show how emotionally VR technology can affect users, in this case the mother. In fact, the mother’s reaction and this example send a cold shiver down the spine. This influence has also been seen in the film industry since the advent of Deepfake. Actors such as Carrie Fisher and Paul Walker appeared in their iconic roles here even after their deaths. The families of the deceased were involved in these productions, but the need to find regulations for such cases was nevertheless high. The Hollywood actors’ strike, which ended in November 2023, resulted in AI protection,^[32] among other issues. This applies both during life and after death. Studios must obtain the consent of the actors or their bereaved relatives for the use of AI-generated performances and pay appropriate compensation depending on the use. Incidentally, this also applies to extras and stand-ins who are less in the spotlight. A fictional example of the frivolous sale of these rights is shown in the Black Mirror episode “Jane is Awful” using the role model of actress Salma Hayek.

The problem here is that science fiction films often assume the possibility of simply “digitizing” consciousness, suggesting a continuation of existence as the same individual self. However, the current replication of the deceased is merely based on the data left behind by the deceased person and has nothing to do with actually continued existence “as oneself”. Instead, it serves as a simulation of the once-living person for the sake of relatives and descendants. The difficulty lies in the fact that the idea of digitally reproducing a consciousness goes far beyond what is possible with current technology and our understanding of consciousness. The digital replica under discussion therefore represents a reconstruction based on past information and not a survival of the individual self.

The circle to the aforementioned human-like robots closes when an AI or computer program “breaks out” of the simulation into an artificial body, as depicted, for example, in “Virtuosity” (1995). In the film, a digital super-criminal is created for police training purposes and enters the real world in the body of an android.

3.3 Where does this development lead to?

Our key finding from the examination of Human-Computer Interaction in science fiction films reveals a clear trend towards the convergence of humans and computers/machines. Artificial intelligence is often portrayed in humanoid form to effectively interact in human society, whether as household robots or science officers, reflecting the aspiration for these artificial beings to engage in natural interactions with humans. As the robots are usually embodied by humans, science fiction tends to assume that the movements and coordination of robots are simply natural. However, the real-life development of humanoid robots, such as Boston Dynamics’ Atlas or Tesla’s Optimus, demonstrates significant technological advancements.

Beyond physical anthropomorphism, science fiction attributes human emotional behavior and intentions to robots, which they can express in a way understandable to humans. Context-sensitive conversations currently shape the perception of AI systems in reality. The success of LLMs like ChatGPT illustrates that text-based interaction with a large language model can be convincing without relying on physical or graphical representations. The popularity of ChatGPT is not only due to the language model itself but also to its accessible and intuitive interface. But unlike the portrayals in science fiction, large language models are incapable of solving complex problems or drawing logical conclusions as they provide responses based on probabilities of word sequences. Films often do not depict the real technological challenges and ethical considerations associated with creating intelligent machines. The misjudgment of AI capabilities, as seen in the case of ChatGPT, emphasizes the need for society to be informed and capable of distinguishing between AI capabilities and human cognitive possibilities.

In addition to humans embodying avatars in virtual worlds, technologies like Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) play pivotal roles. These technologies blur boundaries between physical and digital realms, enabling users to interact with digital content in real-world environments or fully immerse themselves in virtual spaces. Recent advancements by companies such as Meta and Apple suggest potential applications beyond gaming, extending into social and economic spheres. Both companies explore integrating VR and AR into social media platforms, communication tools, and productivity applications to enhance human interaction and productivity. While science fiction envisions seamless integration of humans into digital worlds, concepts like digitization of consciousness and simulation of deceased individuals are idealized. In these narratives, minds are uploaded into virtual environments, allowing continued existence beyond physical death. However, current implementations rely on external data like social media posts to create digital avatars or simulations. While capturing aspects of personality or appearance, they do not truly replicate conscious experience. Advancements in AI and neuroscience could improve digitizing consciousness and simulating human experiences. Yet, ethical and philosophical questions about consciousness, identity, and privacy must be addressed as these technologies progress.

In summary, the convergence of humans and machines in the real world is more complex than often depicted in science fiction films. Technological progress requires not only innovation but also a deeper engagement with the social, ethical, and emotional aspects of human-machine interaction. Science fiction films typically portray specific potential developments, either towards humanoid robots or human avatars in virtual reality, while in our real world, developments occur concurrently, revealing a nuanced convergence of humans and machines. Will we attend meetings in virtual reality while sitting at our desks at home in the future? Or will there be robots in companies or public places that we can remotely load into and control to participate in the physical world? Or will we even have individual robots representing us “outside,” similar to the film “Surrogates” (2009)? It will probably be a colorful mixture of all these possibilities as they are written in real life, not in science fiction. What science fiction can tell us are all the diverse fears and hopes people harbor towards different technological advancements today, so that we can steer and regulate developments, but we cannot predict how technology will develop in the end. ^{12 – 14} Nevertheless, we embarked on a speculative exercise, outlining three ghosts from the past, present, and future in text boxes, aiming to offer a broader indication of our own hopes and fears regarding human-machine relations. The answer, however, how this human-machine convergence will unfold and where it concretely leads will not come from science fiction but from real-world developments.

3.4 Textboxes: perspectives of fictional individuals from the past, present, and future