An Interactive Biography of the Survivor or a Survivor’s Hologram? Novel Methods of Collecting Holocaust Oral Testimony and Their Determinants

2.1 Interactive Biography or Interactive Testimony?

How do the authors of this new initiative define “interactive biography”? Let us begin by retracing the etymology of the term. Recapitulating the long tradition of biographical research exceeds the scope of this essay and attempts at summarizing its sprawling history could never reflect the complex and interdisciplinary character of the field.^[17] Consequently, I will restrict myself to bringing up solely the definition that will serve as a departure point for a subsequent inquiry into the nature of interactive biography. In its most fundamental interpretation, a biography describes the life and work of a usually real-life figure: “a treatise of an encyclopedic character,” covering the life of the person in question.^[18] In her essay, “Porządki w bibliotece” (Tidying the Library), Anna Nasiłowska writes that “biography is a genre addressed to a broader audience, […] on account of its narrative nature” (Nasiłowska 2019), and its pedagogic (Skrzyniarz et al. 2014) aspect is its essential component. An autobiography, therefore, is a particular strain of the biography genre, with the individuals themselves writing the account of their own life. Unlike “biography,” it is the other portion of the term in question that demands a more comprehensive elucidation. The term “interactivity” hails from the field of communication. Victoria Grace Walden states that “[i]nteractivity is an umbrella term which seems to cover everything from simply pushing buttons in a museum to radical forms of democratic participatory that attempt to change social order. Holocaust memory has itself long been interactive” (Walden 2021, 278).

Interactus is Latin for “mutual act,” and denotes entry into interaction. The biggest Polish dictionary, meanwhile, defines the term through the lens of computer sciences and software, as the “capacity to receive user input and react to it.”^[19] Finally, a management encyclopedia lists “the capacity for reciprocal action by parties in communication”^[20] as the basic meaning. These are far from the only meanings of the term, which has multiple theoretical concepts describing it and several divergent interpretations (Kiousis 2002, 355–383;^[21] Downes and McMillan 2000, 157–179; Bucy 2004, 373–383). The aforementioned interactivity^[22] has long been acknowledged as a fundamental concept underpinning new media. One of its definitions was proposed by Yuping Liu and L. J. Shrum, whose paper What Is Interactivity and Is It Always Such a Good Thing? Drew on a variety of fields, including psychology and social sciences, to interrogate the nature of interactivity and outline the conditions under which it can emerge and exert an influence (see Liu and Shrum 2002, 53–64). The authors begin by bringing up earlier findings on the topic at hand, including the conceptual work of Jonathan Steuer, who suggested defining interactivity as “the degree to which users of a medium can influence the form or content of the mediated environment” (Steuer 1992, 73–92) and a “stimulus-driven variable, and is determined by the technological structure of the medium.”^[23] In their definition of interactivity, Liu and Shrum include the conditions under which it may emerge. They write about three potential ways of defining interactivity, depending on whether it is focused on interaction between two users, a user and a machine, or a user and a message (Liu and Shrum 2002, 54). Then, they suggest another three-pronged taxonomy, this time based on the dimensions of interactivity: active control, two-way communication, and synchronicity (see Liu and Shrum 2002, 54–55). Dominik J. Leiner and Oliver Quiring also point out one aspect of interactivity previously unmentioned here, namely what it means to the user of a medium (Leiner, Quiring 2008, 127–155). According to the authors, it’s the effectiveness of producing a reaction, impact on the course of events, responsiveness of new visual objects to input and user control over the communication process that primarily constitute the meaning of interactivity for the user (see Leiner, Quiring 2008, 130).

Drawing on all the theoretical concepts recapitulated above, I formulate my own operative interpretation of interactivity for the purpose of this essay. Herein, the concept is defined as synchronous, simultaneous, and reciprocal action between at least two actants. In this particular case, the interaction will be of the user – machine variety, and manifested for the user in the ability to shape the medium in real time, and for the device in the capacity to receive and process information, and react to input. A key role in this interpretation of the process will be played by the total time^[24] of the exchange and the total time it takes for a stimulus to produce feedback.^[25] As Stephen Smith added,

the Dimensions in Testimony project is not an attempt to introduce interactivity to Holocaust memory, but rather a project driven by a desire to preserve the interactivity already inherit to listening and responding to survivor testimony. Smith’s comment acknowledges that this face-to-face learning encounter is already experiential and active, not didactic and passive (Walden 2021, 278).

How then do the scholars behind the Dimensions in Testimony define the interactive biography? The project’s authors say that the methodology underpinning the interactive biography concept drew inspiration from the fact that survivors during their engagement with audiences not only told their personal histories but also engaged in “question and answer” sessions. While the concept was fleshed out in Digital Holocaust Memory, Education and Research edited by Victoria G. Walden (2021), no efforts have yet been undertaken to exhaustively interrogate its theoretical foundations. An attempt at a definition can be found in the writings on the USC Shoah Foundation website, and attendant findings of Sanna Stegmaier and Svetlana Ushakova, who argue that the interactive biography “enables viewers to ask questions of the survivor and hear responses in real-time, lifelike conversation.”^[26] The two scholars say that:

The Russian and German team developed their respective terms separately, they came to similar conclusions regarding terminology. For instance, both teams did not want to reproduce the English term “biography” as its transnational communication lacks the nuances of German and Russian expression, which refer to both the product of the interview as testimony and the role of interviewee as witness to history. The Russian-language team calls the interview “интерактивное историческое свидетельство,” which translates to English as “interactive historic testimony.” Similarly, the German team agreed on “interaktives Zeitzeugnis.” As a composite of the words “time” and “testimony,” the term “Zeitzeugnis” has no direct translation in English. Unlike the US concepts of “survivor” or “witness,” “Zeitzeuge” situates the survivors’ stories between history and biography, translating as “testimony of time.” This allowed the team to reflect on both the role of the interviewee as witness to history and the product of the interview as testimony. At the same time, however, the term situates the project in a controversial debate about the particularity of the Holocaust. While “Zeuge” has been ascribed to the particularly Jewish experience of World War II, “Zeitzeuge” and its respective product “Zeitzeugnis” apply to all witnesses of World War II (Stegmaier and Ushakova 2021, 90–91).

Given the attempts at definition brought up above, I would like to reach back to the term “biography,” discussed in the opening of this section. If we assume that it is a written account of a person’s life (bíos is Greek for “life,” while gráphō is “write”), who then writes down the life’s story of survivors? We could arguably call biographers those who interviewed the survivors, reconstructing the courses of their lives through hundreds of questions.^[27] But this does not make them biographers in the strict sense of the word, as their efforts are more of a trigger that sets off the survivors’ own telling of their story. Although recorded earlier, it is the survivors (or, more precisely, their two-dimensional representations) meeting with the viewers that precipitates the public retracing (via software) of their lives. The issue grows complicated here, as in the aforementioned survivor-viewer interaction; it is the latter that triggers the responses of the two-dimensional representation of the former. Furthermore, the course of the retelling depends on the choices of the viewer, and is constructed in response to the questions, which push the narrative forward, but without necessarily preserving its specific causal order, potentially leading to achronological testimonies. One of the project’s characteristic features is its non-linear narration. Explaining the guidelines behind the project, the production process, and the premises underpinning the first recording captured in 2014, Stephen Smith said: “[w]hat makes this so different is the nonlinear nature of the content. We have grown used to hearing life histories as a flow of consciousness in which the interviewee is in control of the narrative and the interviewer guides the interviewee through the stages of his or her story.”^[28] Consequently, in the context of the two-dimensional survivor representations, should not we be calling the form an interactive interview, in which the interviewee tells their story in response to questions posed by the interviewer? The museum visitor stands in for the interviewer, whose questions (although asked during the recording) are removed as they are replaced by another interlocutor. Personally, in reference to the Dimensions in Testimony project, I suggest using the term “interactive testimony,” as, on the one hand, it reflects the reciprocal interaction of the viewer and the recorded survivor, while on the other emphasizes the narrativity of the survivor’s responses and the (auto)biographical character of their retelling. It contains a trace of their synchronous mutual interaction and what the team working on the German version of the project has noticed – namely that the recorded narratives of the survivors are not just their biographies, but first and foremost a testimony of the times of war that they lived through – their fates interwoven with a specific moment in history. “What ultimately matters in all processes of witnessing, spasmodic and continuous, conscious and unconscious, is not simply the information, the establishment of the facts, but the experience itself of living through testimony, of giving testimony” (Laub 1992, 85).

2.2 Technology and Interactivity, or on Programmed Testimony

Although the creators of the project stress that the technology – albeit crucial for the final effect – remains secondary to the stories offered by the recorded survivors, I would still like to take a moment to focus on the technological aspect of the project to explore the mechanisms behind the innovative method. Here, I will be drawing primarily on the work of scholars from the University of Southern California’s Institute for Creative Technologies, including Antona Leuskiego, Davida Trauma, Rona Artsteina, Alesii Gainer i Ariego Shapiro,^[29] whose writings informed the methodological foundation of my interpretation of the project platform’s operational framework. At the core of the platform is the NPCEditor, which Leuski and Traum (2021) call a:

system for building and deploying virtual characters capable of engaging a user in spoken dialog on a limited domain. The dialogue may take any form as long as the character responses can be specified a priori. […] At the core of the system is a state of the art statistical language classification technology for mapping from user’s text input to system responses.^[30]

To put it more simply, it is a piece of software that selects responses to user queries from a pre-arranged repository, similarly to popular voice assistants. Consequently, “NPCEditor combines the functions of Natural Language Understanding (NLU) and Dialogue Management – understanding the utterance text and selecting an appropriate response.”^[31] Naturally, all of the replies of the virtual witness are actually recorded statements previously uttered by a person, without any additional lines generated by the software. However, the responses are still selected by an algorithm from the repository throughout the act of storytelling or during conversations with visitors. “The interface of Dimensions in Testimony demonstrates a previously unknown form of engaging with archival material. By combining the aim to preserve survivors’ legacies with making their testimony accessible with interactive technology” (Stegmaier and Ushakova 2021, 66). This is the first project of its kind to allow engaging the public with witness testimony using voice recognition and featuring an interface that designed to prevent users from realizing that they are drawing on what is essentially an archive (see Stegmaier and Ushakova 2021, 66). Using the aforementioned natural language technologies, “the Dimensions in Testimony system transforms asked questions into search terms. The system then matches the search terms to the most appropriate interviewee response to your question and plays back the associated video clip, resulting in a conversational-like experience.”^[32] After her encounter with the virtual Pinchas Gutter, Hillary Jackson wrote that it felt:

similar to using Siri on an Apple device. When the system doesn’t recognize a question, or if you’ve not spoken clearly, a polite Gutter says, "Just repeat that." But instead of interacting with a computer, the experience is more human. Gutter’s responses are authentic. They’re just recorded with future dialogue in mind. His account feels natural, like an in-person conversation. It isn’t edited down and there are 15–20 second pauses where Gutter collects his thoughts. While his response to the question above is moving, it goes in a slightly different direction than anticipated, much like many in-person conversations do.^[33]

And indeed, “the hologram of Pinchas engages with the audience; it answers their questions; it reacts to the external stimuli” (Zalewska 2016, 29). The representation also seems to be creative, emotionally engaged, singing songs from his childhood, for example. “However, it also reaches its cognitive limits: it is able to give only what it has previously received from its makers” (Zalewska 2016, 29). On the one hand, the announcements accompanying promotional efforts for the project promised that:

Gutter not only will retell his own story, but also will be able to answer questions from his audiences about the general Holocaust experience. If asked how life was at Auschwitz, for example, he will respond with “I wasn’t at Auschwitz, but I can tell you about my experiences at Bergen–Belsen” – responding as if he has agency, subjectivity. (O’Neill 2016, 46)

On the other, however, we know that even this redirection had been programmed into the software, essentially reducing the survivor to a recording that is a mere animate representation, which no longer has need of its maker (see Sierzputowski 2018, 130), but requires only a custom software suite. The interview conducted for the purpose of the project enabled the creation of a pool of behaviors and responses, which were then coded to be played back after certain triggers. “Rather, the computer and its sensors ‘hear,’ ‘see,’ and ‘speak’ through him [the survivor – author’s note]. His movements, the way he turns to look at his questioner, the tone and pace of his voice, are all programmed as a set of differential responses to certain types of sound patterns” (O’Neill 2016, 46). The representations themselves – including their outfits, hairstyles, and overall appearance – are also hard-coded into the software. In one of his interviews, Pinchas Gutter said: “I had to wear the same clothes and I had three pairs (sic) of the same jackets, the same shirts, the same trousers, the same shoes.”^[34] This is another constraint of the recording process, forcing the survivors to wear the exact same outfit to each session.^[35] In his interview from December 29, 2021, Steven Frank, a survivor recorded for the Forever Project, said, “clothing was important. No stripes or square patterned shirts for filming purposes. Also shoes!!” ^[36] Later on, he added, “at the end of one session in the FP and the beginning of the next session one had to be in exactly the same position, and this was carried out with ‘forensic’ precision for the smooth continuation of the filming. If not done there could be a ‘shudder’ (slight movement) between stop and restart.”^[37] These remarks suggest a tension between the human and non-human aspects of the project. We could argue that what we are dealing with here is not only a programmed or directed testimony, but a media-hibernated body, artificially preserved beyond the normal passage of time.

2.3 Engaging with Interactive Survivor Representations in Practice

The USC Shoah Foundation takes great care “to not edit, alter, manipulate, or censor any response the interviewee gave. The Dimensions in Testimony post-production methodology has been entirely created to preserve the integrity of the interviewee’s voice, a promise made to the interviewees and their families.”^[38] In “New Dimensions in Testimony Demonstration,” the aforementioned scholars, headed by Ron Artstein, explained the technicalities of the project, writing that ”a typical installation is run on a 15-inch MacBook Pro with Retina display, connected via HDMI to an external monitor or television” (Artstein et al. 2016, 34). For the purpose of the presentation they used from 22-inch for the personal interaction to a large size theater projector, but the most preferred display – as the authors write – “is an 80-inch high definition television in vertical orientation. This allows showing the speaker at approximately life size, making it appropriate for one-on-one and small group interaction, as well as large group interaction in a theatre setting.” (Artstein et al. 2016, 34).

For small, informal demonstrations in a quiet setting, we have had good results using the MacBook Pro’s built-in microphone for audio capture, and the built-in trackpad as a push-to-talk button. In more challenging environments we use a Sennheiser HSP-4 headworn microphone, which works well to isolate the user’s speech from the background noise. (Artstein et al. 2016, 34)

The Foundation also supervises partner institutions and exhibitions that use its recordings to ensure that the representations are always used in an ethical manner. It is primarily about how the objects are displayed, in what setting of the exhibition, whether survivor and his story are properly exposed to ensure the best impact, and properly care from the museum.^[39]

A sample encounter with an interactive survivor representation unfolds as follows: the survivor tells their life story over a dozen minutes or so, after which the museum visitor can ask them questions about the Holocaust, their lives before and after the war, and what they think about the problems of the modern world. Consequently, this manner of visual presentation differs significantly from typical displays of video testimonies in museums. The latter mostly involve playing back a recorded interview conducted by someone else, whereas Dimensions in Testimony actually produces the impression of having an actual conversation with a two-dimensional representation of a survivor, with genuine questions being asked and prompting answers from a pre-programmed interlocutor instead of just triggering pre-recorded footage. For example, the Gutter database contains answers to anywhere between 20,000 and 30,000 questions.^[40] Still, Maria Zalewska writes, “Since the set of pre-programmed questions is fixed, the hologram’s account does not meet the criteria for a live witness testimony” (Zalewska 2016, 29). This means that the encounter entails not an actual, here-and-now conversation, but a pre-arranged exchange or just one of a variety of strictly plotted conversations.

The USC seems to be promoting this new method of bringing survivors and recipients together as a self-sufficient experience (see Frosh 2018, 363), whereas Kansteiner, drawing on his research, writes that the conversation it entails only appears spontaneous, and that the holograms of the survivors are very good at telling stories about the past, but stumble when asked about the present (Kansteiner 2017, 122). And indeed, when Lesley Stahl of CBS’ 60 Minutes spoke on April 5, 2020, with a recording of Aaron Elster (who died in 2018), aside from asking about his life story, which prompted a flowing story from the representation, question him over the day’s weather, curious about his reaction, to which he only replied: “I’m actually a recording. I cannot answer that question.”^[41] In another interview, the hologram of Pinchas Gutter replied “Can you rephrase that, please?” when the software could not parse the question and generate an appropriate answer,^[42] clearly indicating the limitations of this form of interaction.