From idle to interaction – assessing social dynamics and unanticipated conversations between social robots and residents with mild cognitive impairment in a nursing home

2.1 Research approach and stages

Our study was conducted in a large nursing home over 4 months, including a 10-week evaluation phase, and followed a co-design approach. This involved residents, their carers, a former manager, and researchers from various disciplines in an active exercise to obtain helpful information on the daily activities, memories, social environments, experiences with technology, and attitudes towards its use.

Developing appropriate technologies, such as exergames ⁴⁶ performed by humanoid robots, to facilitate physical activity and interactions in the context of nursing home residents and their professional caregivers requires involving potential users from the outset. ^{18 , 47 , 48} It is necessary to recognize different viewpoints, institutional habits, social and individual routines, and how they combine with technology. Therefore, this study was set up as a design case study, as originally outlined by Wulf et al. ⁴⁹ This approach consists of three phases (see Figure 1): (1) a pre-study involving an empirical analysis of existing individual and social practices within the empirical setting; (2) the co-design of innovative ICT-based artifacts informed by the findings of the pre-study; and (3) a long-term investigation into the interaction with and appropriation of the developed technological solution. ^{49 , 50}

Figure 1:

Timeline of the study.

In the pre-study phase, we examined existing practices, organizational and social perspectives, and the challenges residents and their care networks face in their daily environments. The second phase involved an iterative process of developing and deploying robotic-based prototype applications for the Pepper robot and its tablet. These were integrated within the nursing home, allowing residents and caregivers to utilize the system on a daily basis. We conducted a 10-week evaluation during the final phase, deploying the system in the nursing home environment. The sensibility of this environment requires close collaboration with the care home and the participants. Therefore, we visited the nursing home twice a week, conducted training sessions, and regularly spoke to participants within their environment. This phase concentrated on developing a concrete understanding of how it is possible to use technological solutions effectively in ways that meet the needs of caregivers and residents.

2.2 Technical infrastructure and implementation

Based on the apps available for the Pepper robot in the previous work, ^{18 , 51 , 52 , 53} we developed more memory games, reaction games, and some cognitive games for people with MCI. These new apps were focused on the system’s effectiveness in improving health, its integration into daily routines and the contexts of use of a potential system for older users. ^{54 , 55}

Based on our findings in early intervention, we re-designed the prototype apps to meet the special needs of the residents and caregivers. This reformed the latest design, applications and development of the digital intervention scenarios. In what follows, we describe the modifications and upgrades we made to the final apps. Our robotic-based system consists of several technical components. First, the technical setup is centered around the humanoid robot Pepper. Our digital applications were developed and selected from sports science and gerontology fields and were further inspired by practical recommendations from our pre-study. The aim of the applications was to support physical activity and cognitive resources, as well as to remain as autonomous as possible, lessening dependence upon professionals providing care. The program includes a set of exercises that are necessary for the execution of everyday activities (e.g., climbing stairs, carrying bags, and sitting-to-standing transfers). The movement applications included exercises for the arms, legs, head and torso. They were accompanied by music to promote well-being and motivation during the exercises. The applications concerning cognitive activation included a variety of games combining movement, creativity, and cognitive tasks to encourage the participants to engage not only in individual quizzes but also in group discussions and dynamics. “Snow Waltz”^[1] is one of the movement applications designed in this process. It encourages residents to follow the robot’s dance movements. The other application called “Proverbs.” belongs to the cognitive training category. The app’s logic is to present the first part of a proverb and have the residents complete the proverb sentence.

3.1 Data collection and analysis

Regarding data collection, we conducted semi-structured interviews and design workshops with different residents, caregivers, social caregivers, and the former nursing home manager. Then, we used the Praxlab ⁵⁶ approach to gather valuable information among the mentioned actors to identify their expectations, interests, and ideas. Praxlab utilizes long-term participatory research in real-world living environments to understand and design technology that aligns with older adults’ socio-cultural needs and diverse experiences. ^{48 , 56 , 57} The qualitative data of the pre-study and appropriation study consisted of audio recordings and field notes collected during the interviews and workshops as well as video recordings of 15 (out of 17) group sessions with Pepper (ca. 11 h), following video analysis to obtain more detailed and comprehensive insights also into non-verbal interactional dimensions. ^{58 , 59} However, the overall qualitative data analysis was performed using a reflexive Thematic Analysis (TA) approach for all materials, ⁶⁰ followed by a more detailed sequential analysis (see Ref. ⁶¹ ) of only thematically pre-selected sequences, which was conducted at the end. We used TA to analyze the data from interviews and videography. This process began with transcribing and iterative coding of the data to identify patterns and relationships. With that, it was possible to identify key themes and find connections between the ideas of the participants and others involved in the study. We identified the following principal themes during the coding sessions: group-based interaction, attitudes and gender affirmations toward the social robot, emotions and reactions of residents to the robot, reactions from caregivers and individual conversations with Pepper in idle mode. These overarching themes were derived from our original codes. Coding differences were discussed and eliminated by adding, editing, or deleting codes according to the outcome of the discussion.

3.2 Participants and setting

In our study, 10 residents (see Table 1), one social caregiver and the former nursing home manager were involved. They engaged in interviews, design workshops, and interventions that took place in the robotic-based group sessions. The participants, aged between 78 and 91, included nine females and one male. The participants were chosen on a voluntary basis. Participants were excluded from the study if: they were suffering from chronic diseases such as cardiovascular illness or cancer; or their physical fitness was impaired such that they could not move without assistance.

In addition, we engaged with the doctors, family members, and other caregivers of the participants in the nursing home. We do not list them here, as these interactions were informal and short-lived.

The study took place in a large nursing home in Siegen, Germany, with a PraxLab approach. The interventions with the residents were done within the nursing home in the common areas.

Group sessions

The interventions were designed to take place in a group atmosphere. A supervisor started the program and motivated the participants to follow the robot’s instructions. Figure 2 shows how participants and researchers were seated. The robot stood in the center of a semicircle of participants so everyone could see it. A camera was set up behind the robot so that a video analysis could be made later. The researchers sat beside the participants at a table, taking field notes and observing the robot’s performance. The interventions took place in a room surrounded by window walls. This meant that people walking along the corridor could see into the room. On the other hand, the participants sat overlooking the terrace of the nursing home, so they were not so much distracted by what was happening in the corridor.

Figure 2:

Sketch structure of the interventions in the nursing home.

Here, training sessions were conducted in groups of 4–5 participants. Pepper planned to conduct the moderated sessions interactively. An example session sequence from cognitive training is that Pepper starts the session by greeting and introducing: “Hello everyone. I would like to play a game with you. I know a few proverbs. Unfortunately, I have forgotten the second part of my proverbs. I need you to help me complete them. Here we go!” The robot then starts with the first proverb: “The first part of the proverb is: ‘Rome wasn’t built …’.” The program then jumps to the Speech Recognition Box, where the correct solution is stored. The participants are then encouraged to respond via speech, completing the phrase with: “in a day.” When this happens, the robot says: “That was exactly right. Very good. Would you like to guess another proverb?”

In total, we conducted 17 visits with an average duration of 1 h each. From time to time, informal and professional caregivers, as well as social workers, also participated (mostly passively) in the moderated sessions.

3.3 Ethical, legal, and social considerations

We ensured the privacy and data security of the participants. The interviews with the residents and caregivers were conducted in consideration of the guidelines on privacy and personal data from the university’s ethical commission. It included informed consent forms, a directory of processing activities and an ethical application. These documents were prepared according to legal provisions and were valid only when combined with a personal declaration of consent, which the participants could revoke at any time. The mentioned documents were part of the ethical approval issued by the commission of the University of Siegen. The study was done in accordance with the Helsinki Declaration to ensure the health and well-being of the participants. We did this by involving health professionals in the preparation of the study and by including them in the study itself. We followed their guidance in regard to the health and well-being of the residents. Because of its sensors, especially Pepper’s cameras, the system was only connected to the internal network for updates to protect the privacy of residents, staff, and visitors.

4.1 Group based interaction

In the first session, only one participant appeared in the intervention. There was no pressure on them to participate in each session. However, the caregivers informed us that they had forgotten about the project. The participation increased with time. Often, we had to pick up some residents for the course. By increasing the number of participants in the course, those who had participated in the previous session became less active in the group setting. When we asked them whether the robot’s performance was unclear or if they needed more instructions, one participant answered: “That’s gymnastics. He [the robot] can do it beautifully.” (P7) It seemed that they were initially not confident to imitate the robot when they were in a large group. To break the ice, one researcher started imitating Pepper’s movement and asked participants to join them if they liked it. They then slowly started to follow Pepper’s movements. In some interventions, it becomes clear that participants only felt encouraged to do the exercises themselves once another participant had started. In one session, for example, P1 and P8 started doing the exercises, after which the other participants also mirrored the robot.

During the physical exercises, it quickly became apparent that the residents mirrored the movements shown by the robot. Participants performed those exercises according to their physical abilities. It was noticeable that in most cases, the residents performed only the movement that the robot could demonstrate based on its capabilities. A verbally announced leg movement, which Pepper could not perform, was rarely performed by them. Only if a participant carried out the robot’s verbal instructions would the others follow too. However, this behavior has changed over time. The participants not only mirrored the physical exercises performed by the robot, but also sometimes imitated the body language of the Pepper robot while speaking,^[2] without these being part of the exercises. This showed that the residents were strongly fixated on the visual level and the physicality of the robot during the movement exercises and that acoustic instructions from the robot tended to be much less followed. However, this fixation on the visuals may also have led to one participant frequently waving at the robot during the movement exercises instead of continuing to perform the exercises. For example, during one session, P2 tried to establish personal contact with the robot and attract its attention.

In the middle of the study, the music and dance applications were integrated into the physical movement activities and rhythmically coordinated with the respective music. With these applications, it took a little longer for participants to perform the movements, but all participants took part in the movement dances. In contrast to the behavior during the movement exercises, the participants interact more with each other during the movement dances. For example, they laugh at each other and applaud each other after the dances. These reactions are probably due to the music. Besides the physical activity, the quiz game with regional and general knowledge questions was played in some courses. It was reported by the caregivers that this game made the participants talk to each other about topics outside the courses. They could even remember some events related to those questions and talk about the events from the past with whoever they had contact with.

At the end of the study, some participants no longer needed to mirror the robot or follow the instructions. In one exercise, Pepper instructed the participants to stand up and walk in a circle in the room. As Pepper has no legs, it could not demonstrate this exercise, but instead, it started showing a pre-recorded movie of the participants walking in the room. However, after the initial instruction, P3 moved his legs appropriately without paying attention to the movie displayed on Pepper’s tablet. The other participants then joined in the second round after watching her. The regulated sequence of the interventions affected the participants, as shown by the fact that the participants in the later interventions continued to move around after the physical exercises had been announced so that they could carry out the exercises correctly and not hit each other. In the last sessions, participants already knew what they must do when Pepper announced the name of the exercises. For example, they moved their chairs apart when the robot announced movement exercises to have enough space for their arms. Most of the participants also carried out the movements demonstrated by the robot more confidently. In most cases, the participants seemed to accept the robot as an exercise instructor during the activities, as they mirrored its movements. They saw the exercises as helpful in staying physically fit.

4.2 Attitudes and gender affirmations toward the social robot

Most participants had no contact with robots prior to the first intervention, and only two participants had brief previous experiences. P9 describes the robot as something new and interesting: “That was always a closed book. It’s not my world. But now I have the chance here. Maybe I’ll learn something.” Despite the uncertainty, P9 wanted to learn and get to know something new, displaying a certain level of curiosity that we also observed with other participants.

A reoccurring irritation in contact with the robot was the topic of gender. The participants tried to categorize the robot in familiar role patterns. In the first intervention, one participant asked herself whether Pepper was a woman or a man. As a result, she humanized the robot and tried to assign it a gender, as in her opinion, a robot should also fit into this role. However, she came to the conclusion that Pepper could embody both genders. The gender assignment also took place with other participants. There, Pepper was described as a “nice man,” although the robot had introduced itself as Paula before each intervention and spoke in a more feminine voice. While most of the participants perceived the robot as human-like, the gender remained unclear: “It’s more of a woman. […] It can be a man and a woman” (P9); elsewhere, she referred to the Pepper robot as a little animal. P2 described the robot as a nice man that she likes: “I think he’s a nice man.” One participant even described Pepper as a friend. Unlike all the other participants, she also wanted to spend time alone with Pepper. This became evident when she was in the room a few times too early and talked to the robot or played with it. None of the participants used the term ‘machine.’

4.3 Emotions and reactions of residents to the robot

The emotions that the participants in our study displayed towards the robot were quite diverse. P1, in particular, showed many non-verbal reactions. She waved to the robot almost once or twice in each session when the robot looked at her. She even referred to the robot as a friend and often stopped her exercises to wave at the robot. On the other hand, P2 was visibly startled when the robot started talking to her for the first time. She only settled down when P1 had a conversation with the robot, and later also initiated gestural greetings towards the robot. P8 was quite afraid of the robot and did not want the robot anywhere near her in the beginning. When the robot asked the participants if they wanted to do any exercises, P8 responded: “Okay, just do not come to me.” P5 was also skeptical at the beginning, stating in her first sessions: “I can’t relate to that little animal at all.” This was also the case for P3, who commented during the same session that “now he’s looking at you intensely, isn’t he? I don’t know anything about that kind of stuff. I’m not learning any more about it either.” However, after the researcher had shown her how to use the tablet, she came to a different conclusion: “I guess practice makes it perfect.” Some positive emotions were also evoked in the participants. In some exercises, the participants were thrilled when Pepper announced the exercise they wanted to perform, and they were good at it. Each robot’s apps were designed to get motivational courage when they responded or answered correctly. In another session, P3 commented on this again: “I’d like to do better. That’s fun for me.” P1 giggled in response to praise from the robot. She kissed the robot several times in two sessions when the robot congratulated her after guessing the correct answer on the quiz game. P1 also looked grimly at the robot when it switched off in intervention four. However, she is also playfully cheeky with the robot, which becomes apparent when she sticks out her tongue at the robot or toasts him with glass to attract its attention when the body of Pepper is pointed at the other participant, but its head is pointed at her (P1). At the beginning of each intervention, the robot greeted the participants and asked how they felt. When Pepper asked them a question in the quiz, the participants did not seem to know whether they should answer him or not. This became visible to us when they looked at each other questioningly. In the middle of interventions, the participants got familiarized with the robot and then usually nodded to the robot to acknowledge their readiness for the exercises when the robot gave instructions or made statements. One participant even demanded that the robot should make her healthy. For her, a robot should have more of a medically useful character.

4.4 Reactions from caregivers

The staff at the nursing home where the study took place were very interested in the project. Some watched from the outside or even took part in the interventions. They also saw a preventive benefit in the applications, such as the ‘catching the rabbit’ game,^[3] as it could improve the resident’s ability to react. The staff reported that it became apparent that the residents with MCI had a lot of fun interacting with the robot. This is important information because they know the test subjects and their behavior very well during other activities without robots. It was also noted that interaction with the robot changed over the course of the study. The residents with MCI perceived the robot as increasingly natural. One employee compared the residents’ interaction with the robot to the way they interact with stuffed animals or dolls. The robot evoked emotions in the residents. The employees were asked whether they could imagine using the Pepper robot in the long term in the nursing home. On this point, the employees all said that they could imagine it, but only if accompanied by a person who is familiar with handling the robot and the residents.

4.5 Individual conversation with Pepper in idle mode

On several occasions, before and during the interventions, there were situations in which the participants conversed with the robot. This occurred both alone and in group sessions, as reported by caregivers and observers on site. This was often not captured on video, as the recording usually started once all participants had arrived. On one occasion in the third session, while the other participants were still being picked up, P1 was filmed conversing alone with the robot (R) for more than 10 min (see Figure 4). Meanwhile, a researcher came to the room to check on her, but did not stay with her. She used the time to talk to the robot, and the following transcript shows the opening sequence of this conversation in Table 2. Even before the beginning of the transcript, when P1 was wheeled into the room, she immediately began to greet the robot by waving to it. The moderator then placed the robot in front of her and left the room to look for further participants, which is when the transcript begins.

Sitting in a wheelchair in front of the robot, P1 starts greeting Pepper (R) verbally (lines 2, 4), asking if the robot can remember her from the previous session (lines 6, 7). However, Pepper is only turned on in standby mode, and since there is no running application, it does not verbally respond to her questions. Nevertheless, the robot is orienting toward her voice (and to the lights in the room), makes smaller movements with its head, and is blinking with its eyes (by turning off and on of the LEDs in its eyes). These orientations initially (seem to) correspond sequentially to P1’s greeting attempts, since Pepper is moving its head in her direction (line 3) and initially seems to respond to her nodding (line 5). However, there are no further mutual orientations or verbal responses from the robot. She continues to observe him and starts to mirror the robot’s expressions, commenting on Pepper’s blinking eyes jokingly (lines 9, 10). Emphasizing loudly her name and that she can make big eyes as well, initiates the robot to reorient (searching for a potential standing participant in the room) looking abruptly up to the right (line 11). This seems to have surprised and startled P1, who exclaimed with a kind of response cry: “Huch!” (line 12), before laughingly realizing that everything seems to be okay, while Pepper also reorients its head towards her.

But even without any further verbal response from the robot, resident P1 kept talking to the robot for several more minutes, and some of her further statements are quoted in the following Table 3.

She tries to get the robot to speak by asking it to say: “good day” and begins to introduce herself again to the robot (line 13). After the robot still does not give any answers, she tells Pepper that she would get sad about not getting any responses (line 14). Then, she begins to look for reasons why the robot is not talking to her today (line 15). She speaks to the robot as if it is a child, uses childlike expressions, worries about the robot’s well-being, and asks if he is doing well. P1 knows that the robot had played that song in the previous session and is trying to encourage Pepper to sing it for her again (line 16). Although, in those moments, the robot does not respond, its autonomous mode, which is activated by default, still makes the robot show some gestures and head movements that only at times seem to correspond to her verbal attempts to establish contact. Despite the robot’s idle behavior, it seems to remain interesting to her, and she continues to ask the robot more questions (line 17). With a curious facial expression, she inquires if the robot could smoke too and starts to tell a rather private story to the robot (line 18). This specifically occurs when she gets comfortable with the robot and starts sharing very personal things, telling Pepper that her husband has died because of smoking. However, when the other residents enter the room, this flow is interrupted and she stops talking to Pepper.

5.1 Mismatching interactions of robotic systems and nursing residents

The interactions between robots and residents frequently exhibit mismatches within our study. Tian and Oviatt ⁶² view these as errors and divide them into performance errors and social errors, with social errors referring to social behavior (e.g., interrupting a user mid-sentence) and performance errors (e.g., not hearing someone) referring to the inability of a robot to perform a certain task. For example, we observed residents speaking or making gestures without receiving a response from the robot. In this case, the robot is not able to understand the social situation. Additionally, the robot does not recognize changes in situations, remaining in its current program until it finishes or is manually stopped, oblivious to any situational shifts, such as someone sharing a relevant anecdote. The robot also cannot comprehend a change in interaction partners, such as when a resident starts talking to another resident instead of to the robot. Group dynamics are unrecognizable to the robot, which responds only to what its sensors and programs can detect. Although algorithms exist to make sense of sensor data, they cannot distinguish whether the input is directed at the robot or someone else nearby. ^{12 , 63}

These mismatches also occur in reverse (robot to resident) (cf. ⁶⁴ ). For example, if the robot says something and the resident does not notice or hear it, the robot waits for a response, unable to understand that its prompt was not received by the interaction partner. Although humans also experience misunderstandings during interactions, the robot’s design exacerbates these issues. The robot’s human-like appearance suggests certain capabilities, and fiction further contributes to the belief that robots possess specific interactive skills. People also tend to extrapolate from successful interactions, assuming that if a robot responds appropriately to one question (like “How are you?”), it will do so for other questions as well, which is often not the case, as the robot might be programmed to answer only that specific question. This leads to unrealistic expectations, such as residents who expect the robot to shake hands in response to a greeting, which it cannot do. This type of behavior has also been observed in other areas, such as the automotive sector, where users place too much trust in the capabilities of autonomous driving. ⁶⁵

Another aspect of this mismatch is the understanding of each other. The standard robot’s voice was not adapted to nursing home residents. Many participants simply did not understand the robot’s voice because, in the beginning, either it spoke too quickly, the tone was too high, or they were not used to this type of artificial voice. This critical design mismatch was corrected during the design phase, and Pepper’s voice was modulated before the evaluation phase. This shows that the human-like appearance of the robot does not necessarily translate into a human-like and adaptable voice in different situations that can be understood accordingly. This is a very important detail that is crucial for this user group. ⁶⁶

Ideally, caregivers (or moderators) should clarify these mismatches and explain the robot’s limitations. Especially since people with MCI have a high level of trust in robots, as Pino et al. point out. ⁶⁷ However, such mismatches occur often and have already been explained by caregivers many times. Over time, caregivers may become desensitized and stop explaining why the robot is not responding appropriately. This is not out of malice, but due to the frequency of such incidents and the multiple tasks caregivers must handle. It may even seem easier to ignore a resident’s confusion than to explain every situation, as the explanation itself can disrupt the flow of interaction, which – as the idle situation with P1 shows – is not necessarily interrupted by such mismatches.

5.2 Nonverbal (re-)actions and storytelling

Nonverbal reactions to the robot are shown most significantly by P1. She waves to the robot very often. In some interventions, she also throws kisses with her hands or whistles at the robot, which is typical behavior with dogs. She also sticks her tongue out at the robot and looks at it grimly when it switches off. With these gestures, the participant tries to attract the robot’s attention. She treats it like a living being, such as a dog or a child. It is striking that she repeats these gestures over and over again in several interventions, even though the robot shows no reaction to them. This behavior has also been observed in animal therapy with dementia patients. Much is also communicated via gestures and facial expressions and less via speech. ⁶⁸ The robot also did not respond to questions and requests. Nevertheless, these situations occurred repeatedly. Here, too, the focus was on gaining the robot’s attention through various requests. In the first five interventions, most participants were unsure whether to talk to or answer the robot. This could be due to fear of the unknown. In Europe, people were initially rather skeptical and fearful of robots. ⁶⁹ In Germany, by contrast, 41 % of people can now imagine being cared for by a robot. ⁷⁰ The participant who showed the robot many nonverbal reactions apparently had no fear of the robot. She often spoke to the robot when she was alone in the room. She introduced herself to the robot and kept asking questions. She also told the robot how her husband died and where he was buried. Other studies have also observed this storytelling (telling secrets) to robots. Residents without cognitive impairments were observed to tell robots sensitive information that they had not previously told a caregiver or relative. ^{39 , 40 , 42 , 43} In another study, children were asked whether they would tell robots secrets. This showed that half of the children would be willing to do so. This included significantly more girls than boys. They had hoped to find a friend in the robot, as a result. ⁷¹ The participant who told her stories to the robot also called it a friend. The reasons why she told the robot very private things about herself, among other things, are unknown. However, it should be noted that she did not share private information of this kind with anyone else during the study. As in Paro’s research, this information can be used for biographical work with people with dementia. ⁷² Biographical work with people with dementia is very important so that they can better identify themselves when other people talk to them about past events or former hobbies and carry out corresponding activities. ⁷³

5.3 Investigating design affordances in social interaction

The individual interaction between P1 and Pepper in its idle mode allows us to further scrutinize social robotic design affordances in their interactional dimensions. Although Pepper actually does not respond to resident P1 in what would usually be considered a meaningful interaction, P1 continuously makes sense of its assumed and displayed responsiveness in the opening sequence of the interaction, such as: Pepper orienting its head towards her, blinking with its eyes, and nodding in a sequentially relevant place that becomes meaningful for P1, who nods accordingly (see transcript on Table 2) – and then initiates the described further storytelling to the robot. Rudaz et al. ⁷⁴ have shown how important such pre-beginnings and openings, i.e., the first moments of co-presence between participant and robot, are for the possible interactional achievement and the emergence of robots as social agents versus inanimate objects. They point out that these first moments of the encounter are often not recorded in human-robot studies, but should systematically be considered since the initial behavior of the robot can be of relevance for subsequent sequential trajectories, as is the case in our example. Furthermore, Pelikan and Broth ⁷⁵ show not only how interactional troubles and mismatches persist (and could be tackled), but also how participants adjust their turns to the limited capacities of robots. In the case of Pepper’s idle mode, these capacities are limited to a bare minimum; however, this does not stop P1 from continuing to probe, engage, and open up to the robot. Mutual orientation and presumed eye contact in combination with minor movements (that can become sequentially relevant) appear already to provide the achievement of mutual responsiveness that is crucial for media of cooperation (cf. ⁷⁶ ). As in the case of the success of ELIZA, an ethnomethodological and conversation analytic perspective offers a re-specification of human-machine and human-robot interaction that zeros in on the actual practices in its use, thereby explicating technology reliance on human social interactional competencies. ^{77 , 78 , 79 , 80} They show “how that technology is embedded in and relies on human social practices for its sense” (33], p. 6; cf. also 34]). In this vein, Lipp ⁸¹ , p. 660 argues that it is not so much robots caring “for (older) people but rather, the other way around – people need to care for robots.” Thus, re-situating the interfacing between robots and humans, as already Suchman ⁸² suggested to rethink and reconfigure the design of computational systems in light of interactions with them (see also ⁸³ : 57ff., ⁷⁵ ). In view of this, the social agency as well as the thing-like character of robots – also in regard to its materiality – can be seen as an actual interactional accomplishment. ^{38 , 84} However, the discussion of the particular case in this paper calls for and highlights the relevance of further studies that more systematically analyze video data in the nursing context in its interactional and praxeological detail.

5.4 Ethical reflections

The use of technology with vulnerable users, such as nursing home residents, is a sensitive topic that requires ethical reflection. While robots are now often viewed as supportive ‘tools’ for specific tasks of caregivers (e.g., Ref. ⁸⁵ ), one of the main drivers of using robots in nursing homes still is to achieve some sort of automation. However, it is questionable whether robots can be an adequate solution to address the shortage of skilled staff and the increase in people in need of nursing. ^{86 , 87} Instead, the main questions are what role they can play to help the field and how we can ensure humane care. ⁸⁶ Among other things, care work is about being empathetic, attentive, and caring towards the patient or resident. Many people are unsure whether a robot should be used in care due to the aforementioned requirements, which are extremely important in this field. It is also questionable whether robots will ever be able to replace human beings. In the technical setup that we had during this study, we can say with certainty that the robot cannot replace a human care worker and can only serve as an assistant. There are no time savings through the use of the robot in the study, but there is an increase in capabilities. Robots can have many applications installed, providing residents with a wide variety of entertainment and training options. However, there needs to be some form of moderation – a person to assist residents in using the robot. In order to recognize the extent to which a robotic system can be used in care, Grunwald recommends informing and educating everyone involved in detail. ⁸⁸ Only then can well-informed and “ethically responsible decisions be made at a social level”. ⁸⁹ Due to the wide variety of robot types that could be used in care, it is not possible to make a blanket ethical judgment, which should not be the goal in the first place. Ethical judgments are always culturally rooted and differ based on robot design, robot functionality, user group, and context. ^{90 , 91} Therefore, it should be addressed at several points during development and implementation. ^{92 , 93}

The person with MCI could, for example, feel threatened in their privacy if they are monitored by a robot. ^{94 , 95} When using socially interactive robotics in the environment of people with MCI, the question arises as to whether the affected person is deceived by this robot. For example, if the robot Pepper satisfies the psycho-social needs of the person by creating some sort of perceived social bond, which may be seen as a deception by the robot or the way people introduce it. People with MCI are often at risk of having difficulties distinguishing between a machine and a living being. Studies have shown that positive feelings are evoked in people with dementia through interaction with Paro. ^{23 , 24 , 25} It is questionable whether this can be seen as a deception of people with dementia and whether that is a legitimate approach. ⁹⁶ However, when users know that they interact with a machine, it may still evoke positive emotions that are also perceived as okay to feel. ⁹⁷ In this situation, residents may still struggle to navigate the decision-making process. However, it is essential to prioritize enabling residents, even those with mild cognitive impairments, to make informed decisions about their interactions with robots. Therefore, the interventions must be carefully designed to provide clear and accessible information about the robot’s nature and function to minimize the risk of misunderstandings and potential emotional harm, such as feelings of betrayal or disappointment. Hence, the nature of the robot should become apparent only later.

Although some participants in the interventions referred to the Pepper robot as a living being using terms such as “little man” or “little animal”, they mostly spoke of the “robot” in conversations about Pepper. Based on our data, we believe that there has been no confusion that the robot is a person or an animal (see also Ref. ⁹⁸ ). Those terms were rather some sort of nicknames. According to Kolling, the fact that the robot evoked emotions in the participants can be considered legitimate, ⁹⁷ as the participants appear to create an emotional bond based on conscious interactions.