Privacy framework for context-aware robot development

1 Introduction

Social robots become increasingly present within the domestic environment, for service tasks as well as for educational and entertainment purposes [1]. Robots will be part of our daily lives soon, and their tasks will not only be related to household tasks such as vacuum cleaning or lawn mowing. Moreover, they will become an assistant in a variety of daily life situations. Several studies examined preferred features for a robot and identified tasks such as reminders, information provision, healthcare supervision, cooking assistance, cognitive and social support, conversational dialogues, music playing, smart home integration, entertainment, and much more [2,3,4]. Therefore, robots are commonly equipped with various sensors such as cameras or microphones and are connected to the Internet to access information. These sensors allow the robot to perceive its environment to move around and interact with the inhabitants to provide assistance [5]. The way of robot behavior within the home context raises new ethical challenges and risks that need to be addressed [6,7].

These concerns include, e.g., autonomy, responsibility, trust, loss of control, loss of privacy, loss of personal liberty, and ethical considerations towards the human–robot relationship [8,9, 10,11]. Most of the mentioned ethical concerns are strongly related to privacy. The more tasks and features a robot provides, the more extensive and complex the amount of collected data becomes. A simple vacuum cleaning robot cleans the floor in a random way. In contrast, a smart vacuum cleaning robot with additional sensors for obstacle recognition and home mapping can gain data about the whole domestic environment. Being additionally able to start the robot via other smart home devices provides data about other devices within the network [12].

The robots can move around autonomously and record whenever and wherever they want. Thereby, the gathered and processed information might consists of sensitive data of users, not only about the users themselves but also about their environment [13]. The robot will know about habits, preferences, hobbies, diseases, and personal facts about the user. Furthermore, the robot maps the home and controls smart home elements. The robot taking control over other systems leads to the discussion of safeguarding this sensitive information and calls for responsible privacy-protecting research.

Privacy does not only concern threats from outside, (cyber-)attacks, or hackers. In addition, risks imposed by robot manufacturers and service providers have to be considered, e.g., misuse or unauthorized selling of data [14]. Privacy and data protection are fundamental human rights established in the Charter of Fundamental Rights of the EU. Furthermore, the processing of personal data needs to be transparent for users, and processing needs to be lawful. The principles of purpose limitation and data minimization need to be followed ref. [15]. This limitation means that the companies providing the robot and its services cannot track everything they want, but a specific purpose is required. The general data protection regulation (GDPR) [16] enforces a mindful confrontation with privacy-respecting development to ensure the rights of users in the EU. Responsible research means that ethical principles, as well as privacy, need to be a central part of the development process besides safety and security [17]. The “pervasiveness and intrusiveness of robots” in the home environment require a conscious examination of critical situations to minimize potential risks [18].

In this article, a privacy framework is proposed in the context of a domestic robot. Privacy methodologies from general IT services are adapted to present a practice-oriented concept for the Privacy by Design (PbD) principles [19]. The privacy framework is related to a user-centered design concept to provide an overview of interrelated stages. Although the user plays an important role, user involvement will not be the focus of this article. However, the focus is solely on the methodologies that simplify a risk assessment according to privacy. The privacy framework aims at a risk assessment for social robots, besides safety and security issues. Thereby, methods are presented to meet the principles for privacy-respecting development.

Following, Section 2 gives an overview of existing privacy principles for service robots. The main concepts turned out to be theoretical approaches without specific instructions, whereas they equate privacy with security and only focus on threats from outside. Section 3 proposes the stages of the risk management cycle and introduces the important methodologies. The proposed process and the methods are then exemplary demonstrated on a use case. A robot capable of social interaction and cognitive support is taken as a use case scenario to emphasize the importance of data protection. The importance of privacy in the context of robots’ sensors, integrated services for data processing, and user information derived in such a scenario is provided.

2 Privacy principles in the context of social robots

As privacy for social robots, especially for the home context, gains more and more interest, there are various ideas and first approaches to taking care of this topic. A first concept was provided by Denning et al. [20]. A list of questions shall identify specific privacy- and security-related aspects that need to be considered for design decisions. The questions concentrate on social, environmental, and technical factors such as intended functionalities, users and context, actuators and sensors, and the threatening of privacy, physical integrity, physical safety, or psychological vulnerabilities.

Other concepts focus on risks that various sensors imply [21,22, 23,24]. Lera et al. classify sensors according to the level of “privacy leakage” ranging from 1 to 5 (very low to very high) with the focus on disclosure of personal and environmental information [21]. A camera is classified as level 4 as it can disclose private images of the person and the environment. It is pointed out that a fusion of several sensors is considered the most dangerous. In addition, it is important to consider where data processing is conducted, as recorded raw images need to be analyzed. Eick and Anton do not only point out potential risks but also highlight the importance of sensor selection [24]. Researchers and developers must select the required sensors carefully depending on the identified requirements and the specified context. The aim is only to select the sensors that are needed. A careful sensor selection was already emphasized in the context of integration of a privacy-relevant model for robotic development [23]. Cerrudo and Apa gave additional examples of potential risks in private homes of certain sensors such as espionage or smart home manipulation [22].

Furthermore, within a workshop conducted by Rueben et al., the topic of privacy was categorized into seven relevant specified research fields [25]: “data privacy; manipulation and deception; trust; blame and transparency; legal issues; domains with special privacy concerns; and privacy theory.” As the main focus for legal issues, they highlighted a so-called “privacy education for users.” The importance of interdisciplinary collaboration is additionally emphasized.

An exchange across disciplines is beneficial, as shown with the PbD principle. The original PbD approach was developed by Cavoukian et al. [19]. The concept encompasses seven essential design principles for privacy. The seven principles are meant to establish basic rules for meaningful development. The PbD concept is discussed contrary as the complexity and variability of privacy issues are not discussed, and systematic methods are not appointed, which can be used to identify certain risks [26,27]. Nevertheless, this generalized and theoretical design concept was adapted in the context of robots for domestic use by the lawyer Pagallo called robot-cloud by design [28]:

1. “We have to view data protection in proactive rather than reactive terms, making PbD preventive and not simply remedial.

2. Personal data should be automatically protected in every robotic system as its default position.

3. Data protection should consequently be embedded into the design of the application.

4. The full functionality of the principle which follows from (2) and (3), allows a positive–sum or win–win game, making trade-offs unnecessary (e.g., privacy vs security).

5. A cradle-to-grave, start-to-finish, or end-to-end life cycle protection ensures that privacy safeguards are at work even before a single bit of information has been collected by the machine.

6. No matter the technology or business practices involved, the design project should make data protection mechanisms visible and transparent to both robotic users and providers.

7. Finally, the principle “requires architects and operators to keep the interests of the individual uppermost by offering such measures as strong privacy defaults, appropriate notice, and empowering user-friendly options” [19]. In other words, robot-cloud by design requires individual-focused respect for user privacy.”

This overview shows that there are already existing theoretical concepts and approaches which outline the important issues for privacy-sensitive and privacy-respecting robotics. Nevertheless, there are no specified and formalized models and methodologies that put the theoretical approaches into practice in the context of social robots. The realizability and practical use of the proposed approaches are missing in the complex field of robotics. Furthermore, the theoretical approaches are insufficient regarding privacy as security issues foreground the concepts. Privacy is not only about risks from outside and being threatened by hackers or data theft. Moreover, it is about protecting sensitive information towards the robot manufacturer and accompanied service providers against, e.g., misuse [29]. Hence, this is an important research area in the context of social robotics. In the next section, further details are provided about PbD in general and methodologies as a basis for a quantified assessment.

3 Risk management cycle

As the PbD approach already proposes, the integration of privacy concepts is of fundamental importance. Methodologies from standard IT services are transferred to fit the needs for the development of social robots to propose a practice-based concept and ensure a privacy-sensitive and privacy-respecting development. The chosen methodologies for privacy enhancement are embedded into a risk management cycle consisting of the three steps: risk identification, risk analysis, and risk mitigation. This risk-based approach can also be applied in the context of a Data Protection Impact Assessment (DPIA) addressing the relevant steps and methods to be used for risk assessment [30,31]. Figure 1 shows the relevance of the risk management cycle in relation to the five steps of the design thinking process [32]. The design thinking process serves to clarify the correlation of process steps and the provided methodologies for privacy. Mainly, the design thinking process steps are similar to the development process steps of the ISO9241-210. It is mentioned that the user should play an essential decision-making role and is meant to participate during the entire cycle actively. Participation means that the user can express potential concerns and be made aware of other risks. But this is not the focus of this article, as it aims to propose the methodologies for risk assessment in robotics. Thereby, the three steps of the privacy framework pursue the goals:

1. Risk identification carves out potential privacy risks of defined functionalities based on necessary sensors and required information.

2. Risk analysis as a second step deals with a reasonable measurement of benefits and risks and the analysis of privacy requirements for the chosen development concepts.

3. Adequate measures for privacy protection meet the analyzed privacy requirements and need to be determined and implemented in the step of risk mitigation.

Figure 1

Privacy framework integrated into the design thinking process.

4 Privacy framework

The use case of an interactive robot will be investigated as an example, and two use case scenarios are contrasted to highlight the importance of context awareness for privacy development. The robot can interact and talk with the user as a friend. It can recognize the user and customize its behavior and conversational content. Moreover, we can think of an advanced smart assistance device equipped with a camera (and can move autonomously). As this use case focuses on investigating privacy risks accompanied by the requirement of social and verbal interaction, the exploration of risks that correlate with autonomous movement will not be presented in detail in this article.

4.1 Risk identification

At the beginning of a development process, it is about the identification of needs and requirements. Thereby, the first step empathize of the design thinking process concentrates on the understanding of all constraints that are involved, such as target group, context, and resulting requirements. Risk identification determines potential risks for every identified need, connected to the empathize step.

Given the example of an interaction partner, which is a universal requirement, potential risks need to be figured out. As a specific interaction example, the features that have already been identified as needs are used [2,3, 4,37]. These features require certain capabilities of the robot. The two main requirements that the interactive robot is meant to fulfill as a communicative interaction partner are voice recording and informational processing and access to be able to answer appropriately. An essential feature for an interaction partner is the ability to record and analyze voice. The most relevant sensor for this requirement is at least one microphone. Furthermore, user recognition shall be a fundamental part of the robot. This can be implemented in various ways, which are explained in the relevant subsection in more detail.

4.1.1 Voice recording

For voice and speech recognition, at least one microphone is required. The amount of sensors does not influence the potential risks. All recorded information should be classified as sensitive. Table 1 gives an overview of the potential risks. An installed microphone can imply privacy risks concerning the types action and behavior, communication, thoughts and feelings, location, and space and association. Privacy of communication is always affected as the robot is used as an interaction partner. The other privacy types can be affected depending on the content of the robot. The main issue is not the fact that the user is being recorded. It is about the type of information and its sensitivity that are recorded. Additional features that utilize sensitive information via speech-based commands induce further risks (Figure 2). A connected calendar to manage appointments provides information about action and behavior or location and space. The ability to make calls via this robot gives information about associations. Cooking aid provides information about possible intolerances. These risks do not imply that no feature should be realized. However, as any feature causes risks, protection has to be ensured.

Table 1

Types of privacy risks in relation to sensors: (1) privacy of the person; (2) privacy of action and behavior; (3) privacy of communication; (4) privacy of data and image; (5) privacy of thoughts and feelings; (6) privacy of location and space; (7) privacy of association

Sensor	1	2	3	4	5	6	7
Microphone		X	X		X	X	X
Camera		X		X		X	X

Figure 2

Robot companion that can be used for informational services and for social interaction.

4.1.2 User recognition

As mentioned earlier, there are different possibilities to realize user recognition. As a microphone is an essential sensor, this could be easily done by speech recognition [38]. A microphone is not sufficient in this context to pave the way for a context-sensitive robot. As an additional advantage, the robot should recognize a person, e.g., when entering a room, or the robot should be able to identify situations where it is advisable to offer support or call for help. These additional requirements ask for a camera. Section 3.2 defines the risks going along with an integrated camera. The features that enable the robot to react and behave situationally also entail privacy risks. Privacy of data and image is endangered, as well as action and behavior, location and space, and associations. Privacy of associations is at risk if, e.g., the robot recognizes persons like relatives or friends besides the users.

4.1.3 Informational process and access

The recorded information needs to be processed and analyzed to make the robot answer a question or fulfill a given task. Depending on the task, some features can be processed locally, such as reminders and timers – complex verbal interactions demand significantly more computational resources. They are typically provided externally for efficient processing. Besides identifying single words and sentences, additional information needs to be derived as the context of the sentence and possible background information, the pitch of the voice, and much more. Hence, as the number of features for which local processing power is sufficient is minimal, it is not discussed further. The analysis of the recorded voice data can lead to integrating additional providers, e.g., music streaming services, which also require privacy measures. Playing music would require a connection to streaming services, and weather and news reports need internet access to retrieve the up-to-date information. The discussion of a social robot with overall interaction capabilities implies gathering information about the users, habits, and life situations. On the basis of this pool of data, the robot can establish a meaningful conversation.

A simplified procedure for data processing, storage, and integrated services is presented in Figure 3. It gives an overview of potentially involved parties and providers of a robotic system and how they exchange information. The recorded raw data are sent to the cloud or another external server of one of the involved providers. The information is analyzed based on pre-trained linguistic models to identify relevant information, and a computed answer is sent back to the robot. Depending on the task, it might be necessary to incorporate other involved providers to receive additional required information and access. The raw data itself might be used to improve the analysis techniques and results of the model. Thereby, risk identification does not only concentrate on the protection of data against theft but also on the protection against misuse on the side of all involved companies and providers [29]. As an example, the user asks the robot to turn on the music. The voice command is transferred to the companies data centers. After processing and analysis, the command is transmitted to affiliated services, e.g., streaming services to turn on the music.^[1]

Figure 3

Interaction of service providers based on data processing and the flow of informations. ML, machine learning, NLP, natural language processing.

Typically, this step contrasts the different implementation techniques, and at the end, a final decision needs to be selected. The use case does not incorporate many sensors, but the robot’s capabilities entail potential risks. Therefore, the risk of network connectivity is necessary to highlight the importance of maintaining, storing, and receiving relevant information for the interaction. Concomitantly, involved providers and third parties need to be investigated as they all represent potential threats. The next step deals with an analysis of requirements for protection to achieve a privacy level that is suitable and appropriate.

4.3 Risk mitigation

Risk mitigation takes place within the prototyping phase. The user can test several implementation techniques from the user for the specific use case scenarios. This involvement gives future users the chance to participate in the design decision and thereby create an intuitively operable robot and modifiable according to one’s preferences.

Availability is a relevant goal for an interactive robot, and both use case scenarios. Measures to ensure availability provided by the SDM^[3] are, e.g., data backups, redundancy of hard- and software components, repair strategies, alternative processes, and protection against external influences. These measures would have to be carried out by the relevant service providers (Figures 4 and 5). Availability needs to be ensured for all involved service providers. It needs to be checked that all service providers fulfill the availability requirements and that the measures of the involved stakeholders are also compliant with the requirements and the principles of user protection. Even though availability is considered of higher importance, confidentiality needs to be ensured as well. To ensure confidentiality, measures such as authentication, encryption, and authorization proved to be useful concepts to prevent others from gaining access to personal data [22].

Figure 4

Mitigation strategies for respective interrelations in the context of health care supervision or work support.

Figure 5

Mitigation strategies for respective interrelations in the context of entertainment.

For transparency as the second relevant protection goal for both use case scenarios, the most relevant aspect deals with the robot’s feature of recording. It needs to be visible for the user when the robot is recording. A flashing can realize this LED, indicating a recording microphone. Furthermore, an essential component for transparency concentrates on the transparency of data flows. It should be visible and understandable for the user, how and which information is transferred, used, and stored, especially when third parties are involved in integrating more features (Figure 3). If certain features are not available, it should still be possible to inform the user about these issues. In the context of the second use case, transparency also becomes important in terms of data tracking. It needs to be comprehensible which health care status information is forwarded to the medical app or the doctor or how specific work processes were chosen and carried out. Unlinkability should be ensured in a way that the integrated services do not communicate and exchange data among each other when not necessary.

Integrity and intervenability are assigned two different levels of importance within the two use case scenarios. Intervenability is seen as more relevant and important within the use case of an entertainment robot (Figure 5). Preferably, this would lead to mitigation strategies where the user can decide the “working principle” of the robot. Beyond a status LED, a button helps to turn on/off the microphone or the camera. In addition, a privacy dashboard where information of activities is listed transparently needs to allow unlinking or disabling certain features that are not required or wanted.

In the health care scenario, integrity is more relevant as, e.g., emergency processes or medication reminders should not be modifiable in an easy way. Modifications should be allowed only using authentication techniques, as changes could cause harm and damage if done improperly without permission. In the case of the second use case scenario, where the camera plays an important role in monitoring the users and their activities, it might be a useful measure to use image filter techniques just to analyze the relevant aspects of the data. Several manipulation techniques can be applied to concentrate on the essential content, e.g. blurring, sketching, or redaction [39,40,41].

Furthermore, context awareness can not only be discussed depending on the intended use case scenario. Context-awareness is additionally part of robot’s behavior. For the entertainment use case scenario, this would require that the robot only listen and record when and where it is allowed to, and the user allows it. For example, video recording within the bathroom and bedroom should be prohibited by default to safeguard privacy. The microphone should only record when the user speaks to the robot. To control this, three different listening principles for smart speakers exist so far, divided into manually activated, speech activated, and always on [42]. The always-on solution should be avoided for this use case scenario. Users should rather be able to determine when to start a conversation, for example, by pressing a button with the help of keywords or using a gesture recognizable via camera.

For the second use case scenario, switching off the camera should not be possible as it is used for emergency monitoring. Nevertheless, as an emergency might also occur within a bathroom or bedroom, the user needs to make clear to the robot to not record over a while, e.g., when having a shower. Emergency monitoring while not using the camera can be realized with a question, and the user has to answer.

Overall, the way of intervenability needs to be determined together with the user. By pressing a button or using a regulator, the user can switch on or off specific sensors. Still, it can also be realized using an interface for the robot that is manipulable via a touchpad or voice commands. There is no specific available solution. Therefore, it is of utmost importance to propose various solutions to decide together with the user.

5 Discussion and conclusion

This use case proposed the relevance of risk assessment, especially for security and privacy protection. In the scenario of an interactive robot that should be seen as a companion, the topic of trust, honesty, and reliability are of utter importance [43] as the robot will receive a lot of very personal and sensitive information. This feeling of companionship sounds illusory, as a robot is a technical device developed for commercial purposes. The use case referred to the robot as a companion, which in our opinion sounds misleading. Of course, it is intended to trust the robot to some extent. Otherwise, a user will not use the robot at all with accompanied functionality. Nevertheless, we should be aware of interacting with a robot and not with a human friend. Moreover, it is a device that pretends to be our friend, but the manufacturers behind it are in charge of all the user’s data. Therefore, it becomes even more important to raise users’ awareness toward protection in certain situations.

The privacy framework consists of risk identification, risk analysis, and risk mitigation functions as guiding help. It presents the relevant issues that need to be investigated for a privacy-respecting development. It is not only about risks that are accompanied by certain installed sensors. Furthermore, it is not only about the investigation of potential external threats and thefts initiated by hackers. In contrast to the concept of “judicious sensor selection” of Eick and Anton [24], this use case could show that the risks do only partly arise with the use of high-risk sensors, for instance, cameras [21]. Rather, it is about the overall and holistic protection of personal and sensitive information against theft and misuse on the side of the involved service providers. Risk-afflicted issues occur with the installation of an internet connection. A robot companion does not necessarily need to be equipped with many sensors. Being equipped with microphones does already entail a collection of information that can be highly sensitive. In robotics research so far, this issue involves the considerations of (cyber-)security protection. Privacy protection is equally important and has become a highly relevant topic with robots in domestic environments and private spaces.

The privacy framework was introduced as a practice-oriented approach to enhance the PbD concept. The privacy risk assessment covers five principles of the PbD concept. It involves the proactive view on data protection (1), data protection by default (2), the embedding into the design process (3), the visibility and transparency (6), and principle (7) when the user is involved. The accomplishment of the full functionality and a win–win development for principle (4) can only be successful when all stakeholders, specifically the users, participate in giving input. The end-to-end cycle protection (5) can only be covered by information protection within all involved service providers. All of them are obliged to carry out such a risk assessment to achieve this principle. Overall, the proposed concept offers a first approach for privacy-respecting development.

Thereby, it is emphasized that the user must be involved throughout the entire development process. Even though this was not the focus of this article, user participation is important. Only the user can provide an insight view and give constructive feedback on putting things into practice. Another advantage of user participation is the contribution of potentially unconventional ideas towards robot development and certain implementation techniques. Concerning privacy, the proposed methodologies can also be used in user participation as they are easy to apply. In addition, the user knows about privacy and security issues and potential risks and can be aware of them. Such awareness enables users to get confronted with the topic of privacy for sensitization. Furthermore, the chance to co-determine gives users a new perspective toward robots, resulting in positive effects toward trust and companionship.