Human-centered design of a privacy assistant and its impact on perceived transparency and intervenability

2.1 Human-centered design

Human-centered design deals with the approach of focusing on the user, their needs, capabilities, and limitations during the design of products and systems. ¹² To gain a comprehensive understanding, it is suggested to employ a combination of qualitative and quantitative methods in usability studies. ¹³ Usability and UX aspects are often assessed using quantitative methods such as the System Usability Scale (SUS) ¹⁴ and the User Experience Questionnaire (UEQ). ¹⁵ However, a substantial part of the mixed-methods approach is about incorporating qualitative methods to gain a comprehensive understanding of users and their interactions with a system. ¹³ Nevertheless, a strict dichotomy between qualitative and quantitative research methods is argued against based on the dichotomy’s lack of foundation and purpose. ¹⁶ Consequently, qualitative-oriented content analysis often involves quantified scaling techniques such as frequency analysis, which involves counting assigned codes based on their occurrence. Other effective analysis techniques include valence and intensity analyses, as well as contingency analysis. ¹⁶

Proper human-centered design of a privacy assistant builds on strong requirements research. Designing accordingly to the needs and preferences of the stakeholders will result in a usable system that helps people achieve their goals. One has to understand not only the goals of the users, but also the context in which they are trying to reach them. The context is made out of their capabilities and skills, their knowledge, their attitude towards aspects of privacy and data protection as well as their limitations. Including the human perspective in a privacy assistant’s design helps to implement the goals and intentions stated in the general data protection regulation (GDPR). The regulation is designed to empower individuals to make informed decisions about how their data is used and to provide them with the option to intervene if they choose.

2.2 Data protection

When using self-measurement technology part of the most common use cases for people is to share their self-measurement data within their social network but also with certain service providers. ¹⁷ People collect digital data about their bodies through technical means, such as smartwatches, to analyze and monitor athletic performance, their sleep, or their health. ⁴ As a result of sharing this kind of data through social networks, people achieve a sense of recognition and social belonging. ¹⁸ Whereas when people decide to share their data with service providers it is usually for a different incentive. It is already a frequent practice to share private data with health insurance companies or insurance agencies in exchange for discounted benefits or other financial incentives. ¹⁷ However, research shows that users are often unaware of the extent of data being collected about them, particularly when third parties aggregate this information without their explicit knowledge. ¹⁹ Aside from the benefits and incentives mentioned, there are potential risks associated with self-tracking and data sharing. Potential risks include identity theft, profiling, locating of users or stalking, embarrassment and extortion, and corporate use and misuse. ²⁰ Even though a portion of users are aware of the risks, most are not, and they tend to assign little to no importance to the matter. ²¹ Additionally, another study found that their participants lacked the motivation to protect their personal information from external threats or exploitation. ²² Conversely, when transparency tools reveal the extent of personal data exposure over time, user attitudes shift. When users were shown a longitudinal view of their online tracking, they became more concerned about privacy risks and were more likely to take protective actions, such as limiting third-party data access. ²³ Similarly, other sources report a growing demand for protecting one’s private data and therefore it is crucial to explore ways to safeguard one’s information. ^{24 , 25}

According to the protection goals from Hansen and colleagues, transparency is critical for the user to intervene in a targeted manner. ²⁶ However, users are usually not sufficiently educated about privacy legislation and the hazards involved. ⁹ As a result, the criteria for successful privacy protection can be described as comprehension of data processing (transparency) and the ability to act against it (intervenability). Both of these criteria form the core principles of the GDPR, ²⁷ introduced to the European Union member states in 2018. Implementing these principles, however, is not without challenges for companies and service providers. Businesses in data-driven markets often struggle to strike a balance between protecting user privacy and driving value creation, yet they increasingly recognize transparency and intervenability as crucial for building trust and ensuring GDPR compliance. ²⁸ This highlights the necessity of privacy assistants’ goals of promoting transparency and enabling intervenability. Finally, to be truly effective, privacy-enhancing technologies must evolve beyond basic transparency toward user-tailored and actionable interventions, ensuring that individuals are not just informed about risks but also empowered to mitigate them effectively. ²⁹

2.3 Related usability research

Self-tracking applications, reveal significant usability and UX limitations in areas such as transparency, data control, and privacy management. Recent research highlights several critical gaps where these applications fail to deliver a seamless, human-centered experience regarding privacy. ^{7 , 30 , 31 , 32} Firstly, there is a lack of accessible privacy controls; many self-tracking apps provide limited and hard-to-navigate settings, making it challenging for users to control what personal data is collected, shared, or retained. This issue undermines user trust and often leads to confusion and frustration, as users struggle to manage their privacy preferences effectively. ⁷

Self-tracking apps frequently lack real-time privacy notifications that would inform users about active data processing, resulting in a lack of awareness about ongoing data collection. Without these notifications, users may assume their data is handled securely by default, missing essential information about data sharing or third-party involvement, which ultimately reduces transparency. ³⁰ In line with that, there is a lack of ex-post transparency options in self-tracking applications. Ex-post transparency tools, which offer visibility into data processing after the fact, such as historical access logs or data usage reports, are rarely integrated. Without these retrospective tools, users cannot review or intervene in data handling practices based on historical data usage, which limits their ability to modify permissions and hold the app accountable. ^{30 , 31}

Finally, another key issue is that privacy policies in self-tracking apps are often complex, lengthy, and written at a high literacy level, making it difficult for the average user to understand how their data is being used or protected. This complexity can lead users to skip these policies altogether or feel distrustful, creating a gap in informed consent. ³²

In summary, self-tracking apps often fall short in delivering a human-centered privacy experiences, with key issues including inaccessible privacy settings, absence of real-time data notifications, overly complex privacy policies, missing transparency tools, and limited retrospective data visibility. These gaps underscore the need for a more intuitive and accessible approach to privacy management, which would promote greater user autonomy, satisfaction, and trust in these applications. Our proposed prototype addresses these significant gaps by implementing a suite of features designed to enhance user autonomy and engagement with privacy management. Key functionalities include easy personalization of privacy settings, allowing users to effortlessly control what personal data is collected, shared, and retained. Additionally, real-time notifications inform users whenever data processing contradicts their established preferences, enhancing awareness and transparency. The prototype also provides assistance for users to intervene in undesired data processing, making it easier to exercise their data subject rights. Furthermore, adaptable data protection content enables users to tailor privacy information to their own knowledge level, helping them better understand and manage their data handling practices. Collectively, these features create a more intuitive, accessible privacy experience that supports user control, informed consent, and trust in self-tracking applications.

3.1 Process overview

The application prototype was created based on previous extensive requirements research and under close observation of usability and UX experts, who provided feedback to revise the prototype iteratively. The study aimed to ensure representative results for a diverse user population matching the use context analysis. To achieve this, 20 participants were selected for the in-person study. The study sample consisted of eight female and twelve male participants. Participant ages ranged from 18 to 45 years (six participants aged 18–25, eight participants aged 26–35 and six participants aged 36–45). Although the recruitment process did not exclude participants above 45, no individuals in that age group took part in the study. Audio recordings of the test sessions were made to facilitate later evaluation and categorization of user feedback.

3.2 Privacy assistant prototype

The creation of the application’s high-fidelity prototype employed various design solutions that were gathered based on specific usage requirements, framework conditions, and project demands. These solutions were then evaluated using wireframes and sketches. The iterative process involved expert feedback to refine the solution, resulting in three primary features or application functionalities, all centered around user flow and functionality. The primary goal of the application is to educate users about privacy and provide insights into the data processes involved when using self-measurement technology. It empowers users to intervene by exercising their data subject rights or adjusting privacy settings for their self-measurement devices. Figure 1 showcases the prototype’s main functions. The first main functionality shows data protection instances, which conflict with the user’s preset data protection preferences. This will help address transparency of personal data use and encourage easy-to-use options of intervenability. The second feature, named “Helpdesk” assists users in adjusting their data protection preferences, and addresses data protection conflicts and tutorials. The third feature is an information center that provides the user with easy-to-process content about the topics related to the features in the application.

Figure 1:

Prototype main pages: privacy dashboard (two tabs: “Found Conflicts” and “Devices & Data”), helpdesk (three tiles: “Data Protection Preferences”, “Data Subject Right Templates” and “Device Tutorials”) and information center (headers: “DSGVO (GDPR)”, “Important Terminology” and “Data Protection in Everyday Life”).

When a user first starts using the application, they are guided through an onboarding process. During this process, they have the option to add their self-measurement technology to the application. The assistant then analyzes and presents the types of data being used, shared, or collected in a structured, device-specific overview integrated into the app’s dashboard. In the setup, the application prompts the user to define their data protection and privacy preferences. Based on their set preferences the application alerts the user when data is being collected, shared or processed they opted to protect. To simplify setting adjustments, three default intensity levels for privacy settings are suggested in line with the project’s privacy personas: ³³ (1) lenient privacy, (2) balanced privacy, and (3) strict privacy. Additionally, the setup allows users to personalize the application’s text content according to their familiarity and desired level of detail. This is done by selecting your personal data protection skill-level with the help of three identifying descriptions (Figure 2).

Figure 2:

Setting your personal data protection skill-level (translation: personalize according to your skill level: beginner: “I have little experience with data protection regulations. I prefer clear and simple explanations.” Intermediate: “I am somewhat familiar with data protection topics. I prefer a balance between details and clarity.” Expert: “I am well-informed about data protection topics. Detailed articles and in-depth content are not a problem for me”).

After the setup, the user gets forwarded to the privacy dashboard which also functions as a landing page. This page consists of two tabs: “Data Protection Conflicts found” and “Devices & Data.” A “Data Protection Conflict” is a warning or notification from the application that certain devices are using or collecting data types inconsistent with the user’s privacy preferences. These conflicts are categorized by relevance and can be resolved by exercising data subject rights, adjusting device settings, or allowing the data to be used, which overrides the privacy preferences for that particular type of data. The Privacy Helpdesk is accessible through the second page in the bottom bar’s navigation structure. Here, users can modify their privacy preferences or access GDPR templates. Additionally, the Privacy Helpdesk offers tutorials for self-measurement device settings. The final main feature page is the Information Center, which contains articles, introductions, and explanations related to privacy topics. Users can engagingly explore GDPR-related content or broaden their knowledge by reading current articles on data protection.

3.3 Comprehensive user test

The content of the user test was divided into six sections. Subsequently, they are addressed in chronological order based on their appearance in the user test. Starting with a pre-questionnaire, then moving on to prototype-based user test tasks, quantitative and qualitative questionnaires, A/B test pairs, cooldown questions in the form of a semi-structured interview, as well as adapted questionnaires on the topic of privacy and GDPR knowledge.

First, participants were asked to fill out a preliminary questionnaire. This questionnaire aimed to assess their demographics and their relationship with self-measurement technology. The latter assessment was achieved by having participants identify with short forms of the project’s defined user personas. ³³ Following the pre-questionnaire, the participants were instructed to execute a set of user tasks with the given prototype. A red route importance ranking was conducted based on the proportion of users and frequency of use to gain insights into the most crucial features. Participants were asked to answer post-task questions after each user test task to encourage them to report on perceived problem sources, preferences, and needs.

In the next parts of the user test, two standardized usability and UX questionnaires were employed. Both the SUS and the UEQ were used due to their specific strengths and weaknesses. Whereas the SUS provides a straightforward way to assess the perceived usability of a product, it does not cover the user perception of security, which is shown to be critical to identity management and data protection. ³⁴ Consequently, the UEQ was used to expand the assessable range of user-centered aspects.

Another study goal is to assess the application’s impact on the user’s perceived transparency and intervenability concerning data privacy. Two A/B test pairings of relevant application pages were created for this purpose. Participants were encouraged to become acquainted with the two versions of a pair before describing which of the two versions they preferred and why. The differences in the A/B versions were linked to privacy transparency and the extent of intervenability. For example, version A’s screen presented substantially more information than version B, indicating a better level of transparency. The initial A/B pair depicted the application’s dashboard. Its’ key difference was in the portrayal of the significance of a discovered “Data Conflict”. Version A presented this as comprehensive continuous text, whereas version B assigned a significance value to the data conflict using a traffic light system. Similarly, the extent of descriptive text was adjusted in the second A/B set, and the usage of icons was contrasted. The differences in levels of intervenability were designed based on the clustering of similar intervention options and the way they were presented. One version offered direct links to specific data subject right templates, while the other only suggested that the conflict could be resolved through the helpdesk functionalities. The participants in the interview expressed their remarks regarding the respective versions, which were categorized and assessed from the transcript.

At the end of the user test, participants were asked a series of cooldown questions to capture a reflection for the entire application prototype. Part of the questions included a holistic evaluation of the application rated on a five-point Likert scale (“How did you like the application overall?”). Upon this, positive and negative aspects of the application were inquired (“What about the application did you like specifically?”, “What about the application did you not like?”), as well as reasons that would prevent participants from using the application (“What would be things that would hinder you from using the application?”) and wishes for the future state of the application (“What are your wishes for a future state of the application?”). The last questions related to the participants’ preference regarding the frequency and type of application notifications (“Would you like to receive notifications from this application (e.g. about data protection conflicts found or changes to the status of the sanity check)? If yes, how often?”) as well as an assessment of the participants whether the use of the application has an influence on their perceived transparency and intervenability on the topic of data protection or in which situations they would use such an application (“Imagine you were to use such an application in your everyday life. What would change in terms of your transparency and ability to intervene on the subject of data protection? In which situations would you like to use it?”).

To further assess the participants’ privacy attitude (PRIV-Questionnaire) and their knowledge regarding GDPR contents (GDPR-Questionnaire), two short questionnaires each containing four items were utilized. Their items are reported in Table 1. Items from the PRIV-Questionnaire employed a seven-point Likert scale ranging from “do not agree at all” to “completely agree”. The questions asked about the participants’ understanding of how companies manage personal data, the relevance of privacy to online companies, a ranking of the relevance of personal privacy compared to other topics, and the level of perceived concern about a threat to one’s privacy. Similarly, the GDPR questionnaire also used a seven-point Likert scale, the difference being that each item was assigned individual scale labels. Participants were asked about their familiarity with the GDPR and their level of knowledge about its contents. Finally, they were asked to rate their level of knowledge compared to the rest of the population and to indicate their level of clarity about the features of the GDPR.

3.4 Qualitative data analysis

Lastly, a qualitative valence analysis of the transcribed user test audio was conducted using deductive and inductive approaches. This approach follows the recommendations of Mayring and by those recommendations, ¹⁶ a set of categories was established based on initial correlation assumptions. Following that, a comprehensive coding system was created through systematic observations and iterative refinement. This coding system included aspects of usability, UX, sources of errors, participants’ attitudes toward privacy issues, and their perceptions of transparency and privacy intervention options through the interactive system. Textual items collected were carefully examined and classified as positive or negative statements based on their context, allowing for a more nuanced analysis of participant feedback. Figure 3 presents an overview of the coding system.

Figure 3:

Qualitative coding system.

4.1 Privacy and data protection

Regarding privacy attitude and GDPR knowledge, the participants predominantly described themselves as individuals who are sensitive to how companies manage their data (priv_item1). Specifically, they placed foremost importance on their privacy when it comes to internet companies (priv_item2). Furthermore, personal privacy emerged as a significant concern compared to other topics (priv_item3). At the same time, there was a broader distribution of responses, indicating varying levels of concern about threats to their privacy (priv_item4).

The majority of participants reported a moderate understanding of the GDPR, with responses falling in the middle of the seven-point Likert scale for gdpr_item1, 2, and 4 while showing a slight tendency towards lower knowledge levels when comparing themselves to the rest of the population (gdpr_item3). Qualitative data analysis reflected the participants’ interest in data protection and privacy issues and their assessment of the usability of opt-out options within the prototype. Almost every participant reported at least a basic understanding of the GDPR, indicating a certain level of familiarity with the regulations. However, their interest in data protection issues varied significantly, with attitudes ranging from considering data protection as the “most important aspect” to regarding to it as “obsolete”.

4.2 Usability and user experience

The high-fidelity prototype’s overall usability was assessed using the SUS. Among participants an “best imaginable” SUS rating (Md = 87.5, M = 85.25, SD = 12.14) was reported, according to the adjective rating system. ³⁷ With a standard deviation of 12.14, there is some variation in participant responses, but overall, the scores are clustered relatively closely around the high mean score, suggesting consistency in usability ratings. The radar chart gives further input on the specific questions from the SUS, that performed particularly well (Questions 2, 4 and 8) and those that fell behind the average (Questions 1 and 9) (Figure 4).

Figure 4:

System usability scale results (adjective-, grade- and acceptability-scale, conclusiveness, radar chart). ³⁸

Furthermore, the UEQ was utilized to assess the overall UX of the high-fidelity prototype. The sample exhibited positive mean values for all subscales of the UEQ. The subscales of attractiveness (M = 1.93, SD = 0.65), transparency (M = 2.24, SD = 0.76), efficiency (M = 1.95, SD = 0.86), dependability (M = 1.78, SD = 0.73), and stimulation (M = 1.51, SD = 0.89) all received positive evaluations according to the benchmarking criteria, with mean scores well above +0.8. ³⁹ While still being positively rated, the subscale of Novelty (M = 1.38, SD = 1.06) did not fall within the “excellent” category but the “good” category instead, indicating room for improvement in terms of innovative aspects (Figure 5). ³⁹

Figure 5:

User experience questionnaire results on benchmark. ³⁹

4.3 User preferences and impressions

This chapter explores user preferences and impressions based on qualitative feedback gathered from transcriptions analyzed through a structured coding system. A total of 1,167 codes were assigned, capturing both positive and negative mentions across various categories. By organizing feedback according to specific topics, this chapter highlights the key themes. All quotations in this section are drawn directly from participant responses.

1. Usability Strengths: In general, the prototype received favorable comments and was often referred to as “intuitive” and “goal-oriented.” Participants appreciated the “lean design,” which appears “modern and refreshing,” as well as the easy learnability aspect. With a few exceptions, the information architectures were mostly characterized as “suitable” and “understandable” or similar expressions (about 75 % of coded comments). The participants considered the structure and flow of the UX as good (82 out of 92 coded comments). Aspects that have contributed to the usability of the application to a particular extent, according to participants, are the “straightforwardness of processes,” support structures such as pre-filled text fields or activation codes, “recommendation features” as assistance in decision-making, and the personalization of the application’s content through the definable privacy skill level.

2. Terminology Challenges: Negative effects on the UX occurred due to “the length of the setup” and the terminology of functions of the application or labels of some individual interaction fields (22 out of the 26 coded comments). The most frequently criticized labels were the main functions “Privacy Helpdesk” and “Information Center”. Many participants had a firm idea of a “Helpdesk”, which did not match the actual functions of the prototype. They expected the Helpdesk to be the place where they could get assistance concerning application features. Shorter versions of “Data Protection Administration” could be more suitable labels. However, “almost any name that does not evoke fixed expectations” is suitable, according to the participants. When it came to labeling the Information Center, it was not so much the name itself that was criticized, but rather the abbreviation “Info” used in the application prototype. This abbreviation was also expected to have a different function. In line with the previous recommendation for action, a renaming to show the complete title (“Information Center”) and a distinguishable icon could improve the user experience and prevent confusion.

3. User Expectations and Data Sharing: 18 out of 26 coded comments about expectations, including anticipated, during and after the interaction, were favorable for the prototype and expressed that the expectations of the participants were mostly met. Namely, the presentation of the user interface, the interactive behavior, and the functionalities of the prototype were in line with the anticipations of the user group. The application was suitably designed for the given sample and in the given context of use. Participants’ opinions split on the question of whether they wanted to view the raw data of their self-measurement technology in the privacy assistant. On the one hand, such connectivity was desired, and on the other hand, sharing self-measurement data with a third-party application was strictly rejected. However, only a fraction of participants commented on that topic (six out of the 20 participants).

4. Effectiveness and Efficiency: Besides user satisfaction, there are two more aspects to usability: perceived effectiveness and perceived efficiency. Coded comments for the aspect of effectiveness were 100 % positive (25 out of 25 coded comments), whereas the aspect of efficiency reports an almost fully positive result of 52 positive comments of out 55. The participants’ comments on effectiveness and efficiency emphasized the ease and speed of successfully completing their planned actions, thanks to the “intuitive structure” and “flow of the system”. According to the participants, the presetting of data protection preferences in the setup process and the fast navigation structures contributed significantly to efficiency. The only limiting factor for efficiency occurred when filling out the templates for data subject rights. It was criticized that only the template is downloaded instead of sending it directly from the application to the person responsible. This finding contributed to a better understanding of the goals of people.

5. A/B Test Findings: The findings on user preference concerning the A/B tests revealed that about 75 % of users prefer visual cues and icons over text when trying to solve a given task. The presentation of the A/B tests aimed at presenting one version with a high information content and one with a lower one, thus varying the level of transparency and intervenability. Preferences differed regarding the transparency level. Some prefer the lower reading effort and describe the information content as “sufficient”. Others would like to be presented with a concise text that can be expanded via user action. According to the participants, a positive effect on the perceived intervenability was achieved by the “grouping of similar functions” for affiliation and coherence of the different options. Additionally, the use of icons was preferred over the absence of icons. The icons would “assist with recognition of the functions and reduce reading effort”. This trend toward providing less information, however, was bound on the premise that key information would remain transparent to the user.

6. Privacy Settings Clarity: Regarding the clarity of the user interface, the breakdown of the privacy preference settings attracted negative attention. The “mental effort required to find the changes” between the setting levels was criticized. The breakdown currently shows both the “unprotected” and the “protected” data types. An improved presentation could be achieved by an overview table that shows the preferences next to the data types and a staggered presentation of existing changes between the levels. However, one aspect that was without exception considered positive was the color design of various interaction fields, warning messages, and related interface elements in general (41 out of 41 coded comments). Especially the design of the warning messages for irreversible actions was perceived as “effective”.

7. Accessibility and Personalization: Both the post-task question about participants’ suggestions for improvement and the cooldown question about wishes for the future state of the application encouraged demands to be made to a special extent reaching a total of 97 coded comments. It is important to note, that these comments were participant-specific and therefore usually not mentioned as many times across the sample. However, demands that were named multiple times included the following: In line with accessibility aspects, the demand was expressed to enable text input by speech or the output of text content by text-to-speech systems. These would reduce barriers for physically impaired people and provide a way to fully use the application for that user group. Further, in the area of accessibility, requests were made for a “senior mode”. This should be characterized by “large font sizes and simpler labels”. Another requirement is the desire for a tutorial to explore the functions and contents of the application when using it for the first time. However, such help structures should be “repeatable even after longer use of the application, yet not mandatory to start with”. A set of demands can be summarized from statements about the personalizability of the application. For example, the extension of the “Information Center Skill-Level” to all application content was discussed, so that users could choose between a detailed “expert mode” and a less extensive “fast mode”. One personalization that could have a significant positive impact on the user experience, according to participants, is the “smart customization” of recommendations and ratings. This meant that after solving a privacy conflict, the application would ask the user whether the rating was found to be appropriate or needed to be changed. Thus, over time, a “near-perfect fit” to user preferences could be achieved. The same adaptation was also conceivable for some participants for the contents of the information center. The application would then adjust the article selection based on feedback according to the user’s fields of interest. To speed up the process, some participants confirmed that they would answer a short sequence of assessment questions if this would result in noticeable differences in recommendations. A final personalization was mentioned by participants regarding notifications. There is a desire not only to turn notifications on or off depending on the topic but also to have the option of receiving them at defined times or after certain intervals. In addition, participants have requested functions for saving information. This function was compared to a “digital wallet”. It would contain all the information needed to fill out a data subject rights template (e.g., personal address, address of the persons responsible, identifiers of various manufacturers).

8. Transparency and Intervenability: In a final assessment of perceived transparency, 17 of the 20 participants confirmed that using the tested application would have a positive effect on perceived transparency. The application was considered a good “interface for data protection” and to present information in a user-friendly way. Those who felt that the application would not affect their perceived transparency cited either their strong and extensive knowledge or their scarcity of exposure to privacy issues as reasons for this assessment. An equal number, but not the same participants, reported in the final evaluation about the perceived intervenability when employing the application that it would have a positive impact. The opportunities for intervention appeared “meaningful, effective, and usable” to the participants. At the same time, the same reasons for the lack of influence on intervenability were presented as before for perceived transparency.