Evaluating GDPR right to information implementation in automated insurance decisions

2.1 GDPR: transparency and right to information about automated decision making

The right to access personal data collected is set out in Article 15 GDPR 1 h. It states that data subjects have the right to gain information about

the existence of automated decision-making, including profiling, [..] and, [..] meaningful information about the logic involved, as well as the significance and the envisaged consequences of such processing for the data subject. ¹⁸

Article 12 GDPR further sets out general rules that apply to the obligation to provide such information. Accordingly, the information must be provided

[..] in a concise, transparent, intelligible and easily accessible form, using clear and plain language [..]

While there is some research on the right to access in general (see esp. Pins et al. ¹⁹ ), the specifics of automated decision making are not yet targeted, despite increasing automation. A person who is subject to automated decision-making in a particular matter, according to Article 15(1)(h), has the right to know that such automated decision-making exists and, in this case, additionally has the right to obtain:

1. meaningful information about the logic involved, as well as

2. the significance and the envisaged consequences of such processing for the data subject.

Obtaining information about this matter is thus part of the right of access, which, however, must provide more information, including purposes of processing, categories of data processed, recipients.

Given the lack of clear judicial precedent in the GDPR for guidelines on the information process, Wachter et al. ²⁰ interpret the right to explanation for automated decision-making more as a right to information. While Temme et al. ²¹ acknowledge that algorithms can be highly complex, they argue that increased transparency could make them more compliant with data protection principles. However, Dexe ²² contends that transparency does not imply unrestricted access to all information. Instead, transparency should enable data subjects to assess whether the processing is appropriate and lawful. This approach aligns with the interests of data subjects, as they can only process a limited amount of information. ²³ Consequently, Wauters et al. ²⁴ emphasize the importance of usable privacy techniques, such as visualization methods, to present privacy information more concisely and comprehensibly.

A few empirical studies on the meaningfulness and usability of information request processes have already been conducted. These include, among others, the studies by Mahieu, ¹¹ the multi-country study by Norris, ¹² or the study commissioned by the Federal Ministry of Justice and Consumer Protection to investigate online services. ¹ The studies came to very different conclusions. In some cases, non-compliance with data protection regulations was found. In some instances, the controllers did not respond at all or their responses did not contribute to more (perceived) transparency for consumers. ^{12 , 19 , 22 , 25} The ambiguous research results show that there is a need for further research in this area. Therefore, the present study also focuses particularly on the question of how the process of providing information can be designed in a more user-friendly manner, as Pins et al. have first prototypically defined it. ¹⁹

2.2 Deceptive patterns in GDPR

Deceptive patterns in UX design refer to manipulative user interface strategies that intentionally mislead users into making choices that benefit service providers, often at the users’ expense. These patterns have gained attention due to their ethical implications and prevalence across various digital platforms. The following sections outline key aspects of deceptive patterns based on recent research.

Nie et al. ²⁶ classify deceptive patterns into 64 distinct types, including Complete Obstruction and Obfuscation, which hinder users from performing desired actions like account disabling. ²⁷ Nie et al. ²⁶ also propose a comprehensive framework, the Deceptive Pattern Analysis Framework (DPAF), to categorize these patterns and assess their impact on users.

Research indicates that deceptive patterns can adversely affect user mental health, particularly among vulnerable demographics, leading to feelings of frustration and helplessness. ²⁸ The pervasive use of deceptive patterns necessitates further research and potential regulation, especially in the social media industry, to protect user interests and promote ethical design practices. ^{27 , 29}

Research on deceptive patterns in data protection, particularly concerning data subject rights, has gained significant attention in recent years. The European Data Protection Board (EDPB) has played a crucial role in this area by publishing guidelines on recognizing and avoiding deceptive patterns, e.g. for social media. ³⁰ These guidelines identify six categories of deceptive patterns: overloading (providing too many options or information), skipping (designing to make users overlook aspects), stirring (appealing to emotions or using nudges), obstructing (hindering or blocking users in their process), fickle (using design to make navigation harder), and left in the dark (hiding information or controls). Each category represents different ways in which user interfaces can manipulate or hinder users’ ability to exercise their data protection rights effectively. The guidelines also provide examples and illustrations of design patterns that violate GDPR provisions, offering recommendations specifically for social network operators to ensure compliance.

2.3 The role of transparency and automation in the insurance industry

Insurance companies, offering intangible services that are delivered in the future, rely heavily on a trusting relationship with their customers. Industry experts often refer to this as a “business of trust”, since insurance is fundamentally based on promises. Insurers sell contingent promises to pay, often at a distant and unspecified point in the future. Transparency in decision-making mechanisms is a key factor in enhancing this trust. ³¹

Household insurance is widely prevalent in Germany and is offered by various market participants, allowing for broad coverage of major insurance providers. Moreover, household insurance is a relatively simple form of insurance. The contract structure is essentially similar across different service providers, enabling good comparability. Another aspect is the feasibility of participant recruitment: Household insurance is widespread in Germany. Simultaneously, information about household insurance is not based on highly sensitive data, such as health data, ensuring that willingness to disclose data for the study is not unnecessarily limited. Due to the increasing demand for data and the growing complexity of calculations, the societal relevance of this topic is rising. Through the right to information, affected individuals should be able to comprehend the calculation of their household insurance. This study focuses on household insurance due to its prevalence, comparability, and the balance it offers between data sensitivity from a participants’ perspective and societal relevance, making it an ideal subject for examining transparency in automated decision-making within the insurance sector. Our research question thus is:

What current design practices in information request processes contribute to or hinder consumer understanding of automated decision-making systems and may thus pose deceptive patterns?

3.1 Conduct

Methodologically, like previous studies, ¹⁹ we divide the process of exercising the right to information about automated decision making into the five steps of (1) finding, (2) defining, (3)requesting, (4) receiving and (5) analyzing. Once a request is made, organizations have 30 days to respond, as per GDPR. Accordingly, our study is divided into two main interventions:

1. Observation of finding, defining and requesting information with subsequent interview

2. Receiving and joint analysis of the data followed by an interview.

In the first phase of the experiment, we pursued three objectives:

1. Preliminary interview

   We addressed general attitudes and expectations regarding data protection, particularly the right to information about automated decision-making.

2. Application

   We observed participants as they attempted to submit an information request.

3. Evaluation

   Finally, we discussed the process up to the point of application submission with the participants.

Initially, study participants were asked demographic questions, self-assessed technical affinity, and their attitudes towards data protection in general. At this point, households were already aware from recruitment that we intended to exercise the right to information about the existence and logic of automated decision-making. In the second step, we tasked them with independently attempting to assert their rights with their household insurance provider. No specific instructions were given regarding the procedure. We encouraged participants beforehand – and if necessary, during observation – to explicitly vocalize their thoughts in a think-aloud manner during their process. For a final evaluation, questions were also prepared in a semi-structured interview guide. After the information request was successfully submitted, we discussed with the participants their assessment of the information process up to that point.

When searching for relevant information to make the request, participants were only instructed to use the internet or their own documents as search sources. Telephone inquiries were intentionally not considered. The information request was sent using the study participants’ email addresses, which were also known to the respective insurance companies. This was intended to avoid potential identity verification queries from the insurance company and to ensure the process was treated as a normal customer inquiry. To obtain more comparable results, all insurance companies received a template request with the same wording.

The second phase included the customers receiving their answer and a joined investigation. After sending the email, we had to wait for a response from the insurers. Accordingly, we asked the households to inform us upon receiving a reply to schedule a second appointment, or, if there were follow-up questions from the insurance company, to clarify these together. We asked the households not to look into the response yet, but to do this together with us. This way, the receipt of the data itself and the initial reaction to it could also be observed. The objectives in the second phase were:

1. Data receipt

   We observed and discussed topics related to the phase of receiving, such as data retrieval and response style.

2. Data analysis

   We observed and discussed the participants’ own analysis of the received information and their classification in terms of support for their explanatory approaches. We analyzed the provided information from the perspective of UX design guidelines.

The requests for information to the insurance companies took place in spring 2021. Accordingly, all requests for information were sent out in the same period in which the interviews were conducted. All 12 interviews took place at the premises of the study participants. During the interviews, the participants had their own household insurance documents and their preferred end device (smartphone, tablet or laptop) was available to them in order to create a familiar working environment. During the interviews, both the soun d and the screen of the participants’ computers were recorded. The pre-installed Windows 10 screen recording function was used for the screen recording. The participants’ consent was obtained before the interview. Reference was made to the pseudonymization of the data (Table 1).

3.2 Participants

A total of 12 people (8 female, 4 male) took part in the study. The average age was 47 (min. 24, max. 69). The technical affinity of the participants is mixed (3 low, 3 medium, 6 high). None of the respondents are data protection nor IT experts, nor experts in the insurance business. The prerequisite for participation in the study was the possession of household contents insurance and the willingness to be interviewed with a subsequent request for information from the insurance company. Only two participants were insured with the same insurance company. This meant that a total of 11 insurance policies were covered. Participation was voluntary and was not compensated.

3.3 Analysis

Qualitative content analysis was used to evaluate the data collected. This is an evaluation method that processes texts that arise during data collection in the context of social science research projects. ³² The aim is to identify, condense and describe phenomena and concepts from the area of the research question(s) on the basis of a defined database (here: the interviews conducted). The audio and screen recordings made during the interviews serve as data material. These were then transcribed. Deductive coding based on Mayring’s ³² qualitative content analysis was used to analyze and categorize the transcripts, in which two of the authors coded the material independently of each other and combined the existing codes in a second step. The aim of the analysis was to analyze the usability of the existing information processes of the insurance companies and to identify possible deceptive or bright patterns. The responses of the insurance companies themselves have not been analyzed specifically, but rather served as a tool to spark discussion and reflection among participants. We generally describe the characteristics of the answers to the right to information on automated decision making in chapter 4.4.

4.1 Attitudes and expectations prior to the request for information

While some participants were aware of the GDPR in principle, no one stated that they had ever exercised their right to information. Participants who did not know anything about the GDPR or their corresponding rights were briefly informed about it. When asked about their expectations of the right to information, only two participants were optimistic and stated that the insurance company would certainly make the information process easy. This included T9, who said:

After all, the insurance company is obliged to make the whole thing easy, so I think they will. They want to keep or satisfy the customer.

The remaining participants were more pessimistic and did not believe that the request for information would be made easy. T2, T3, T4, T11 and T12 stated that their insurance company will certainly be skeptical if they receive a request for information. T4 commented:

I can well imagine that the insurance company will ask why I want to know. So, if for example, I’m unhappy with something or there’s a problem.

T2 suspected that the type of response could also depend on the case handler themselves:

[…] it certainly depends on who you end up with, i.e. what kind of expert advisor.

When asked whether the insurance company’s answers would be written in language that laypeople could understand, six participants expressed skepticism about this (T2, T3, T4, T5, T11, and T12), citing different reasons. While T2 fears technical terms from the insurance industry and IT (“I think they will use a lot of technical terms.”), T11 draws attention to the complexity of the topic of data protection:

The content will definitely be difficult to understand, just as you would expect from many data protection texts.

4.2 Phase 1: finding, tailoring and submitting the application

The first part of the information process, i.e. until the data is requested, is made up of three parts: The information search, the tailoring of the request and the information request itself.

4.3 Attitudes and expectations following the request for information

After the process, the participants were asked what expectations they now had of the answers. Many participants were rather pessimistic; the previous process had obviously already lowered their expectations. T10 let it be known how the search in the data protection information had impressed him:

The answer will probably look like the privacy information on the insurance site, small print with lots of blah blah blah.

Participants already had some suggestions for improvement:

Something like this could be made easier, e.g. just read it online and tick what you want. With the current method, I have to work out everything myself (T3).

T3 also pointed out that the layout of the letter and the legal jargon would not be accessible to everyone and that it could possibly make an impression on the recipients and possibly even influence the results:

I would have written that in simple sentences, I’m not a data protection expert. But I think that’s [the style] I would have received the answer then. We are now sending a proper sample request with details of the legal text etc., so they will make more of an effort to answer sensibly.

Here the participant brings a personalization of the information into play – albeit in a negative sense, because of the professionally formulated application. Whether this would of course be used strategically by the company or rather subconsciously by a clerk remains to be seen.

4.4 Phase 2 – receiving data: characterizing the answers

A total of 12 requests for information were sent out. One insurance company (V9) was contacted twice. Ten insurance companies responded to the request for information. One insurance company (V4) did not respond to the request for information. After two months, T4 called his insurance company and received the answer that the data protection e-mail address (via which the request for information was sent) was not read by any employee. The request for information was then sent to the insurance company’s main e-mail address. By the end of the study, T4 had not received a reply to the second request for information either.

As can be seen in Figure 1, there are large time discrepancies between sending the request for information and receiving the response. The length of the responses also varies considerably. Figure 1 shows that 10 of the 12 requests for information were received within four weeks. This means that nine out of 11 requested insurance companies met the data protection requirement in Art. 12 para. 3 sentence 1, according to which the response should not take longer than one month. Only V6 only replied after just over 4 months and V4 did not send a reply at all.

Figure 1:

Characterizing answers by time, extend of answers and medium.

While the shortest response consisted of three sentences, the longest response was six pages long.

4.5 The information provided and its contribution to transparency

This section describes users’ experiences in dealing with insurers’ responses and the data received, if any.

5.1 Disappointed expectations about “data protection as added value”

The idea of positioning data protection as added value in customer communication is not entirely new, but it has gained renewed attention due to the extensive changes brought about – not exclusively, but certainly predominantly – by the GDPR. Our study shows that some of our participants, too, expected companies to please their customers in the area of data protection and data subject rights, and to ensure a positive customer experience. This expectation was already dampened by the appearance of the privacy notices and further disappointed by the partly evasive, partly refusing attitude of the insurance companies. As Pins et al. ¹⁹ demonstrate, the right to information about an automated decision is not an isolated case in user-unfriendly design.

5.2 Deceptive pattern in process design

For a long time, there have been indications that texts and information from the data protection domain are difficult for users to understand. The prototypical example for this are privacy policies, which have been long researched. ^{33 , 34 , 35} While this may be due to legal language and the aim of achieving legal certainty, the situation is different for other use cases: For several years, cookie banners have even been actively designed against users’ interests. Some research contributions have already discovered that the process design for the right to information lacks some of the most basic interaction patterns and usability criteria. The identification and research of deceptive design patterns has also dramatically gained importance. Our study provides indications of what such patterns may look like for exercising data subject rights. Finding the contact point was relatively easy. Almost all participants also received responses in principle. However, these differed fundamentally in terms of the companies’ willingness to provide explanations. In our exploratory study, we were thus able to identify several phenomena among the largest German insurers in the household insurance sector that constitute implementations of deceptive patterns. In the following, we (1) discuss reasons provided by insurance companies to limit or deny answers to the request. We then discuss deceptive patterns in (2) the process of exercising the right to information, and (3) the design of answers provided. As a classification scheme, we use the Guidelines of the European Data Protection Board, ³⁰ which identified the deceptive patterns of “overloading”, “skipping”, “stirring”, “obstructing”, “fickle”, and “left in the dark”.

5.3 Limitations and future work

At this point, it should be noted that the study only focused on the information request process via the insurance companies’ websites, and the request was only made via email. The usability recommendations therefore only apply to this type of request. The selection of participants was an ad-hoc sample and could potentially benefit from a strategic planning of different user groups in a second step to provide more targeted results in this regard. Moreover, the study includes only a small number of participants, which does not allow for immediate generalization of the problems. However, this is not the goal of qualitative-exploratory studies. In a first step, we aimed to identify existing requirements and deceptive pattern through in-depth qualitative analysis, thereby making them accessible for further research. The present study is therefore an important starting point and only enables further research in the area, which could, for example, be more evaluative-quantitative in nature. A subsequent task is, for instance, to investigate the robustness of the identified deceptive patterns using quantitative methods. A similarly related limitation is the fact that we mainly acquired only one person per insurer. However, this was a conscious decision, as the behavior of the insurance companies should be quite standardized due to defined processes, and thus different behaviors of the organization could be better found through the study design. Future work could validate our hypothesis here. Another venue of interest regards a systematic investigation of personalization, particularly of the differences in information disclosures with different formulations of requests and to any (un)conscious unequal treatment that may exist. Last, but not least, the instances of deceptive design identified in our studies align well with the existing EDPB framework and thus provide first insights into potential issues with deceptive design specific to the insurance industry. Our findings therefore can be a starting point to draft potential solutions or even “bright patterns” in the future.