Tackling Hate Speech in the Digital Space: Germany’s Plans on an Act Against Digital Violence and its Impact on Ethno-Cultural Minorities

2.1 Legislative Context

On 17 February 2024, the European Union’s (EU) Digital Services Act (DSA)^[2] became applicable. Together with the Digital Markets Act (DMA),^[3] which already applies since 2 May 2023, the EU has comprehensively revised its regulatory framework for digital platforms.^[4] In the wake of these sweeping legislative changes, the German government launched an initiative to supplement EU legislation with specific measures to enhance the judicial enforcement of the subjective rights of those affected by digital violence.^[5] These plans include replacing certain provisions of the 2017 Netzwerkdurchsetzungsgesetz (Network Enforcement Act),^[6] which the DSA has largely rendered obsolete, with the Digitale-Dienste-Gesetz (Act on the implementation of the DSA) and the Gesetz gegen digitale Gewalt (Act against Digital Violence). While the Federal Government submitted a draft of the former bill to the Bundesrat in December 2023 (Drucks. 676/23), the draft of the latter bill, originally announced for fall 2023, is still pending.^[7] As of now, there is only a concept paper^[8] published by the Federal Ministry of Justice in April 2023, which outlines the key elements of the latter bill and has already attracted considerable academic attention (Härting and Adamek 2023, 316–320; Lück 2023, 740–746; Maurer 2024, 257–263; Panahi 2023, 556–62; Schäfer 2023, 734–40; Sehl, Pfleger, and Suliak 2023; Valerius 2023, 142; Weck 2023, 12–15). The Ministry’s paper proposes strengthening private information procedures, allowing victims to obtain court orders to suspend social media accounts, and requiring social networks to appoint a person in Germany authorised to receive service. These measures are general in nature and not specifically tailored to the needs of minorities. However, if thoughtfully implemented, they could bolster efforts against digital hate speech and enhance the protection of individual rights, thereby having a significant impact, particularly for minority groups, who are disproportionately targeted online (cf. Topidi 2019; The UN Special Rapporteur on Minority Issues 2021, 21; Faloppa et al. 2023; FRA 2023, 35 et seq.).

2.2 Scope of the Initiative

Unlike existing approaches to platform regulation that primarily focus on the behavior of gatekeepers, the initiative shifts focus to the (judicial) enforcement of subjective rights. As outlined by Schäfer (2023, 734 et seq.), Panahi (2023, 557), Lück (2023, 741), Valerius (2023, 142–3) and Maurer (2024, 259), the Ministry’s initiative draws on a notion previously unfamiliar to the German legal system: “digital violence”. This term is paraphrased as “violations of the right of personality in the digital space” (BMJ 2023a, 1), with examples including “insults, threats or defamation” (BMJ 2023b, 1 [translations by the authors]). In addition, one proposal regarding the private information procedure even extends to all absolute rights, including the right to establish and carry on a business (Recht am eingerichteten und ausgeübten Gewerbebetrieb) (BMJ 2023a, 3). Overall, however, the paper should not be interpreted as introducing a new category of individual rights infringements, nor as altering the established rules of democratic discourse (BMJ 2023a, 2; Schäfer 2023, 734–5; Weck 2023, 12; critically, Panahi, supra note 11, p. 562).

This approach has been criticized, e.g. by GFF (2023, 1, 3), Maurer (2024, 259–6) and Lück (2023, 742), for being overly broad: By citing an untruthful restaurant review as a violation of the right to establish and carry on a business, it would lack any discernible connection to the protection against ‘digital violence’. Conversely, the proposal has been considered too narrowly focused on individual rights, leaving a gap where hateful contents do not violate subjective rights but rather general legal interests protected by objective law, e.g. in cases of incitement (GFF 2023, 1, 3, 5; Lück 2023, 742–3). On the other hand, Schäfer (2023, 736–7), argued that the concept of personality rights, as recognized by the German Federal Court of Justice (Judgement of 14 June 2022 – VI ZR 172/20), may be broad enough to cover hateful content affecting entire groups, provided that members of that group can be individualized. Examining similar debates, one might even consider whether alternative concepts, such as ‘online harm’ (as discussed by Farrand 2024), would be more suitable for effectively addressing the issue at hand. This range of opinions underscores the need for further conceptual clarification. In particular, the Ministry will need to consider whether its focus on individual rights sufficiently addresses all behaviours it seeks to regulate under the notion of ‘digital violence’ (cf. Maurer 2024, 260).

2.3 Proposed Measures

The Ministry proposed three specific measures: The first is to modify the existing information procedure under the Telekommunikation-Telemedien-Datenschutz-Gesetz (Telecommunications and Telemedia Data Protection Act)^[9] to facilitate the identification of individuals publishing infringing content. This is achieved by expanding the scope of the information procedure to the disclosure of user data, e.g. IP addresses, and extending its application to messenger services (BMJ 2023a, 2–4. Cf. Panahi 2023, 557–8; Schäfer 2023, 735 et seq.; Lück 2023, 742 et seq.; Weck 2023, 12; Maurer 2024, 257–8). However, as Lück (2023, 743) points out, in some cases (e.g. shared access), disclosing IP addresses does not necessarily lead to an individualized person, and Maurer (2024, 260) notes that there are relatively simple ways to hide IP addresses. In addition, the information procedure shall be extended to infringements of all absolute rights (BMJ 2023a, 3). In this context, concerns have been raised that this extended procedure could be misused to target anonymous speech that, while unwelcome, does not violate any rights (GFF 2023, 1, 3, 5; Lück 2023, 742–3; Maurer 2024, 261).

The second proposal introduces a new instrument allowing courts, upon request, to order social networks to suspend accounts used to disseminate hateful content (BMJ 2023a, 4–5; Lück 2023, 742 et seq.; Maurer 2024, 258; Panahi 2023, 559–60; Schäfer 2023, 738–9; Weck 2023, 12–3). According to the concept paper, this measure is subject to the following conditions: First, content moderation must have proven insufficient, and there must be a risk of recurring serious violations of personality rights. Second, the account holder must be notified of the blocking request and given an opportunity to be heard. Third, the suspension must be proportionate and limited to a reasonable duration. Given the severity of this measure, it has been appropriately noted that the legal threshold should be set relatively high (Schäfer 2023, 739). However, the proposal lacks clarity on whether only a specific account can be blocked, or if other accounts held by the same individual (potentially even on related platforms) can also be suspended to reduce the risk of circumvention. Furthermore, it is unclear under which circumstances a suspension would be possible for a first offence (Lück 2023, 744; Schäfer 2023, 739).

The third proposal requires social networks to appoint a person in Germany authorized to receive service of process (BMJ 2023a, 6; Lück 2023, 746; Maurer 2024, 258; Panahi 2023, 560; Schäfer 2023, 739–40; Weck 2023, 12–3). While this obligation already existed under previous legislation,^[10] the proposal now extends it to the service of extrajudicial documents, such as requests to remove illegal content. The Ministry considers this to be an essential prerequisite for informing service providers of infringing statements and for properly documenting this information in the event of legal disputes with foreign-based providers.^[11]

2.4 Legal Obstacles to the Initiative

2.5 Fundamental Rights Obligations

The Ministry’s proposal also necessitates a balance among several fundamental rights, including the general right of personality, freedom of speech, informational self-determination, and the privacy of telecommunications. While some of these issues have been outlined in the above-mentioned works (Cole and Ukrow 2023, 41–2; Lück 2023, 743–4; Panahi 2023, 558, 560; Weck 2023, 13–4), little attention^[18] has been paid to the specific situation of minority groups, particularly regarding potential fundamental rights requirements for strengthening enforcement measures against hate speech. The foundation for such considerations lies in the well-researched (e.g. FRA 2023, 22; Frosio and Geiger 2023, 31–77; Grote and Wenzel 2022, 124; Korpisaari 2022, 352–377; Topidi 2019) case law of the European Court of Human Rights (ECtHR) on hate speech and freedom of expression.^[19] In this context, it is firmly established that the most serious forms of hate speech fall outside the protection of Article 10 by virtue of Article 17 of the European Convention on Human Rights (ECHR).^[20] Expressions that do not meet this threshold may be restricted in accordance with the general principles outlined in Article 10 para. 2 of the ECHR (cf. Grote and Wenzel 2022, 124; FRA 2023, 22). In addition, the Strasbourg Court has substantiated its case law in two relevant aspects: first, regarding the liability of private entities, i.e. platform administrators and account holders, and second, concerning the possible obligations of the state to take action against hate speech.

With its landmark decision in Delfi v Estonia (appl. no. 64569/09), the Grand Chamber of the ECtHR provided notable clarifications on the balance between freedom of expression and the protection against hate speech in the digital space (including the liability of platform administrators), which were more recently refined in a different factual context in Sanchez v France (appl. no. 45581/15; cf. Kupsch 2021; Korpisaari 2022, 352–77; FRA 2023, 22–3). In the latter case, after recalling the general principles governing the question whether a particular interference is “necessary in a democratic society” (Article 10 para. 2 of the ECHR), the ECtHR stated:

Since tolerance and respect for the equal dignity of all human beings constitute the foundations of a democratic, pluralistic society, it follows that, in principle, it may be considered necessary in certain democratic societies to penalise or even prevent all forms of expression that propagate, encourage, promote or justify hatred based on intolerance (including religious intolerance), provided that any ‘formalities’, ‘conditions’, ‘restrictions’ or ‘penalties’ imposed are proportionate to the legitimate aim pursued (appl. no. 45581/15, para. 149).

The Court then reiterates its principles on hate speech, as set out in Perinçek v Switzerland (appl. no. 27510/08, 204–8), and emphasizes:

Bearing in mind the need to protect the values underlying the Convention, and considering that the rights under Articles 10 and 8 of the Convention deserve equal respect, a balance must be struck that retains the essence of both rights. While the Court acknowledges that important benefits can be derived from the Internet in the exercise of freedom of expression, it has also found that the possibility of imposing liability for defamatory or other types of unlawful speech must, in principle, be retained, constituting an effective remedy for violations of personality rights (appl. no. 45581/15, 162 [emphasis added]; see also appl. no. 64569/09, 110).

In Sanchez v France, the court even went so far as to extend the private liability to the (non-)removal of hateful third-party comments on the politician’s Facebook comment page, which he administered (cf. Kupsch 2021; Korpisaari 2022, 369). The idea, that states may impose liability on platform administrators that fail to remove hateful contents was further reinforced in the case of Zöchling v Austria (appl. no. 4222/18; critically Tuchtfeld 2023). While it is important to bear in mind the differences between platform administrators, as addressed in the above-mentioned cases, and hosting service providers, who are covered by the DSA’s liability privilege, the ECtHR provides some clarity on the criteria for identifying unlawful contents, on which the DSA is silent (see Frosio and Geiger 2023, 65; Korpisaari 2022, 372 et seq.).

However, in addition to issues of liability and content moderation – both of which are governed by different legal standards depending on the actors involved – ECHR case law could support calls to strengthen national legislation on the enforcement of individual rights in cases of hate speech. A particularly relevant case in this context is Beizaras and Levickas v Lithuania (appl. no. 41288/15), which dealt with hateful and homophobic comments on a social networking platform (cf. Kundrak 2020, 19–39; Peters and Altwicker 2022, 104). After emphasising that “pluralism and democracy are built on genuine recognition of, and respect for, diversity” (para. 107), the ECtHR recalled

the States’ positive obligation to secure the effective enjoyment of the rights and freedoms under the Convention. This obligation is of particular importance for persons holding unpopular views or belonging to minorities, because they are more vulnerable to victimisation […].

Positive obligations on the State are inherent in the right to effective respect for private life under Article 8; these obligations may involve the adoption of measures even in the sphere of the relations of individuals between themselves. While the choice of the means to secure compliance with Article 8 in the sphere of protection against acts of individuals is in principle within the State’s margin of appreciation, effective deterrence against grave acts where essential aspects of private life are at stake requires efficient criminal-law provisions […].

The Court has acknowledged that criminal sanctions, including against the individuals responsible for the most serious expressions of hatred, inciting others to violence, could be invoked only as an ultima ratio measure […] (appl. no. 41288/15, 108, 110, 111).^[21]

The court went on recalling its case law on the positive dimension of Article 14 of the ECHR, the notion of discrimination and the required burden of proof. It concluded that the relevant authorities failed “to discharge their positive obligation to investigate in an effective manner whether the comments regarding the applicants’ sexual orientation constituted incitement to hatred and violence” (appl. no. 41288/15, 129). Although it has been argued that it would be “far-fetched” to conclude that the ECtHR has generally created a “new positive obligation to investigate and prosecute hate speech“ (Kundrak 2020, 35), the case demonstrates that states that fail to take action against clearly hateful content after being informed of it are in breach of the Convention.

This case law could encourage member states to explore new avenues for the effective protection of personality rights online, especially where the current regime of content moderation and criminal sanctions (with their limited deterrent effects), proves insufficient (see the data at Sehl, Pfleger, and Suliak 2023). Specifically, in cases of manifestly unlawful hate speech against particularly vulnerable persons, such as members of minorities, fundamental rights considerations may therefore argue for the introduction of a legal basis for court orders aimed at terminating or preventing an infringement, as envisaged by the German government. Furthermore, it would make a notable contribution to Germany’s compliance with its obligations under international law, such as those arising from Article 6 of the Framework Convention for the Protection of National Minorities.^[22] For the reasons outlined above, it is important to ensure that such legislation is limited to this purpose and does not override the liability regime applicable to service providers as such, as provided for in the DSA.

Against this background, the proposal’s victim-centered approach to strengthening individual rights enforcement adds a crucial layer to the complex interplay among gatekeepers, regulatory authorities, users and courts. By equipping those affected with the necessary legal instruments, this proposal empowers individual users to assert their rights through state courts, particularly in cases where previous content moderation and mitigation strategies have proven inadequate. Although this represents only a small fraction of the broader legislative efforts to foster a socially acceptable (i.e. non-harmful) online environment, it encourages the actorness of the corresponding stakeholders. Enhancing the role of users – especially those from minority groups frequently exposed to harmful content – by improving their access to effective legal remedies therefore offers a promising complement to existing approaches.

3.1 Comparative Perspectives

Most national and supra-national legislative initiatives in the direction of addressing digital forms of violence include within their regulatory scope content that sexually victimizes children and intimate content circulated without consent. However, they are equally concerned with hate speech, incitement to violence, promotion of genocide or terrorism which are of direct linkage to ethno-cultural minority groups and their members.

The German government’s concept paper on digital violence (BMJ 2023a) comes at a time when digital patterns of anti-minority discrimination and hate speech highlight stigmatisation, assaults on individuals’ worth, and dehumanisation. Racialisation processes online additionally create an impact on their victims, regardless of whether they constitute instances of everyday racism or more significant attacks. They also tend to conflate within racialised hostility ethnic, racial, and religious backgrounds. As worryingly, minority group members can understand online hate speech and racism as inevitable and as a direct consequence of one’s online visibility (Bivens 2017, 880–898; Christin and Lu 2023, 1–24; Duffy, Poell, and Nieborg 2019, 1–8; Topidi and Metcalfe 2024a). Within this framework, intentionality to cause harm is relevant and can be understood as the extent to which respondents experience online hate as reflecting discriminatory intentions either on an individual or a group basis.

Within a complex frame where the online-offline dimensions of discrimination and harm become blurred, the proposed legislation contributes to the discussion about the role of and expectations from technology when regulating harm and violence produced online. At the same time, the legislative project in question confirms the observation that technology has shifted from being a distinct sector of regulation to becoming an additional policy layer for all publicly regulated processes and goods.

From the perspective of ethno-cultural minority groups, the online space has become, as mentioned, a source of discrimination (see the important findings of Haimson et al. 2021; Haugen 2023; Noble 2018; Topidi and Metcalfe 2024b). The question that the proposed legislation reiterates is what could be the adequate means to support the exercise of rights of vulnerable individuals and groups that are experiencing harm and oppression online. More than that, the same set of proposed rules more broadly inscribe in the efforts to strike the right balance between innovation and rights protection. It is also noticeable, that the projected legislation comes after the enactment of the EU DSA, suggesting the willingness of the German legislator to complement EU law with national legislation with all the legal caveats already discussed in previous sections.

Overall, the approach pursued by the previous attempts at regulation, the NetzDG as well as the more recent DSA at EU level, rely on a strict regulatory frame based on the ‘notice and action’ model. The model obliges big platforms to block or delete harmful content within prescribed periods. They are also required to have in place accessible and user-friendly individual complaint mechanisms for illegal online content. Lastly, they are under an obligation to report criminal content to law enforcement authorities. This is not, however, the only regulatory approach to the question adopted by states at the moment: the UK and Canadian examples have adopted a more ‘system-based’ approach that is characterized by the duty of platforms to act responsibly and the propensity to look at the issue from a more systemic and ex ante decision-making perspective (Rinceanu and Stephenson 2024). Such duty implies detailed risk assessments, mitigation strategies as well as regular evaluation of their efficiency (Rinceanu and Stephenson 2024). For example, the Canadian Bill C-63 seeking to enact the Online Harms Act, tabled in February 2024,^[23] considers the establishment of a distinct regulatory body, named the Digital Safety Commission, whose task is to consider complaints and conduct independent investigations of cases in ways that facilitate regulatory oversight of platforms and Big Tech while aiming to distribute regulatory power more evenly. Compared to other similar online safety acts, the Canadian example adopts a vague definition of regulated services, leaving considerable room to regulation implementation.^[24] It additionally emphasizes the need for platforms to report on their moderation systems and practices in a way that allows a better understanding of the interrelationships at play and preempt regulatory challenges of the future. It has drawn inspiration from Australia’s Online Safety Act (OSA) 2021.

The earlier Australian Act against online harm has been designed around the expectation that online service providers should become proactive in protecting users against harmful online content.^[25] The Act, similarly, had taken a risks- and harms-based approach that is coordinated through an eSafety Commissioner.^[26] The latter’s tasks include prevention, protection and identification of emerging trends in online harm. The Commissioner in question also enjoys discretionary powers including those of making regulations. The Australian Act additionally introduces the power to require internet service providers to block access to violent content, the duty of online platforms and service providers to detect and remove illegal or restricted content as well as duty-based measures against tech companies requiring them to take reasonable steps to maintain a safe and free from violence online space. The Act is currently subjected to independent review to determine its effectiveness by virtue of Section 239A of the OSA, with findings expected in late October 2024 (Global Compliance News 2024). The review was brought forward by one year, among other reasons, due to the need to enhance the protection against hate speech in conjunction with other legislative initiatives. Indicatively, the Australian government is considering enhancing hate speech legislation through a planned Religious Discrimination Bill. The regulatory intersection between the OSA and religious discrimination at federal level suggests the limitation of the current national regime against online harm and violence. It could also be construed as a warning mechanism calling for comprehensive and coordinated regulatory regimes in this area.

In sum, the common characteristic feature observed in both the Canadian and Australian regulatory attempts places a designated regulatory body in a position that enjoys broad authorities with a lack of oversight, relying on a widely framed duty of care instilled on tech companies and platforms (Anjum 2024). At the moment, the broad common direction of regulatory efforts within national contexts suggests the shifting of the basis of access to information in cases of digital violence from platforms’ internal terms of service to the law, which has been a constant claim of vulnerable groups in the past few years. Such a shift has repercussions for the regulatory regime of the online space as it presents platforms with increased levels of accountability of the content they moderate.

Still, regardless of the approach chosen to online harm mitigation, the key regulatory issues concern the definition of online violence and harmful content, the economic incentives for platforms to moderate content more efficiently, the risks of platform overreach in terms of potential violations and the chilling effect on the exercise of individual rights and the transparency and accountability concerns as matters of good governance. The extent to which any approach considers the regulation of online spaces an issue of ‘mass speech administration’ as opposed to an individualized system of error correction is not irrelevant either (Hasinoff and Schneider 2022; Peterson-Salahuddin 2024, 1–13). This is because the implications of online harm and violence defy individual cases of offenders and victims and point to a more complex web of systemic interrelationships that largely replicate discriminating patterns of the offline world. Broadly speaking, the definition of harmful content, including in Canadian Bill C-63 or the German proposal, focuses on harms that users experience when viewing content posted by others. It neglects how platforms contribute to the virality and proliferation of harmful content through algorithmic and other moderation tools (Hrynyshyn 2024). The systemic dimensions of stigmatizing and discriminating processes online echo thus discrimination and disadvantage targeting minority groups in the physical world.

In comparative terms, among the most crucial hurdles in the regulation of online violence one can additionally identify the failure to provide legal grounds obliging online platforms to cooperate in the identification of perpetrators (HateAid 2021) together with the absence of the definition of what constitutes digital violence obstructing the maintenance of user safety online. So far, platforms arbitrarily have been withholding information from law enforcement authorities, even following the issuing of court orders. On an exceptional basis, their cooperation is forthcoming in cases where negative publicity is at stake (Topidi and Metcalfe 2024a). In the absence of the knowledge on how digital technologies are affecting the social fabric of societies, which is of direct correlation and impact to minority groups, digital violence continues to spread through polarization, alienation and mental as well as physical forms of violence. Outstanding issues that are largely left unaddressed by the initiatives in question concern victim-centered, intersectional regulatory solutions to online violence requiring frameworks that are built around consent, confidentiality, intersectionality and accountability.

3.2 Responses to the Public Consultation Process: The Minority Group Angle in the German Context

A survey of the reactions to the public consultation process launched in 2023 and sourced from a variety of civil society actors that are engaged in ethno-cultural minority group policy making and representation reveals a number of (recurring) structural policy concerns with the current proposal in similar terms to the ones discussed in the previous section. These points concern conceptual, procedural as well as governance related points that the proposed legislation touches upon, particularly from the perspective of ethno-cultural minority groups in Germany.