Natural Language Processing for Social Good: contributing to research and society

2.1 Trustworthiness

One of the key issues in state-of-the-art NLP technology and particularly LLMs remains to be factual correctness. Despite all advances, LLMs still tend to “hallucinate”, i.e. produce texts that are factually incorrect (see [7] for a more comprehensive definition of hallucination). It seems unlikely that we will be able to comprehensively address this issue on the current path of scale, which is purely based on ever larger data sets to train on and models with an ever larger number of parameters. Despite incremental improvements in factual correctness, mistakes will always happen. The fact that LLMs have no explicit notion of certainty (or even factuality) when generating texts makes it difficult to detect the unavoidable errors. This limits the unsupervised application of LLMs in domains in which they could have significant positive impact and remains one of the biggest challenges in NLP research.

Instead of relying solely on scaling, we need to investigate novel approaches like post-generation fact-checking or leveraging explicit representations of knowledge, e.g. in the form of knowledge graphs, in order to not just improve the factual correctness of the output generated by LLMs, but also the verifiability (and hence the trustworthiness) of LLMs and their outputs.

Another important factor to build trust are the processes in which the use of AI technology is embedded. Human oversight, in the form of human-in-the-loop approaches, is one of the most effective measures to build trust in AI systems, as we have shown in the context of higher education in [8].

2.2 Perspectives and disagreement

Most Machine Learning (ML) approaches rely on finding a single “truth” or “gold standard”, e.g. by majority vote and removal of “outliers” during the annotation process. This assumed truth is subsequently used for the training and evaluation of ML models. Only recently, questioning this assumption of a single truth that can be found in an annotation process has gained attention within the NLP community (see e.g. [9], [10], [11], [12]).

Particularly for inherently subjective tasks, like the detection of offensive language, it seems reasonable to assume that such an objective truth does not exist. In other domains, like politics, eradicating divergent positions is not only factually wrong but can become a thread for our pluralistic society as AI models become more deeply embedded in our daily lives. But even in domains that are traditionally seen as more “factual”, like medicine or law, experts regularly disagree on decisions and conclusions. Instead of trying to remove these instances, we should try to leverage the information such a disagreement between expert annotators can provide.

In my research, I investigate how we can include different perspectives, opinions, and standpoints in all steps of the ML pipeline:

Starting with the annotation of data, where we have to separate noise, such as honest mistakes and attention slips during the annotation, from genuine disagreement that can provide valuable insights using, by using annotation processes that allow for such distinctions, as outlined in [9]. Followed by the training of ML models, where we have to develop new approaches that can be trained on a range of (potentially disagreeing) inputs rather than a single truth. E.g. by using probability-based multi-label methods, ensemble approaches, or instruction tuning, as exemplified in [13], [14].

And finally, the presentation of results from such models, which can no longer rely on just presenting the most likely option, but has to be a representation of the different opinions and annotations that went into the model. Presenting such a more nuanced and comprehensive results is also preferred by potential users, as we have shown in [13].

2.3 Evaluation

Evaluating Natural Language Generation (NLG) systems and their outputs is particularly challenging. It is rarely possible to classify such outputs in a binary correct/incorrect schema and the overall quality of a text depends on many factors, like fluency, readability, grammaticality, and (factual) correctness [15]. Conducting evaluations that consider all these factors is time-consuming and expensive. Therefore, researchers often rely on using automated evaluation metrics that are much cheaper to apply. Although these metrics are designed to correlate with human judgments on specific tasks and domains, they are often used in cases where their applicability and validity are questionable [16]. While these problems have existed since the first NLG systems were built, LLMs have added a further challenge to current evaluation practices due to the fact that they are trained on such vast amounts of data that often we cannot be sure anymore that they have not been trained on the test data included in benchmark datasets (and sometimes we can even be reasonably sure that they have been) [17].

Independent of such problems, purely metric-based evaluations can have limited significance with regard to the benefits a system can provide in practical application. Even if a system is very good at performing a certain task, it does not necessarily add significant value in practice, e.g. if the task has limited relevance or if the system is not properly integrated into existing workflows. To evaluate the potential benefits a system can have in a real-world application, task-based evaluations in realistic contexts are indispensable. In [18], for example, we conducted a task-based evaluation with real drivers of a behavior-change support systems that supports driving safety. In [19], we conducted a task-based evaluation of a system that can automatically detect potentially void clauses in consumer contracts with experts from consumer protection NGOs.

From a commercial perspective, the current practices in NLP evaluation often provide little insight into the economic viability of a system. We need frameworks that are able to quantify whether an AI system is beneficial from an economical perspective by taking into account its benefits (savings, service improvements, reduced waiting time, …) but also its potential costs (error frequency, costs of specific error classes, …) in order to allow companies to make informed decisions about the use of AI systems in production.

2.4 Access to justice

I believe that one of the most important contributions research can make to ensure AI is benefiting our society as a whole is to actively seek and work on applications that focus on positive impact, e.g., along the lines of the UN Sustainable Development Goals. One focus of my work is therefore on applications of NLP that support access to justice. Legal matters are often complicated to assess for ordinary citizens and seeking legal advice from trained professionals is expensive. NLP technology has the potential to lower the barrier to access to justice and legal advice. However, most existing so-called “Legal Tech” solutions focus on supporting companies rather than consumers, contributing to an already existing imbalance of power between both groups.

In my research, I work with legal scholars and practitioners, as well as NGOs, like the German “Verbraucherzentralen”, in order to support consumer protection. In the AGB-Check project, for example, which was funded by the German Federal Ministry of Justice and Consumer Protection, we developed a tool that is able to support consumer protection lawyers in the assessment of clauses from Terms and Conditions from online shops by automatically identifying the topic of a clause and clauses that are potentially void [20], [21], [22].

Many of the issues described above, like trustworthiness, but also other aspects that are currently at the center of ongoing NLP research like explainability or multi-modality are particularly important for applications in the legal domain and other domains that are relevant for our society, like medicine or education. Working on such use cases is therefore not only relevant from a societal perspective but also from a scientific one.