Improving Access to Scientific Literature with Knowledge Graphs

1 Introduction

Scientific libraries must adapt to the changing requirements of science. The digitization of scientific working methods, processes and forms of publication is a central challenge. The methods of scholarly communication have been more or less static text articles for centuries. Although these can now be reproduced electronically as PDF or HTML and quickly accessed via the Internet, the basic representation as unstructured, static articles has not changed fundamentally. On the other hand, other information domains have changed fundamentally and developed completely new digital forms of representation. The only remaining encyclopedia, for example, is Wikipedia, which is not simply a digital PDF copy of an analog encyclopedia, but has realized completely new forms of processing, representation and organization for encyclopedic content, thus enabling, for example, the realization of encyclopedia versions for hundreds of languages and a wide variety of target groups in a completely new depth and breadth. Further examples of entirely digital information services include

1. digital map applications (such as Google Maps^[1] or OpenStreetMaps^[2]), which have now almost completely replaced physical street maps

2. Online stores and e-commerce applications with completely new search, evaluation and data networking functions instead of the classic mail order catalogs,

3. digital communication applications, which have made telephone books obsolete, for example.

All these examples illustrate that analog forms of representation (books and documents) have not simply been “pseudo-digitized” as PDF, but have been realized completely new as digital-born applications. Such “digital-first” applications are based on a fundamentally new, structured and data-oriented information organization and thus enable completely new support through intelligent search and filter functions, the integration of diverse additional information and services, crowdsourcing etc.

The currently still only pseudo-digitized scientific exchange of information is roughly comparable to a situation in which we would have to pick out products from PDF catalogs sent by an e-mail or find our way to our vacations on a PDF road map, which causes great problems for scientific work:

1. We are confronted with a constantly growing number of scientific publications, which of course can be produced faster with digital tools. In the field of technology and natural sciences, for example, the number of publications per year has almost doubled within a decade.^[3]

2. Due to the dramatic growth in the number of publications, the quality of peer reviews is often insufficient: On the one hand, there are too few qualified reviewers with sufficient time, and on the other hand, it is increasingly difficult to determine the review, i.e. the contribution of the research

3. The majority of scientific publications cannot be reproduced by researchers (and often even by the authors themselves).^[4] A main reason for this is the unstructured presentation in static PDF articles, where important information may be missing.

4. Different research approaches can hardly be compared due to the unstructured presentation, which makes it extremely difficult to determine the state of research, especially for younger or interdisciplinary researchers.

5. The unstructured presentation of research results does not allow for any or only very insufficient machine support. Research contributions cannot be effectively searched, filtered or visualized. Assistance systems such as those already available for everyday situations with Google Now or Alexa are currently unthinkable for coping with the flood of scientific information.

Of course, there are a number of initiatives that have set themselves the goal of addressing these problems. However, it seems that often the symptoms are being worked on rather than addressing the fundamental problems. In part, such solutions seem to be also based on false assumptions.

One misconception, for example, is that text and data mining can solve the problem of indexing and exploring scientific articles. Fully automated text mining and natural language processing methods alone will not provide sufficient accuracy for the extracted information for real use. Such methods often achieve only medium precision and recall for the recognition of entities (named entity recognition). The actual performance highly depends on the domain-while standardized terms (e.g. genes, datasets, or countries) can be discovered more reliably, the recognition of other more vaguely defined entities (e.g. materials, processes) is very error-prone.^[5] For relation extraction, which is essential for improved machine support, the correct results are often hardly one third of extracted relations, which are not sufficient to realize reasonable applications in most cases.

Another misconception is that the scientific information can only be organized with fully automated procedures and possibly machine learning. Machine learning methods only work where sufficient training data is available. This is not the case with the structured extraction of scientific results from unstructured articles and will not be possible for the foreseeable future. With crowd-sourcing, or rather expert sourcing, we could, however, master the structured organization of scientific information-possibly supported by machine learning methods. Initially, neither a complete processing of the scientific literature nor the involvement of a large percentage of scientists is necessary. It would initially be sufficient if only a relatively small percentage of scientists were involved in the curation and organization and if, under certain circumstances, only a few research problems and special fields were covered. This would be sufficient for potential applications in these areas and could lead to the establishment of a network effect later on, thus covering broader scientific fields. A good example in this regard is OpenStreetMaps, a crowdsourcing application with which a few thousand collaborators have created an open world map that is much more detailed than commercial offerings in many areas. Thanks to its innovative data organization, it can be used for a wide range of applications from disaster control and navigation to mobility for the disabled and bicycle maps. Applied to scholarly communication, we therefore need a new form of representation of scientific knowledge that is highly semantically structured and allows for large-scale collaboration between specialist scientists, knowledge engineers, information scientists, librarians and users, while at the same time enabling an evolutionary transition from previous scientific publication and incentive systems. The Open Research Knowledge Graph developed by the TIB and partners, which we present in this article, aims precisely to incorporate these requirements.

2 Overview of the Open Research Knowledge Graph

The following we give a brief overview of the most important features of the Open Research Knowledge Graph (ORKG).

Structured description of the research contributions. The ORKG allows describing the research contributions which is traditionally described in scientific articles in a structured and semantic way. For this purpose, articles are added to the ORKG by retrieving (or manually adding) key metadata of the article via DOI from CrossRef and then describing the content of the research articles using specialized input fields. Such structured content descriptions of scientific contributions should describe the addressed research problem, the materials and methods used, and the results achieved in such a way that the contribution becomes comparable with other articles addressing the same research problem. The semantic description follows the RDF subject-predicate-object paradigm and can be flexibly extended by users to include their own additional predicates (properties or attributes) at any time. A suggestion function makes it easy to find and reuse existing predicates and entities. Figure 1 illustrates the structured input of a research article on the effect of ozone on plant growth.

Figure 1

Structured input of a research contribution in ORKG

Templates. The structured description of research contributions is often not an easy task, because the description of scientific findings is complex and based on expert knowledge. On the one hand, it must be decided in which granularity a research contribution should be described. On the other hand, research contributions dealing with the same problem should be comparable. For this reason, the ORKG supports the possibility to create templates that specify the structure of scientific information. Templates can then be reused in the description of research contributions to facilitate data entry and ensure comparability.

SOTA Comparisons. The ORKG enables the automated comparison of research contributions that deal with a specific problem. Comparisons support users in obtaining a state-of-the-art overview. A classic example in computer science is a comparison of the best/worst-case performance of sorting algorithms or the precision and recall of algorithms for vehicle recognition in images. For researchers in virology and epidemiology it is interesting to be able to compare the reproduction numbers of different viruses. Such comparisons provide an overview of key information on a research problem over dozens or hundreds of papers and are thus a valuable tool for obtaining an overview of the state of the art in a field.

Graph Visualization. Since the ORKG is a knowledge graph, research contributions can also be visualized as graphs. The graph visualization is a sophisticated user interface for visual exploration of a scientific contribution and is therefore a way to interact with ORKG content. The graph is automatically arranged optimally on the screen. Nodes can be easily expanded, collapsed or removed. Users can also search for information directly in the graph.

Figure 2

Dynamic graph visualization in ORKG

Observatories. The ORKG relies heavily on expert content curation and knowledge organization. In order to pool disciplinary expertise, we developed the ORKG Observatories. Observatories bring together experts from different institutions that curate and organize ORKG content for a specific discipline or research problem. Observatories and their experts can contribute in many ways. In addition to adding and describing contributions or curating existing contributions, observatories play a crucial role in the organization of knowledge in a research area. Observatories can, for example, create templates relevant to a specific field. In this way, observatories help to ensure the creation of high-quality and comparable structured scientific knowledge for their field. Since knowledge curation and organization is resource-intensive, ORKG acknowledges the contributions of experts and corresponding observatories and institutions are made prominently visible in the ORKG, the provenance information in Figure 3 shows how the research contributions of an observatory in the ORKG has been acknowledged.

Figure 3

Provenance information of a research contribution in ORKG (box on the right)

Abstract Annotator. An abstract annotator tool has been developed to automatically extract key information from the abstracts of scholarly articles. Different natural language processing and machine learning techniques have been employed to annotate the abstract in an efficient way. While adding the paper, users can use the abstract annotator which could help them to extract the important information such as research problems, methods and materials used to solve the problem. Once these annotations are extracted these can be added in ORKG for a particular paper. Moreover, we plan to automate the content curation as much as possible by implementing the text summarization techniques. Text summarization would summarize the content such as methodology while ensuring to preserve the key information elements.

Question Answering. Answering scientific questions with text is an important part of any research lifecycle. The acquisition of knowledge and appropriate answers is hardly possible due to the following main reasons: machine inactionable, ambiguous and unstructured content in publications. We have also developed a question answering system in ORKG which maps the natural language queries to the graph and finds the answer for that query.^[6] Furthermore, analyzing and searching data from tables is a difficult aspect so a question answering system for ORKG comparisons has also been introduced. Users or authors can ask questions from comparisons to find the relevant information.

Knowledge integration. The structured and semantic description of the knowledge enables a simplified knowledge re-use. The comparisons described above are only one type of knowledge re-use. In fact, in science, knowledge in literature is re-used in countless ways. For example, to support this diversity, ORKG implements a web-based interface (REST API), which can be used with the Python programming language. This allows loading ORKG content (individual article descriptions and comparisons) into a data analysis environment such as Jupyter notebooks, to process it and to create domain-specific applications and visualizations. This allows to easily create data visualizations, but also to implement complex data-enabled activities, which integrate ORKG-data with other data integrators, data interpreters, models, etc.

2.1 Extraction from Survey and Review Articles

A method to populate ORKG is to leverage already existing scholarly knowledge from survey articles. Survey articles, also called review articles, present an overview of the state-of-the-art for a specific research area. The overviews are generally manually curated and of high quality. Additionally, surveys indicate what the current trends are and are therefore providing relevant information for researchers.^[7] Within survey articles, the overviews are often presented in (semi-)structured tabular format. Compared to generating a knowledge graph from natural text, building a graph from a table is more straightforward because of the already existing structure.

To import survey articles within ORKG, we employ a human-in-the-loop approach. The full approach and evaluation are described in Oelen et al. (2020b). While most of the steps are conducted automatically, the human curator is responsible for fixing potential extraction errors and adding additional metadata. All steps required to import a table are integrated in a single User Interface (UI). We now discuss the individual steps required to import a survey article and indicate where human labor is required. Firstly, the curator uploads a PDF version of the survey article. Afterwards, the region of the table is selected (as depicted in Figure 4). Only tables that list related work (i.e., tables with references) should be selected. The selected table is extracted using Tabula, a state-of-the-art PDF table extraction tool.^[8] The extracted table is presented in a spreadsheet editor, as displayed in Figure 5. The curator is responsible for fixing extraction errors and for formatting the table in such a way that it is suitable for import. This means that each row should represent a single paper, in some cases a table has to be normalized to accomplish this. Once the table is formatted by the curator, the references are extracted using GROBID. GROBID is a state-of-the-art bibliographic data extraction tool that supports parsing references in PDF articles.^[9] In this step, the curator is responsible for validating the extracted references, and if needed, adding missing references. Multiple columns are appended to the table to include the parsed reference data (among others, the title, authors and publication date).

Figure 4

Select the table region within the PDF article to perform the table extraction

Figure 5

Spreadsheet editor to fix extraction errors, perform table formatting and to extract references

The resulting knowledge graph for a single paper is depicted in Figure 6. The figure depicts an illustrative example table with three rows. For each row, a separate paper subgraph is generated. Based on the first column of the table, the metadata of the cited paper is extracted (the resulting data is colored blue in the figure). The remaining tabular data is displayed in orange colored nodes. The predicates of the orange nodes are defined by the table header. Finally, the white nodes are ORKG specific concepts. This includes the Research Contribution and the Research Problem.

Figure 6

The resulting graph for a single paper comprising extracted metadata (blue), the tabular data (orange) and ORKG specific concepts (white)

The method of extraction survey tables provides a means of relatively quickly populating a knowledge graph without the need of domain experts. To evaluate our approach, we used the previously described method to import 160 survey tables.^[10] This resulted in a total amount of 2,626 papers that were ingested in ORKG. In the same work, we estimated that approximately 20 % of recently published survey articles use tables to represent reviewed articles. Importing a table in ORKG is not the final step; on the contrary, it provides a first step to create more comprehensive and dynamic overviews of the literature. One of the weaknesses of the current review method is that published reviews are static.^[11] When survey articles are published, they are rarely updated and do therefore not always accurately represent the current state-of-the-art. Once a survey is imported in ORKG, it enables researchers to add their results to the literature comparison (by means of adding an additional row). Additionally, a survey could be extended by including more comparison criteria (by adding extra columns). This highlights another benefit of this approach, namely the ability to collaboratively work on literature overviews. Not only is the point of view of a few authors represented in a comparison, but of the whole community.

3 Knowledge Graph Use Cases

In this section we will show how knowledge graphs and in particular the ORKG can be used to structure information for three different areas. For each domain an ORKG comparison is generated to give an overview of the state of the art for this particular example.

3.1 Computer Science

In computer science, a number of recurring characteristics of scientific contributions are of interest to the ORKG. These properties include evaluation results (F-measure, precision, recall), data sets, benchmarks, model properties and implementation approaches. These properties are applicable to a considerable number of computer science articles and are of interest to many researchers. In this example we show how these recurring properties are used in the field of Question Answering (QA). The goal of the QA task is to automatically provide answers to questions in natural language. To organize QA-related information in ORKG, methodological approaches and evaluation results of different papers and the implemented approaches are represented and compared. An interesting aspect of these problems is that the compared QA systems and the evaluation results of these systems are not originally published in the same articles or by the same authors. However, due to the dynamic aspects of the ORKG it is possible to establish links between the systems and evaluations afterwards. Figure 7 shows a comparison of different QA systems and their evaluation results in terms of precision, recall and F-measure in the QALD benchmark. Each QA system is presented in a separate article published in different journals but can be compared directly with other approaches. It is therefore possible to get a quick overview of the state of the art and the performance of the systems. Furthermore, the comparisons of these evaluation results provide information about the progress for a specific research area.

Figure 7

Comparison of question-answer systems and their evaluation results in the lower part

3.2 Epidemiology and Covid-19

The Covid-19 pandemic also leads to an abundance of daily new articles in research. To support Covid-19 research, many publishers have decided to publish open access articles related to Covid-19. While access is crucial, organizing the information published in articles is essential for effective research. Unfortunately, however, this is extremely time-consuming and thus a major obstacle. An example of a fundamental research finding that is spread over numerous published (preprint) articles is the Covid-19 baseline reproduction number R0, the respective 95 % confidence interval, the location of the population under study and the observation period. In contrast to the conventional document-based publication of such information in natural language texts, tables or figures, ORKG enables to publish such information in a structured, semantic form. Information can thus be evaluated automatically and across publications. Machine readable scientific knowledge opens a number of interesting possibilities. As already mentioned, the structured semantic representation of scientific knowledge in the knowledge graph makes it possible to automatically create literature comparisons. Figure 8 illustrates the comparison for our use case of the Covid-19 basic reproduction number. In addition, ORKG comparisons can develop further in contrast to static PDF overview articles (surveys or reviews). If new literature on the basic reproduction number is published, it is easy to continuously expand such a comparison, which thus continues to reflect the current state of knowledge in a comparable way.

Figure 8

Automatic comparison of basic reproduction figures published in the literature

However, the true strength of knowledge graphs becomes apparent when the graph is used for further comprehensive data analyses. We demonstrate this by connecting Jupyter notebooks to ORKG to take advantage of the flexibility of data science environments and programming languages such as Python to visualize Covid-19 comparison data. Figure 9 shows a possible visualization of the R0 values and the 95 % confidence interval over time. Of course, data from other sources can be used for such analyses, for example statistical data on the death rate in this case.

Figure 9

Visualization of the R0 values and the 95 % confidence interval over time as data analysis on the ORKG knowledge graph

3.3 Materials Science

Finally, we present an example from electrochemistry, materials and engineering sciences. Silicon is an important element of modern technology. It is widely used in the production of metal alloys, optical fibers, solar elements, advanced ceramics, batteries, microchips and numerous other beneficial applications. For solar and electronic devices, there is a demand for solar-grade silicon (SoG-Si) with a purity of 99.9999 % or electronic silicon with an even higher purity. Silicon electrochemistry in molten salts has recently attracted much attention due to its potential to produce Solar Grade Silicon with a negligible carbon footprint. The comparison shown in Figure 10 provides a comprehensive overview of several parameters such as silicon dioxide precursor, electrolyte, contact electrode or temperature of the experimental conditions of silicon electrochemical reduction in molten electrolytes. In this way, researchers can easily analyze relevant parameters used in the process specifications for the generation of silicon surface structures.

Figure 10

ORKG comparison for work on silicon electrochemical reduction in molten electrolytes in materials science

4 Integration of library communities and services

Libraries can play a central role in the further development and curation of scholarly knowledge graphs. Not only do scientists need to be supported by AI and text mining procedures, but the information science core competencies in knowledge organization and formal indexing are also indispensable for the success of scholarly knowledge graphs. The Open Research Knowledge Graph is therefore intended to become a comprehensive and broad knowledge infrastructure for the collaboration of specialists, information scientists, librarians and users. In order to involve as many stakeholders and multipliers as possible, we are currently working on various strategies:

1. ORKG is a radically open research infrastructure according to the principles of open science, open data and open source: all software, information and data are available under open licenses. The ORKG provides a comprehensive programming interface (API), which can easily be used to implement specific applications based on the ORKG infrastructure.

2. In subject-specific observatories, specialist scientists organize research work on relevant research questions in a clearly defined field together with expert speakers. We want to make their commitment visible and recognized by prominently featuring the persons and organizations involved in the respective entries in the ORKG.

3. We plan to publish comprehensive and well-documented comparisons of the state of research on a specific research question as independent publications in an Open Access journal. Already now, such comparisons can be provided with a DOI in the ORKG and can be published citable.

4. (Semi-) automatic procedures are based on good authoritative reference data, this part of the intellectual development remains the task of library departments. Various ontologies and standards data can then help to structure knowledge in graphs.

We would be very pleased if we could win further collaborators in the library community who, together with specialist scientists, maintain observatories, link existing infrastructures with the ORKG or implement new subject-specific applications (e.g. elements for specialist information services) based on the ORKG infrastructure.

5 Related Scholarly Knowledge Graph Projects

In addition to the ORKG, there are a number of other scholarly knowledge graph projects, such as the Microsoft Academic Graph,^[12] the Springer Nature SciGraph,^[13] Papers with Code^[14] and ResearchGraph.^[15] However, these initiatives primarily focus on integrating and organizing bibliographic metadata. Another related initiative targeting the deep analysis of publications is SemanticScholar.^[16] Semantic Scholar functions mainly like a search engine but focuses primarily on highly automatized artificial intelligence approaches to augment the key-word search and thus lack highly structured and semantically rich descriptions of the research contributions.

Figure 11

Comparison of different scholarly knowledge graphs in the Open Research Knowledge Graph^[17]

6 Conclusion^[17]

We need to go beyond organizing scholarly communication just on the surface by managing and integrating bibliographic metadata. Going further with knowledge graphs, scientific findings can be represented semantically in a human and machine readable way according to the FAIR principles. Concepts that are still deeply hidden in unstructured publications are given a unique identification with persistent identifiers and can be semantically linked to other relevant concepts or artifacts. The semantically structured ORKG content can be easily accessed in many ways via the web, API, data dump or query interfaces and the open licenses.

Despite the potential and benefits of leveraging knowledge graphs for scholarly communication, we are still at the very beginning of development and there are still many open questions to be answered: How can we increasingly involve specialist scientists in the curation process? Do the semantic curations techniques scale for broad subject areas and can a continuous evolution of semantic representation be achieved? How can incentive systems for scientists be adapted or transformed in a meaningful way for knowledge graph contributions? Which tasks are performed by libraries?