UDM (Unified Data Model) for chemical reactions – past, present and future

Introduction

UDM overview

UDM addresses a critical gap in the FAIRification of chemical synthesis data – the lack of their comprehensive and standardized digital representation. UDM provides a hierarchical model and controlled vocabularies to describe chemical reactions on the one hand and a well-defined file format to represent them on the other.

The Unified Data Model covers compounds and their properties, reaction conditions, preparations and outcomes, analytical data, literature references, and legal and licensing information. UDM files are defined by applying a pragmatic approach for data storage and relying on a proven, standard technology—XML. The entire data model, data types, value constraints and units of measure are implemented using an XML Schema [5]. It allows the use of widely available, generic XML tools for validating, querying and transforming UDM documents.

UDM files can use different representations of molecular structures, reaction diagrams and textual descriptions of reaction recipes. To reduce the cost of long-term adoptions, the file format aims to be stable (fewer major releases rather than a more continuous approach) and allow flexibility and openness to store additional data not foreseen by the UDM team or too specific for individual applications. Such scenarios are supported by several extension points built into the UDM. In the most basic case, they allow storing vendor- or process-specific data, which are only required to be well formatted XML document sections. In a more advanced case, derived, backwards-compatible versions of UDM can be created by extending the original UDM XML schema and validating the data stored in the extension points. The original Pistoia Alliance license was explicitly designed to reflect this model. The current MIT license offers even more flexibility.

How did UDM come about?

Reaxys^® [6, 7] is a commercially available chemical information system originally consisting of the Gmelin and Beilstein handbooks which recorded systematically organized chemical data starting from the 19th century. Up to date, Reaxys continuously adds chemical information from peer-reviewed journals, patents and conference abstracts using manual and automatic content processing pipelines. Its organization of chemical information in the Reaxys databases is based on three pillars: citations, substances and reactions.

Reaxys and Roche understood that integrating the reaction data of the publicly available Reaxys information with all inhouse available data sources would improve productivity and save time for researchers as they would be able to launch a single search across multiple data systems and find information across them, including the related information, for example, via citations.

Therefore in 2012, Roche started the development of the Unified Data Model (UDM). Its goal was the integration of the different data formats from their various internal and externally licensed databases together with the reaction data from their ELNs into one common data model (UDM) to let this data be joined with the reaction data provided within Reaxys.

The overall data reaction migration consisted of five major steps: Collect, Unify, Canonicalize, De-duplicate and Extract (Fig. 1). Reactions were collected and unified into the single common UDM format. Using canonicalization, the reactions were de-duplicated so that frequently occurring reactions were represented by the same reaction scheme. The data was processed, and the essential information was extracted and migrated into Reaxys. The Reaxys User Interface (UI) was used for the simultaneous searches in both data sources [7, 8].

Fig. 1:

Reaction migration steps utilizing UDM.

The first UDM version was implemented based on the RD file format. In UDM RDfiles all molecular structures and their data were explicitly added to each reaction section where it belonged to. This lead to redundant and potentially inconsistent representation of identical molecules and citations from time to time. To remove these redundancies the UDM was further developed into an XML format where molecules and citations were de-duplicated, stored in a separate data block within the UDM file and referenced in the related reaction sections by the “XMLise” step. The adaption of these modifications and necessary extension by Reaxys for their upload process eventually constituted the UDM 1.0 version.

In the following few years, Elsevier has further developed the UDM schema with the help of other integration customers. The UDM was extended to cover additional properties that are more linked to molecules that come with a reaction. In October 2017, the latest Reaxys UDM version (UDM 3.6) was provided to the Pistoia Alliance to make it more generic and to let it be extended to other experiment types.

Subsequent developments and releases of UDM within the “Pistoia Alliance UDM” project were driven by the project founders (Elsevier, Roche, BIOVIA, GSK, Novartis, and BMS) and the UDM team members (see Table 1) under the governance by Pistoia Alliance.

The UDM development team expanded and included other stakeholders such as pharma, chemical, and cheminformatics companies, as well as ELN providers as illustrated in Fig. 2.

Fig. 2:

UDM stakeholders.

During the Pistoia Alliance governance, the UDM team met online on a regular basis to discuss requirements, new developments, extensions to the format and new releases. There were also dedicated sessions during European and US Pistoia Alliance meetings, as well as a dedicated workshop in Amsterdam sponsored by Elsevier in 2019 where publisher and ELN provides met to exchange their experiences about reaction handling.

Developing successive versions of the Unified Data Model was guided by the requirements for new features gathered by the project team members without removing existing features already extensively used by some organizations. The result was that some of them were preserved for backward compatibility even though a better, more generic approach was identified and implemented. A good example is the handling of molecular properties. The pre-4.0 UDM version had dedicated XML entities to store values such as cLogP, the number of hydrogen bond donors or acceptors, the PSA (polar surface area) and so on. In UDM version 5, a concept of a generic molecular PROPERTY was introduced (with additional fields describing how it was obtained). It could be used to store the properties mentioned above. However, some stakeholders heavily rely on the original property entities, and therefore they are still present. In future UDM releases, such elements may be marked as deprecated or obsolete.

Fig. 3 illustrates vital milestones in the development of the UDM format under the umbrella of the Pistoia Alliance.

Fig. 3:

UDM development milestones as Pistoia Alliance project.

From the beginning of the Pistoia Alliance UDM project, the team was well aware of the critical importance and challenges of adopting the UDM. The group involved representatives from key informatics suppliers and their customers actively contributing to the specification and many of them declared future support for the format. For example, BIOVIA has implemented the UDM file format released in December 2020 in the Pipeline Pilot 2021 version. Considering the flexible nature of the product and its widespread adoption, it provides the ability to process UDM files to a large number of life-science organisations. To our knowledge, no other commercial ELN provider has adopted the UDM exchange format; however, the Chemotion ELN, a free ELN developed by Karlsruhe Institute of Technology (KIT) for the NFDI4Chem initiative, has incuded the UDM implementation on its development roadmap [9]. The support from other ELN providers for their adoption of the UDM exchange format is ongoing and will be essential for the successful adoption of the format.

The entry barrier to implementing the UDM is lowered by the fact that it is an XML-based file format with a well-defined schema, and one can use existing XML libraries and tools to convert, process, and query UDM files. This approach has been used by several organisations which adopted the UDM to ingest reaction data into Elsevier Reaxys. Furthermore, a sample Python code is included in the UDM distribution (and could be found on the UDM GitHub site), implementing the conversion of SPRESI RD files to UDM. The UDM team has also prepared a proposal for a toolkit facilitating various aspects of processing and curation of reactions in the UDM format and is looking for external funding to deliver it.

Unified Data Model

At its top level the Unified Data Model consists of the following five components:

• 1. The version of the UDM file format

• 2. Legal information about the data set

• 3. List of citations referenced by individual reaction records

• 4. List of molecules involved in reactions in the data sets in various roles: reactants, products, catalysts, solvents or generic reagents. Similar to the citations, they are referenced from different parts of the reaction hierarchy.

• 5. Reaction hierarchy – the core part of the UDM.

A high-level schema of the UDM is presented in Fig. 4.

Fig. 4:

Top-level elements of the Unified Data Model.

As mentioned earlier, the UDM XML file format is described by an XML schema that defines the allowed elements, their types and additional constraints on their values. The XML schema is also an XML document adhering to another XML schema. The XML Schema provides a rich set of pre-defined simple data types that can be associated with individual elements, for example, date, string, integer, positive integer, non-negative integer, negative integer, non-positive integer, decimal etc. UDM uses them to control values of elements like the DOI (Digital Object Identifier):

<xs:element name=“DOI” type=“xs:string”>

Or the YEAR of publication:

<xs:element name=“YEAR” type=“xs:positiveInteger”>

Another application of simple data types are enumerations employed by the UDM to create controlled vocabularies:

• 1. List of countries and their two- and three-letter codes based on ISO 3166 [10] with additional, frequently used names.

• 2. Reaction classes from the RXNO name reaction ontology [11].

• 3. Analytical methods and result types are taken from Allotrope Foundation Taxonomies (AFT) and on based preferred labels for physical characterization assay entities [12, 13]

• 4. Similarly, the vocabulary of units of measure is aligned with Allotrope Foundation Ontologies (AFO) and based on unit abbreviations [12, 13].

There are also complex, user-defined types (equivalent to records or structures in many programming languages) that can specify additional constraints on element values, define element attributes or group together multiple elements. The UDM MF (molecular formula) element illustrates the first case (Fig. 5) when strings are allowed to contain <sub> and <sup> XML tags to support subscripts and superscripts:

Fig. 5:

XML schema element defining molecular formula.

A simple example of using an additional attribute can be found in the VOLUME element, where the unit attribute represents the corresponding unit of measure. The volume units in the example below (Fig. 6) and other units of measurement are based on the Allotrope Foundation Ontologies (AFO) [13] with extensions to support units missing the AFO and special cases like “μL” and its English transliteration “uL”.

Fig. 6:

XML schema element defining volume of substance with the associate unit of measure.

Finally, authorDetails is a relatively straightforward complex data type grouping other types that represents both publication authors and scientists directly performing reactions (as signed in ELNs). It is shown in Fig. 7.

Fig. 7:

Example grouping of XML elements into a sequence.

The XML Schema datatype system may seem to be complex at first sight, but it allows expressing many real-life scenarios, for example:

• 1. Numeric values where an exact value or a range value (minimum and/or maximum) are specified like pressure, time etc.

• 2. An extension of the above when the actual value changes from a minimum to its maximum at a specified increased speed, e.g., temperature of the reaction.

• 3. Equivalents of reactants that can be specified as numeric values (percentages by default) or using a special “excess” keyword.

Example UDM file

The following sections discuss a simplified UDM file describing synthesis of 2-nitrobenzoic acid from Reaxys.

At the top level, the UDM file consists of the five components mentioned above and most of them group the similar elements: citations, molecules or reactions as presented in Fig. 8.

Fig. 8:

High-level structure of a UDM XML file.

UDM version

The UDM_VERSION element specifies the UDM file format (XML schema) version used to encode the reaction data.

Legal information

Many of the widely used chemistry file formats are not designed to contain machine-readable legal information, which is essential for FAIRification and sharing of data. The UDM team recognized that it was important to clarify the allowed uses of a UDM data set and introduced an XML block making it explicit. It follows the Creative Commons ALM (Author, License, Machine-readability) recommendation, and it specifies the name of the dataset (TITLE), who should be attributed for the dataset (PRODUCER and COPYRIGHT) and how the data can be used (LICENSE) as presented in Fig. 9.

Fig. 9:

LEGAL block of a UDM file.

CITATIONS section

The CITATIONS block contains a list of preferable de-duplicated literature or patent references identified by unique IDs. The IDs are used to reference the citations from other parts of the UDM document in a particular molecule or reaction variation elements. The newest version of the UDM (6.0) allows the direct inclusion of complete citations instead of referencing them by ID. This feature was introduced to simplify the transformation of smaller RDfile sets, but it is not recommended as a general practice. A sample CITATIONS section is show in Fig. 10.

Fig. 10:

Example of a CITATIONS block containing a single literature reference.

MOLECULES section

Molecules are represented in two locations within the UDM file: the RXNSTRUCTURE entity containing reaction diagram described in a later section and the MOLECULES block. The latter stores common properties of molecules involved in the reaction in various roles: reactants, products, solvents, catalysts or other types of reagents. Although the list of molecules is expected to be de-duplicated based on their structures to improve the data quality and to remove redundancy, it is not a hard requirement. Similar to citations, molecules have unique identifiers used to reference them from within the reactions. The UDM version 6.0 can also be directly embedded within reactions (again, this is not a recommended practice).

The de-duplication of molecular entities based on their structures eliminates the risk of inconsistent property values to be specified for the same compound. It also simplifies integration with molecule-centric applications, such as compound registry systems. For example, in a typical outsourcing scenario, reaction information provided by CROs is often ingested into two systems: reactions are stored in the customer’s ELN, and individual molecules are centrally registered.

Each molecule is described by its structure, various names and identifiers and an extensible list of properties. Initially, UDM allowed only molfiles to represent molecular structures. Still, later version added support for InChI [14], SMILES [15, 16], CDXML [17] and WLN—Wiswesser Line Notation [18, 19]. The inclusion of the last one was a light-hearted tribute to an important milestone in the history of the representation of chemical structures. Interestingly, PubChem [20, 21] contains over six thousand historical compounds with specified WLN. An even more exciting outcome of this tribute was an open-source WLN parser published by Sayle et al. [22] and contributed to both RDKit and Open Babel.

An example of an extremely simplified MOLECULES block containing two molecules is shown in Fig. 11.

Fig. 11:

Simplified example of the MOLECULES block.

REACTIONS section

The REACTIONS section is at the very heart of the Unified Domain Model, and it contains a list of reactions with all the related data. A reaction is specified by a general single-step chemical transformation as a reaction diagram and a list of involved reactants and products. The reaction diagrams can be stored using one of the following common representations: RXN, RInChI, Reaction SMILES or CDXML. Multi-step reactions are not supported by the current UDM version 6.0, however, an UDM extension to describe multi-step reactions has been developed and is on the roadmap for release in version 6.1.

Each reaction has one or more variations that provide detailed information about how it was performed—its stoichiometric amounts, conditions, used solvents, catalysts and the outcomes (yield, purity and other analytical results). This approach is similar to many legacy reaction databases, which relied on the MDL ISIS/Host technology. An important part of a reaction variation is the CONDITIONS section which serves two purposes:

• 1. It stores an optional preparation section—a textual description of how the synthesis was performed. This part of the document may store the content in any format (plain text, HTML/XML, serialized binary documents) to support various approaches used by ELNs.

• 2. Following that, there can be zero, one, or multiple condition groups that gather conditions applied simultaneously. The purpose of this approach is to allow a precise description of the reaction stages.

Fig. 12 below illustrates the use of the CONDITIONS section.

Fig. 12:

Sample schematic UDM preparation section with a set of UDM condition groups.

The UDM also supports compound samples that are child nodes of compound playing different roles in a reaction like reactants, products, etc. The compound samples are used primarily to associate analytical results.

Fig. 13 contains a simplified example of REACTIONS section.

Fig. 13:

Sample reaction variation for the synthesis of 2-nitrobenzoic acid.

In the above example the reaction is represented as an RInChI string and the REACTANT_ID and PRODUCT_ID fields are references to molecules involved in the reaction. The VARIATION block below contains details about an individual experiment. The same chemical reaction may be performed in various conditions or using different catalysts and they are represented as individual VARIATIONs. As there may be experiment-specific data for individual chemical entities, for example, product yield, they are represented as more comprehensive elements such as REACTANT, PRODUCT and so on. Within these elements compounds from the MOLECULES block are referenced using their identifiers (<MOLECULE MOL_ID="3587155"/>) or alternatively complete compound information can be embedded since version 6.

UDM versus other formats

There are several file formats currently used to represent reaction data: RDfile (BIOVIA), ELN XML (Perkin Elmer), CMLReact, Open Reaction Database Schema and, in the near future, RInChI.

UDM challenges and how to address them

There are a couple of important challenges that any open data format will need to address. One is the adoption, and another is a proper guidance provided to the developing community on best practices and creation of compatible extensions to the original model. This is why the governance is of vital importance for any format that wants to become a standard. There are, however, opportunities with the open data format that allow various groups to experiment and to try how it fits with the original format/model. For example, there is a publication by M. Kappler and C. Lowden on BioChemUDM in this issue of the journal. Community experiments are crucial for the evolution of any data standard. After all, the standard is used by people and organizations who are the most interested in developing it.

To address the adoption challenge, one of the latest Post Pistoia Alliance UDM team activities included a qualitative and detailed investigation on pain points of chemical reaction data processing within a small group of chem/data scientists. The questions asked by the UDM project team included:

1. Can you give example(s) of your workflows when you extract, combine, store, analyze and use data?

2. What problems do you face when extracting, combining, storing, analyzing, and using data?

3. How do you communicate with external and internal parties in terms of data exchange?

4. Where in your existing workflows do you see use-cases for using UDM exchange format?

5. What value will UDM bring to your work? What problem will it solve?

6. What would you need to overcome/what could help adopting the UDM format?

What we have learned in terms of problem statements can be summarized in the diagram below (Fig. 14).

Fig. 14:

Reconfirmed pain points in reaction data exchange, management, and analysis.

As quantitative statements are not possible to make due to the small size of the group, we were, however, able to identify the use cases and to discuss the value of the UDM. For example, reaction data integration from various ELNs is still a challenge for many pharmaceutical companies and they spend a good amount of time cleaning and integrating the data into one data repository. Reaction data preparation for ingestion into in-house repositories from other data sources including ELNs take a lot of time too due to the disparity of the data. Data exchange with CROs and pharma companies is suboptimal and most of it is happening via file sharing (e.g., PDF documents). It is impossible to collect information from these files into one database. Data cleanup and preparation for the modelling project is also tedious and time consuming due to lack of a common standard. Another increasingly important activity is data exchange between various applications, for example, from the database where a scientist has searched for a reaction of interest to the import of this reaction in their synthesis predictive tool to be able to modify the product or the reactant molecule and to run a new prediction based on the modifications. Last but not least, the data exchange from different laboratory devices is hampered by the vendor specific formats which are not interoperable.

The value of the UDM is defined by these use cases and how they help solving each issue, so what are the adoption barriers that slow down the adoption. A few important are elaborated on in the next paragraph.

The users rarely see the format being included in the ELNs or other applications dealing with chemistry data up to now. For example, if one draws and searches a molecule, collects additional information on the synthesis and conditions and wants to export it into another tool, UDM export and import buttons would simplify the work. The other barrier is that direct interconversion tools from the UDM to RDfile or vice versa or JSON to UDM and vice versa have not yet been developed due to a lack of funding so that workarounds like protocols of Biovia’s Pipeline Pliot must be used for this transfer process. In addition, UDM was focusing on the reactions and analytics data and, for example, bioassay data extension was planned afterwards and is still on the UDM’s roadmap. However, it is also understood from the discussions that the senior management in many pharma companies begin to understand the problem, the scale and the value of having a data standard once large data integration projects are planned. Although, the situation is changing as AL/ML methods developed by many pharma companies require lots of efforts to clean the data, lots of time spent on harmonization and integration of the data and the involvement of people with cheminformatics and data management skills. To address the adoption challenges, the UDM team is reaching out to ELN providers and other stakeholders to discuss UDM export/import functionalities in their tools.

UDM next steps

Our vision is UDM to become a common data exchange format used by different informatics systems dealing with chemical reactions so that data can be seamlessly exchanged without losing content and observing intellectual property constraints. Fig. 15 illustrates this vision by representing an ideal information flow between various types of systems that either use the core UDM or UDM with additional vendor extensions, e.g., for synthetic robot data specific to the used equipment. This would remove the need of constant processing and curation of data files using slightly different data models or data formats which is the situation still present in many organisations.

Fig. 15:

Vision of UDM usage at scale.