Data flow in clinical laboratories: could metadata and peridata bridge the gap to new AI-based applications?

Introduction

In recent years, clinical laboratories have experienced significant improvements in several technological tools and instrumentation. Laboratory information systems (LIS), initially introduced in the 1970s, have rapidly evolved from simple software to a sophisticated part of the entire information technology (IT) system, available in clinical laboratories, able to retrieve and exchange information with several instrumental middleware, other laboratories LIS as well as hospital information systems and regional databases [1]. In general, the increased capabilities of certain LIS, coupled with the advancements in various information technologies, the introduction of cost-effective chemical, physical and mechanical sensors into analytical instruments, and the improved integration with other digital tools, have led to an increase in the volume of data produced in clinical laboratories [2].

Considering the total testing process (TTP) or the so-called “brain-to-brain loop” [3], the analytical phase includes only a part of the extensive information stream generated during the entire cycle of testing. Apart from patients’ test results and demographic details, LIS might also register many additional pieces of information, such as the plain names of tests, audit trail records, technical or medical validations, as well as ward names for inpatients or the general practitioner for outpatients. Data generated during the TTP may also include other details or information of importance [4]. As an example, considering the pre-preanalytical and the preanalytical phases, a large volume of data may contain the time and types of samples collected, the handling conditions, including the temperature and time of transportation, the centrifugation conditions, the number of aliquots generated, etc. [5], 6]. During the analytical phases, relevant information is often recorded by LIS in addition to test results. These might include hemolysis (H), icterus (I) and lipemia (L) indexes (which might be a unique value, or a value associated for each executed test), the assay calibration curves, sample dilutions, analysis repetitions and technical validation rules (accepted or violated). For analyses based on “-omics”, other data, often of large volume, are generated, encompassing mass spectra or proteomics, metabolomics and lipidomics, and sequence files for next-generation sequencing. Certain LIS might incorporate or integrate data from the laboratory quality control (QC) system; these include not only internal quality controls (IQC), but also external quality assessments (EQA) as well as additional resources about the entire process of verification and validation of analytical methods [7]. Finally, there are some LIS, more specifically designed for genetic testing or for microbiology or transfusion medicine, which are capable of integrating with different instrumental software or sophisticated pipelines to convert thousands of sequences into clinically useful information [8].

During the post-analytical phase, interpretative comments, the sample storage or sample re-analyses (e.g. in case of follow-up testing), additional tests requested by clinical wards and details on the scrutiny of urgent results with the issue of provisional report and its delivery to the requesting clinician are usually generated and are also registered by LIS.

The aim of this opinion paper from the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group on Artificial Intelligence (WG-AI) is to discuss qualitative types of data produced in each phase of TTP, and propose how to divide laboratory-generated information into distinct data categories.

The importance of collecting and sharing high-quality data

The short statement “garbage in, garbage out” recognizes that poor quality of data would lead to unreliable outputs, emphasizing the utmost importance of improving and maintaining data quality [9]. In healthcare research, the collection of high-quality, reliable data is essential in the field of statistics and artificial intelligence (AI)-models, especially machine learning (ML) methods, which also include deep learning techniques for generating new clinically relevant algorithms for advancing medical knowledge, illness detection and personalized treatments [10]. The development of highly reliable ML models usually requires large sets of data for training, testing, and validation; however, these large datasets can often only be obtained by combining a multitude of sources. Additional reasons motivate the merging of data. Firstly, in many circumstances, even for large clinical laboratories, it is unrealistic to retrieve large datasets as required by ML, and the under-representation of specific patient cohorts in testing data could represent a biasing issue; examples are cases where ML algorithms are applied to pediatric records, rare diseases or in situations with a high imbalance in the number of studied individuals within groups. On the other hand, outside sources (e.g. from other centers or other laboratories) have to be merged with local data to support external validation. Third, the in praxi continuous performance monitoring of ML-based algorithms (e.g. accuracy, sensitivity and specificity) over time might be of higher quality when using externally gathered data, different from the source (e.g. the laboratory) where they are usually applied [11], 12]. This latter consideration may be of particular interest to engineers and data scientists, since the complex interaction between AI-models and real-world conditions (including the instrument variability and bias) can lead to unexpected and harmful behavior [13]. Thus, the possibility of collecting high-quality data from multiple, unparalleled resources is of utmost importance and should be encouraged by scientific communities, and through national and international guidelines [14].

Defining properties of data attributable to the total testing process

It is undoubtedly true that laboratory TTP-generated data could benefit from a redefinition of terms, especially considering their potential use outside clinical purposes, such as in the fields of data science for statistical applications and the generation of ML tools.

The term metadata was first announced by the National Information Standards Organization [15] and it was defined as “attributes that are necessary to locate, fully characterize, and ultimately reproduce other attributes that are identified as data” [16]. Thus, metadata are additional information that can be enclosed with data, which clearly and unambiguously describe the data as well as their full provenance [17]. This definition is reminiscent of the statement given by the National Institute of Standards and Technology (NIST): “Information describing the characteristics of data including, for example, structural metadata describing data structures (e.g. data format, syntax, and semantics) and descriptive metadata describing data contents” [18]. Metadata are often called “data about information” or “information about information” [19]. Both definitions are not contradictory and highlight some possible relevant features of metadata applicable to the laboratory medicine field. These valuable technical pieces of information improve the “reproducibility”, “comparability” and “harmonization” of data, requirements that are relevant for assessing the quality and the validity of laboratory test results. In addition, considering the scientific fields related to data science, such as ML approaches, metadata not only represent useful information on the robustness of data but may also facilitate data sharing and improve usability. Interestingly, the concepts of comparability and reliability are commonly associated with clinical laboratory findings as well.

Beyond metadata, other forms of data (excluding laboratory test results, which are deemed the “primary data”), e.g. reference intervals, clinical decision limits, etc., may in some way affect the TTP and thus warrant special consideration. Our suggestion is to designate data, which neither constitute primary data nor metadata, as “peridata”. Peridata can be characterized as data facilitating the accurate interpretation of test results. Hence, peridata may enrich mostly the clinical validity of test results, also by offering comparative frameworks, benchmarks, or additional layers of validation. Hence, peridata are information relevant for the interpretation of the results within the clinical context, making that data actionable for the patients’ care. It is essential to consider that since peridata may also be derived from databases different from those typically obtained from clinical laboratories, it has to be ensured that informed consent has been obtained from the patient or a legal representative of the patient. This consideration applies only to certain peridata. However, the requirement for obtaining consent may vary depending on the regulations of different European and non-European countries.

Based on the previous definitions of metadata and peridata in the clinical laboratory setting, it should be clearly stated which kind of information generated throughout the TTP could be defined as metadata or peridata. Therefore, through consensus among the members of the EFLM WG-AI, a distinction between metadata and peridata was identified, as reported in Tables 1–2 and Figure 1.

Figure 1:

Types of data generated during the phases of the total testing process, subdivided by primary data (test results) and metadata/peridata. HIL, hemolysis, icterus and lipemia indexes; TAT, turnaround time; EQA, external quality assessment; IQC, internal quality control.

A practical, real-world example is provided in the Supplementary Material to demonstrate the data records retrieved from the LIS, instrument middleware, quality management system (QMS), or electronic health records (EHR). This was done for a synthetic patient result derived from the clinical laboratory of one of the authors. All available data registrations were categorized into meta- or peridata. The results are reported in Supplementary Tables 1–4. It should be noted that not all data types were retrievable directly from the LIS, and that additional data sources needed to be consulted to obtain the different meta- and peridata missing values. This means that for the registration of relevant meta- and peridata in the LIS, laboratories might need to invest time, efforts and costs for their creation, storage, and retrieval. These investments will have to be expended by the LIS user (who needs to enter data that currently remains unregistered), the LIS producer (who needs to create additional storage and interfaces for registration and retrieval), or the LIS buyer (who needs to pay a surplus for a meta- and peridata-capable LIS or additional instrument driver connections that feed data into the LIS).

The FAIR data approach

The concepts of findability, accessibility, interoperability, and reusability (FAIR) within the context of scientific data are paramount for making healthcare data (and thus laboratory medicine data) optimally usable for clinical and research purposes [20]. The FAIR principles aim to address challenges associated with data sharing and reuse in research, ensuring that data are not only generated for a specific purpose but also has the potential to contribute to broader scientific advancements across disciplines [21]. It is noteworthy that the FAIR principles could be extended beyond the realm of collected data to encompass the entirety of a research study, offering even greater applicability and benefits [22].

Ensuring the findability of data establishes a foundation for effective data discovery, enabling researchers to locate and access relevant datasets efficiently. Indeed, datasets should be easily findable (e.g. by using unique identifiers for the dataset and by promoting use of online repositories); second, once data are found, they should be easily accessible. This means that the data, along with its meta- and peridata, should be promptly accessible using adequate permissions.

Finally, data should be interoperable and reusable [23]. Interoperability involves the capability for data to be seamlessly integrated and merged with other datasets. Reusability, on the other hand, implies that data are designed to be reused for future purposes [24]. In the context of medical laboratory data, ensuring interoperability and reusability entails providing clear and comprehensive meta- and peridata, documenting the data, the steps involved in data creation and processing, and adopting widely accepted standard formats to ease interpretation and promote reuse. Adopting the FAIR principles could create positive synergy with the validation of AI models. Specific benefits of FAIR AI research could be a) an improvement of generalizability, by exposing AI models to diverse, FAIR datasets, b) the guarantee of scientific correctness of AI algorithms, especially when they are applied to different contexts or different labs [25], c) reduce the overall cost of research [26], d) the evaluation of real-world performance monitoring of AI models [27]. The latter point gains significance in assessing the potential growing volume of AI tools available both for improving instrumental technologies and in clinical practice. This attention has grown in response also to recent studies highlighting concerns about the robustness and generalizability of FDA-cleared AI models for research and clinical use [28]. The establishment of a publicly available international repository of laboratory-medicine datasets that may serve as the benchmark sets for AI/ML model performance evaluation may resolve this problem. Further solutions could be represented by synthetic datasets, as already demonstrated for hematological laboratory analyses [29].

Necessity of standard preanalytical coding for biospecimens to improve data reliability

Standardization and harmonization are key elements pursued by national and international laboratory societies and agencies to render medical diagnostics comparable and usable for national and international databases (e.g. European Health Data Space). Systems like Logical Observation Identifiers Names and Codes (LOINC) [30] or Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT) [31] have been around for some time and have been improved over time. While LOINC focuses on variables such as the type of sample material, the analytical method, the unit of measurement, etc., SNOMED-CT includes also clinical content for medical documentation with greater granularity than LOINC. However, none of these systems take pre- or postanalytical meta- or peridata of laboratory test results into account. Terminology standards are crucial for ensuring semantic interoperability in healthcare data and for guaranteeing efficient interactions across databases. However, the existence of multiple standards limits the achievement of a consensus, and data interoperability can be enhanced by cross-mapping LOINC and SNOMED-CT or using SNOMED-CT as a reference terminology to bridge various standards [32].

Interestingly, both LOINC and SNOMED-CT prove to be highly pertinent, since they encompass valuable information that could be regarded as either meta- or peridata. The latter consideration emphasizes the significance of meta- and peridata; while certain features of LOINC and SNOMED-CT are recognized as relevant for ensuring data robustness, their applicability in the context of data analysis or for ML modeling remains relatively limited. Another important system is the Observational Health Data Sciences and Informatics (OHDSI), which is fundamental for the interrelationship among databases, assuring anonymization and aggregation of data. OHDSI offers many open-source tools to support the standardization of data analytics allowing the interaction across two or more databases using the Common Data Model (CDM) [33].

As of today, to the best of our knowledge, the only system that may be considered a valid approach to document preanalytical variables in a standardized fashion is the Sample PREanalytical Code (SPREC) system for biobanking samples, developed by the International Society for Biological and Environmental Repositories (ISBER) Biospecimen Science Working Group [34]. A standardized methodology for constructing a coding system tailored to medical laboratories could be developed enlisting national or international laboratory associations and working groups active in these topics. However, in anticipation of the ever-increasing use of data for AI modeling, it is crucial that variables potentially influencing the test results and/or its interpretation be documented alongside the result itself, as they could represent important meta- or peridata. Therefore, immediate actions have to be taken to develop such a catalog, ideally by implementation into existing coding systems. With respect to the data structure, either SNOMED-CT or a SNOMED-CT/LOINC combination would be the most reasonable candidates.

Conclusions and remarks

This study delves into key facets regarding data produced in clinical laboratories and the distinctive attributes defining them in the context of AI applications. Significant advancements in LIS and IT have ushered in an era of unprecedented data availability, emphasizing the pivotal role of laboratories in generating patient data, promptly applicable in ML models. Nevertheless, data derived from laboratories TTP extend beyond mere test results; it encompasses various additional pieces of information, with a substantial portion involving meta- or peridata. Understanding and appreciating the significance of additional information linked to test results may be improved by referring to a part of TTP laboratory data as meta- and peridata. This information is truly valuable for many applications, including those using ML specifically. Initially, this approach could facilitate the design of new clinical studies, feature collections, and information recording. However, achieving this purpose can be challenging due to limited access to clinical data. A recent European survey organized by WG-AI of the EFLM, collecting 195 replies from European countries, revealed that about 50 % of laboratory professionals lack full access to clinical data, and another study reported that about 65 % of laboratories do not have such access [35], 36].

Secondly, metadata could improve the reusability of data, aligning with the FAIR principles and creating a positive synergy for the validation of AI models. The adoption of these could yield several advantages in the development of future AI applications in healthcare, mitigating financial waste associated with fragmented efforts. For this purpose, clinical laboratories and Scientific Societies should stimulate the sharing of original experimental data, for reproducibility and comparability studies and other uses. Thirdly, standardization processes could be enhanced by focusing on a combination of meta- and peridata, whilst the development of LIS software could be improved by enabling the prompt visualization of all peridata available, particularly in clinical or technical validation. Metadata and peridata are, therefore, crucial for ensuring data quality, results accuracy, and their proper interpretation, ultimately aiming to benefit patient care [37], 38].

While the terms meta- and peridata are usually more confined to the domain of data science, they should be viewed as essential elements within the data management ecosystem, enabling researchers and clinicians to comprehend the nuances and context of primary data. For these reasons, it is the responsibility of the domain experts to define the kind of information useful for enriching the data with meta- or peridata to comprehensively describe the data production process (see Table 2). The redefinition of data types in the field of laboratory medicine is becoming increasingly essential, especially with the growing utilization of advanced computational methods, such as ML algorithms, to facilitate the creation of clinically valuable datasets.