EFLM checklist for the assessment of AI/ML studies in laboratory medicine: enhancing general medical AI frameworks for laboratory-specific applications

1 Type of laboratory data described

Laboratory test results are not mere numerical outputs, but rather the final products of complex pre-analytical, analytical and post-analytical processes that may vary widely across institutions, methods, and devices, ultimately reflecting a patient’s (patho-) physiological levels of proteins, metabolites, or other biomolecules. The differences in pre/post and analytical processes have significant implications for the development and generalizability of AI models. Many laboratory analytes – particularly in the fields of clinical chemistry and immunochemistry – exhibit method dependency, meaning that their measured values can differ substantially depending on the analytical principle, the reagent system, the calibrator used, and the instrumentation.

For example, consider the lack of specificity associated with Jaffe methods compared to enzymatic assays in the measurement of serum creatinine, and the substantial impact that the standardization process has had on improving the quality and comparability of its measurement [18]. Similar challenges affect other measurands, particularly those measured using immunoassays, such as hormones, cardiac biomarkers, drugs, and tumor markers. Even when analytes are nominally the same (e.g., troponin I, procalcitonin or vitamin D), assays from different manufacturers may not be interchangeable due to lack of standardization and variability in antibody specificity [19], [20], [21]. In addition, analytes within the fields of toxicology or therapeutic drug monitoring, may be measured with the gold-standard method of mass-spectrometry, or the less costly and more automatable method of immunochemistry. Apart from the mentioned methodological differences, the measurement uncertainty of these two methods may differ substantially [22]. Similarly, enzymatic activities, such as alanine and aspartate aminotransferase (ALT, AST), alkaline phosphatase (ALP), or lactate dehydrogenase (LDH), are sensitive to assay conditions including temperature, pH, and buffer composition [23], [24], [25]. Consequently, values obtained from different platforms or under different analytical conditions may not be analytically equivalent, even if expressed in the same units. This is particularly relevant for assays based on immunoassay principles, where results from different manufacturers (e.g., Abbott, Roche, Siemens) can diverge substantially due to differences in antibody specificity, calibration processes, and detection technologies. Therefore, indicating the method category alone (e.g., “immunoassay”) is insufficient. For meaningful interpretation and potential reproducibility, it is essential to provide detailed information about the analytical method, platform, reagent manufacturer, and, where relevant, specific assay configurations.

Equally important is the explicit reporting of the unit of measurement for each laboratory variable, as it directly influences the numerical scale and interpretability of the data. Notably, a recent initiative for recommending the extensive adoption of the International System of Units (SI) has been promoted, because standardization is an essential tool for laboratory ISO 15189:2022 accreditation [26]. In some cases, values expressed in different units are not interconvertible due to fundamental differences in measurement principles. For instance, D-dimer may be reported in Fibrinogen Equivalent Units (FEU) or D-dimer Units (DDU), two non-equivalent systems, with approximately a two-fold difference between them [27]. However, without knowing which unit is used and the specific assay calibration, no reliable conversion is possible. Another example is HbA1c, which can be reported either as a percentage (%), following the NGSP/DCCT standard, or in mmol/mol, according to IFCC recommendations [28]. While a mathematical relationship exists between the two, it is non-linear and depends on population-level regression equations; applying it blindly, especially in the absence of assay standardization, may introduce conversion errors. Similarly, hormone concentrations for insulin or parathyroid hormone (PTH) are variably reported in mass units (e.g., µg/L) or international units (e.g., mIU/L), but no universal conversion factor exists due to assay-dependent differences in molecular structure recognition and calibration [29], 30]. In such cases, unit inconsistency may lead ML models to misinterpret identical biological values as distinct or fail to generalize across datasets. Furthermore, inconsistencies in measurement units can be particularly troublesome in various contexts, including AI applications, as they are often difficult to detect even during data curation, despite rigorous consistency checks conducted by experts or data scientists. Therefore, documenting the unit of measurement is not merely a formal detail, but a prerequisite for reliable data integration, model training, and cross-site reproducibility.

Despite their obvious importance, these aspects are rarely reported in ML studies using laboratory data. Recent reviews have highlighted that laboratory data used in AI models are often insufficiently characterized, with missing information on measurement methods, units, traceability, and analytical platforms [5], 9]. Such omissions hinder reproducibility, impair cross-site validation, and may lead to models that are overfitted to the idiosyncrasies of a particular laboratory environment.

In order to ensure meaningful interpretability and comparability of laboratory-derived data across studies and clinical settings, it is not sufficient to mention the analytical method category alone (e.g., “immunoassay”), as considerable variability exists across manufacturers and platforms even within the same methodological class. Therefore, the following details should be systematically documented:

1. The analytical method used, specifying the exact measurement principle, assay type, and manufacturer (e.g., electrochemiluminescence immunoassay, Roche Elecsys^® platform);

2. The specific platform or analyzer including instrument model, software version, and, where possible, reagent lot;

3. The units of measurement (e.g., mmol/L, µg/L), preferably expressed using the Unified Code for Units of Measure (UCUM) to ensure interoperability, along with conversion factors where applicable [12].

This level of transparency is particularly critical when models are trained in multicenter settings or intended for deployment across different healthcare systems. In such cases, method-related differences can introduce systematic shifts that compromise model accuracy and reliability if not properly accounted for. Therefore, we recommend that this information be explicitly required when laboratory data are used as input features in AI model development.

2 Variability addressed (biological and analytical)

One of the fundamental challenges in applying AI to laboratory medicine lies in the inherent imprecision of the data, which stems from both biological and analytical sources [31]. Laboratory results are subject to biological variation – including within-subject biological variation (CV_I) and between-subject biological variation (CV_G) components – as well as analytical variation (CV_A), which arises from measurement procedures conducted under variable conditions. These sources of variation not only affect the interpretability of laboratory tests in the clinical context but also compromise the reproducibility and generalizability of AI models trained on such data.

In clinical practice, laboratory data are intrinsically “noisy”, unavoidably affected by stochastic fluctuation and procedural inconsistencies due to biological and analytical variations. Indeed, even under standardized conditions, repeated measurements in different samples from the same individual can vary due to physiological rhythms, transient metabolic states, or pathological fluctuations. Analytical variation (namely imprecision and bias), in turn, may be introduced through differences in instrumentation, reagents, calibration materials, environmental conditions, or operator technique. As Coskun highlights [31], such variability, particularly if not explicitly modeled during algorithm development, can introduce significant uncertainty and reduce model performance, especially in external validation or deployment across diverse laboratory settings. Despite its critical impact, imprecision is rarely incorporated explicitly into the design or validation of AI workflows, limiting their clinical robustness and applicability.

Recent literature has emphasized the importance of accounting for analytical and biological variations, termed instantial variability in a mathematical view and defined as a form of uncertainty arising from the case-dependent, context-specific nature of each data point [32]. Unlike classical notions of variation, associated with population-level heterogeneity or measurement error, instantial variation encompasses the intrinsic fluctuation observed within a single subject over time or under differing conditions, reflecting both biological and analytical sources of variability. Ignoring analytical and biological variability may result in ML models that are overfitted to the training distribution and fail to generalize to new instances, even when these instances belong to the same individual measured at a different time or under slightly altered conditions. Interestingly, a mathematical model has been developed by Campagner et al. for dealing with analytical and biological variations in AI models [31].

Moreover, laboratory data often exhibits ‘soft heterogeneity’, context- and method-dependent shifts in meaning, that structured terminologies (e.g., Logical Observation Identifiers Names and Codes (LOINC) for test identity/method and Unified Code for Units of Measure (UCUM) for units) help make explicit [10], 32]. Two values referring to the same analyte may carry different clinical significance depending on pre-analytical contextual factors such as, for example, the sampling conditions (e.g., fasting vs. postprandial state, resting vs. post-exercise). These contextual elements, also referred to as peridata [14], are not always explicitly encoded in the data, yet they influence the true meaning and downstream interpretability of measurements. As demonstrated in recent studies [10], 33], standard AI models may be ill-equipped to handle such fine-grained variability, which can substantially compromise robustness, particularly in external validation or deployment scenarios. Explicitly representing this variability is therefore essential for the development of clinically trustworthy, context-aware AI systems.

To mitigate the risks associated with model miscalibration, lack of robustness, and limited generalizability in the context of laboratory medicine data, developers of AI-based clinical decision support systems should adhere to a series of best practices grounded in biostatistical principles and uncertainty-aware modeling [34]. We mention three:

1. Report variability metrics: Developers should quantify and disclose key sources of variability, including CV_A, CV_I, and CV_G variation. The CV_A can be derived from laboratory quality control or validation data. CV_I and CV_G can be derived from databases and literature, e.g. the EFLM Database on Biological Variation [35], or own experiments. These variation sources should be included in model cards or data sheets to contextualize the model’s reliability and limitations. In the interpretation of serum biomarkers, significant intra-individual variation over time may lead to divergent classification outcomes if such variability is not explicitly acknowledged or controlled for.

2. Account for individual variation: Temporal fluctuations within individuals, including both predictable patterns (e.g., circadian rhythms) and non-predictable noise, a phenomenon also referred to as individual variation (IV), represents important sources of variability. These may reflect a combination of systematic (predictable) and random (non-predictable) components. Strategies such as repeated measurements, temporal smoothing, or longitudinal designs can improve model robustness. Failure to account for such variability may lead to misclassification, even when values fall within the biologically normal range [10], 32].

3. Model uncertainty explicitly: Techniques such as data augmentation reflecting known CV_A/CV_I, probabilistic models, or ensemble methods can capture and propagate uncertainty. These approaches have demonstrated improved resilience to IV-related perturbations [36].

3 Use of metadata and peridata

As described above, the clinical and analytical validity of laboratory data used in AI models cannot be properly assessed without consideration of the contextual information that accompanies the test result. In this regard, two categories of supporting data are particularly relevant: metadata and peridata. According to the recent EFLM position paper [17], metadata is defined as:

“Data derived from the testing process that describe the characteristics and the requirements that are relevant for assessing the quality and the validity of laboratory test results.”

Peridata, on the other hand, is:

“Data derived from the testing process that are relevant for the interpretation of the results within the clinical context, making that data actionable for the patients’ care.”

We recommend encoding laboratory observations with LOINC and representing measurement units with the UCUM. LOINC’s six-axis model (Component, Property, Time, System, Scale, Method) makes method and context dependence explicit [37], while UCUM provides machine-readable, unambiguous units [12]. Contextual factors (e.g., fasting status) should be represented as separate LOINC-coded observations or controlled peridata fields linked to the primary result. For related clinical concepts (e.g., disorders, procedures, findings) linked to laboratory results, we recommend using Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) [38], while continuing to use LOINC for test/observation identifiers and UCUM for units.

These elements are often available in laboratory information systems but are rarely exported, structured, or reused in the context of AI development. Metadata includes descriptors such as test name and code, analytical method, instrument identifier, units of measurement, reference intervals, calibration traceability, and timestamps. Peridata encompass variables such as sample type and quality (e.g., hemolysis, lipemia), pre-analytical conditions (e.g., time from collection to analysis), clinical setting (e.g., emergency vs. outpatient), and the circumstances under which the test was ordered.

In the absence of this structured information, laboratory results are treated as decontextualized numerical values, which risks compromising the interpretability, generalizability, and clinical usability of AI models. For instance, a troponin result obtained from a point-of-care device in the emergency department may not be directly comparable to one measured in a central laboratory using a high-sensitivity assay. Similarly, glucose values may differ depending on the delay between sampling and centrifugation or by using different additives in sample containers [39], an issue that peridata could help detect and correct for.

To enhance the semantic depth and interoperability of laboratory-derived features in AI workflows, it is essential to:

1. Extract, structure, and integrate key metadata and peridata fields as part of the AI development pipeline

2. Include them in the data curation and during the model development phases

3. Use them to support interpretability, auditability, and harmonization across datasets

For example, for a serum glucose result, include the LOINC code and version, the UCUM unit, the measurement method (if using a method-specific LOINC), instrument/reagent identifiers, and a linked LOINC-coded observation for fasting status.

The inclusion of metadata and peridata also aligns with FAIR (Findable, Accessible, Interoperable, Reusable) data principles, supporting responsible data stewardship in clinical AI applications [17], see Section 6: “FAIR Data Sharing and Feature Reproducibility” for practical reporting expectations. Their systematic incorporation can substantially enhance both scientific reproducibility and clinical relevance, thereby strengthening the translational value of AI models in laboratory medicine.

4 Harmonization and standardization of laboratory measurements

In the context of AI model development based on laboratory data, it is not sufficient to describe how each test result is generated (method, unit, instrument); it is also necessary to understand whether those results are analytically comparable across different laboratory settings [40]. One example of a strategy for improving data harmonization is the use of an international normalized ratio (INR) for reducing the between-laboratory variation, regardless of the type of assay used [41].

Laboratory medicine, unlike many other domains of clinical data, has invested considerable effort in establishing standardization and harmonization protocols to reduce inter-method variability and ensure the traceability of results to reference systems [42]. However, these efforts are analyte-specific, unevenly implemented, and incomplete for many clinically relevant tests [43], 44].

Standardization refers to the establishment of metrological traceability to recognized reference methods or materials, enabling full result comparability regardless of the testing system used [45]. Harmonization, in contrast, aims to align results across different methods in the absence of a formal reference [46]. Numerous widely used analytes, such as glucose, creatinine, HbA1c, and cholesterol, have established metrological traceability and are generally suitable for multicenter AI model training [47], 48]. In contrast, many immunoassay-based measurements (e.g., TSH, troponins, BNP, vitamin D) still suffer from significant method-dependent variability due to differences in reagent composition, antibody specificity, and calibration practices [49], 50].

Another example is the measurement of Lp(a). While current evidence demonstrates that increased Lp(a) is a causal risk factor for cardiovascular outcomes, care must be given to the interpretation of Lp(a) results because Lp(a) assays are not yet internationally standardized [51], 52]. In addition, two different measurement units are currently being used. One unit measures Lp(a) particle numbers in nmol/L and the other measures Lp(a) mass in mg/dL. Since the mass of Lp(a) particles is variable, a direct conversion between the two can only be an approximation [53]. The recent EuBIVAS (European Biological Variation Study) provided robust evidence confirming the substantial biological and methodological variability in Lp(a) measurement and reinforced the need for updated, standardized analytical performance specifications to ensure clinical utility [52].

This variability may go unnoticed in single-center studies but becomes critical in multicenter research, federated learning – a privacy-preserving approach that trains models across sites by exchanging model updates rather than raw data – , or real-world implementation where both semantic and procedural proximity between datasets can materially affect observed performance [54], 55]. A model trained on non-harmonized data may inadvertently capture local analytical biases instead of biologically meaningful patterns, limiting its clinical reliability and portability. From a clinical perspective, the lack of harmonization forces patients to rely on a single laboratory over time to ensure result reproducibility, which may compromise continuity of care, limit diagnostic flexibility, and exacerbate healthcare inequities. Ensuring inter-laboratory comparability is therefore not only a methodological necessity, but also a clinical and ethical imperative.

Only by assessing whether a laboratory feature can be meaningfully compared across environments can we ensure the safe deployment and equitable performance of AI systems in laboratory medicine.

In addressing the complexity of laboratory data for AI/ML applications, it is essential to recognize that metrological traceability alone may be insufficient to ensure meaningful analytical comparability across settings. In fact, even when traceability is declared by the manufacturer, results may remain non-interchangeable due to methodological differences of performance, including the matrix effects, and variable uncertainty propagation in routine clinical use [56]. This perspective aligns with long-standing concerns in laboratory medicine that high-level harmonization efforts, while conceptually rigorous, may not fully capture the analytical heterogeneity observed in real-world conditions, particularly in multi-analyzer or multicenter environments. Thus, the inclusion of explicit documentation on analytical methods, uncertainty estimation, and harmonization strategies is not merely a best practice but a prerequisite for building trustworthy, generalizable AI models in laboratory medicine.

To mitigate these risks, we recommend that studies developing AI models using laboratory data:

1. Explicitly state whether the analytes used are standardized or harmonized, and describe the level of traceability to reference systems;

2. Provide information on calibration hierarchies, traceability chains, or use of certified reference materials;

3. Identify analytes lacking harmonization as method-dependent and validate associated models accordingly;

4. Document terminology mapping strategies (e.g., local-to-standard code conversions), while taking into account ongoing initiatives by EFLM Committees on the Exchange of Laboratory Data [57], aimed at advancing semantic interoperability through frameworks such as LOINC and SNOMED CT.

This information builds upon the descriptive requirements outlined in point 1 but adds an essential dimension of inter-site comparability.

5 External validation with attention to dataset similarity and cardinality

External validation is widely recognized as a critical step in the evaluation of AI models in healthcare [3], 58], 59]. It provides evidence that a model performs well not only on the data it was trained on but also on independent datasets that reflect real-world variability. In laboratory medicine, this step is especially important due to the heterogeneity of testing practices, instrumentation, population characteristics, and pre-analytical conditions and all other above-mentioned variables across institutions [33], 50].

Despite its importance, external validation is often overlooked or inadequately reported in studies using laboratory data for AI development. Moreover, when validation is performed, little attention is typically paid to the semantic and procedural similarity between training and validation datasets, which is a key factor in interpreting validation results. Here, we use semantic proximity to denote the degree to which variables have the same meaning across datasets – encompassing analyte definition/coding (e.g., LOINC), units (e.g., UCUM), reference intervals, and relevant contextual factors (e.g., fasting status) – and procedural proximity to denote the similarity of data-generating procedures, including pre-analytical conditions, measurement methods, instruments/reagents, and calibration/traceability practices; by cardinality we mean the size and structure of the datasets (e.g., numbers of subjects, episodes, and measurements per subject, class prevalence, and missingness). As highlighted in recent methodological work, including the concept of meta-validation proposed by Cabitza et al., both the cardinality (sample size and structure) and the semantic and procedural proximity between datasets must be considered to assess whether a validation is genuinely informative [10].

Laboratory data is influenced by local institution-specific practices that affect feature distributions in non-trivial ways. Even standardized tests may differ in reference intervals, calibration, or measurement traceability. Without detailed documentation and comparison of these parameters, performance metrics derived from external validation may be misleading, either by inflating model robustness when datasets are overly similar, or by underestimating generalizability when measurement systems are poorly harmonized.

In addition to the original statements reported on the original validation checklist, we recommend that studies using laboratory data [7]:

1. Clearly specify whether external validation was performed and describe the selection criteria and characteristics of the validation dataset

2. Report on analytical comparability between datasets, including methods, units, and instruments used

3. Characterize population overlap and distributional shifts between development and validation cohorts

4. Consider adopting meta-validation strategies to assess the relevance and representativeness of the validation cohort relative to the development data

These practices are essential for developing AI systems that are not only accurate under controlled conditions but also resilient to the variability, introduced by real-world clinical laboratory environments. In line with EFLM priorities, attention to external validation is not only a scientific imperative, but a practical necessity for ensuring the clinical applicability of AI tools in laboratory medicine.

6 FAIR data sharing and features reproducibility

Transparency and reproducibility are foundational principles in scientific research and are particularly critical in the development of AI models intended for clinical use. While much attention is given to model architecture and performance metrics, the data processing workflows – including how laboratory features are curated, transformed, and documented – often remain insufficiently described. This lack of transparency poses a significant barrier to model replication, regulatory review, and clinical adoption.

The application of the FAIR principles – Findable, Accessible, Interoperable, Reusable – to laboratory data is still emerging but is increasingly recognized as essential for robust and trustworthy AI development, although practical adoption remains variable [60]. Although this field is still in its early stages, a growing number of frameworks are starting to support FAIR aligned data practices, including the use of cloud-based platforms for data management, integration, and sharing [61], 62]. FAIRness goes beyond simply sharing raw data: it encompasses the ability to trace how variables were included in the model, how outliers were handled, how missing data were imputed, and whether domain-specific transformations (e.g., normalization to reference intervals, log transformations for skewed distributions) were applied. In the laboratory context, where each variable may carry implicit assumptions related to measurement units, calibration, or biological variation, detailed documentation of the data pipeline is essential for interpretability and reproducibility.

Importantly, this point does not duplicate the requirement to describe laboratory data characteristics (as in point 1 and 4), nor does it address inter-laboratory comparability (as in point 5). Instead, it focuses on ensuring that the entire data processing and modeling pipeline can be audited and reused, whether by external researchers, regulators, or clinical collaborators.

To meet this objective, we recommend to:

1. Provide comprehensive documentation of all preprocessing steps, transformations, and feature derivations used during model development

2. Make available, where ethically and legally permissible, code repositories, data dictionaries, and mapping files (e.g., local-to-standard code conversions)

3. Clearly trace the provenance of each variable – from raw measurement to model input – be clearly traceable

4. Where direct data sharing is not feasible, supply synthetic datasets, anonymized test cases, or detailed mock schemas to illustrate data structure and preprocessing logic

By adhering to these practices, developers align with current expectations for open science and responsible AI [60]. Moreover, such transparency fosters collaboration, facilitates regulatory review, and accelerates scientific progress by enabling others to build upon validated laboratory-AI pipelines.