Assessment of measurement uncertainty of immunoassays and LC-MS/MS methods for serum 25-hydroxyvitamin D

Introduction

Standardization of total 25-hydroxyvitamin D assays is essential for ensuring consistent clinical and epidemiological interpretations [1]. Standardization efforts aim to minimize inter-assay variability, establish reliable concentration thresholds for defining vitamin D status, and provide a framework for vitamin D supplementation guidelines. The Vitamin D Standardization Program (VDSP) and its Vitamin D Standardization-Certification Program (VDSCP), led by the Centers for Disease Control and Prevention (CDC) and the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) [2], are key initiatives promoting standardization and method comparability across laboratories. These programs have supported the development of three Joint Committee for Traceability in Laboratory Medicine (JCTLM) recognized reference measurement procedures (RMPs), which are maintained at the National Institute of Standards and Technology (NIST) [3], Ghent University [4], and the CDC [5].

To ensure meaningful standardization, analytical performance specifications (APS) must be established. The APS historically applied in vitamin D measurement standardization are those proposed by Stöckl et al. [6], as endorsed by the clinical laboratory community and agreed upon at the 1999 Stockholm Consensus Conference on Analytical Goals in Laboratory Medicine [7]. The VDSP has incorporated these APS into its certification process, whereby a 25-(OH)D method is considered “certified” if its calibration bias remains within±5 % of the CDC RMP and its imprecision, expressed as the coefficient of variation (CV), is<10 % [8]. The VDSCP also evaluates individual sample pass rates, i.e., the percentage of samples meeting bias criteria. Recent evaluations have shown that LC-MS/MS methods achieve higher pass rates (40–92 %, mean 65 %) compared to immunoassays (10–45 %, mean 27 %), highlighting the substantial variability between methods [9].

While standardization efforts focus primarily on bias and imprecision, measurement uncertainty (MU) provides a complementary approach for evaluating assay performance. MU integrates both systematic and random errors, offering a more comprehensive assessment of an assay’s reliability. In this manuscript, MU refers to the expanded measurement uncertainty (U), calculated from the combined standard uncertainty (u_c), which itself incorporates both standard uncertainty due to imprecision (u_IMP) and bias (u_BIAS), as defined by ISO 20914. The JCTLM Task Force on Reference Measurement System Implementation (JCTLM TF-RMSI) has proposed APS specifically targeting combined MU, defining a desirable threshold of ≤10 % and a minimum acceptable threshold of ≤15 % for clinical samples [10]. Additionally, the IFCC–IOF Committee on Bone Metabolism (IOF-IFCC C-BM) has also proposed a complementary approach based on MU to derive APS for 25-(OH)D based on the 2014 Milan Strategic Conference “Model 1” [11]. The APS we proposed are based on the physiological variation of 25-(OH)D concentrations over time. Specifically, data from the European Biological Variation Study (EuBIVAS) [12] demonstrated that, in healthy individuals, 25-(OH)D concentrations naturally increased by 31.6 % over a 10-week period (April-June) [13]. This increase is primarily driven by seasonal effects, where progressive sun exposure during spring and summer enhances cutaneous vitamin D synthesis, leading to a biologically expected rise in circulating 25(OH)D levels. To evaluate whether analytical methods can reliably detect such physiologically meaningful changes, an assay should exhibit MU that allows for statistically significant differentiation of this increase. The minimum required MU for this purpose can be determined using the following formula:

where Z represents the Z-score, corresponding to different levels of statistical confidence. Since this variation represents an increase, the statistical approach must be one-sided. According to Fraser [14], the probability of detecting a significant rise can be classified as.

1. Likely detection (p>0.80) → Z=0.84

2. More than likely detection (p>0.90) → Z=1.28

3. Very likely detection (p>0.95) → Z=1.65

4. Virtually certain detection (p>0.99) → Z=2.33

Thus, an assay capable of detecting a 31.6 % increase in 25-(OH)D over 10 weeks with these probabilities must have an MU of ≤26.5 % (likely detection), ≤17.4 % (more than likely detection), ≤13.6 % (very likely detection) and ≤9.6 % (virtually certain detection), respectively.

In this multicentric study, we used the University of Ghent (UGhent) RMP to evaluate the performance of two distinct LC-MS/MS methods and 13 immunoassays. Performance was compared against the APS defined by the VDSP, JCTLM TF-RMSI and IOF-IFCC C-BM approach based on physiological variation of 25-(OH)D over 10 weeks. Finally, we introduced a novel graphical approach based on MU, providing an intuitive visualization of method performance to complement traditional APS evaluations.

Materials and methods

Results

According to the UGhent RMP, serum total 25-(OH)D concentrations in the 17 pooled and eight single-donor samples ranged from 3.4 μg/L to 73.7 μg/L, with a mean of 31.7 μg/L. One sample contained a significant amount of 25-(OH)D2 (∼5 μg/L), as measured by LC-MS/MS methods, and was excluded since UGhent RMP did not provide 25-(OH)D2 measurements in this study. Another low-concentration sample was removed because its 25-(OH)D3 level was at or below the LOQ of several methods. Additionally, one sample from the single-donor set had concentrations below the LOQ of the Ortho system (<8 μg/L) and was excluded from the evaluation of this method only.

The mean imprecision (CV%) and mean bias (%) for serum total 25-(OH)D based on four individual replicate measurements, as well as VDSP performance criteria by assay, are presented in Table 2. Briefly, both LC-MS/MS methods and four automated immunoassays (Beckman-Coulter, DiaSorin, Fujirebio and IDS iSYS and ELISA – even if in one laboratory out of two for these three latter) passed the VDSP criteria for bias and imprecision. The individual sample pass rate, representing the percentage of determinations with bias between ± 5 %, ranged from 0 % (for Diazyme and Roche in one laboratory) to 87 % for the UMC Amsterdam LC-MS/MS. Table 2 also provides a detailed breakdown of bias, imprecision, and measurement uncertainty across participating laboratories. Significant inter-laboratory variability was observed for certain methods. With the Abbott assay, one laboratory exhibited a bias nine times higher than the other, suggesting considerable variability in calibration or operational conditions. Similarly, for DiaSorin, the CV in one lab was three times higher than in the other, indicating substantial differences in precision. Among the two laboratories using Ortho Diagnostics, one displayed extreme bias above 60 %, which remained uncontrolled even after recalibration and reruns with new samples. In ELISA-based methods, operator variability was evident despite all technicians being experienced and using identical instruments under controlled environmental conditions.

To further assess assay performance, measurement uncertainty of the methods was also evaluated against the different APS frameworks described above. The two LC-MS/MS and the methods from Beckman, DiaSorin, Fujirebio, IDS iSYS and Snibe met the JCTLM TF-RMSI desirable APS for standard Mu on clinical samples (≤10 %) in all the laboratories whereas the methods from Abbott and the IDS ELISA were successful in one out of two centres; All the other methods failed to meet these APS. The “minimum acceptable” APS, set at (≤15 %) was met by all methods, except four of them: Siemens, Ortho and Diasource (in one out of two participating laboratories) and Affimedix.

Figure 1 presents, for all methods combined, a set of correlation graph where the x-axis represents x(UGhentRMP) and the y-axis represents the mean of the 4 replicates for each sample included in the study. Each data point is displayed with an error bar corresponding to the calculated unexpanded Mu (%). V-shape tolerance limits were applied to each graph at x(UGhent)± 9.6 %, x(UGhent)± 13.6 %, x(UGhent)± 17.4% and x(UGhent)± 26.5 %. These limits represent the MU required for an analytical method to detect a 31.6 % physiological change in 25-(OH)D concentrations after a 10-weeks period with a probability of 99, 95, 90 and 80 %, respectively. The V-shaped zones provide a graphical representation of each method’s ability to track physiologically relevant changes. An assay is considered more clinically useful if its data points and MU bars fall within the narrower zones (e.g., within the±9.6 % or±13.6 % limits). LC-MS/MS methods consistently remained within the most stringent V-shape boundaries, confirming their ability to reliably track changes in vitamin D concentrations. In contrast, several immunoassays exceeded the broader V-shaped limits, indicating that their MU might compromise their ability to detect meaningful fluctuations in 25-(OH)D levels over time. Detailed individual plots for each method are provided in Supplementary Figures S1–S13.

Figure 1:

(A) Combined relative expanded measurement uncertainty (CREMU) across all immunoassays and LC-MS/MS methods. The CREMU (%) is plotted on the y-axis against the UGhent reference measurement procedure (RMP)-assigned 25(OH)D concentrations on the x-axis. Each method’s mean CREMU is shown, along with horizontal reference lines corresponding to the predefined analytical performance specifications (APS). This panel illustrates how methods perform across the concentration range and allows direct comparison between pooled samples (red dots) and single donations (black dots). (B) Combined correlation graphs with measurement uncertainty (MU) and V-shaped acceptance zones for all methods. Each data point represents the mean of four replicates per sample, with the x-axis showing the UGhent RMP-assigned value and the y-axis showing the method-specific mean. Vertical error bars indicate the calculated unexpanded MU (%). Superimposed V-shaped zones delineate thresholds of ±9.6 %, ±13.6 %, ±17.4 %, and ±26.5 % around the RMP value, reflecting the MU required to detect a 31.6 % physiological change in 25(OH)D concentrations over 10 weeks with probabilities of 99 , 95, 90, and 80 %, respectively. Methods with data points and MU bars within narrower zones (e.g., ±9.6 % or±13.6 %) are more likely to detect clinically meaningful changes reliably. LC-MS/MS methods consistently remain within the tightest zones, while some immunoassays exceed the broader limits, indicating potential limitations for longitudinal monitoring. Red dots represent pooled samples; black dots indicate individual single donations. Detailed method-specific plots are provided in Supplementary Figures S1–S13.

An alternative graphical representation of the results is also shown in Figure 1. In this approach, the CREMU (%) is plotted on the y-axis against the x-axis, which represents UGhent RMP. The mean CREMU (%) is displayed for each method, along with horizontal reference lines corresponding to the previously mentioned limits. This graph effectively illustrates how the methods behave across different concentrations and allows for an immediate comparison between pooled samples and single donations, as well as differences between methods.

For instance, Abbott shows a significant increase in CREMU at higher 25-(OH)D concentrations in one out of the two laboratories, likely due to high cross-reactivity with other vitamin D metabolites, such as 24,25-(OH)₂ vitamin D. In the case of Ortho, the discrepancy between the two laboratories is quite pronounced, possibly due to an incorrect calibrator value assignment in one of the lots. Siemens exhibits a notably high CREMU in the low concentration range, particularly for single donations, which may be attributed to a matrix effect – though the pools with similar low concentrations also show elevated CREMU. Lastly, the figure highlights the excellent performance of LC-MS/MS methods, alongside certain immunoassays such as Fujirebio and IDS.

Discussion

In this study, we have used eight single donations of serum and 17 pools made from remnant samples spanning the measurement range to compare the performances of thirteen immunoassays and two distinct LC-MS/MS methods with the RMP from the University of Ghent. The samples have been measured in duplicate on two consecutive days in two different laboratories and we calculated the imprecision (CV%), the bias and the measurement uncertainty for each data point obtained with each method.

The 25-(OH)D imprecision and bias of most of these methods had previously been estimated by Wise et al. according to the VDSP APS (mean bias <|5 %| and mean CV<10 %) in 50 single-donor samples containing 25-(OH)D2 and 25-(OH)D3 or 25-(OH)D3 alone [17]. The results showed that three of the 12 methods they tested failed to pass, mainly due to a mean bias that exceeded the allowable limit of |5 %|. In our study, both LC-MS/MS, Beckman and DiaSorin were able to pass the VDSP criteria whereas this was the case for one lab (or technician) out of two for Fujirebio, Abbott, IDS iSYS and IDS ELISA. All the other methods failed. The reason for failing to pass the VDSP was the bias, which was higher than±5 %. Of note, one of the labs failed with Fujirebio while presenting a CV of 1.7 % and a bias of −5.6 % whereas a lab using DiaSorin succeeded with a CV of 9.9 % and a bias of 4.6 %.

The sample pass rate, which is an interesting metrics proposed by the VDSCP to evaluate the percentage of individual samples that pass the VDSP APS, is also of interest. If the two LC-MS/MS methods and one lab using IDS iSYS presented a high sample pass rate (from 65.2 to 87.0 %), the percentage dropped to around 50 % for the other iSYS system and Fujirebio to 30 % for the tests who succeeded the VDSP (Beckman and DiaSorin) and 20 % or lower for the other methods.

The JCTLM TF-RMSI has recently proposed APS for vitamin D measurement, establishing desirable and minimum thresholds for measurement uncertainty at≤10 % and ≤15 %, respectively [10]. We applied these APS to evaluate the ability of different methods to meet these criteria. Our findings show that slightly more than half of the tested methods met the≤10 % target, while only four methods failed to stay within the≤15 % limit, suggesting that this threshold may be relatively permissive. When applying the IFCC-IOF C-BM approach-assessing a method’s likelihood of detecting a physiological change based on its uncertainty [13] - we found a strong alignment with the TF-RMSI APS. Specifically, methods classified as virtually certain, very likely, or more than likely to detect this change corresponded to those meeting the<10 % APS, whereas those classified as only likely (or failing to detect the change) did not meet this APS.

Another key objective of this study was to assess the feasibility of using MU and novel graphical representations to visually evaluate assay performance. Traditional assessments of analytical performance often rely on separate evaluations of bias and imprecision and do not always provide an intuitive representation of whether an assay can reliably detect physiologically relevant changes in 25-(OH)D concentrations. By integrating MU into graphical representations, the approach we propose allows for a more holistic assessment of method performance, combining analytical errors into a single visualization. One of the major advantages of these graphs is that they provide an immediate and intuitive overview of an assay’s reliability, making it easier to compare different methods at a glance. Instead of requiring a detailed examination of multiple performance metrics, laboratorians and clinicians can quickly assess whether an assay’s MU is low enough to ensure the reliable detection of clinically relevant changes in vitamin D status. While numerical APS thresholds remain essential for standardization and certification, we believe that graphical representation enhances the interpretation of assay reliability by offering a more clinically meaningful perspective. Future standardization efforts should consider integrating both numerical MU assessments and intuitive visual tools to provide a more comprehensive evaluation of method performance.

Our study has however some weaknesses. It was conducted in only two laboratories per method, raising the question of whether these labs are fully representative of broader clinical and research settings. This is particularly relevant given that substantial inter-laboratory differences were observed for some immunoassays, despite their use under routine conditions. Also, while the two LC-MS/MS methods in this study showed comparable performance, in-house developed LC-MS/MS methods can exhibit significant variability in accuracy and precision across different laboratories. Therefore, caution should be exercised when generalizing these findings to other in-house LC-MS/MS methods that were not included in this study. Another limitation is that the commutability of the samples (i.e. the generated pools and the single donations) has not been established. Furthermore, most of the single donations had 25-(OH)D concentrations lower than 20 μg/L, whereas of the majority of the pools exhibited concentrations higher than 20 μg/L. Matrix effects are thus possible even though, visually, we did not observe overt discrepancies in behavior between the pools and the single donations. The nature of the anonymized samples, both single donations and pools (by definition), that were used in this study can also be questioned. The single donations came from apparently healthy subjects from the United-States, and no data were available on the ethnicity of the subjects. It is well known that vitamin D binding protein (VDBP) is highly polymorphic, and that allelic combinations differ in their affinities for vitamin D metabolites, circulating concentrations, and geographical and ethnic distribution [18]. The pools were made of remnant anonymized single serum samples coming from the routine practice of the CHU de Liege and it is very well possible that some of these samples were from patients suffering from CKD or hemodialyzed patients. It is well known that immunoassays for vitamin D can be affected by CKD [19], 20] which might explain differences observed in this study compared to external quality assessment schemes and similar studies using donations from healthy individuals only [21], 22]. Pregnancy, especially in the last trimester, can also affect VDBP concentrations [19], but the CHU de Liège does not have a maternity ward and there is highly improbable that a substantial number of samples coming from pregnant women were included in the pools.

Also, in this study, the estimation of MU was primarily based on a bottom-up approach, using within-laboratory imprecision (uRw) and bias relative to the RMP. Although ISO 20914 recommends a top-down approach incorporating all uncertainty sources along the traceability chain, including the uncertainty associated with reference materials (uref) and calibrator value assignment (ucal), these data could not be easily retrieved. Moreover, no MU estimates are provided by manufacturers in the kit inserts or validation documentation (a limitation we regret and believe should be addressed), making a full top-down estimation impractical. Since the bias component in our MU calculation is derived from comparison against the RMP, it implicitly covers part of the uncertainty linked to the calibration hierarchy. Therefore, for laboratories that do not design measurement procedures from scratch, a bottom-up estimation represents a pragmatic and acceptable approach to assess MU.

The study has also many strengths. The protocol we used is similar to the VDSCP and the study was performed in a total independence from the manufacturers, in a large number of experienced laboratories situated in various countries across Europe. Moreover, each method was run in at least two laboratories or two different technicians (except Affimedix). The results of this study should however not be over interpreted. As already mentioned, the excellent performance of the LC-MS/MS is not per definition representative for all LC-MS/MS methods and the performance of the immunoassays depicted in this study is only relevant for the lots of reagents and calibrators used.

Overall, this study highlights the persistent variability in 25-(OH)D assay performance despite ongoing standardization efforts. While LC-MS/MS methods have demonstrated superior accuracy and lower measurement uncertainty, certain immunoassays also met the established analytical performance specifications. The study underscores the importance of integrating measurement uncertainty into performance evaluations and introduces novel graphical representations to enhance the interpretation of assay reliability. However, limitations such as inter-laboratory variability, potential matrix effects, and the absence of commutability assessments should be considered when interpreting the findings. Future standardization initiatives should further explore the use of uncertainty-based metrics and visualization tools to improve method comparability and ensure clinically meaningful assessments of vitamin D status.