Home Looking beyond linear regression and Bland-Altman plots: a comparison of the clinical performance of 25-hydroxyvitamin D tests
Article
Licensed
Unlicensed Requires Authentication

Looking beyond linear regression and Bland-Altman plots: a comparison of the clinical performance of 25-hydroxyvitamin D tests

  • Gellert Karvaly EMAIL logo , Katalin Mészáros , Krisztián Kovács , Attila Patócs , Zoltán Sipák and Barna Vásárhelyi
Published/Copyright: August 15, 2016
Become an author with De Gruyter Brill
Submit Manuscript Author Information
Clinical Chemistry and Laboratory Medicine (CCLM)
From the journal Clinical Chemistry and Laboratory Medicine (CCLM) Volume 55 Issue 3

Abstract

Background:

The systematic evaluation of the clinical concordance of various 25-hydroxyvitamin D (25OHD) testing methods is presented. The need for this approach is raised by the discrepancies in the analytical performance of the available assays.

Methods:

The analytical and clinical performance of six automated 25OHD assays and an in-house liquid chromatography-tandem mass spectrometry (LC-MS/MS) method was investigated. Leftover serum samples (n=162, SA: n=114) were analyzed and all 21 assay combinations were evaluated. The utility of Cohen’s κ values was assessed by transforming them into minimum percentage agreement (MPA). McNemar’s hypothesis test was employed for testing the symmetry of the disagreeing classification outcomes within each method pair.

Results:

Depending on the assay method, the ratio of results classified as positive (<20 ng/mL) was 13.5%–40.0%. The percentage agreement (PA) was 74.1%–92.6%. Compared to other methods, significantly more hypovitaminosis cases were delivered by DiaSorin Liaison® 25 OH vitamin D Total (DL) and significantly fewer by IDS-iSYS 25-Hydroxy Vitamin DS (II). The strongest clinical concordance was exerted by II vs. LC-MS/MS. The κ-derived MPA showed close similarity to the PA scores. McNemar’s tests confirmed the asymmetry of the disagreement in the classification in 14 method combinations.

Conclusions:

The presented approach allows the prediction of the clinical consequences of a 25OHD method transfer. Differences in the clinical classification of assay results are likely encountered when transferring to a new method, even between assays standardized according to the Vitamin D Standardization Program (VDSP) Reference Method Procedure (RMP).

Keywords: 25-hydroxyvitamin D; hypovitaminosis D; immunoassay; liquid chromatography-tandem mass spectrometry; method comparison; multicriteria evaluation
Corresponding author: Gellert Karvaly, DPharm, PhD, Department of Laboratory Medicine, Semmelweis University, 4 Nagyvárad tér, H-1089 Budapest, Hungary, Phone: +361.2351600/extension 55545, Fax: +361.4591500/extension 55545

  1. Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

  2. Research funding: The study was supported by grant ED_14-1-2014-0002.

  3. Employment or leadership: None declared.

  4. Honorarium: None declared.

  5. Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

References

1. LeBlanc ES, Zakher B, Daeges M, Pappas M, Chou R. Screening for vitamin D deficiency: a systematic review for the U.S. Preventive Services Task Force. Ann Intern Med 2015;162:109–22.10.7326/M14-1659Search in Google Scholar PubMed

2. Cashman KD, Dowling KG, Skrabáková Z, Gonzalez-Gross M, Valtuena J, de Henauw S, et al. Vitamin D deficiency in Europe: pandemic? Am J Clin Nutr 2016;103:1033–44.10.3945/ajcn.115.120873Search in Google Scholar PubMed PubMed Central

3. Moreau E, Bächer S, Mery S, Le Goff C, Piga N, Vogeser M, et al. Performance characteristics of the VIDAS® 25-OH Vitamin D Total assay – comparison with four immunoassays and two liquid chromatography-tandem mass spectrometry methods in a multicentric study. Clin Chem Lab Med 2016;54:45–53.10.1515/cclm-2014-1249Search in Google Scholar PubMed

4. Saleh L, Mueller D, von Eckardstein A. Anaytical and clinical performance of the new Fujirebio 25-OH vitamin D assay, a comparison with liquid chromatography-tandem mass spectrometry (LC-MS/MS) and three other automated assays. Clin Chem Lab Med 2016;54:617–25.10.1515/cclm-2015-0427Search in Google Scholar PubMed

5. LeGoff C, Cavalier E, Souberbielle J-C, González-Antuña A, Delvin E. Measurement of circulating 25-hydroxyvitamin D: a historical review. Practical Lab Med 2015;2:1–14.10.1016/j.plabm.2015.04.001Search in Google Scholar PubMed PubMed Central

6. Robinson S, Canavan M, O’Donnell MJ, Mulkerrin E. Vitamin D supplementation – clarity required regarding treatment regimens and target plasma levels. Q J Med 2014;107:327–9.10.1093/qjmed/hcu018Search in Google Scholar PubMed

7. LeFevre ML. Screening for vitamin D deficiency in adults: U.S: Preventive Services Task Force recommendation statement. Ann Intern Med 2015;162:133–40.10.7326/M14-2450Search in Google Scholar PubMed

8. Cavalier E, Lukas P, Crine Y, Peeters S, Carlisi A, Le Goff C, et al. Evaluation of automated immunoassays for 25(OH)-vitamin D determination in different critical populations before and after standardization of the assays. Clin Chim Acta 2014;431:60–5.10.1016/j.cca.2014.01.026Search in Google Scholar PubMed

9. Braun A, Chang D, Mahadevappa K, Gibbons FK, Liu Y, Giovaucci E, et al. Association of low serum 25-hydroxyvitamin D levels and mortality in the critically ill. Crit Care Med 2011;39:671–7.10.1097/CCM.0b013e318206ccdfSearch in Google Scholar PubMed PubMed Central

10. Mansbach JM, Ginde AA, Camargo CA. Serum 25-hydroxyvitamin D levels among US children aged 1 to 11 years: do children need more vitamin D? Pediatrics 2009;124:1404–10.10.1542/peds.2008-2041Search in Google Scholar PubMed PubMed Central

11. Earthman CP, Beckman LM, Masodkar K, Sibley SD. The link between obesity and low circulating 25-hydroxyvitamin D concentrations: considerations and implications. Int J Obes (Lond) 2012;36:387–96.10.1038/ijo.2011.119Search in Google Scholar PubMed

12. Wyness SP, Straseski JA. Performance characteristics of six automated 25-hydroxyvitamin D assays: mind your 3s and 2s. Clin Biochem 2015;48:1089–96.10.1016/j.clinbiochem.2015.08.005Search in Google Scholar PubMed

13. Thienpont LM, Stepman HC, Vesper HW. Standardization of measurements of 25-hydroxyvitamin D3 and D2. Scand J Clin Lab Invest Suppl 2012;243:41–9.Search in Google Scholar

14. Binkley N, Sempos CT. Standardizing vitamin D assays: the way forward. J Bone Mineral Res 2014;29:1709–14.10.1002/jbmr.2252Search in Google Scholar PubMed PubMed Central

15. Carter GD. Accuracy of 25-hydroxyvitamin D assays: confronting the issues. Current Drug Targets 2011;12:19–28.10.2174/138945011793591608Search in Google Scholar PubMed

16. Janssen MJ, Wielders JP, Bekker CC, Boesten LS, Buijs MM, Heijboer AC, et al. Multicenter comparison study of current methods to measure 25-hydroxyvitamin D in serum. Steroids 2012;77:1366–72.10.1016/j.steroids.2012.07.013Search in Google Scholar PubMed

17. Bedner M, Lippa KA, Tai SS. An assessment of 25-hydroxyvitamin D measurements in comparability studies conducted by the Vitamin D Metabolites Quality Assurance Program. Clin Chim Acta 2013;426:6–11.10.1016/j.cca.2013.08.012Search in Google Scholar PubMed PubMed Central

18. Simpson CA, Cusano AM, Bihuniak J, Walker J, Insogna KL. Effect of 25(OH) vitamin D reference method procedure (RMP) alignment on clinical measurements obtained with the IDS-iSYS chemiluminescent-based automated analyzer. J Steroid Biochem Mol Biol 2015;148:41–6.10.1016/j.jsbmb.2014.09.013Search in Google Scholar PubMed

19. van den Ouweland JM, Beijers AM, Demacker NM, van Daal H. Measurement of 25-OH-vitamin D in human serum using liquid chromatography tandem-mass spectrometry with comparison to radioimmunoassay and automated immunoassay. J Chromatogr B 2010;878:1163–8.10.1016/j.jchromb.2010.03.035Search in Google Scholar PubMed

20. de Koning L, Al-Turkmani MR, Berg AH, Shkreta A, Law T, Kelogg M. Variation in clinical vitmin D status by DiaSorin Liaison and LC-MS/MS in the presence of elevated 25-OH vitamin D2. Clin Chim Acta 2013;415:54–8.10.1016/j.cca.2012.09.002Search in Google Scholar PubMed

21. Koivula MK, Matinlassi N, Laitinen P, Risteli J. Four automated 25-OH total vitamin D immunoassays and commercial liquid chromatography tandem-mass spectrometry in Finnish population. Clin Lab 2013;59:397–405.10.7754/Clin.Lab.2012.120527Search in Google Scholar PubMed

22. Cavalier E, Rousselle O, Ferrante N, Carlisi A, Le Goff C, Souberbielle JC. Technical and clinical evaluation of the VITROS® Immunodiagnostic Products 25-OH Vitamin D Total assay – comparison with marketed automated immunoassays and a liquid chromatography-tandem mass spectrometry method. Clin Chem Lab Med 2013;51:1983–9.10.1515/cclm-2013-0138Search in Google Scholar PubMed

23. Moon HW, Cho JH, Hur M, Song J, Oh GY, Park CM, et al. Comparison of four current 25-hydroxyvitamin D assays. Clin Biochem 2012;45:326.10.1016/j.clinbiochem.2011.12.025Search in Google Scholar PubMed

24. Holmes EW, Garbincius J, McKenna KM. Analytical variability among methods for the measurement of 25-hydroxyvitamin D. Am J Clin Pathol 2013;140:550–60.10.1309/AJCPU2SKW1TFKSWYSearch in Google Scholar PubMed

25. Lippi G, Salvagno GL, Fortunato A, Dipalo M, Aloe R, Da Rin G, et al. Multicenter comparison of seven 25OH vitamin D automated immunoassays. J Med Biochem 2015;34:344–50.10.2478/jomb-2014-0054Search in Google Scholar PubMed PubMed Central

26. R Development Core Team (2008). R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. Available at: http://www.R-project.org.Search in Google Scholar

27. Stöckl D, Dewitte K, Thienpont LM. Validity of linear regression in method comparison studies: is it limited by the statistical model or the quality of the analytical input data? Clin Chem 1998;44:2340–6.10.1093/clinchem/44.11.2340Search in Google Scholar

28. Freeman J, Wilson K, Spears R, Shalhoub V, Sibley P. Performance evaluation of four 25-hydroxyvitamin D assays to measure 25-hydroxyvitamin D2. Clin Biochem 2015;48:1097–104.10.1016/j.clinbiochem.2015.05.021Search in Google Scholar PubMed

29. Wielders JP, Carter GF, Eberl H, Moris G, Roth HJ, Vogl C. Automated competitive protein-binding assay for total 25-OH vitamin D, multicenter evaluation and practical performance. J Clin Lab Anal 2015;29:451–61.10.1002/jcla.21793Search in Google Scholar PubMed PubMed Central

30. Li L, Zeng Q, Yuan J, Xie Z. Performance evaluation of two immunoassays for 25-hydroxyvitamin D. J Clin Biochem Nutr 2016;58:186–92.10.3164/jcbn.15-61Search in Google Scholar PubMed PubMed Central

31. Banerjee M. Beyond kappa: a review of interrater agreement measures. Canadian J Stat 1999;27:3–23.10.2307/3315487Search in Google Scholar

32. Nofuentes JA, del Castillo JD. Comparing two binary diagnostic tests in the presence of verification bias. Comput Stat Data Anal 2006;50:1551–64.10.1016/j.csda.2005.01.006Search in Google Scholar

33. Sim J, Wright CC. The Kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther 2005;85:257–68.10.1093/ptj/85.3.257Search in Google Scholar

34. 510(k) Substantial Equivalence Determination Decision Summary k142373. U.S. Food and Drug Administration, 22 December 2014.Search in Google Scholar

35. 510(k) Substantial Equivalence Determination Decision Summary k140554. U.S. Food and Drug Administration, 19 December 2014.Search in Google Scholar

36. 510(k) Substantial Equivalence Determination Decision Summary k133156. U.S. Food and Drug Administration, 24 July 2014.Search in Google Scholar

Supplemental Material:

The online version of this article (DOI: 10.1515/cclm-2016-0536) offers supplementary material, available to authorized users.

Received: 2016-6-19
Accepted: 2016-7-5
Published Online: 2016-8-15
Published in Print: 2017-3-1

©2017 Walter de Gruyter GmbH, Berlin/Boston

Articles in the same Issue

  1. Frontmatter
  2. Editorials
  3. Targeting errors in microbiology: the case of the Gram stain
  4. Time for a holistic approach and standardization education in laboratory medicine
  5. Reviews
  6. Serum uric acid levels and risk of prehypertension: a meta-analysis
  7. Lactic acidosis: an update
  8. Mini Review
  9. Progress and impact of enzyme measurement standardization
  10. Opinion Paper
  11. Critical comments to a recent EFLM recommendation for the review of reference intervals
  12. IFCC Paper
  13. Quality Indicators in Laboratory Medicine: the status of the progress of IFCC Working Group “Laboratory Errors and Patient Safety” project
  14. Genetics and Molecular Diagnostics
  15. Evaluation and comparison of three assays for molecular detection of spinal muscular atrophy
  16. General Clinical Chemistry and Laboratory Medicine
  17. Risk analysis of the preanalytical process based on quality indicators data
  18. Analytical and clinical validation of the new Abbot Architect 25(OH)D assay: fit for purpose?
  19. Looking beyond linear regression and Bland-Altman plots: a comparison of the clinical performance of 25-hydroxyvitamin D tests
  20. Next-generation osmotic gradient ektacytometry for the diagnosis of hereditary spherocytosis: interlaboratory method validation and experience
  21. Diagnosis of sphingolipidoses: a new simultaneous measurement of lysosphingolipids by LC-MS/MS
  22. Monitoring nicotine intake from e-cigarettes: measurement of parent drug and metabolites in oral fluid and plasma
  23. Development of a rapid and quantitative lateral flow assay for the simultaneous measurement of serum κ and λ immunoglobulin free light chains (FLC): inception of a new near-patient FLC screening tool
  24. A real-world evidence-based approach to laboratory reorganization using e-Valuate benchmarking data
  25. Cancer Diagnostics
  26. Utility of proGRP as a tumor marker in the medullary thyroid carcinoma
  27. Cardiovascular Diseases
  28. Can non-cholesterol sterols and lipoprotein subclasses distribution predict different patterns of cholesterol metabolism and statin therapy response?
  29. Infectious Diseases
  30. Improving Gram stain proficiency in hospital and satellite laboratories that do not have microbiology
  31. Diabetes
  32. Volumetric absorptive microsampling at home as an alternative tool for the monitoring of HbA1c in diabetes patients
  33. Corrigendum
  34. EFLM Recommendation
  35. Corrigendum to: Recommendation for the review of biological reference intervals in medical laboratories
  36. Letters to the Editor
  37. Evaluation of the trueness of serum alkaline phosphatase measurement in a group of Italian laboratories
  38. Innovative software for recording preanalytical errors in accord with the IFCC quality indicators
  39. Mixing studies for abnormal coagulation screen – the current trend
  40. Identification of 5-fluorocytosine as a new interfering compound in serum capillary zone electrophoresis
  41. How to define reference intervals to rule in healthy individuals for clinical trials?
  42. Evaluation of the performance of INDEXOR® in the archive unit of a clinical laboratory: a step to Lean laboratory
  43. Evaluation of an automated chemiluminescent immunoassay for salivary cortisol measurement. Utility in the diagnosis of Cushing’s syndrome
  44. Analysis of hemolysis, icterus and lipemia in arterial blood gas specimens
  45. Comparison of three routine insulin immunoassays: implications for assessment of insulin sensitivity and response
https://doi.org/10.1515/cclm-2016-0536
Keywords for this article
25-hydroxyvitamin D; hypovitaminosis D; immunoassay; liquid chromatography-tandem mass spectrometry; method comparison; multicriteria evaluation

Articles in the same Issue

  1. Frontmatter
  2. Editorials
  3. Targeting errors in microbiology: the case of the Gram stain
  4. Time for a holistic approach and standardization education in laboratory medicine
  5. Reviews
  6. Serum uric acid levels and risk of prehypertension: a meta-analysis
  7. Lactic acidosis: an update
  8. Mini Review
  9. Progress and impact of enzyme measurement standardization
  10. Opinion Paper
  11. Critical comments to a recent EFLM recommendation for the review of reference intervals
  12. IFCC Paper
  13. Quality Indicators in Laboratory Medicine: the status of the progress of IFCC Working Group “Laboratory Errors and Patient Safety” project
  14. Genetics and Molecular Diagnostics
  15. Evaluation and comparison of three assays for molecular detection of spinal muscular atrophy
  16. General Clinical Chemistry and Laboratory Medicine
  17. Risk analysis of the preanalytical process based on quality indicators data
  18. Analytical and clinical validation of the new Abbot Architect 25(OH)D assay: fit for purpose?
  19. Looking beyond linear regression and Bland-Altman plots: a comparison of the clinical performance of 25-hydroxyvitamin D tests
  20. Next-generation osmotic gradient ektacytometry for the diagnosis of hereditary spherocytosis: interlaboratory method validation and experience
  21. Diagnosis of sphingolipidoses: a new simultaneous measurement of lysosphingolipids by LC-MS/MS
  22. Monitoring nicotine intake from e-cigarettes: measurement of parent drug and metabolites in oral fluid and plasma
  23. Development of a rapid and quantitative lateral flow assay for the simultaneous measurement of serum κ and λ immunoglobulin free light chains (FLC): inception of a new near-patient FLC screening tool
  24. A real-world evidence-based approach to laboratory reorganization using e-Valuate benchmarking data
  25. Cancer Diagnostics
  26. Utility of proGRP as a tumor marker in the medullary thyroid carcinoma
  27. Cardiovascular Diseases
  28. Can non-cholesterol sterols and lipoprotein subclasses distribution predict different patterns of cholesterol metabolism and statin therapy response?
  29. Infectious Diseases
  30. Improving Gram stain proficiency in hospital and satellite laboratories that do not have microbiology
  31. Diabetes
  32. Volumetric absorptive microsampling at home as an alternative tool for the monitoring of HbA1c in diabetes patients
  33. Corrigendum
  34. EFLM Recommendation
  35. Corrigendum to: Recommendation for the review of biological reference intervals in medical laboratories
  36. Letters to the Editor
  37. Evaluation of the trueness of serum alkaline phosphatase measurement in a group of Italian laboratories
  38. Innovative software for recording preanalytical errors in accord with the IFCC quality indicators
  39. Mixing studies for abnormal coagulation screen – the current trend
  40. Identification of 5-fluorocytosine as a new interfering compound in serum capillary zone electrophoresis
  41. How to define reference intervals to rule in healthy individuals for clinical trials?
  42. Evaluation of the performance of INDEXOR® in the archive unit of a clinical laboratory: a step to Lean laboratory
  43. Evaluation of an automated chemiluminescent immunoassay for salivary cortisol measurement. Utility in the diagnosis of Cushing’s syndrome
  44. Analysis of hemolysis, icterus and lipemia in arterial blood gas specimens
  45. Comparison of three routine insulin immunoassays: implications for assessment of insulin sensitivity and response
Sign up now to receive a 20% welcome discount
Institutional Access
How does access work?
Have an idea on how to improve our website?
Please write us.
© 2025 De Gruyter Brill
Downloaded on 23.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/cclm-2016-0536/html?lang=en
Scroll to top button