Home Error simulation modeling to assess the effects of bias and precision on bilirubin measurements used to screen for neonatal hyperbilirubinemia
Article
Licensed
Unlicensed Requires Authentication

Error simulation modeling to assess the effects of bias and precision on bilirubin measurements used to screen for neonatal hyperbilirubinemia

  • Jose C. Jara Aguirre , Andrew P. Norgan , Walter J. Cook and Brad S. Karon EMAIL logo
Published/Copyright: January 20, 2021
Become an author with De Gruyter Brill
Submit Manuscript Author Information
Clinical Chemistry and Laboratory Medicine (CCLM)
From the journal Clinical Chemistry and Laboratory Medicine (CCLM) Volume 59 Issue 6

Abstract

Objectives

Error simulation models have been used to understand the relationship between analytical performance and clinical outcomes. We developed an error simulation model to understand the effects of method bias and precision on misclassification rate for neonatal hyperbilirubinemia using an age-adjusted risk assessment tool.

Methods

For each of 176 measured total bilirubin (TSBM) values, 10,000 simulated total bilirubin (TBS) values were generated at each combination of bias and precision conditions for coefficient of variation (CV) between 1 and 15%, and for biases between −51.3 μmol/L and 51.3 μmol/L (−3 and 3 mg/dL) fixed bias. TBS values were analyzed to determine if they were in the same risk zone as the TSBM value. We then calculated sensitivity and specificity for prediction of ≥75th percentile for postnatal age values as a function of assay bias and precision, and determined the rate of critical errors (≥95th percentile for age TSBM with <75th percentile TBS).

Results

A sensitivity >95% for predicting ≥75th percentile bilirubin values was observed when there is a positive fixed bias of greater than 17.1 μmol/L (1.0 mg/dL) and CV is maintained ≤10%. A specificity >70% for predicting <75th percentile bilirubin values was observed when positive systematic bias was 17.1 μmol/L (1 mg/dL) or less at CV ≤ 10%. Critical errors did not occur with a frequency >0.2% until negative bias was −17.1 μmol/L (−1 mg/dL) or lower.

Conclusions

A positive systematic bias of 17.1 μmol/L (1 mg/dL) may be optimal for balancing sensitivity and specificity for predicting ≥75th percentile TSB values. Negative systematic bias should be avoided to allow detection of high risk infants and avoid critical classification errors.

Keywords: error simulation model; neonatal hyperbilirubinemia; serum bilirubin; transcutaneous bilirubin
Corresponding author: Brad S. Karon, MD, PhD, Department of Laboratory Medicine and Pathology, Mayo Clinic, 200 First Street SW., Rochester, MN 55905, USA, Phone: +1507- 538- 4921, Fax: +1507- 538- 7060, E-mail:

  1. Research funding: None declared.

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Ethical approval: The samples were collected as part of a separate study previously approved by the Mayo Clinic Institutional Review Board.

References

1. Maisels, MJ, Bhutani, VK, Bogen, D, Newman, TB, Stark, AR, Watchko, JF. Hyperbilirubinemia in the newborn infant > or =35 weeks’ gestation: an update with clarifications. Pediatrics 2009;124:1193–8. https://doi.org/10.1542/peds.2009-0329.10.1542/peds.2009-0329Search in Google Scholar PubMed

2. Guidelines for detection, management and prevention of hyperbilirubinemia in term and late preterm newborn infants (35 or more weeks’ gestation) - Summary. Paediatr Child Health 2007;12:401–18.10.1093/pch/12.5.401Search in Google Scholar PubMed PubMed Central

3. American Academy of Pediatrics Subcommittee on Hyberbilirubinemia. Management of hyperbilirubinemia in the newborn infant 35 or more weeks of gestation. Pediatrics 2004;114:297–316.10.1542/peds.114.1.297Search in Google Scholar PubMed

4. Rennie, J, Burman-Roy, S, Murphy, MS, Guideline Development, G. Neonatal jaundice: summary of NICE guidance. BMJ 2010;340:c2409. https://doi.org/10.1136/bmj.c2409.Search in Google Scholar PubMed

5. Bhutani, VK, Johnson, L, Sivieri, EM. Predictive ability of a predischarge hour-specific serum bilirubin for subsequent significant hyperbilirubinemia in healthy term and near-term newborns. Pediatrics 1999;103:6–14. https://doi.org/10.1542/peds.103.1.6.Search in Google Scholar PubMed

6. Karon, BS, Teske, A, Santrach, PJ, Cook, WJ. Evaluation of the BiliChek noninvasive bilirubin analyzer for prediction of serum bilirubin and risk of hyperbilirubinemia. Am J Clin Pathol 2008;130:976–82. https://doi.org/10.1309/ajcprx1e3nwcxhmz.Search in Google Scholar PubMed

7. Kolman, KB, Mathieson, KM, Frias, C. A comparison of transcutaneous and total serum bilirubin in newborn Hispanic infants at 35 or more weeks of gestation. J Am Board Fam Med 2007;20:266–71. https://doi.org/10.3122/jabfm.2007.03.060172.Search in Google Scholar PubMed

8. Taylor, JA, Burgos, AE, Flaherman, V, Chung, EK, Simpson, EA, Goyal, NK, et al.. Utility of decision rules for transcutaneous bilirubin measurements. Pediatrics 2016;137:e20153032. https://doi.org/10.1542/peds.2015-3032.Search in Google Scholar PubMed PubMed Central

9. Boyd, JC, Bruns, DE. Quality specifications for glucose meters: assessment by simulation modeling of errors in insulin dose. Clin Chem 2001;47:209–14. https://doi.org/10.1093/clinchem/47.2.209.Search in Google Scholar

10. Karon, BS, Boyd, JC, Klee, GG. Glucose meter performance criteria for tight glycemic control estimated by simulation modeling. Clin Chem 2010;56:1091–7. https://doi.org/10.1373/clinchem.2010.145367.Search in Google Scholar PubMed

11. Karon, BS, Boyd, JC, Klee, GG. Empiric validation of simulation models for estimating glucose meter performance criteria for moderate levels of glycemic control. Diabetes Technol Therapeut 2013;15:996–1003. https://doi.org/10.1089/dia.2013.0086.Search in Google Scholar

12. Lyon, AW, Kavsak, PA, Lyon, OA, Worster, A, Lyon, ME. Simulation models of misclassification error for single thresholds of high-sensitivity cardiac troponin I due to assay bias and imprecision. Clin Chem 2017;63:585–92. https://doi.org/10.1373/clinchem.2016.265058.Search in Google Scholar

13. EBMcalc: An educational medical reference. Neonatal hyperbilirubinemia assessment. https://ebmcalccom/NewbornHyperbilihtm [Accessed 24 Dec 20].Search in Google Scholar

14. Bromiker, R, Bin-Nun, A, Schimmel, MS, Hammerman, C, Kaplan, M. Neonatal hyperbilirubinemia in the low-intermediate-risk category on the bilirubin nomogram. Pediatrics 2012;130:e470–5. https://doi.org/10.1542/peds.2012-0005.Search in Google Scholar

15. Dai, J, Parry, DM, Krahn, J. Transcutaneous bilirubinometry: its role in the assessment of neonatal jaundice. Clin Biochem 1997;30:1–9. https://doi.org/10.1016/s0009-9120(96)00131-2.Search in Google Scholar

16. Sheridan-Pereira, M, Gorman, W. Transcutaneous bilirubinometry: an evaluation. Arch Dis Child 1982;57:708–10. https://doi.org/10.1136/adc.57.9.708.Search in Google Scholar PubMed PubMed Central

17. Rubaltelli, FF, Gourley, GR, Loskamp, N, Modi, N, Roth-Kleiner, M, Sender, A, et al.. Transcutaneous bilirubin measurement: a multicenter evaluation of a new device. Pediatrics 2001;107:1264–71. https://doi.org/10.1542/peds.107.6.1264.Search in Google Scholar PubMed

18. Taylor, JA, Burgos, AE, Flaherman, V, Chung, EK, Simpson, EA, Goyal, NK, et al.. Discrepancies between transcutaneous and serum bilirubin measurements. Pediatrics 2015;135:224–31. https://doi.org/10.1542/peds.2014-1919.Search in Google Scholar PubMed PubMed Central

19. Jara Aguirre, JC, Wockenfus, AM, Krekelberg, BJ, Fine, KL, Cook, WJ, Karon, BS. Implementation of a reformulated Roche(R) bilirubin Gen.3 reagent did not affect the relationship between BiliChek transcutaneous and Roche total serum bilirubin. Clin Biochem 2019;63:148–51. https://doi.org/10.1016/j.clinbiochem.2018.10.001.Search in Google Scholar PubMed

20. Lo, SF, Doumas, BT, Ashwood, ER, College of American Pathologists. Bilirubin proficiency testing using specimens containing unconjugated bilirubin and human serum: results of a College of American Pathologists study. Arch Pathol Lab Med 2004;128:1219–23.10.5858/2004-128-1219-BPTUSCSearch in Google Scholar PubMed

21. Lo, SF, Doumas, BT. The status of bilirubin measurements in U.S. laboratories: why is accuracy elusive? Semin Perinatol 2011;35:141–7. https://doi.org/10.1053/j.semperi.2011.02.008.Search in Google Scholar PubMed

22. Gu, D, Wang, Y, Ren, B, Wang, L, Zhang, K, Yuan, Y. Comparison of three routine methods for the measurement of serum bilirubin in a China laboratory. Clin Lab 2018;64:1485–90. https://doi.org/10.7754/clin.lab.2018.180333.Search in Google Scholar PubMed

23. Lano, IM, Lyon, AW, Wang, L, Ruskin, R, Lyon, ME. Comparative evaluation of neonatal bilirubin using Radiometer whole blood co-oximetry and plasma bilirubin methods from Roche Diagnostics and Ortho Clinical Diagnostics. Clin Biochem 2018;53:88–92. https://doi.org/10.1016/j.clinbiochem.2017.12.009.Search in Google Scholar PubMed

24. Kuzniewicz, MW, Greene, DN, Walsh, EM, McCulloch, CE, Newman, TB. Association between laboratory calibration of a serum bilirubin assay, neonatal bilirubin levels, and phototherapy use. JAMA Pediatr 2016;170:557–61. https://doi.org/10.1001/jamapediatrics.2015.4944.Search in Google Scholar PubMed

25. Medicare. Medicaid and CLIA programs; regulations implementing the clinical laboratory improvement Amendments of 1988 (CLIA). Fed Regist 1992;28:7002–186.Search in Google Scholar

26. Karon, BS, Wockenfus, AM, Hartung, KJ, Scott, RJ, Carter, SD, Jaffe, AS. Comparing analytical outliers and the percent of emergency department patients with results above the 99th percentile upper reference limit for 2 conventional and one high sensitivity troponin assay. Clin Biochem 2018;53:104–9. https://doi.org/10.1016/j.clinbiochem.2018.01.001.Search in Google Scholar PubMed

Supplementary Material

The online version of this article offers supplementary material (https://doi.org/10.1515/cclm-2020-1640).

Received: 2020-11-02
Accepted: 2020-12-28
Published Online: 2021-01-20
Published in Print: 2021-05-26

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Articles in the same Issue

  1. Frontmatter
  2. Editorial
  3. COVID-19: which lessons have we learned?
  4. Review
  5. Global FT4 immunoassay standardization: an expert opinion review
  6. Mini Review
  7. Mechanism of bilirubin elimination in urine: insights and prospects for neonatal jaundice
  8. Opinion Paper
  9. Laboratory medicine in the COVID-19 era: six lessons for the future
  10. EFLM Paper
  11. How to meet ISO15189:2012 pre-analytical requirements in clinical laboratories? A consensus document by the EFLM WG-PRE
  12. General Clinical Chemistry and Laboratory Medicine
  13. External quality assessment of M-protein diagnostics: a realistic impression of the accuracy and precision of M-protein quantification
  14. Error simulation modeling to assess the effects of bias and precision on bilirubin measurements used to screen for neonatal hyperbilirubinemia
  15. NT-proBNP levels in preeclampsia, intrauterine growth restriction as well as in the prediction on an imminent delivery
  16. Serum N-glycan fingerprint nomogram predicts liver fibrosis: a multicenter study
  17. Hematology and Coagulation
  18. Performance of digital morphology analyzer Vision Pro on white blood cell differentials
  19. Cardiovascular Diseases
  20. Prognostic implication of elevated cardiac troponin I in patients visiting emergency department without diagnosis of coronary artery disease
  21. Relationships between renal function variations and relative changes in cardiac troponin T concentrations based on quantile generalized additive models (qgam)
  22. Diabetes
  23. Association of hemoglobin H (HbH) disease with hemoglobin A1c and glycated albumin in diabetic and non-diabetic patients
  24. Comparative study of i-SENS glucometers in neonates using capillary blood samples
  25. Infectious Diseases
  26. Evaluation of four commercial, fully automated SARS-CoV-2 antibody tests suggests a revision of the Siemens SARS-CoV-2 IgG assay
  27. Vitamin-D levels and intensive care unit outcomes of a cohort of critically ill COVID-19 patients
  28. Does mid-regional pro-adrenomedullin (MR-proADM) improve the sequential organ failure assessment-score (SOFA score) for mortality-prediction in patients with acute infections? Results of a prospective observational study
  29. Letters to the Editors
  30. Global FT4 immunoassay standardization. Response to: Kratzsch J et al. Global FT4 immunoassay standardization: an expert opinion review
  31. Free-thyroxine standardization: waiting for Godot while well serving our patients today
  32. Prevalence of SARS-CoV-2 antibodies in health care personnel of two acute care hospitals in Linz, Austria
  33. Pediatric evaluation of clinical specificity and sensitivity of SARS-CoV-2 IgG and IgM serology assays
  34. Prevention and control of COVID-19 in the penitentiary of Florence
  35. Pooling for SARS-CoV-2-testing: comparison of three commercially available RT-qPCR kits in an experimental approach
  36. Very high SARS-CoV-2 load at the emergency department presentation strongly predicts the risk of admission to the intensive care unit and death
  37. Next-generation sequencing and RT-PCR to identify a 32-day SARS-CoV-2 carrier
  38. 25-Hydroxyvitamin D concentrations in COVID-19 patients hospitalized in intensive care unit during the first wave and the second wave of the pandemic
  39. Laboratory findings in a child with SARS-CoV-2 (COVID-19) multisystem inflammatory syndrome
  40. Discrepant cardiac troponin results in a young woman
  41. Response to: towards the rational utilization of SARS-CoV-2 serological tests in clinical practice
  42. Congress Abstract
  43. 12th National Scientific Congress SPML,29–31 October 2020, Online, Portugal
https://doi.org/10.1515/cclm-2020-1640
Keywords for this article
error simulation model; neonatal hyperbilirubinemia; serum bilirubin; transcutaneous bilirubin

Articles in the same Issue

  1. Frontmatter
  2. Editorial
  3. COVID-19: which lessons have we learned?
  4. Review
  5. Global FT4 immunoassay standardization: an expert opinion review
  6. Mini Review
  7. Mechanism of bilirubin elimination in urine: insights and prospects for neonatal jaundice
  8. Opinion Paper
  9. Laboratory medicine in the COVID-19 era: six lessons for the future
  10. EFLM Paper
  11. How to meet ISO15189:2012 pre-analytical requirements in clinical laboratories? A consensus document by the EFLM WG-PRE
  12. General Clinical Chemistry and Laboratory Medicine
  13. External quality assessment of M-protein diagnostics: a realistic impression of the accuracy and precision of M-protein quantification
  14. Error simulation modeling to assess the effects of bias and precision on bilirubin measurements used to screen for neonatal hyperbilirubinemia
  15. NT-proBNP levels in preeclampsia, intrauterine growth restriction as well as in the prediction on an imminent delivery
  16. Serum N-glycan fingerprint nomogram predicts liver fibrosis: a multicenter study
  17. Hematology and Coagulation
  18. Performance of digital morphology analyzer Vision Pro on white blood cell differentials
  19. Cardiovascular Diseases
  20. Prognostic implication of elevated cardiac troponin I in patients visiting emergency department without diagnosis of coronary artery disease
  21. Relationships between renal function variations and relative changes in cardiac troponin T concentrations based on quantile generalized additive models (qgam)
  22. Diabetes
  23. Association of hemoglobin H (HbH) disease with hemoglobin A1c and glycated albumin in diabetic and non-diabetic patients
  24. Comparative study of i-SENS glucometers in neonates using capillary blood samples
  25. Infectious Diseases
  26. Evaluation of four commercial, fully automated SARS-CoV-2 antibody tests suggests a revision of the Siemens SARS-CoV-2 IgG assay
  27. Vitamin-D levels and intensive care unit outcomes of a cohort of critically ill COVID-19 patients
  28. Does mid-regional pro-adrenomedullin (MR-proADM) improve the sequential organ failure assessment-score (SOFA score) for mortality-prediction in patients with acute infections? Results of a prospective observational study
  29. Letters to the Editors
  30. Global FT4 immunoassay standardization. Response to: Kratzsch J et al. Global FT4 immunoassay standardization: an expert opinion review
  31. Free-thyroxine standardization: waiting for Godot while well serving our patients today
  32. Prevalence of SARS-CoV-2 antibodies in health care personnel of two acute care hospitals in Linz, Austria
  33. Pediatric evaluation of clinical specificity and sensitivity of SARS-CoV-2 IgG and IgM serology assays
  34. Prevention and control of COVID-19 in the penitentiary of Florence
  35. Pooling for SARS-CoV-2-testing: comparison of three commercially available RT-qPCR kits in an experimental approach
  36. Very high SARS-CoV-2 load at the emergency department presentation strongly predicts the risk of admission to the intensive care unit and death
  37. Next-generation sequencing and RT-PCR to identify a 32-day SARS-CoV-2 carrier
  38. 25-Hydroxyvitamin D concentrations in COVID-19 patients hospitalized in intensive care unit during the first wave and the second wave of the pandemic
  39. Laboratory findings in a child with SARS-CoV-2 (COVID-19) multisystem inflammatory syndrome
  40. Discrepant cardiac troponin results in a young woman
  41. Response to: towards the rational utilization of SARS-CoV-2 serological tests in clinical practice
  42. Congress Abstract
  43. 12th National Scientific Congress SPML,29–31 October 2020, Online, Portugal
Sign up now to receive a 20% welcome discount
Institutional Access
How does access work?
Have an idea on how to improve our website?
Please write us.
© 2025 De Gruyter Brill
Downloaded on 24.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/cclm-2020-1640/html
Scroll to top button