Computer simulation approaches to evaluate the interaction between analytical performance characteristics and clinical (mis)classification: a complementary tool for setting indirect outcome-based analytical performance specifications
-
Hikmet Can Çubukçu
Abstract
Simulation-based approaches for setting indirect outcome-based analytical performance specifications (APS) predominantly involve test repetition through analytical reruns or resampling. These methodologies assess the agreement between original and simulated measurement results, determining the APS corresponding to pre-established performance thresholds. For APS related to imprecision and bias, both analytical performance characteristics (APCs) are typically considered in simulations, whereas for APS regarding measurement uncertainty, bias is excluded in alignment with traceability standards. This paper introduces the “APS Simulator,” a novel tool designed to complement the existing APS Calculator by simulating APS under various scenarios involving imprecision, bias, and measurement uncertainty. The APS Simulator facilitates simulations using distinct analytical rerun and resampling models, enabling laboratory professionals to explore a wide range of performance levels for their specific needs. While the APS Simulator provides valuable insights, significant challenges remain in the broader application of indirect outcome-based APS. These include incorporating sources of diagnostic uncertainty, setting appropriate thresholds for performance metrics, validating clinical decision limits, and accounting for population data characteristics. Addressing these limitations will be essential to enhancing the standardization and robustness of APS determination. The source code and desktop application for the APS Simulator are freely available at https://github.com/hikmetc/APS_Simulator, providing a user-friendly platform for researchers and clinicians to further explore these methodologies.
Acknowledgments
This study drawed inspiration from the research conducted by the EFLM Task Group on Performance Specifications Based on Outcome Studies and the EFLM Working Group on Accreditation and ISO/CEN Standards. Their invaluable contributions will aid laboratory medicine specialists in developing standardized procedures to achieve outcome-based APS.
-
Research ethics: NCHS Research Ethics Review Board (ERB) Approval for publicly available data is available at: https://www.cdc.gov/nchs/nhanes/irba98.htm. Accesed: 1 October 2022.
-
Informed consent: NHANES 2017-March 2020 Pre-Pandemic Brochures and Consent Documents are Available at: https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/documents.aspx?Cycle=2017–2020 Accesed: 1 October 2022.
-
Author contributions: The author has accepted responsibility for the entire content of this manuscript and approved its submission.
-
Use of Large Language Models, AI and Machine Learning Tools: None declared.
-
Conflict of interest: The author states no conflict of interest.
-
Research funding: None declared.
-
Data availability: The source code and desktop application for the APS Simulator are freely available at https://github.com/hikmetc/APS_Simulator. NHANES 2017-March 2020 Pre-pandemic data was used for demonstration, which is available at:https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?cycle=2017–2020. Accesed: 1 October 2022.
References
1. Çubukçu, HC, Vanstapel, F, Thelen, M, van Schrojenstein Lantman, M, Bernabeu-Andreu, FA, Meško Brguljan, P, et al.. APS calculator: a data-driven tool for setting outcome-based analytical performance specifications for measurement uncertainty using specific clinical requirements and population data [eng]. Clin Chem Lab Med 2024;62:597–607. https://doi.org/10.1515/cclm-2023-0740. Epub 20231117 as.Suche in Google Scholar PubMed
2. Chai, JH, Ma, S, Heng, D, Yoong, J, Lim, WY, Toh, SA, et al.. Impact of analytical and biological variations on classification of diabetes using fasting plasma glucose, oral glucose tolerance test and HbA1c [eng]. Sci Rep 2017;7:13721. https://doi.org/10.1038/s41598-017-14172-8. Epub 20171020 as.Suche in Google Scholar PubMed PubMed Central
3. Çubukçu, HC. Test Repetition From the Viewpoint of Biological Variation [en]. Acıbadem Üniv Sağlık Bilim Derg 2022;13:1–5. https://doi.org/10.31067/acusaglik.1023238.Suche in Google Scholar
4. Thelen, MHM, van Schrojenstein Lantman, M. When bias becomes part of imprecision: how to use analytical performance specifications to determine acceptability of lot-lot variation and other sources of possibly unacceptable bias. Clin Chem Lab Med 2024;62:1505–11. https://doi.org/10.1515/cclm-2023-1303.Suche in Google Scholar PubMed
5. International Organization for Standardization. ISO/TS 20914:2019 Medical laboratories – Practical guidance for the estimation of measurement uncertainty. Geneva, Switzerland.Suche in Google Scholar
6. Smith, AF, Shinkins, B, Hall, PS, Hulme, CT, Messenger, MP. Toward a framework for outcome-based analytical performance specifications: a methodology Review of indirect methods for evaluating the impact of measurement uncertainty on clinical outcomes [eng]. Clin Chem 2019;65:1363–74. https://doi.org/10.1373/clinchem.2018.300954. Epub 20190823 as.Suche in Google Scholar PubMed PubMed Central
7. Çubukçu, HC. The trade-off between bias and imprecision: debunking the myth of bias as the primary culprit in patient result misclassification [eng]. Am J Clin Pathol 2024;161:513–6. https://doi.org/10.1093/ajcp/aqad172.Suche in Google Scholar PubMed
8. Loh, TP, Markus, C, Lim, CY. Impact of analytical imprecision and bias on patient classification [eng]. Am J Clin Pathol 2024;161:4–8. https://doi.org/10.1093/ajcp/aqad115.Suche in Google Scholar PubMed
9. Plebani, M. Errors in clinical laboratories or errors in laboratory medicine? [eng]. Clin Chem Lab Med 2006;44:750–9. https://doi.org/10.1515/cclm.2006.123.Suche in Google Scholar PubMed
10. Lippi, G, Simundic, AM. The EFLM strategy for harmonization of the preanalytical phase [eng]. Clin Chem Lab Med 2018;56:1660–6. https://doi.org/10.1515/cclm-2017-0277.Suche in Google Scholar PubMed
11. Can Çubukçu, H, Vanstapel, F, Thelen, M, Bernabeu-Andreu, FA, van Schrojenstein Lantman, M, Brugnoni, D, et al.. Improving the laboratory result release process in the light of ISO 15189:2012 standard [eng]. Clin Chim Acta 2021;522:167–73. https://doi.org/10.1016/j.cca.2021.08.013. Epub 20210818.Suche in Google Scholar PubMed
12. Plebani, M. Harmonizing the post-analytical phase: focus on the laboratory report [eng]. Clin Chem Lab Med 2024;62:1053–62. https://doi.org/10.1515/cclm-2023-1402.Suche in Google Scholar PubMed
13. Barrett, BJ, Fardy, JM. Evaluation of diagnostic tests [eng]. Methods Mol Biol 2021;2249:319–33. https://doi.org/10.1007/978-1-0716-1138-8_18.Suche in Google Scholar PubMed
14. Horvath, AR, Bell, KJL, Ceriotti, F, Jones, GRD, Loh, TP, Lord, S, et al.. Outcome-based analytical performance specifications: current status and future challenges [eng]. Clin Chem Lab Med 2024;62:1474–82. https://doi.org/10.1515/cclm-2024-0125.Suche in Google Scholar PubMed
© 2025 Walter de Gruyter GmbH, Berlin/Boston
Artikel in diesem Heft
- Frontmatter
- Editorials
- The journey to pre-analytical quality
- Manual tilt tube method for prothrombin time: a commentary on contemporary relevance
- Reviews
- From errors to excellence: the pre-analytical journey to improved quality in diagnostics. A scoping review
- Advancements and challenges in high-sensitivity cardiac troponin assays: diagnostic, pathophysiological, and clinical perspectives
- Opinion Paper
- Is it feasible for European laboratories to use SI units in reporting results?
- Perspectives
- What does cancer screening have to do with tomato growing?
- Computer simulation approaches to evaluate the interaction between analytical performance characteristics and clinical (mis)classification: a complementary tool for setting indirect outcome-based analytical performance specifications
- Genetics and Molecular Diagnostics
- Artificial base mismatches-mediated PCR (ABM-PCR) for detecting clinically relevant single-base mutations
- Candidate Reference Measurement Procedures and Materials
- Antiphospholipid IgG Certified Reference Material ERM®-DA477/IFCC: a tool for aPL harmonization?
- General Clinical Chemistry and Laboratory Medicine
- External quality assessment of the manual tilt tube technique for prothrombin time testing: a report from the IFCC-SSC/ISTH Working Group on the Standardization of PT/INR
- Simple steps to achieve harmonisation and standardisation of dried blood spot phenylalanine measurements and facilitate consistent management of patients with phenylketonuria
- Inclusion of pyridoxine dependent epilepsy in expanded newborn screening programs by tandem mass spectrometry: set up of first and second tier tests
- Analytical performance evaluation and optimization of serum 25(OH)D LC-MS/MS measurement
- Towards routine high-throughput analysis of fecal bile acids: validation of an enzymatic cycling method for the quantification of total bile acids in human stool samples on fully automated clinical chemistry analyzers
- Analytical and clinical evaluations of Snibe Maglumi® S100B assay
- Prevalence and detection of citrate contamination in clinical laboratory
- Reference Values and Biological Variations
- Temporal dynamics in laboratory medicine: cosinor analysis and real-world data (RWD) approaches to population chronobiology
- Establishing sex- and age-related reference intervals of serum glial fibrillary acid protein measured by the fully automated lumipulse system
- Hematology and Coagulation
- Performance of the automated digital cell image analyzer UIMD PBIA in white blood cell classification: a comparative study with sysmex DI-60
- Cancer Diagnostics
- Flow-cytometric MRD detection in pediatric T-ALL: a multicenter AIEOP-BFM consensus-based guided standardized approach
- Impact of biological and genetic features of leukemic cells on the occurrence of “shark fins” in the WPC channel scattergrams of the Sysmex XN hematology analyzers in patients with chronic lymphocytic leukemia
- Assessing the clinical applicability of dimensionality reduction algorithms in flow cytometry for hematologic malignancies
- Cardiovascular Diseases
- Evaluation of sex-specific 0-h high-sensitivity cardiac troponin T thresholds for the risk stratification of non-ST-segment elevation myocardial infarction
- Retraction
- The first case of Teclistamab interference with serum electrophoresis and immunofixation
- Letters to the Editor
- Is this quantitative test fit-for-purpose?
- Reply to “Is this quantitative test fit-for-purpose?”
- Short-term biological variation of coagulation and fibrinolytic measurands
- The first case of Teclistamab interference with serum electrophoresis and immunofixation
- Imlifidase: a new interferent on serum protein electrophoresis looking as a rare plasma cell dyscrasia
- Research on the development of image-based Deep Learning (DL) model for serum quality recognition
- Interference of hypertriglyceridemia on total cholesterol assay with the new CHOL2 Abbott method on Architect analyser
- Congress Abstracts
- 10th Annual Meeting of the Austrian Society for Laboratory Medicine and Clinical Chemistry (ÖGLMKC)
Artikel in diesem Heft
- Frontmatter
- Editorials
- The journey to pre-analytical quality
- Manual tilt tube method for prothrombin time: a commentary on contemporary relevance
- Reviews
- From errors to excellence: the pre-analytical journey to improved quality in diagnostics. A scoping review
- Advancements and challenges in high-sensitivity cardiac troponin assays: diagnostic, pathophysiological, and clinical perspectives
- Opinion Paper
- Is it feasible for European laboratories to use SI units in reporting results?
- Perspectives
- What does cancer screening have to do with tomato growing?
- Computer simulation approaches to evaluate the interaction between analytical performance characteristics and clinical (mis)classification: a complementary tool for setting indirect outcome-based analytical performance specifications
- Genetics and Molecular Diagnostics
- Artificial base mismatches-mediated PCR (ABM-PCR) for detecting clinically relevant single-base mutations
- Candidate Reference Measurement Procedures and Materials
- Antiphospholipid IgG Certified Reference Material ERM®-DA477/IFCC: a tool for aPL harmonization?
- General Clinical Chemistry and Laboratory Medicine
- External quality assessment of the manual tilt tube technique for prothrombin time testing: a report from the IFCC-SSC/ISTH Working Group on the Standardization of PT/INR
- Simple steps to achieve harmonisation and standardisation of dried blood spot phenylalanine measurements and facilitate consistent management of patients with phenylketonuria
- Inclusion of pyridoxine dependent epilepsy in expanded newborn screening programs by tandem mass spectrometry: set up of first and second tier tests
- Analytical performance evaluation and optimization of serum 25(OH)D LC-MS/MS measurement
- Towards routine high-throughput analysis of fecal bile acids: validation of an enzymatic cycling method for the quantification of total bile acids in human stool samples on fully automated clinical chemistry analyzers
- Analytical and clinical evaluations of Snibe Maglumi® S100B assay
- Prevalence and detection of citrate contamination in clinical laboratory
- Reference Values and Biological Variations
- Temporal dynamics in laboratory medicine: cosinor analysis and real-world data (RWD) approaches to population chronobiology
- Establishing sex- and age-related reference intervals of serum glial fibrillary acid protein measured by the fully automated lumipulse system
- Hematology and Coagulation
- Performance of the automated digital cell image analyzer UIMD PBIA in white blood cell classification: a comparative study with sysmex DI-60
- Cancer Diagnostics
- Flow-cytometric MRD detection in pediatric T-ALL: a multicenter AIEOP-BFM consensus-based guided standardized approach
- Impact of biological and genetic features of leukemic cells on the occurrence of “shark fins” in the WPC channel scattergrams of the Sysmex XN hematology analyzers in patients with chronic lymphocytic leukemia
- Assessing the clinical applicability of dimensionality reduction algorithms in flow cytometry for hematologic malignancies
- Cardiovascular Diseases
- Evaluation of sex-specific 0-h high-sensitivity cardiac troponin T thresholds for the risk stratification of non-ST-segment elevation myocardial infarction
- Retraction
- The first case of Teclistamab interference with serum electrophoresis and immunofixation
- Letters to the Editor
- Is this quantitative test fit-for-purpose?
- Reply to “Is this quantitative test fit-for-purpose?”
- Short-term biological variation of coagulation and fibrinolytic measurands
- The first case of Teclistamab interference with serum electrophoresis and immunofixation
- Imlifidase: a new interferent on serum protein electrophoresis looking as a rare plasma cell dyscrasia
- Research on the development of image-based Deep Learning (DL) model for serum quality recognition
- Interference of hypertriglyceridemia on total cholesterol assay with the new CHOL2 Abbott method on Architect analyser
- Congress Abstracts
- 10th Annual Meeting of the Austrian Society for Laboratory Medicine and Clinical Chemistry (ÖGLMKC)
