Sample size and rejection limits for detecting reagent lot variability: analysis of the applicability of the Clinical and Laboratory Standards Institute (CLSI) EP26-A protocol to real-world clinical chemistry data
-
Sollip Kim
Abstract
Objectives
To maintain the consistency of laboratory test results, between-reagent lot variation should be verified before using new reagent lots in clinical laboratory. Although the Clinical and Laboratory Standards Institute (CLSI) document EP26-A deals with this issue, evaluation of reagent lot-to-lot difference is challenging in reality. We aim to investigate a practical way for determining between-reagent lot variation using real-world data in clinical chemistry.
Methods
The CLSI EP26-A protocol was applied to 83 chemistry tests in three clinical labs. Three criteria were used to define the critical difference (CD) of each test as follows: reference change value and total allowable error, which are based on biological variation, and acceptable limits by external quality assurance agencies. The sample size and rejection limits that could detect CD between-reagent lots were determined.
Results
For more than half of chemistry tests, reagent lot-to-lot differences could be evaluated using only one patient sample per decision level. In many cases, the rejection limit that could detect reagent lot-to-lot difference with ≥90% probability was 0.6 times CD. However, the sample size and rejection limits vary depending on how the CD is defined. In some cases, impractical sample size or rejection limits were obtained. In some cases, information on sample size and rejection limit that met intended statistical power was not found in EP26-A.
Conclusions
The CLSI EP26-A did not provide all necessary answers. Alternative practical approaches are suggested when CLSI EP26-A does not provide guidance.
Research funding: None declared.
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Competing interests: Authors state no conflict of interest.
Ethical approval: The local Institutional Review Board deemed the study exempt from review.
References
1. CLSI. Tubes and additives for venous and capillary blood specimen collection; approved standard, 6th ed. CLSI document GP39-A6. Wayne, PA: Clinical and Laboratory Standards Institute; 2010.Search in Google Scholar
2. Don-Wauchope, AC. Lot change for reagents and calibrators. Clin Biochem 2016;49:1211–12. https://doi.org/10.1016/j.clinbiochem.2016.04.003.Search in Google Scholar PubMed
3. Holzel, W. Analytical variation in immunoassays and its importance for medical decision making. Scand J Clin Lab Invest Suppl 1991;205:113–9. https://doi.org/10.3109/00365519109104609.Search in Google Scholar PubMed
4. Martindale, RA, Cembrowski, GS, Journault, LJ, Crawford, JL, Tran, C, Hofer, TL, et al.. Validating new reagents: roadmaps through the wilderness. Lab Med 2006;37:347–51. https://doi.org/10.1309/BRC6Y37NM3BU97WX.Search in Google Scholar
5. Thaler, MA, Iakoubov, R, Bietenbeck, A, Luppa, PB. Clinically relevant lot-to-lot reagent difference in a commercial immunoturbidimetric assay for glycated hemoglobin A1c. Clin Biochem 2015;48:1167–70. https://doi.org/10.1016/j.clinbiochem.2015.07.018. 'PMCID: PMC6223607'.Search in Google Scholar PubMed
6. Thompson, S, Chesher, D Lot-to-lot variation. Clin Biochem Rev 2018;39:51–60. PMID: 30473592; PMCID: PMC6223607.Search in Google Scholar
7. Kim, HS, Kang, HJ, Whang, DH, Lee, SG, Park, MJ, Park, JY, et al.. Analysis of reagent lot-to-lot comparability tests in five immunoassay items. Ann Clin Lab Sci 2012;42:165–73. PMID: 22585613.Search in Google Scholar
8. CLSI. Measurement procedure comparison and bias estimation using patient samples; approved guideline, 3rd ed. CLSI document EP09-A3. Wayne, PA: Clinical and Laboratory Standards Institute; 2013.Search in Google Scholar
9. Cho, MC, Kim, SY, Jeong, TD, Lee, W, Chun, S, Min, WK. Statistical validation of reagent lot change in the clinical chemistry laboratory can confer insights on good clinical laboratory practice. Ann Clin Biochem 2014;51:688–94. https://doi.org/10.1177/0004563214520749.Search in Google Scholar PubMed
10. Liu, J, Tan, CH, Loh, TP, Badrick, T. Detecting long-term drift in reagent lots. Clin Chem 2015;61:1292–8. https://doi.org/10.1373/clinchem.2015.242511.Search in Google Scholar PubMed
11. Katzman, BM, Ness, KM, Algeciras-Schimnich, A. Evaluation of the CLSI EP26-A protocol for detection of reagent lot-to-lot differences. Clin Biochem 2017;50:768–71. https://doi.org/10.1016/j.clinbiochem.2017.03.012.Search in Google Scholar PubMed
12. CLSI. User evaluation of between-reagent lot variation; approved guideline. CLSI document EP26-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2013.Search in Google Scholar
13. Aarsand, A, Fernandez-Calle, P, Webster, C, Coskun, A, Gonzales-Lao, E, Diaz-Garzon, J, et al.. The EFLM Biological Variation Database. https://biologicalvariation.eu/ [Assessed 27 Apr 2020].Search in Google Scholar
14. Desirable Biological Variation Database specifications. https://www.westgard.com/biodatabase1.htm [Assessed 15 Dec 2019].Search in Google Scholar
15. Ricos, C, Alvarez, V, Cava, F, Garcia-Lario, JV, Hernandez, A, Jimenez, CV, et al.. Current databases on biological variation: pros, cons and progress. Scand J Clin Lab Invest 1999;59:491–500. https://doi.org/10.1080/00365519950185229.Search in Google Scholar PubMed
16. Braga, F, Infusino, I, Panteghini, M. Performance criteria for combined uncertainty budget in the implementation of metrological traceability. Clin Chem Lab Med 2015;53:905–12. https://doi.org/10.1515/cclm-2014-1240.Search in Google Scholar PubMed
17. Braga, F, Panteghini, M. The utility of measurement uncertainty in medical laboratories. Clin Chem Lab Med 2020. https://doi.org/10.1515/cclm-2019-1336.Search in Google Scholar PubMed
18. CLIA document. 42 CFR Part 493. Clinical Laboratory Improvement Amendments of 1988 (CLIA) Proficiency Testing Regulations Related to Analytes and Acceptable Performance. https://www.federalregister.gov/agencies/centers-for-medicare-medicaid-services [Accessed 15 Dec 2019].Search in Google Scholar
19. German Medical Association. Guidelines of the German Medical Association on quality assurance in medical laboratory testing. https://www.egms.de/static/en/journals/lab/2015-6/lab000018.shtml [Accessed 15 Dec 2019].Search in Google Scholar
20. CLSI. Evaluation of precision of quantitative measurement procedure; approved guideline, 3rd ed. CLSI document EP05-A3. Wayne, PA: Clinical and Laboratory Standards Institute; 2014.Search in Google Scholar
21. Panteghini, M, Sandberg, S. Defining analytical performance specifications 15 years after the Stockholm conference. Clin Chem Lab Med 2015;53:829–32. https://doi.org/10.1515/cclm-2015-0303.Search in Google Scholar PubMed
22. Sandberg, S, Fraser, CG, Horvath, AR, Jansen, R, Jones, G, Oosterhuis, W, et al.. Defining analytical performance specifications: Consensus Statement from the 1st Strategic Conference of the European Federation of Clinical Chemistry and Laboratory Medicine. Clin Chem Lab Med 2015;53:833–5. https://doi.org/10.1515/cclm-2015-0067.Search in Google Scholar PubMed
23. Weykamp, C, Secchiero, S, Plebani, M, Thelen, M, Cobbaert, C, Thomas, A, et al.. Analytical performance of 17 general chemistry analytes across countries and across manufacturers in the INPUtS project of EQA organizers in Italy, the Netherlands, Portugal, United Kingdom and Spain. Clin Chem Lab Med 2017;55:203–11. https://doi.org/10.1515/cclm-2016-0220.Search in Google Scholar PubMed
24. Algeciras-Schimnich, A, Bruns, DE, Boyd, JC, Bryant, SC, La Fortune, KA, Grebe, SK. Failure of current laboratory protocols to detect lot-to-lot reagent differences: findings and possible solutions. Clin Chem 2013;59:1187–94. https://doi.org/10.1373/clinchem.2013.205070.Search in Google Scholar PubMed
25. van Rossum, HH. Moving average quality control: principles, practical application and future perspectives. Clin Chem Lab Med 2019;57:773–82. https://doi.org/10.1515/cclm-2018-0795.Search in Google Scholar PubMed
Supplementary Material
The online version of this article offers supplementary material (https://doi.org/10.1515/cclm-2018-0795).
© 2020 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- 10.1515/cclm-2020-frontmatter1
- Mario Plebani turns 70
- Editorial
- Measurement uncertainty and the importance of correlation
- Reviews
- Untangling the association between prostate-specific antigen and diabetes: a systematic review and meta-analysis
- The emerging role of cell senescence in atherosclerosis
- Mini Review
- Minimum retesting intervals in practice: 10 years experience
- Opinion Paper
- Cardiotoxic effects and myocardial injury: the search for a more precise definition of drug cardiotoxicity
- EFLM Papers
- The CRESS checklist for reporting stability studies: on behalf of the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group for the Preanalytical Phase (WG-PRE)
- Thoughts and expectations of young professionals about the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM)
- Guidelines and Recommendations
- Evidence on clinical relevance of cardiovascular risk evaluation in the general population using cardio-specific biomarkers
- Genetics and Molecular Diagnostics
- A systematic evaluation of stool DNA preparation protocols for colorectal cancer screening via analysis of DNA methylation biomarkers
- General Clinical Chemistry and Laboratory Medicine
- The new IVD Regulation 2017/746: a case study at a large university hospital laboratory in Belgium demonstrates the need for clarification on the degrees of freedom laboratories have to use lab-developed tests to improve patient care
- Evaluation of serum electrolytes measurement through the 6-year trueness verification program in China
- Feasibility for aggregation of commutable external quality assessment results to evaluate metrological traceability and agreement among results
- Sample size and rejection limits for detecting reagent lot variability: analysis of the applicability of the Clinical and Laboratory Standards Institute (CLSI) EP26-A protocol to real-world clinical chemistry data
- Effects of calcium dobesilate (CaD) interference on serum creatinine measurements: a national External Quality Assessment (EQA)-based educational survey of drug-laboratory test interactions
- Mass spectrometric sample identification with indicator compounds introduced via labeled sample tubes
- Automation and validation of a MALDI-TOF MS (Mass-Fix) replacement of immunofixation electrophoresis in the clinical lab
- A new strategy implementing mass spectrometry in the diagnosis of congenital disorders of N-glycosylation (CDG)
- Analytical evaluation of four faecal immunochemistry tests for haemoglobin
- Reference Values and Biological Variations
- Study of total error specifications of lymphocyte subsets enumeration using China National EQAS data and Biological Variation Data Critical Appraisal Checklist (BIVAC)-compliant publications
- Mass spectrometric quantification of urinary 6-sulfatoxymelatonin: age-dependent excretion and biological variation
- Establishment of international autoantibody reference standards for the detection of autoantibodies directed against PML bodies, GW bodies, and NuMA protein
- Hematology and Coagulation
- Appropriate thresholds for accurate screening for β-thalassemias in the newborn period: results from a French center for newborn screening
- Cardiovascular Diseases
- Analysis of B-type natriuretic peptide impurities using label-free data-independent acquisition mass spectrometry technology
- Diabetes
- Hemoglobin variants in southern China: results obtained during the measurement of glycated hemoglobin in a large population
- Potential of MALDI-TOF mass spectrometry to overcome the interference of hemoglobin variants on HbA1c measurement
- Acknowledgment
- Acknowledgment
- Letter to the Editors
- Measurement uncertainty estimation for derived biological quantities
- Non-reproducible cardiac troponin results occurring with a particular reagent lot
- Successfully meeting analytical expectations for the fast 0/1-h algorithm for NSTEMI by internal control procedures for cardiac troponin T
- Performance of perpendicular drop versus tangent skimming gating of M-protein in proficiency testing challenges
- Analytical evaluation of Reebio procalcitonin latex-enhanced immunoturbidimetric assay on the HITACHI Labospect 008AS
- Assessment of intraindividual agreement in prolactin results after post-polyethylene glycol precipitation test for the estimation of macroprolactin. Should the precipitation procedure be repeated in the same patient?
- Aggregation kinetic and temperature optimum of an EDTA-dependent pseudothrombocytopenia
Articles in the same Issue
- 10.1515/cclm-2020-frontmatter1
- Mario Plebani turns 70
- Editorial
- Measurement uncertainty and the importance of correlation
- Reviews
- Untangling the association between prostate-specific antigen and diabetes: a systematic review and meta-analysis
- The emerging role of cell senescence in atherosclerosis
- Mini Review
- Minimum retesting intervals in practice: 10 years experience
- Opinion Paper
- Cardiotoxic effects and myocardial injury: the search for a more precise definition of drug cardiotoxicity
- EFLM Papers
- The CRESS checklist for reporting stability studies: on behalf of the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group for the Preanalytical Phase (WG-PRE)
- Thoughts and expectations of young professionals about the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM)
- Guidelines and Recommendations
- Evidence on clinical relevance of cardiovascular risk evaluation in the general population using cardio-specific biomarkers
- Genetics and Molecular Diagnostics
- A systematic evaluation of stool DNA preparation protocols for colorectal cancer screening via analysis of DNA methylation biomarkers
- General Clinical Chemistry and Laboratory Medicine
- The new IVD Regulation 2017/746: a case study at a large university hospital laboratory in Belgium demonstrates the need for clarification on the degrees of freedom laboratories have to use lab-developed tests to improve patient care
- Evaluation of serum electrolytes measurement through the 6-year trueness verification program in China
- Feasibility for aggregation of commutable external quality assessment results to evaluate metrological traceability and agreement among results
- Sample size and rejection limits for detecting reagent lot variability: analysis of the applicability of the Clinical and Laboratory Standards Institute (CLSI) EP26-A protocol to real-world clinical chemistry data
- Effects of calcium dobesilate (CaD) interference on serum creatinine measurements: a national External Quality Assessment (EQA)-based educational survey of drug-laboratory test interactions
- Mass spectrometric sample identification with indicator compounds introduced via labeled sample tubes
- Automation and validation of a MALDI-TOF MS (Mass-Fix) replacement of immunofixation electrophoresis in the clinical lab
- A new strategy implementing mass spectrometry in the diagnosis of congenital disorders of N-glycosylation (CDG)
- Analytical evaluation of four faecal immunochemistry tests for haemoglobin
- Reference Values and Biological Variations
- Study of total error specifications of lymphocyte subsets enumeration using China National EQAS data and Biological Variation Data Critical Appraisal Checklist (BIVAC)-compliant publications
- Mass spectrometric quantification of urinary 6-sulfatoxymelatonin: age-dependent excretion and biological variation
- Establishment of international autoantibody reference standards for the detection of autoantibodies directed against PML bodies, GW bodies, and NuMA protein
- Hematology and Coagulation
- Appropriate thresholds for accurate screening for β-thalassemias in the newborn period: results from a French center for newborn screening
- Cardiovascular Diseases
- Analysis of B-type natriuretic peptide impurities using label-free data-independent acquisition mass spectrometry technology
- Diabetes
- Hemoglobin variants in southern China: results obtained during the measurement of glycated hemoglobin in a large population
- Potential of MALDI-TOF mass spectrometry to overcome the interference of hemoglobin variants on HbA1c measurement
- Acknowledgment
- Acknowledgment
- Letter to the Editors
- Measurement uncertainty estimation for derived biological quantities
- Non-reproducible cardiac troponin results occurring with a particular reagent lot
- Successfully meeting analytical expectations for the fast 0/1-h algorithm for NSTEMI by internal control procedures for cardiac troponin T
- Performance of perpendicular drop versus tangent skimming gating of M-protein in proficiency testing challenges
- Analytical evaluation of Reebio procalcitonin latex-enhanced immunoturbidimetric assay on the HITACHI Labospect 008AS
- Assessment of intraindividual agreement in prolactin results after post-polyethylene glycol precipitation test for the estimation of macroprolactin. Should the precipitation procedure be repeated in the same patient?
- Aggregation kinetic and temperature optimum of an EDTA-dependent pseudothrombocytopenia
