Statistical approach for optimization of external quality assurance (EQA) studies of molecular and serological viral diagnostics
-
Leonid Rumer
,
Cristina Domingo
Abstract
Management of viral diagnostic quality is based on external quality assurance (EQA), where laboratories involved in diagnostics of a targeted virus are offered to analyze a panel of blinded samples. The utility of EQAs is compromised because of the absence of an approach to EQA design which upfront defines acceptance criteria and associated statistical analysis ensuring fair and consistent interpretation. We offer a rigorous statistically based approach for EQA planning. Instead of a conventional performance characteristic (the score) which is calculated as the sum of the points for correctly identified samples in a blinded test panel, Youden index is used as the performance measure. Unlike the score, Youden index requires an estimate of sensitivity and specificity and incorporates the relationship of these performance parameters. Based on the assumption that the coordinator is a reputable expert of viral diagnostics, the performance of the coordinator’s laboratory is defined as a proficiency standard for performance evaluation. The immediate goal of EQA is defined as to obtain a statistically reliable estimation for every laboratory whether its performance meets the proficiency standard, while the overall goal is to match every laboratory to its specific performance level. Dependence of informational capacities of test panel from the panel size and content is quantitatively analyzed and the optimal design and informational capacities of both idealized panels (whose size is not restricted by financial factors) and currently feasible panels are considered. Our approach provides the basis both for rational design of currently feasible EQA test panels and for an increased panel size.
Acknowledgments:
The authors are grateful to R. Schädler for excellent assistance in the preparation of the manuscript.
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: None declared.
Employment or leadership: None declared.
Honorarium: None declared.
Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.
References
1. MM14-A2: Design of Molecular Proficiency Testing/External Quality Assessment; Approved Guideline, 2nd ed. 2913 May. Available at http://shop.clsi.org/molecular-methods-documents/MM14.html. Accessed: 21 Jan 2015.Search in Google Scholar
2. European Network for Diagnostics of “Imported” Viral Diseases (ENIVD). Available at: http://www.enivd.de/index.htm. Accessed: 21 Jan 2015.Search in Google Scholar
3. Domingo C, Niedrig M, Teichmann A, Kaiser, Rumer L, Jarman RG, et al. 2nd international external quality control assessment for the molecular diagnosis of dengue infections. PLoS Negl Trop Dis 2010;4:e833. Doi: 10.1371/journal.pntd.0000833.10.1371/journal.pntd.0000833Search in Google Scholar
4. I N S T A N D e. V. Gesellschaft zur Förderung der Qualitätssicherung in medizinischen Laboratorien e. V, WHO Collaborating Centre for Quality Assurance and Standardization in Laboratory Medicine, Deutsche Vereinigung zur Bekämpfung der Viruskrankheiten (DVV), Gesellschaft für Virologie (GfV), Deutsche Gesellschaft für Hygiene und Mikrobiologie (DGHM). Final Report External Quality Assessment Scheme (EQAS) – March/April 2012 Virus Genome Detection-HIV-1 (RNA) (360) (PCR/NAT HIV-1). 2012 May 15. Available at: http://www.instandev.de/uploads/tx_nfextinstandpdf/360_HIV-1_Genome_March_April_2012_EN_Homepage_Version_20120515d.pdf. Accessed: 21 January 2015.Search in Google Scholar
5. Domingo C, Escadafal C, Rumer L, Méndez JA, García P, Sall AA, et al. First international external quality assessment study on molecular and serological methods for yellow fever diagnosis. PLoS One 2012;7:e36291. Doi: 10.1371/journal.pone.0036291.10.1371/journal.pone.0036291Search in Google Scholar
6. Youden WJ. Index for rating diagnostic tests. Cancer 1950;3:32–5.10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3Search in Google Scholar
7. Greiner M, Pfeiffer D, Smith RD. Principles and practical application of the receiver-operating characteristic analysis for diagnostic tests. Prev Vet Med 2000;45:23–41.10.1016/S0167-5877(00)00115-XSearch in Google Scholar
8. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529–36.10.7326/0003-4819-155-8-201110180-00009Search in Google Scholar
9. QIAGEN. Available at: https://www.qiagen.com/de/products/catalog/sample-technologies/rna-sample-technologies/viral-rna/qiaamp-minelute-virus-spin-kit/#. Accessed: 21 January 2015.Search in Google Scholar
10. QIAGEN. Available at: https://www.qiagen.com/de/products/catalog/assay-technologies/complete-assay-kits/human-pathogen-detection/artus-hcv-rt-pcr-kits-ce/#orderinginformation. Accessed: 21 January 2015.Search in Google Scholar
11. McDonald JH. Handbook of biological statistics, 2nd ed. Baltimore (MD): Sparky House Publishing, 2009. Available at: http://www.biostathandbook.com/exactgof.html. Accessed: 21 January 2015.Search in Google Scholar
12. Interactive Stats. Available at: http://statpages.org/confint.html. Accessed: 21 January 2015.Search in Google Scholar
13. Faul F, Erdfelder E, Lang A.-G, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007;39:175–91.10.3758/BF03193146Search in Google Scholar
14. Langton SD, Chevennement R, Nagelkerke N, Lombard B. Analysing collaborative trials for qualitative microbiological methods: accordance and concordance. Int J Food Microbiol 2002;79:175–81.10.1016/S0168-1605(02)00107-1Search in Google Scholar
15. Drosten C, Doerr HW, Lim W, Stöhr K, Niedrig M. SARS molecular detection external quality assurance. Emerg Infect Dis 2004;10:2200–3.10.3201/eid1012.040416Search in Google Scholar
16. Stat Trek. Available at: http://stattrek.com/online-calculator/binomial.aspx#examples. Accessed: 21 January 2015.Search in Google Scholar
17. Biel SS, Donoso Mantke O, Lemmer K, Vaheri A, Lundkvist A, Emmerich P, et al. Quality control measures for the serological diagnosis of hantavirus infections. J Clin Virol 2003;28:248–56.10.1016/S1386-6532(03)00009-XSearch in Google Scholar
18. Donoso Mantke O, Lemmer K, Biel SS, Groen J, Schmitz H, Durand J-P, et al. Quality control assessment for the serological diagnosis of dengue virus infections. J Clin Virol 2004;29:105–12.10.1016/S1386-6532(03)00110-0Search in Google Scholar
19. Niedrig M, Schmitz H, Becker S, Günther S, terMeulen J, Meyer H, et al. First international quality assurance study on the rapid detection of viral agents of bioterrorism. J Clin Microbiol 2004;42:1753–55.10.1128/JCM.42.4.1753-1755.2004Search in Google Scholar PubMed PubMed Central
Supplemental Material
The online version of this article (DOI: 10.1515/cclm-2016-0081) offers supplementary material, available to authorized users.
©2016 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Editorial
- Serum myoglobin immunoassays: obsolete or still clinically useful?
- Reviews
- Kounis syndrome: an update on epidemiology, pathogenesis, diagnosis and therapeutic management
- What do we know about homocysteine and exercise? A review from the literature
- Mini Review
- Osteocalcin as a potential risk biomarker for cardiovascular and metabolic diseases
- Opinion Paper
- Statistical approach for optimization of external quality assurance (EQA) studies of molecular and serological viral diagnostics
- EFLM Article
- Sample collections from healthy volunteers for biological variation estimates’ update: a new project undertaken by the Working Group on Biological Variation established by the European Federation of Clinical Chemistry and Laboratory Medicine
- General Clinical Chemistry and Laboratory Medicine
- Analyte stability during the total testing process: studies of vitamins A, D and E by LC-MS/MS
- Improvement in the predictive ability of the Intermountain Mortality Risk Score by adding routinely collected laboratory tests such as albumin, bilirubin, and white cell differential count
- Cystatin C provides a better estimate of the effect of glomerular filtration rate on serum human epididymis protein 4 concentrations
- Verification of the harmonization of human epididymis protein 4 assays
- Clinical utility of urinary liver-type fatty acid binding protein measured by latex-enhanced turbidimetric immunoassay in chronic kidney disease
- Comparison of three analytical platforms for quantification of the neurofilament light chain in blood samples: ELISA, electrochemiluminescence immunoassay and Simoa
- Reference Values and Biological Variations
- Distribution of antiphospholipid antibodies in a large population-based German cohort
- Cardiovascular Diseases
- Serum protein S100 as marker of postoperative delirium after off-pump coronary artery bypass surgery: secondary analysis of two prospective randomized controlled trials
- Infectious Diseases
- Copeptin predicts 10-year all-cause mortality in community patients: a 10-year prospective cohort study
- Clinical and laboratory findings in the diagnosis of right lower quadrant abdominal pain: outcome analysis of the APPAC trial
- Letters to the Editor
- Why a new algorithm using high-sensitivity cardiac troponins for the rapid rule-out of NSTEMI is not adapted to routine practice
- Optimal collection tubes for plasma glucose determination: confusion reigns supreme
- Long-term stability of serum samples positive for carbohydrate deficient transferrin (CDT) routinely stored at −20 °C
- Seasonal variations in plasma free metanephrine concentrations are not evident in the West of Ireland
- Potential errors in the determination of urinary ammonium by formol titration
- Interference by biological anti-cancer drugs in electrophoretic and immunofixation techniques
- A point mutation in the thiopurine S-methyltransferase gene that led to exon 5 deletion in the transcribed mRNA
- Detection of the heterozygote of hemoglobin Constant Spring by α-thalassemia immunochromatographic strip test
- Automated CH50 liposome-based immunoassay: consideration in dilution and validation of reference interval
- Theranos phenomenon – Part 5: Theranos’ presentation at the American Association for Clinical Chemistry Annual Conference 2016
- Congress Abstracts
- 58th National Congress of the Hungarian Society of Laboratory Medicine
- The 4th Joint EFLM-UEMS Congress “Laboratory Medicine at the Clinical Interface” Warsaw, Poland, 21th–24th September, 2016
- Congress of Clinical Chemistry and Laboratory Medicine
Articles in the same Issue
- Frontmatter
- Editorial
- Serum myoglobin immunoassays: obsolete or still clinically useful?
- Reviews
- Kounis syndrome: an update on epidemiology, pathogenesis, diagnosis and therapeutic management
- What do we know about homocysteine and exercise? A review from the literature
- Mini Review
- Osteocalcin as a potential risk biomarker for cardiovascular and metabolic diseases
- Opinion Paper
- Statistical approach for optimization of external quality assurance (EQA) studies of molecular and serological viral diagnostics
- EFLM Article
- Sample collections from healthy volunteers for biological variation estimates’ update: a new project undertaken by the Working Group on Biological Variation established by the European Federation of Clinical Chemistry and Laboratory Medicine
- General Clinical Chemistry and Laboratory Medicine
- Analyte stability during the total testing process: studies of vitamins A, D and E by LC-MS/MS
- Improvement in the predictive ability of the Intermountain Mortality Risk Score by adding routinely collected laboratory tests such as albumin, bilirubin, and white cell differential count
- Cystatin C provides a better estimate of the effect of glomerular filtration rate on serum human epididymis protein 4 concentrations
- Verification of the harmonization of human epididymis protein 4 assays
- Clinical utility of urinary liver-type fatty acid binding protein measured by latex-enhanced turbidimetric immunoassay in chronic kidney disease
- Comparison of three analytical platforms for quantification of the neurofilament light chain in blood samples: ELISA, electrochemiluminescence immunoassay and Simoa
- Reference Values and Biological Variations
- Distribution of antiphospholipid antibodies in a large population-based German cohort
- Cardiovascular Diseases
- Serum protein S100 as marker of postoperative delirium after off-pump coronary artery bypass surgery: secondary analysis of two prospective randomized controlled trials
- Infectious Diseases
- Copeptin predicts 10-year all-cause mortality in community patients: a 10-year prospective cohort study
- Clinical and laboratory findings in the diagnosis of right lower quadrant abdominal pain: outcome analysis of the APPAC trial
- Letters to the Editor
- Why a new algorithm using high-sensitivity cardiac troponins for the rapid rule-out of NSTEMI is not adapted to routine practice
- Optimal collection tubes for plasma glucose determination: confusion reigns supreme
- Long-term stability of serum samples positive for carbohydrate deficient transferrin (CDT) routinely stored at −20 °C
- Seasonal variations in plasma free metanephrine concentrations are not evident in the West of Ireland
- Potential errors in the determination of urinary ammonium by formol titration
- Interference by biological anti-cancer drugs in electrophoretic and immunofixation techniques
- A point mutation in the thiopurine S-methyltransferase gene that led to exon 5 deletion in the transcribed mRNA
- Detection of the heterozygote of hemoglobin Constant Spring by α-thalassemia immunochromatographic strip test
- Automated CH50 liposome-based immunoassay: consideration in dilution and validation of reference interval
- Theranos phenomenon – Part 5: Theranos’ presentation at the American Association for Clinical Chemistry Annual Conference 2016
- Congress Abstracts
- 58th National Congress of the Hungarian Society of Laboratory Medicine
- The 4th Joint EFLM-UEMS Congress “Laboratory Medicine at the Clinical Interface” Warsaw, Poland, 21th–24th September, 2016
- Congress of Clinical Chemistry and Laboratory Medicine
