Highly accurate and explainable detection of specimen mix-up using a machine learning model

Tomohiro Mitani; Shunsuke Doi; Shinichiroh Yokota; Takeshi Imai; Kazuhiko Ohe

doi:10.1515/cclm-2019-0534

Article

Highly accurate and explainable detection of specimen mix-up using a machine learning model

Tomohiro Mitani , Shunsuke Doi , Shinichiroh Yokota , Takeshi Imai and Kazuhiko Ohe

Published/Copyright: September 12, 2019

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information

From the journal Clinical Chemistry and Laboratory Medicine (CCLM) Volume 58 Issue 3

Abstract

Background

Delta check is widely used for detecting specimen mix-ups. Owing to the inadequate specificity and sparseness of the absolute incidence of mix-ups, the positive predictive value (PPV) of delta check is considerably low as it is labor consuming to identify true mix-up errors among a large number of false alerts. To overcome this problem, we developed a new accurate detection model through machine learning.

Methods

Inspired by delta check, we decided to conduct comparisons with the past examinations and broaden the time range. Fifteen common items were selected from complete blood cell counts and biochemical tests. We considered examinations in which ≥11 among the 15 items were measured simultaneously in our hospital; we created individual partial time-series data of the consecutive examinations with a sliding window size of 4. The last examinations of the partial time-series data were shuffled to generate artificial mix-up cases. After splitting the dataset into development and validation sets, we allowed a gradient-boosting-decision-tree (GBDT) model to learn using the development set to detect whether the last examination results of the partial time-series data were artificial mixed-up results. The model’s performance was evaluated on the validation set.

Results

The area under the receiver operating characteristic curve (ROC AUC) of our model was 0.9983 (bootstrap confidence interval [bsCI]: 0.9983–0.9985).

Conclusions

The GBDT model was more effective in detecting specimen mix-up. The improved accuracy will enable more facilities to perform more efficient and centralized mix-up detection, leading to improved patient safety.

Keywords: anomaly detection; data-mining; delta-check method; laboratory information system; machine learning; patient safety

Corresponding author: Tomohiro Mitani, MD, Department of Biomedical Informatics, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo 113-8655, Japan, Phone: +81-3-5800-6427, Fax: +81-3-5803-1803

Acknowledgments

We would like to thank Editage (www.editage.jp) for English language proofreading.

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: None declared.
Employment or leadership: None declared.
Honorarium: None declared.
Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

References

1. Lippi G, Chance JJ, Church S, Dazzi P, Fontana R, Giavarina D, et al. Preanalytical quality improvement: from dream to reality. Clin Chem Lab Med 2011;49:1113–26.10.1515/CCLM.2011.600Search in Google Scholar PubMed

2. Lippi G, Blanckaert N, Bonini P, Green S, Kitchen S, Palicka V, et al. Causes, consequences, detection, and prevention of identification errors in laboratory diagnostics. Clin Chem Lab Med 2009;47:143–53.10.1515/CCLM.2009.045Search in Google Scholar PubMed

3. Simundic A-M, Church S, Cornes MP, Grankvist K, Lippi G, Nybo M, et al. Compliance of blood sampling procedures with the CLSI H3-A6 guidelines: an observational study by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) working group for the preanalytical phase (WG-PRE). Clin Chem Lab Med 2015;53:1321–31.10.1515/cclm-2014-1053Search in Google Scholar PubMed

4. Yamashita T, Ichihara K, Miyamoto A. A novel weighted cumulative delta-check method for highly sensitive detection of specimen mix-up in the clinical laboratory. Clin Chem Lab Med 2013;51:781–9.10.1515/cclm-2012-0752Search in Google Scholar PubMed

5. Randell EW, Yenice S. Delta checks in the clinical laboratory. Crit Rev Clin Lab Sci 2019;11:1–23.10.1080/10408363.2018.1540536Search in Google Scholar PubMed

6. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: machine learning in Python. J Mach Learn Res 2011;12:2825–30.Search in Google Scholar

7. Chen T, Guestrin C. XGBoost: a scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Confereence on Knowledge Discovery Data Min – KDD 16. 2016;785–94.10.1145/2939672.2939785Search in Google Scholar

8. Health Level Seven International. FHIR [Internet]. [cited 2019 Jul 1]. http://hl7.org/fhir.Search in Google Scholar

9. Lundberg MitaniSM, Erion GG, Lee S-I. Consistent individualized feature attribution for tree ensembles. http://arxiv.org/abs/1802.03888. Accessed: 30 May 2019.Search in Google Scholar

Received: 2019-05-29

Accepted: 2019-08-13

Published Online: 2019-09-12

Published in Print: 2020-02-25

You are currently not able to access this content.

Articles in the same Issue

https://doi.org/10.1515/cclm-2019-0534

Keywords for this article

anomaly detection; data-mining; delta-check method; laboratory information system; machine learning; patient safety