Why specifications for allowable glucose meter errors should include 100% of the data

Jan S. Krouwer

doi:10.1515/cclm-2013-0387

Artikel Öffentlich zugänglich

Why specifications for allowable glucose meter errors should include 100% of the data

Jan S. Krouwer

Veröffentlicht/Copyright: 21. Juni 2013

Veröffentlicht von

Veröffentlichen auch Sie bei De Gruyter Brill

Manuskript einreichen Informationen für Autor*innen

Aus der Zeitschrift Clinical Chemistry and Laboratory Medicine (CCLM) Band 51 Heft 8

The recently published ISO 15197 guideline [1], and the CLSI POCT12-A3 guideline [2] are two new glucose meter performance guidelines. Unfortunately, each of these guidelines fails to include error limits for 100% of the data. For the CLSI guideline, specifications are given for 98% of the results and for the ISO guideline, 99% of the results. The purpose of this article is to discuss why a specification for 100% of the results is appropriate.

I have previously commented [3] that in clinical chemistry, errors are often thought of as a distribution of continuous variables and not as discrete error events. Thus, these glucose meter guidelines provide one set of error limits for 95% of the data and a wider set of limits for either 98% or 99% of the data. There is nothing wrong with this construct. However, one can also consider glucose meter errors as discrete failure events. For example, if a glucose sample reference result was 1.67 mmol/L (30 mg/dL), a serious glucose meter failure event would be if the meter result for that sample was 16.65 mmol/L (300 mg/dL) (e.g., within the E zone of a Parkes error grid [4]). And for each sample, this failure event can either occur or not occur. It makes sense to specify a goal of zero percent of data in the E zone of the Parkes error grid. The ISO and CLSI goals can be thought of as specifying the percentage of results that fall within the A and perhaps B region of an error grid. Thus, the ISO and CLSI goals are based on a distribution of a continuous variable and goals for the higher zones in an error grid, which are missing in the ISO and CLSI guidelines, are based on discrete failure events. The balance of the data (2% for CLSI and 1% for ISO) is allowed to fall in intermediate but not the highest error zones. Note that in other fields, where people think in terms of discrete failure events, one would never set goals in the way that they are done for glucose meters. Thus, one would not specify that nuclear power plants should run without meltdowns 99% of the time nor that correct site surgery should occur 99% of the time. Perhaps a difficulty arises because one can never (statistically) prove the occurrence of zero failures but the goal itself is valid.

Manufacturers worry about the approval process and in performing method comparisons, the possibility of an outlier is perhaps a rationale for the ISO and CLSI glucose meter specifications. However, it is the wrong way to think about the problem. If there were one E zone result in a 125 sample method comparison, this error rate of 0.8% would be acceptable according to either glucose meter standard. On the one hand, if this glucose meter had 1% of the glucose meter market (total market=7.9 billion results per year in the US), this implies over 632,000 dangerous glucose results per year in the US [5]. On the other hand, if this glucose meter company ran 10 million samples with no results in the E zone, this only proves (with 95% confidence) that no more than 29 dangerous results would occur [6]. Thus, proving that rare events do not occur by running method comparisons is not practical.

What is done both in other fields and in clinical chemistry is to perform Failure Mode Effects Analysis (FMEA) and fault tree analysis to enumerate the sources of possible failure events and to implement mitigations where appropriate. These techniques also do not prove zero failures but complement method comparisons which need to be performed anyway.

If an E zone event does occur in a method comparison, this by itself is not a reason to consider the meter to be unacceptable. The most important question is the cause of the outlier and it is possible that no cause will be found. There is a financial constraint to collecting data to decide on meter approval and the use of post-market data, the size of which dwarfs method comparison studies should also be used as a way to monitor meter performance.

Conflict of interest statement

Author’s conflict of interest disclosure: The author stated that there are no conflicts of interest regarding the publication of this article.

Research funding: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Jan S. Krouwer, Krouwer Consulting, 26 Parks Drive, Sherborn, MA 01770, USA, Fax: +1 508 653 2379

References

1. ISO 15197:2013 In vitro diagnostic test systems – Requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. Geneva, Switzerland: International Organization for Standardization, 2013.Suche in Google Scholar

2. CLSI POCT12-A3 point-of-care blood glucose testing in acute and chronic care facilities; approved guideline, 3rd ed. Wayne, PA: CLSI, 2013.Suche in Google Scholar

3. Krouwer JS. Recommendation to treat continuous variable errors like attribute errors. Clin Chem Lab Med 2006;44:797–8.10.1515/CCLM.2006.152Suche in Google Scholar PubMed

4. Parkes JL, Slatin SL, Pardo S, Ginsberg BH. A new consensus error grid to evaluate the clinical significance of inaccuracies in the measurement of blood glucose. Diabetes Care 2000;23:1143–8.10.2337/diacare.23.8.1143Suche in Google Scholar PubMed

5. Assumes that patients who take insulin test 3 times per day – FDA public meeting clinical accuracy requirements for point of care blood glucose meters March 16, 2010. Available from: http://www.fda.gov/downloads/MedicalDevices/NewsEvents/WorkshopsConferences/UCM208598.pdf. Accessed on 23 May, 2013.Suche in Google Scholar

6. Hahn GJ, Meeker WQ. Statistical intervals. A guide for practitioners. New York: Wiley, 1991:104.10.1002/9780470316771Suche in Google Scholar

Published Online: 2013-06-21

Published in Print: 2013-08-01

Artikel in diesem Heft

https://doi.org/10.1515/cclm-2013-0387