Agreement between procalcitonin measurements using the new point-of-care testing ichroma™ reader and the automated Kryptor instrument

Sepsis is a significant public health problem throughout the world, with more than 31 million cases reported annually and a 17% mortality rate [1]. Early detection of infection is essential: the earlier the diagnosis and the antibiotic therapy, the better the chances of survival [2].

Procalcitonin (PCT) has been used in Europe for many years and was also approved for use in the United States by the US Food and Drug Administration (FDA) as a diagnostic aid for sepsis in 2005. It also gained an FDA indication in 2016 for serial use to assess sepsis progression and 28-day mortality risk [3]. However, due to the suboptimal sensitivity and/or specificity of the PCT test, results should always be interpreted alongside the clinical context [4], [5], [6]. A 24-h availability of PCT determinations is desirable [4], [5], [6]. For this purpose, a point-of-care testing (POCT) could be an attractive solution [7], especially in a local small laboratory, when it is physically separated from the central routine laboratory where the state-of-the-art instruments are usually to be found. The critical point is that PCT is usually trend-evaluated, especially in order to guide/monitor antibiotic therapy [4], [5], [6], [8]. Consequently, if a critical patient should be moved from a local hospital to the central hospital, the interchangeability of the PCT measurements using POCT, and those using the central routine laboratory instrument, is mandatory to evaluate the PCT kinetic changes accurately. The aim of this study was to find out if the PCT measurements using the new POCT ichroma™ are interchangeable with those using Kryptor, the well-accepted gold-standard instrument.

To do this, we investigated samples (n=117), processed within 2 h after blood drawing, selected from pre-existing routine samples with a PCT result (in-house method: Liaison XL, Diasorin, Saluggia, Italy). PCT assays were performed daily using firstly Kryptor (BRAHMS AG, Hennigsdorf/Berlin, Germany) and secondly the ichroma™ reader (Boditech Med Incorporated, Gangwon-do, Korea). We tested serum samples because requests for multiple tests (chemistry and immunoassay) are usually made to a local laboratory where the use of only one sample (i.e. serum) tube is not only practically and economically suitable, but also serves to limit patients’ discomfort.

This study was performed according to the Code of Ethics of the World Medical Association (Declaration of Helsinki) for experiments involving humans. The Hospital Institutional Review Board waived the need for informed consent.

Our range of PCT values was 0.08–60.85 ng/mL using Kryptor and 0.13–66.27 ng/mL using ichroma™. Within-run and between-run analytical imprecisions were consistent with claims made by the manufacturer (<10%) (Table 1). Passing-Bablok (PB) regression (Figure 1A, Table 2) did not show a significant deviation from linearity (all p-values from the Cusum linearity test: >0.10) while proportional and constant differences were observed between Kryptor and ichroma™ except for data under 2 ng/mL (intercept contained 0 and slope contained 1 in the 95% confidence interval [95% CI]). The mean bias% was within the desirable quality specification for total error (TE) (<20%) [10] but the 95% CI was very large, exceeding both the TE specification (20%) and the clinical reference change value (about 60%) (Table 3).

Figure 1:

Passing-Bablok regression (A) and Bland-Altman plot (B and C) comparing procalcitonin (PCT) measurements using Kryptor and ichroma™.

The acceptance limits for the bias% were defined a priori (1) by the desirable specifications for the allowable total error (TE), based on the biological variation [11], [12], namely <20% and (2) by the clinically well-accepted daily decrease of 50% change values, useful to suggest the adequacy of the antibiotic treatment [11] or by an increase of 88% during the first 24 h, suggesting the patient has turned out to have an infection [11]. (A) Red-dashed line: identity, blue-dotted line: Passing-Bablok regression. (B) Central bold-dashed red line represents the difference standardized with respect to the mean ([method A – method B]/mean %), with corresponding red-dashed 95% confidence interval. Blue bold-dashed lines represent limits of agreement with respect to the percentage standardized difference. Blue-dashed lines represent 95% confidence intervals for the values of each limit of agreement. (C) Central bold-dashed red line represents the mean difference with corresponding red-dashed 95% confidence interval. Blue bold-dashed lines represent limits of agreement (mean difference plus and minus 1.96 times the standard deviation of the differences). Blue-dashed lines represent 95% confidence intervals for the values of each limit of agreement.

The data distribution shows that the bias% and even more the absolute bias could depend on the concentration (Figure 1B and C). However, even if the data analysis is evaluated in a smaller range (under 10 ng/mL and under 2 ng/mL), the bias% is still not acceptable. Therefore, ichroma™ may not be considered suitable to be used interchangeably with Kryptor for monitoring patients.

However, Cohen’s kappa statistic [14], used to calculate the concordance at clinically relevant cutoffs for bacterial infections, showed that even though the agreement between ichroma™ and Kryptor values at the clinical cutoff of 0.25 ng/mL was moderate [14] (κ=0.725; 95% CI: 0.532–0.917), suggesting caution in the interpretation of the data around this value, the agreement at 0.50 ng/mL (κ=0.878; 95% CI: 0.774–0.982), 2.0 ng/mL (κ=0.983; 95% CI: 0.949–1.016) and 10 ng/mL (κ=0.938; 95% CI: 0.869–1.007) was strong, suggesting that the ichroma™ can be used to support clinicians’ decision-making. Basically the same results were observed for Liaison XL with respect to the ichroma™ [15] instrument.

In conclusion, our data suggest that ichroma™ is not interchangeable with Kryptor, so cannot be mixed with it for monitoring patients; one must choose one instrument only and be consistent. However, while the strong concordance at the clinically relevant cutoffs allows us to consider ichroma™ a suitable option to Kryptor to support clinicians’ decision-making, nevertheless the moderate agreement at the 0.25 ng/mL cutoff recommends caution in interpreting the data around this cutoff.

Moreover, our study has some limitations that need to be considered: (a) we tested the interchangeability between methods in only one laboratory and (b) we used only serum samples. Whole blood and plasma samples need to be investigated in future, possibly in a multicenter study.