External quality assessment of the manual tilt tube technique for prothrombin time testing: a report from the IFCC-SSC/ISTH Working Group on the Standardization of PT/INR

Introduction

For monitoring of oral anticoagulant treatment with vitamin K antagonists (VKA), the prothrombin time (PT) is the primary measurement. To achieve standardization of the PT, the results should be expressed as international normalized ratio (INR). Recently, a modified reference measurement system (RMS) and a sustainable calibration hierarchy for the PT/INR have been proposed [1], 2]. The proposed RMS is still based on calibration using the International Sensitivity Index (ISI) [3]. The proposed RMS and calibration hierarchy are based on a harmonized manual tilt tube (MTT) method to be used for PT measurements with the current and replacement international reference preparation (IRP) for thromboplastin.

There are good reasons for maintaining the MTT as reference measurement procedure. Firstly, all previous IRP for thromboplastins have been calibrated in multicentre studies using the MTT. It is well known that the value of the International Sensitivity Index (ISI) can be influenced by the use of various automated coagulometer instruments [4], [5], [6]. To avoid ISI bias, the IRP must be used with the MTT. Secondly, replacing the MTT by one particular automated instrument procedure for the IRP would require that this particular instrument must be available worldwide for the life of the IRP, e.g. 10–20 years. It is questionable whether any instrument manufacturer could guarantee such a long availability without any change in the instrument’s properties. It has been recognized that ISI calibration by using the classic WHO method [3] (i.e. by using the MTT) is a very labour intensive process typically performed by reagent manufacturers, but which is generally beyond the resources of most laboratories [7]. On the other hand, the availability of the MTT and training of operators can be maintained by a few specialized calibration laboratories [1].

The calibration studies of previous IRP’s for thromboplastin and their use in the calibration of secondary standards were performed with a non-harmonized MTT [8], [9], [10], [11]. Using a non-harmonized MTT resulted in a large PT/INR variation between calibration laboratories [11]. For this reason efforts were made to harmonize the MTT and reduce the between-laboratory variation [1].

Laboratories should participate in external quality assessment (EQA) schemes as part of their total quality system [12]. For this reason we set up a new EQA scheme specifically for calibration laboratories performing the harmonized MTT. The purpose of the present paper is to report the results of the first 10 surveys of the new scheme and to investigate whether there is improvement in performance over time and in comparison with previous multicentre studies. This work was performed under the auspices of the joint International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) – Scientific and Standardization Committee (SSC) of the International Society on Thrombosis and Haemostasis (ISTH) Working Group on the Standardization of PT/INR [2]. The members of the Working Group are recruited from academia, SSC/ISTH (Subcommittee on Control of Anticoagulation), WHO Expert Committee on Biological Standardization, National Institute for Biological Standards and Control (NIBSC), EQA organizations, and In-Vitro-Diagnostic manufacturers.

Materials and methods

Results

Correlation of PT between different operators

The PT results obtained by individual operators in consecutive surveys were displayed graphically (Supplementary Figures 1S–4S). There were considerable differences in PT results between successive surveys which were in several cases similar for the two operators in the same laboratory (Figure 1 and Supplementary Table 3S). For example, there was a significant positive correlation (p<0.001) between the PTs of plasma DF measured by operators no. 1 and no. 2 in laboratory no. 1 (Figure 1). Similar results were observed in other laboratories (see Table 3S in Supplementary Material). Correlations between operators in the same laboratory were stronger for control plasmas in the order AF<BF<CF<DF (Supplementary Table 3S).

Figure 1:

PT (s) measurements (mean of duplicate PTs per day) for control plasma DF by operator 2 (laboratory 1) plotted against the corresponding values obtained by operator 1 (laboratory 1) on the same day. Open symbols: measurements performed in 2022 (5 surveys, n=25); Spearman’s rank correlation coefficient: 0.824 (p<0.001). Black symbols: measurements performed in 2023 (5 surveys, n=25); Spearman’s rank correlation coefficient: 0.798 (p<0.001).

Discussion

In this paper we report the results of an External Quality Assessment of the PT determined with a harmonized manual tilt tube technique by eight operators in 4 laboratories. The study comprised 10 surveys over a time period of nearly two years. This allowed us to see whether there was any change in the performance over time. Some operators improved but others did not (Table 1). There was no important overall difference in CV_O between the years 2022 and 2023 (Table 1). We used Recombiplastin 2G because it is very similar to the current IRP for thromboplastin, recombinant, human (rTF/16). Two lots of Recombiplastin 2G were used, one lot in 2022 and the other in 2023. The PTs obtained with the first lot were slightly different from those obtained with the second lot (Table 4). More clearly we observed higher PT ratios in 2023 as compared to those in 2022 (Table 5). These results suggested that the two lots had slightly different sensitivities with regard to the coagulation factors in the test plasmas. For this reason we analysed the results obtained with the two lots separately.

There were differences between the operators and between the surveys, but in some cases there was a significant positive correlation between the PTs measured by the two operators in the same laboratory (Figure 1). This observation suggests that in those cases differences between surveys could be due to local conditions, e.g. reagent and sample variation. The reagent was reconstituted with diluent freshly on the first day of each survey and the same vial was used by both operators on all other days of the survey. A potential error in reagent preparation in a particular survey may influence both operators’ PT results of the same survey. The plasma samples were thawed freshly by the operators on each day of testing. Reagent and sample between-vial variation was not investigated in the present study. The effect of reagent between-vial variation may be reduced by pooling several vials in future studies.

The frozen plasma samples were stored at −70 °C during the course of this study. Despite this condition, we observed a small increase of the clotting times during the course of the year (Table 3 and Supplementary Table 5S) which may be due to instability of the frozen plasma possibly reinforced by barium sulphate treatment. Although frozen plasmas are stable for short periods of time, some decay does occur even at −74 °C [13], 17]. A slight increase of PT does not influence the comparison between operators in the same survey and the within-operator within-survey variation.

It should be realized that there is no true value for the PT measured with the harmonized MTT. For this reason, we used the mean PT of all operators as consensus value. We calculated each operator’s deviation from the consensus value and the median deviations (Table 2). Some operators had lower median D_O in 2023 than in 2022 but others did not. Comparing the results of 2023 with those of 2022, we cannot conclude that overall performance was improved. Part of the deviations from the PT consensus value (Table 2) may be due to physiological or psychological differences (e.g. reaction time) between the operators, but it may also be due to between-vial differences of the plasma samples or the thromboplastin reagent or the stability of the reagent after reconstitution. Between-vial variation and stability were not controlled in the present study. Between-vial variation and stability of the reagent should be accounted for in future studies.

The between-operator variation and the within-operator variation observed in the present study may be compared to the between-operator variation and the within-operator variation observed in a previous multicentre study for the calibration of international reference thromboplastin preparations rTF/16 and RBT/16 [11]. The present study showed much lower between-operator variation with CV’s not greater than 2.2 % (Table 4), compared to 4.9–9.2 % in the previous multicentre study [11] which we have described in Table 7 for ease of comparison. Furthermore, the median within-operator CV_O values (Table 1) were much lower than the median within-operator CV_w values obtained in the previous multicentre study (Table 7). The operators in the present study used a harmonized MTT which was devised by the same operators and which was exercised by the operators together in two workshops [1]. In contrast, the previous multicentre study was performed by 20 operators who were not trained in a preceding workshop and who used a non-harmonized MTT [11]. The finding that the average CV_O of the eight operators in the present study did not change during the course of the 10 successive surveys suggests that the average CV_O is as low as technically possible.

Overall, within-operator, within-run variation, i.e. repeatability (CV_R) seemed to depend on the plasma used (Table 6). CV_R was greater for plasma AF than for plasmas BF, CF and DF. It is reassuring that CV_R is low in the therapeutic range of anticoagulation levels.

There are a few limitations of this study. First of all, we could not assess the variation of the INR. The INR should be calculated with the Mean Normal PT (MNPT) and the International Sensitivity Index (ISI) [3]. In the present study, each operator’s MNPT was not available and therefore the INR could not be calculated. Although we could not calculate the CV of the INR in a direct way, we may estimate a minimum value using some assumptions. The first assumption is that each operator’s MNPT is equal to the operator’s mean PT obtained with pooled normal plasma AF. The second assumption is that each operator’s ISI is equal to the value obtained by one operator’s local calibration (see Materials and Methods). According to Taberner et al. the CV of the INR is approximated to the CV of the PT ratio multiplied by the ISI [16]. Using Taberner’s approach with an approximate ISI value of 1.04 we obtained approximate coefficients of variation of the INR ranging from 1.1 to 2.1 (Table 5). It should be emphasized that neglecting the uncertainty of the MNPT and ISI results in underestimation of the uncertainty of the INR [18]. Future EQA studies should include determination of the MNPT by each operator at the beginning of the surveys. The second limitation of our study is that the plasma samples were artificially depleted of coagulation factors and had a different composition as compared to clinical samples. In general artificially depleted plasmas are not commutable with clinical samples [19], 20]. As the relative concentrations of the coagulation factors in artificially depleted plasmas may be different from those in clinical samples, it cannot be excluded that the use of artificially depleted plasmas in the present study affected the variation of the PT within and between operators. Nevertheless, previous studies have shown that the coefficient of variation of the PT calibration line slope determined with MTT was similar with either fresh clinical samples or lyophilized artificially depleted plasmas [20], suggesting that the analytical PT measurement variation was similar with both types of samples. The third limitation of our study is that only one type of reagent (i.e., Recombiplastin 2G) was used which has a different composition as compared to the future replacement IRP. We suggest that EQA studies of the MTT should be continued with a reagent similar to the future IRP.

In this context it is interesting to refer to a previous study of INR between-laboratory variation among end users in European countries where a multiplicity of various thromboplastin reagents and procedures were used [21]. The between-laboratory variation ranged from 7.8 to 9.0 % when end users did not perform local calibration and from 5.8 to 7.6 % when end users did perform local calibration. The authors concluded that implementing local calibration is strongly recommended [21]. We wonder whether the above-mentioned level of between-laboratory variation can be reduced further. We suggest that improved standardization of higher-order calibrations by reference laboratories should contribute to improved accuracy of INR results obtained by end users.

The present study has shown that the uncertainty of the PT determined with the MTT depends on the operator. An important question is the maximum allowable uncertainty of the PT which is required for an appropriate analytical performance specification for INR measurements. Recently it was stated that a sustainable metrologically traceable calibration hierarchy for the INR should be based on an international protocol for value assignment with a single primary reference thromboplastin and the harmonized MTT for PT determination [2]. It was stated that the reference measurement system’s uncertainty budget should represent one third of the total standard uncertainty. If we accept that the total standard uncertainty of the INR is 10 %, the uncertainty budget of the reference measurement system would be 3.3 % [2]. What should be the standard uncertainty of the reference measurement system’s PT determination corresponding to an INR standard uncertainty of 3.3 % ? This question can only be answered when the results are available of a multicentre calibration study of the new reference measurement system including the standardized MTT and the replacement primary reference thromboplastin. It is anticipated that this calibration study will be completed by the end of 2024.

In conclusion, there were differences in PT variation within and between the MTT operators in the present EQA study. The variation between operators was much smaller than in a previous multicentre calibration study which is likely due to harmonization of the MTT.