Quality Indicators in Laboratory Medicine: the status of the progress of IFCC Working Group “Laboratory Errors and Patient Safety” project

Introduction

The growing awareness of the importance of the extra-analytical phases in generating reliable laboratory information has prompted clinical laboratories to closely observe all activities in the total testing process (TTP) in order to identify all possible error risks. According to the ISO Guide 73:2009 “Risk is often expressed in terms of a combination of the consequences of an event (including changes in circumstances) and the associated likelihood of occurrence” [1]. Frequently occurring events of low level severity are associated with higher risks, but high severity events, even if isolated/rare can incur even higher risks [2]. It is therefore mandatory for the identification and monitoring of activities with a high risk of error to become a modus operandi in all laboratory procedures. According to the ISO 15189:2012, clinical laboratories should identify critical TTP activities and implement Quality Indicators (QIs) in order to highlight and monitor errors when they occur. QIs, managed as a part of laboratory improvement strategy have proven to be a suitable tool in monitoring and achieving improvement [3], their ultimate purpose being to keep the error risk at a level that minimizes the likelihood of patient harm, given that no activity is completely risk-free. Data available in the literature demonstrate that the effectiveness of this tool is closely linked to the list of QIs chosen, and to: a) data collection method, b) data processing procedure in use, c) appropriate analysis of results, and d) an understanding of the priorities for corrective actions according to performance of the various QIs [4, 5, 6, 7].

The knowledge of error rates is essential for any clinical laboratory as it enables the service to correctly identify of its own risk level, and to compare it with those of other laboratories in order to evaluate its performance in relation to the State-of the-Art (i.e. benchmarking) and identify the priorities for improvement actions. Nevertheless, identifying a suitable State-of-the-Art is a challenging issue, calling for the knowledge of error rates measured worldwide in laboratories that have different organizational and management aspects and contexts, and serve different populations. Moreover, it also depends on the choice of events to keep under control and the procedure that an individual laboratory uses for measurement.

Although many laboratory professionals believe that the systemic use of QIs in Laboratory Medicine may be effective in decreasing errors occurring throughout TTP with a view to enhancing patient safety and meeting the requirements of International Standard ISO 15189 [8], they find it difficult to maintain standardized and systematic data collection along with a high level of interest, commitment and dedication from the entire staff. In order to overcome these problems and identify the State-of-the-Art concerning errors occurring in the TTP, the Working Group “Laboratory Errors and Patient Safety” (WG-LEPS) of the International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) has, since 2008, implemented a project aimed at defining a common Model of QIs (MQI), a harmonized method for data collection, managed as an External Quality Assurance Program (EQAP) in which confidentiality is guaranteed [9, 10, 11].

An MQI issued by a Consensus Conference held in Padova in 2013 and used since 2014. involves a priority score being assigned to each indicator, or assisting laboratories to gradually introduce QIs into routine practice. A criterion to identify Quality Specifications (QSs) for assessing laboratory performance has also been proposed [12, 13].

Aim

This article describes the state of progress of the MQI project. The results reported for each indicator are as follows:

• statistical data of QIs data collected in the 2014, 2015 and 2016 (6 months);

• statistical data of sigma values for data collected in the years 2014, 2015 and 2016 (first 6 months);

• criteria used to define QSs.

The critical aspects highlighted by participants and emerging during the use of QIs use are also described, and future trends considered.

Methodology

All laboratory data collected in 2014, 2015 and in the first 6 months of 2016, were processed on a yearly basis, and the values of the 25th, 50th and 75th percentiles calculated.

A similar procedure was used to estimate sigma values, the short-term formula being applied for the QIs expressed in percentages [14].

The criterion used to define QSs is based on percentile values estimated according to laboratory results. Three levels of performance were identified:

• high, 25th percentile value, representing the best performance;

• medium, 50th percentile value, representing the more frequent/common performance;

• low, 75th percentile representing the worst performance.

When the QIs results measured the desirable events (Post-Comm, Supp-Train, Supp-Cred, Supp-Phys, Supp-Pat), the high level of performance corresponded to the 75th percentile and the low, to the 25th percentile. When the percentile values coincided, it was possible to use a single value.

Results

Table 1 shows all the findings for QIs (in particular, 25th, 50th and 75th percentiles), obtained from data collected by laboratories that consistently participate in the MQI project. Overall, data were received from 59 laboratories in: Argentina, 2; Austria, 1; Brazil, 1; Estonia, 2; Germany, 1; Great Britain, 2; India, 2; Italy, 16; Republic of China, 2; Republic of Croatia, 6; Spain, 2; Switzerland, 2; Serbia, 19; Uruguay, 1.

The short-term sigma for QIs, expressed as a percentage, was estimated in order to identify the quality level of the processes monitored (Table 1). The sigma quality level provides information on the frequency of the occurrence/risk of the various defects. A higher sigma quality level indicates that a process is less likely to generate problems, thus also indicating that the need for checking and inspection is reduced, costs are lower, and customer satisfaction enhanced. The estimation of sigma value is not applicable to QIs results that cannot be expressed in percentages (i.e. minutes or numbers).

The criterion adopted to identify the QSs for each indicator includes the definition of three different performance goals (low, medium, high) according to laboratory results, thus highlighting the most recent error rates collected at a particular time. Information on a performance level based on measures allows each laboratory to establish and compare its placing with that of other laboratories, thus making it possible to plan improvement actions. The use of the 75th percentile as the lower limit seems to be a more practical approach, indicating that performance was poor in less than 25% of participating laboratories. In fact, a high percentage of unsatisfactory performances may discourage some laboratories from attempting to improve quality. On the other hand, if laboratories see they have achieved a higher goal, they are not motivated to undertake improvement actions. Since the improvement actions implemented by the different laboratories are expected to improve over time, the performance goals need to be regularly reviewed (e.g. annually) by analysing the error rate recorded. The knowledge of QSs enables clinical laboratories to identify the most suitable corrective/improvement actions and the relative priorities, whereas it may be excessively challenging to focus improvement projects on all the activities being monitored.

Discussion

Although many laboratories worldwide expressed their willingness to participate in the MQI project, only a few of them systematically entered their own results or used a number of QIs designed to cover all phases of the TTP.

The main QIs used, classified according to the phase of the TTP, are as follows:

• pre-analytical phase: a) unsuitable samples (haemolysed, clotted, inappropriate sample-anticoagulant volume ratio, insufficient volume, wrong container, unlabelled, inappropriate type, not received) and b) misidentified errors (requests and samples);

• intra-analytical phase: a) unacceptable performance in EQAs-PT and b) tests with inappropriate internal quality control (IQC) performance;

• post-analytical phase: a) incorrect reports issued and b) inappropriate TAT (reports delivered outside the specified time, critical values notified after a consensually agreed time, Potassium TAT).

For QIs of Outcome Measures and Support Processes, all indicators proposed in MQI appear to be used in a similar fashion, but only by a small number of laboratories.

Many laboratories justify their inadequate participation in QI data collection by citing the ‘excessive’ number of QIs included in the MQI. However, it is important to bear in mind that:

• QIs should monitor all critical aspects of the TTP, as stated by ISO 15189:2012, and therefore several QIs need to be decided upon;

• to obviate any confusion between indicator and measurements, different measures are often required to ensure that an indicator is appropriately monitored. It is advisable to split an indicator into different measures in order to consider all the events causing a specific error, and to benchmark data entered by different laboratories. Some laboratories, for example, offer their service to outpatients and inpatients, while others perform analyses for only one of these two patient groups. Any comparison of data entered by laboratories that process samples from different patient populations might generate misleading conclusions. This applies in particular to some indicators showing wrong procedures performed by different personnel as the leading cause of the error. Sample collection is a paradigmatic example, wherein the error is typically attributable to clinical ward staff for inpatient samples and to the laboratory and peripheral drawing centre personnel for specimens collected from outpatients.

Rather than deleting some QIs, it might be preferable to revise MQI in order to identify the QIs that should be split into further measurements. For example, in the case of haemolysed samples, the error rate of 1.06 estimated in 2015 included laboratories that used to detect haemolysis by means of serum indices, visual inspection or other unspecified procedures. An error rate of 1.18 was calculated for laboratories using serum indices, but only 0.63 for other facilities using visual inspection. This clearly indicates that it may be advisable to split this indicator into two different measures to prevent misleading conclusions concerning the real burden of haemolysis. The lesson learnt with this QI implies that it might be better to differentiate the various QSs according to the specific detection procedure used in the different laboratories. A similar consideration can be made for indicators used for tests with a CV% higher than the set target (Intra-Var).

As the International Standard on Laboratory Accreditation and approved guidelines do not specify the appropriate number of QIs to be used in the laboratory, and the MQI project does not oblige laboratories to use all QIs proposed, it seems appropriate to include in the MQI all the indicators that appear useful in monitoring critical activities. The individual laboratory should be able to decide how many, and which, QIs are to be adopted.

Another aspect biasing the participation in the project is related to difficulties in data collection, especially when automated collection is unavailable. The laboratory staff may be discouraged and dissatisfied from manual collection of data, since this activity takes time and dedication. The design of dedicated software for automated data collection could hence stimulate a major involvement of the laboratory staff in the project [15].

However, several real difficulties have been acknowledged in the collection of data with some post-analytical QIs. In many cases, identification errors call for enquiry and the active involvement of clinicians/nurses, a challenging requirement in from the viewpoint of time and frequency. In other circumstances, it seems necessary to better specify which events need to measured and how this can be done (i.e. Post-Comm, interpretative comments impacting positively on patient’s outcome).

The laboratories also experienced difficulty in meeting the deadline for collecting and entering data in the MQI-dedicated website. Laboratories are more inclined undertake the retrospective collection of data, with transmission delayed by months or, in extreme cases, a year. As shown in Table 1, the results of some indicators obtained in 2016 have been excluded due to the low number of responses. The failure to comply with these deadlines, in turn, further delays the provision of reports to the clinical laboratories participating the MQI project.

As regards the sigma values estimate for the QIs of intra-analytical phase, significant improvements have been achieved in the last few decades, whereas fewer improvements have been made to the extra-analytical phases (Table 1). The accurate interpretation of the significance of intra-analytical QIs is of crucial importance, as these QIs are not intended to monitor the performance of the analytical procedures, but to reflect the management of unsatisfactory analytical performances. This observation highlights the need for a greater focus on this issue, which is often overlooked. Some laboratories manage (or correct) an error at the same time as its occurrence (i.e. unsatisfactory performance in EQA or IQC), thus overriding the underlying cause(s) or disregarding appropriate improvement actions (risk management).

Due to the type of results and to the lower number of responses, a significant sigma value could not be calculated for the support processes.

In order reduce the error rates in critical TTP procedures, the following some initiatives must be undertaken:

1. involving Scientific Societies of different countries to promote participation of laboratories in the MQI project;

2. involving Accreditation Bodies, so that the MQI may be identified as a suitable tool complying with the ISO 15189:2012 requirements;

3. selecting and nominating a National Leader to coordinate and manage the MQI project;

4. defining guidelines supporting the use of QIs and implementation of improvement actions in clinical laboratories;

5. establishing criteria to ensure that an appropriate list of QIs (number, typology, and frequency of collection of data) is included in the MQI, procedures are processed and laboratory performance evaluated;

6. sharing QIs with other inter-laboratory quality management providers.

Conclusions

The increased value of laboratory information as well as its impact on clinical outcomes is a catalyst in assuring the reliability of test results. The inter-laboratory comparison of QIs is an important component of quality management, since it enables a direct comparison with both other laboratories and established performance specifications (benchmark). The QIs system, which should be part of a coherent and coordinated quality improvement strategy, should be constantly reviewed and updated, comply with accreditation requirements and scientific recommendations, support efforts to continuously improving laboratory performances, enhance the value of both TTP and clinical practice, and be effective in evaluating patient’s outcome. Additional efforts should be made to ensure the effective the use of QIs in clinical laboratories. The MQI project is proving to be an important tool that not only provides the TTP error rate and divulges awareness of the value of QIs in enhancing patient safety, but also highlights the more critical aspects interfering with the widespread and appropriate use of QIs themselves. The dedicated website (www.ifcc-mqi.com), already useful for sharing the list of QIs, showing the frequency of data collection, and providing valuable information, could be further improved as it is of promise as a tool for connecting participating laboratories and stakeholders.