Quality control in the Netherlands; todays practices and starting points for guidance and future research

Introduction

Ensuring adequate quality of reported test results is one of the primary responsibilities of medical laboratories. Medical laboratories have several tools at their disposal to accomplish this, including internal quality control (iQC) measurements, external quality control measurements, and patient-based real-time quality control (PBRTQC) including manual verification of patient results and patient moving averages [1]. iQC is an essential, and perhaps the most prominent, component of the quality assurance for medical tests. iQC is based on the frequent, e.g. daily or within specified run, analysis of control materials with a predetermined target and standard deviation (SD) value for specified (quantitative) measurement systems. The interpretation of these iQC measures is generally based on a statistical approach using SD-based control rules, also referred to as statistical quality control (SQC).

The iQC practice of setting up SQC and the theoretical concepts of iQC have evolved from standard 2SD rules based solely on analytical characteristics, to sigma metric approaches using clinical tolerance limits based SQC set-up via multi- (Westgard-)rules, to the more recently described, risk-based bracketed SQC that also considers the number of patients at risk [2], [3], [4]. With each new development and conceptual advancement, the complexity of SQC increased significantly. In addition, the use of PBRTQC to support iQC for assuring the analytical quality has also gained momentum [5, 6]. With all of these developments, the general medical laboratory is challenged to select the iQC setup that is most appropriate for the laboratory. This becomes even more important when there are no legal or accreditation/certification-required SQC practices as in the Netherlands. Therefore, each medical laboratory must decide separately how to perform the analytical quality assurance. Finally, consistency and comparability of laboratory results is also increasingly a major priority in The Netherlands due to increasing consolidation of individual laboratories into large lab-chain organizations. This often triggers the desire for harmonized iQC procedures covering a large number of analyzers and multiple laboratory sites.

To have an understanding of the current analytical quality assurance practices and the adoption of the newer SQC and PBRTQC concepts, a survey was conducted among medical laboratories in the Netherlands. A second objective of the survey was to extract relevant information, to be incorporated into a revised version of a Dutch guidance document on Six Sigma analytical quality control.

Materials and methods

Results

Long-term management of iQC

Figure 1 shows the results of 3 questions regarding the long-term management of iQC; who evaluates and manages the long-term iQC configuration, is there a separate dedicated evaluation of the iQC settings, and how are between-analyzer differences of the same measurement procedure controlled. For the latter, several laboratories mentioned that EQA analysis is also used for this purpose.

Figure 1:

Questions related to the long(er)-term iQC management.

Discussion

The questionnaire provided relevant insights into the iQC practices performed in the Netherlands. In particular the great heterogeneity of the iQC setup applied in the medical laboratories was considered remarkable. The obtained answers provided some key aspects that are relevant to provide guidance. Furthermore, for some practices performed by the laboratories, such as the determination of SD values, the virtual analyzer methodology and the assurance of between analyzer comparisons, there seems to be a need for additional research to provide the necessary additional insights into best practices.

Recently, several other surveys on the international internal QC practices have been conducted and published. These include a global survey initiated by the IFCC (n=46) [9], a survey amongst academic medical centers in the US (n=21) [10], and by far the largest survey conducted by Westgard QC and published on their website (n=619) [11]. In general, these surveys compared iQC practices around the world and focused on more general aspects, such as, the source of QC materials, the use of Westgard-rules, and QC frequency without providing a methodological basis thereof. The general findings of the IFCC and Westgard surveys were also a large heterogeneity of iQC practices. Both the US-based survey and the Westgard survey showed that within the US, 2SD control used were overwhelmingly used as single iQC rule, although the QC frequency could vary widely [10, 11]. A limitation of these studies was that no information was provided on the methodology and system used to design the SQC. In addition, also more practical details were missing, such as whether a virtual analyzer methodology was used, and what SD values represent and how they are obtained. As this questionnaire was primarily designed to address such practical issues that may be addressed in the guidance document, the questionnaire focused significantly on such more detailed practices. In comparison to the 2SD control rule observation in other surveys, in this survey 14 laboratories reported the use of 1 × 2SD rules; 13 of these 14 also used other SD rules and only a single laboratory responded 1 × 2SD to be the single SQC rule used.

The questionnaire was sent to all medical laboratories in the Netherlands and only a (43 %) minority of the laboratories responded to the request and completed the questionnaire. The responses may be biased in the sense that the laboratories that responded may have a higher than average interest in iQC and have a higher than average iQC standard and use more advanced iQC methods. This is considered a limitation of this study and therefore the results may not accurately represent the true general iQC practice as performed in the Netherlands. There was a representation of different types of medical laboratories, such as those supporting university medical centers, general hospitals and primary care. In addition, laboratories with both single and multiple laboratory sites were included. Therefore, although the results may overestimate the use of advanced iQC methods, the diversity in respondents and the answers to the questionnaire reveal large heterogeneity in the SQC design.

Approximately 30 % of the laboratories base their iQC setup solely on the analytical characteristics of assays and do not apply clinical tolerance limits. This indicates that the use of clinical tolerance limits and the incorporation of patient risk into SQC design has been adopted by the majority of the participating clinical laboratories. Analytical quality control is an explicit requirement for ISO 15189 accreditation. The prevailing ISO 15189:2012 during the questionnaire does not specify into great detail the way iQC is performed. However, the recently published version (ISO 15189:2022) explicitly requires consideration of the intended clinical application of the examination, as the performance specifications can differ in different clinical settings [12]. The 30 % laboratories which base their iQC setup solely on the analytical characteristics of assays will have to change their iQC setup to fulfill the requirements of the revised ISO 15189.

Of particular interest to the authors was the high adoption of any type of virtual analyzer system methodology for establishing and using SQC settings by 89 % of the laboratories. The “virtual analyzer” was considered as a theoretical analyzer that is comprised of the individual analyzers that operate within it. Within the virtual analyzer model, both the uncertainty of the individual analyzers employed and the variability among these instruments are used to define the uncertainty of the virtual analyzer. Unfortunately, no additional in-depth questions about these practices were included in the questionnaire.

When asked about the method used to determine SD and target values, a variety of practices was used. Unfortunately, there is little research and there are no prospective user studies available on how to set, obtain and manage SD and target values on individual analyzers when using a virtual analyzer setup. However, some have recently started to provide more advanced work on this topic [13, 14]. It seems relevant to explore the pros and cons of different approaches and to share best practices. Another interesting observation was the high percentage of alternative systems to support the SQC. In particularly the adoption of patient moving averages or PBRTQC for one or more analytes in 41 % of the clinical laboratories was much higher than the previously reported adoption rate of 10 % in US academic medical centers [10]. In addition, the adaption of peer-to-peer review of iQC results from different vendors has gained momentum. However, how laboratories use these results in the context of EQA to determine true error or bias seems relevant, as peer-to-peer review can allow for much more frequent peer comparison than EQA.

When reviewing the management aspects of iQC, there is also a wide variation in practices. Some use a traffic light system to identify out-of-control situations, while others manually review all QC charts. Furthermore, although most use their LIS to manage iQC, about half of the clinical laboratories use more than one software system to manage QC results. Some even use all of the software systems mentioned (LIS, middleware, analyzer software and 3rd party software) to manage iQC. The latter primarily seems to reflect practices at the UMCs and may be caused by greater specialization and separation of the various, more independent, specialized laboratories as part of the overall clinical laboratory. Also, the management of longer term settings and the evaluation of the settings are highly variable and seem to depend largely on the daily management set-up. Finally, the methods used to ensure the comparability of analyzers also appear to be variable, with some using iQC measures and others using a patient sample to ensure the use of commutable materials for this purpose. Some report using moving averages.

In conclusion, the questionnaire provided many relevant insights into the QC practices applied in clinical laboratories in the Netherlands. In particular, it has provided some relevant starting points for further research and guidance, as for some of these practices there is little or no evidence and guidance available. The most relevant points according to the authors are listed in Table 4. This overview should provide some guidance for the necessary research and perhaps the sharing of some best practices.