Behind the scenes of EQA – characteristics, capabilities, benefits and assets of external quality assessment (EQA): Part I – EQA in general and EQA programs in particular

History of EQA

Though the US military conducted regular surveys of syphilis serology laboratory competence in the 1930s [9], the first published surveys of chemistry and hematology assays were in the late 1940s in the USA and the UK in the early 1950s [10], 11]. These demonstrated wide (2- to 4-fold) variation in results between laboratories, not attributable to the methods used, even with aqueous solutions containing the pure substance of interest, whereas in almost all cases there was no evidence of bias [10], 11]. Sporadic surveys, published and unpublished but too varied and numerous to reference, continued during the 1950s and 1960s, with similar findings. Most were geographically limited, in their scope in terms of analytes, especially in the challenges of producing reports within a meaningful timescale. These surveys were, however, instrumental in raising awareness of the need for quality assurance (QA) and stimulating the development of internal quality control (IQC) techniques adapted from the manufacturing industry [12], 13].

However sophisticated the application of IQC, it became obvious that EQA was essential to attaining and maintaining comparability of results among laboratories. This led to the establishment of national or regional programs in the USA [14], UK [15] and other countries in the late 1960s and early 1970s. EQA services then spread across further disciplines in laboratory medicine [16] and geography [17]. These took advantage of advances in data processing technology to provide timely reports, and emphasize the frequency of distributions, the number of analytes and specimen numbers. The ethos differed between countries, with some driven by legislative requirements (e.g. the Clinical Laboratory Improvement Act (CLIA) in the USA and the Calibration Law in West Germany) [18], 19]. Most, however, followed policies of voluntary participation with the aim of self-improvement based on scientific principles. Though program designs varied across countries, the objectives were to deliver regular services with frequent multi-specimen distributions and rapid feedback through reports, including scoring systems. However, some were restrained by the resources available from the government or the professional societies responsible for delivery [15].

There remained some confusion in terminology, however, as there was a misconception that external programs could provide an element of “control” despite their retrospective nature. This was dispelled by the publication of the outcome of a WHO Europe consensus conference [16], 20]. This clearly differentiated between the roles of the components of analytical quality assurance:

1. IQC as the set of procedures undertaken for the continuous monitoring of laboratory operations and results, to decide whether the results are reliable enough to be released

2. EQA as the system of objectively checking laboratory results by an external agency, including retrospective comparison of a laboratory’s results with those of others, to establish between-laboratory comparability

It also outlined substantial agreement on desirable aspects of program design [16], 20]. By the mid-1980s national EQA services had been established in many countries. Programs continued to develop in succeeding decades, increasing the scope of analytes surveyed and the sophistication of their designs in delivering information helpful to their participants in improving their performance [14].

Legal background to participation in EQA

A study on the impact of regulatory requirements on EQA failure rates shows that different countries have very different regulations and recommendations for participation in EQA schemes in general and the frequency of participation (Table 1) [21]. In most of the 33 countries reported in this regard, there is a clear legal obligation to participate in EQA; to a lesser extent, authorities or other official bodies, such as medical associations, review the participation and performance of individual laboratories in EQA or the EQA provider reports incorrect results to them; to a small extent there are (at least potential) sanctions of a financial nature or by restricting the authorization to carry out examinations with failed EQA (Table 1).

Financing of EQA programs

Providing an EQA service can be expensive due to the procurement of material, analysis of material, the need to verify stability and homogeneity, logistic issues both in sample production and dispatch as well as the support for educational activities to complement the EQA program. The funding source may, therefore, hinder what a particular program can offer and restrict the range of measurands or breadth of challenges.

EQA providers can be classified in terms of their funding as either not-for-profit, usually a professional/medical association or a government agency, or for-profit, usually a commercial organization. Mixed models, like foundation, medical association plus individual person ownership, government plus professional/medical association are also possible. What laboratories pay and if they have a choice of provider can also vary by country and depends on whether or not a government agency, a not-for-profit organization or a commercial company provide the EQA. In many countries, EQA participation is mandatory (Table 1).

There are also fundamental differences in classifying EQA providers on the basis of whether they are regulatory or educational. These include if they provide programs outside their country of origin, the ownership (professional organization, private), type of organization (not-for-profit or for-profit), the range of programs they offer by discipline, measurand, and the level of support they provide to their participants. For an EQA provider to be sustainable it must be able to finance all activities within its goals fully. These include all resources – personnel and material requirements – for program delivery and customer support activities such as education and troubleshooting assistance.

Sources of income may include subscription fees for programs and income from additional material, income from webinars and conferences, grants and, in some cases, direct financial support from the government.

For some EQA organizations, lack of sufficient funding constrains the development and structure of EQA programs. Funding also influences the educational activities that can be provided and the cost to participants. The ability to provide verifiable commutable material or to have reference method target value assignment may be limited because of the funding model.

EQA and ethics

The introduction of personalized medicine requires laboratory medicine to enter the era of precision diagnostics, setting clinical performance specifications to develop and evaluate assays for clinical use [22]. It is the role and ethical obligation of EQA providers to employ contemporary methods that can identify examination procedures capable of meeting analytical performance specifications, and to enable laboratories to determine whether they meet these requirements and whether the analytical service is beneficial for the patient. The applicable standards refer to ethical requirements for the laboratory towards its patients and the EQA scheme provider towards its participants in points 4.1 “Impartiality” and 4.2 “Confidentiality” of ISO 15189:2022 and ISO 17043:2023 and for the laboratory additionally in 4.3 “Requirements for patients” of ISO 15189:2022 [5], 8].

EQA programs play a pivotal role in assessing laboratory performance through standardized evaluations. They provide uniform samples and compare results across institutions, ensuring consistency, accuracy, and reliability of laboratory test results. This assurance is critical for patient outcomes, reinforcing their safety and instilling confidence in the healthcare system.

Ethical participation in EQA programs highlights a laboratory’s commitment to fundamental principles such as patient safety, transparency, accountability, and professional integrity. It ensures that laboratories identify and rectify errors, continuously improve their testing procedures, and adapt to advancements in medical technologies. For instance, as new diagnostic tools and techniques emerge, EQA programs update their standards and evaluations to ensure that laboratories use the most effective and accurate methods. This commitment enhances the quality of patient care and strengthens public trust in the healthcare system.

Confidentiality is a cornerstone of ethical consideration in EQA programs. It ensures the secure handling of patient information and laboratory performance data, reassuring patients, healthcare providers, and laboratories about protecting their data. Ethical practices demand that patient samples and EQA results be protected from misuse, fostering trust among all stakeholders.

Thus, EQA programs are not just a technical necessity but an ethical imperative. They play a vital role in safeguarding patient outcomes, reinforcing public trust, and encouraging continuous learning and quality improvement within laboratories, making them indispensable to modern healthcare [23].

Information management systems in EQA

EQA requires software with various functionalities to manage and administer programs and participants effectively. This functionality may be provided by bespoke development or commercial solutions as a single system or by multiple systems connected to perform specific tasks. Information systems used by EQA providers should be validated to ensure that they are fit for purpose and operate as intended. Data integrity must be ensured, data manipulation or loss must be prevented, and the accuracy of test results must be maintained. To ensure that detailed audit trails, role-based access control, and regular, verified data backups are just as required as redundant systems and infrastructure with regular monitoring and proactive maintenance to prevent disruptions and ensure continuous data access to authorized users. Additionally, as the software may be accessible from the internet for data entry and can also be interfaced with external laboratory information systems (LIS), robust cyber security measures are essential to protect data and systems from malicious attacks, threats, and breaches. Since the structure of software for EQA also provides a good overview of processes running in parallel, software features generally required for most EQA programs are listed in Table 2.

Regulatory and educational/aspirational purposes of EQA

EQA serves different purposes, namely regulatory and educational/aspirational [6], 24]. While the primary regulatory purpose is to identify poorly performing laboratories, the leading educational/aspirational purpose of EQA is to improve the quality of laboratory examination. Regulatory EQA activities usually have wide tolerance limits, whereas for educational EQA activities, these are generally tighter and may be based on clinical outcome data, biological variation or “state of the art” [25]. EQA programs may offer combinations of performance specifications that relate to either regulatory or educational/aspirational aims. Because of the different analytical performance specifications, a laboratory can have acceptable performance in one (regulatory) challenge and unacceptable performance in another (educational) for the same measurand.

The life cycle of an EQA program

All EQA programs have to start somewhere. Once established, the EQA program usually operates on a continuous basis, and on rare occasions, a program is terminated. Whether the EQA service is pre-examination, examination or post-examination, or requires the distribution of physical specimens, or covers derived examinations based on analytical results and an algorithm, EQA service providers are continually looking to expand and improve their services to meet the needs of their users better. Providers’ development and implementation of EQA programs is an intentional undertaking requiring significant resources, considerable research and development, and follows a stepwise process, as illustrated in Figure 2.

Figure 2:

Life cycle of an EQA program. After conception and design, an EQA program comes into its routine run. Ongoing programs regularly present challenges to registered participants in the form of EQA cycles. The performance of individual EQA programs is evaluated on a regular basis and, if necessary, details of programs are adapted to match the needs of participants and technology. However, an EQA program may also reach the end of its life cycle, either because the included measurands have lost their clinical relevance or because the EQA provider decides to close it for various reasons.

Frequency and intensity of EQA

Apart from the Rilibäk, which regulates the minimum frequency of participation in EQA schemes for laboratories in Germany, there are only a few other guidelines regarding frequency and intensity of EQA [27]. For example, concerning screening of donated blood for transfusion-transmissible infections (at least two cycles per year [30]) or blood lead (three samples every two months [31]). One of the reasons for a lack of harmonization in this area is that ISO 15189:2022 suggests that EQA providers should be accredited in compliance with ISO/IEC 17043:2023, which specifies the criteria and procedures the EQA providers are to follow [32]. The choice of frequency by the EQA organization may be influenced by existing information regarding results accuracy and/or harmonization, the availability and price of control materials and the cost of the examination in laboratories. Consequently, EQA providers design their programs with different frequencies of cycles [33].

Laboratories are ultimately responsible for deciding which EQA program they use for their service. The laboratory should take into account the type of service that they are providing (prognosis, diagnosis, screening), the prevalence of the disease and the number of investigations undertaken by the laboratory (workload), the analytical complexity, the error rate of the investigation and the specialist nature of the investigation [34]. The laboratory can then choose an EQA program that meets these requirements. A review of 22 organizations representing 407 programs showed that the median for all examined disciplines was four cycles per year. The responses of this survey were categorized into scientific disciplines, i.e. biochemistry (6 cycles per year median), hematology (three cycles per year median), hemostasis (4 cycles per year median), and microbiology (median three cycles per year). As the authors concluded, the number of EQA cycles (and number of samples) varied widely per year within and between each discipline. Furthermore, there is a consensus that error rates and testing volume play an essential role in establishing the frequency of EQAs [34].

One study has tried to develop a framework for evaluating the frequency of EQA challenges [35]. The aim was to demonstrate the impact of the correlation of EQA data between different samples on the information that can be extracted from EQA results, such as the evaluation of laboratory or method performance. It was shown that the assessment of performance was flawed by the presence of a correlation between EQA results from different samples. Therefore it becomes less beneficial to send more samples per EQA cycle or organize more EQA cycles within a time interval. The authors concluded that there will always be a tension between resources (cost of the program, time to run and analyze the results) and value of appropriate intervention on problems that may increase the risk to patients [35].

The “ideal” frequency of EQA in the context of medical laboratories can vary greatly depending on the specific service a laboratory provides [36]. Both a higher and a lower number of individual samples per cycle seem to have advantages and disadvantages (Table 5). While a higher frequency seems to have more advantages in terms of analytical quality, the advantages of lower frequencies are more economical. The disadvantages seem to be the other way around [35].

Extra-analytic EQA

ISO 15189:2022 requires that the EQA program selected by the laboratory be used to check pre-examination, examination and post-examination processes [5]. Although most errors happen within the pre-examination and post-examination processes, far less emphasis has been put on their quality control and improvement, compared to the analytical parts [36]. The reason might be that pre- and post-examination processes can appear to be particularly hard to control since, contrary to the analytical phase, most steps occur outside of the laboratory.

Another issue with regard to pre-examination and post-examination quality control are difficulties in the acquiring and documenting of such errors and the lack of universally standardized quality indicators (QIs) for evaluating, monitoring and improving these steps within the total testing process [37]. Several organizations provide information and platforms for QI acquisition, documentation and benchmarking [38], 39]. Many existing laboratory information systems (LIS) do not come with a pre-built functionality of recording QIs, making this process partly/mostly a manual one (which is both time consuming and error prone); however, LIS are constantly improving, so potentially, data collection will be easier. Additionally, current coding systems, such as the Logical Observation Identifiers Names and Codes (LOINC) [40] or the Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT) [41] are focused on intra-laboratory (analytical) processes and initiatives like the Standard Preanalytical Code (SPREC) [42] systems are currently used only for biobanking purposes. This makes extra-analytic quality benchmarking intentions almost impossible. Finally, no acceptance criteria have been defined for most – if not all - extra-analytical QIs, leaving laboratories collecting and documenting such data to evaluate them like they would do with an IQC by looking out for drifts and variations of recordings over time.

Currently, most EQA providers who provide pre-examination programs are either 1) surveying the procedure and checking the knowledge of participating laboratories or 2) sending out samples designed to test pre-examination systems (e.g. for serum indices detection, RNA/DNA extraction, etc.), or pre-examination case-reports with either real or fictional data [43], [44], [45], [46]. Recently, a new type of EQA program has been introduced, aimed at improving/maintaining quality of samples sent to the laboratory via pneumatic tube transport [47], [48], [49]. Post-examination data may be collected within examination EQA programs by asking for interpretation of reported results, collection of reference interval information or answering a series of case study questions.

Pre-examination and post-examination programs are designed to look specifically at QIs not only facing challenges with data collection, but also evaluating data and reporting meaningful information back to laboratories. A number of EQA providers use a sigma metric approach. A risk-based scoring system allows laboratories to prioritize action.

Developing specific EQA programs for each extra-analytic error possibility seems unfeasible [50]. Nevertheless, such programs are vital for quality maintenance and improvement. Therefore, it seems reasonable to focus on extra-analytical errors with the highest frequency and/or patient safety risk while considering their applicability. An alternative could be a mandatory constant documentation of selected QIs. Although no QIs are specifically mentioned, the ISO 15189:2022 standard stipulates that pre- and post-examination QIs should be regularly recorded, documented and evaluated. Currently, most laboratories are collecting information on only a few QIs, if any [36], 51], 52], demonstrating once more that there is room for improvement regarding quality management of the extra-analytical phases.

In some cases, it is possible to combine pre-analytical, analytical and post-analytical EQAs where case reports together with control material are circulated, and both pre-analytical, analytical and post-analytical responses are registered, ending up with a diagnosis and how this is reported to the clinicians. This is often done for rare diseases [53].