Behind the scenes of EQA – characteristics, capabilities, benefits and assets of external quality assessment (EQA): Part III – EQA samples

Challenging samples

The clinical usefulness of laboratory test results depends on performance evaluation and detection of errors, including those that affect the sample’s integrity and the presence of interferences in that sample. Specimens received in the laboratory often do not reflect the crystal-clear samples used in many EQA programs and may not always be good representatives of clinical specimens received in the laboratory. The three major sample interferences found to affect the accuracy and reliability of most laboratory results are haemolysis, icterus and lipaemia. Each of these situations is a potential source of biological and analytical bias, which ultimately compromises the reliability of measurements of different parameters in routine clinical chemistry examination. Most manufacturers have now included hemolytic/icteric/lipemic indices (HIL) in their test repertoire to identify these interferents in samples, and several EQA providers have now established EQA programs to assess the performance of IVD-MDs for EQA materials with abnormal HIL indices.

Some EQA programs also assess the susceptibility of examination procedures to potential interferents as part of their standard programs. Examples include the assessment of the effects of therapeutic drugs and their metabolites on the performance of other measurands, hook effects in immunoassays, steroid cross-reactivity in immunoassays, effects of ascorbic acid in urinalysis, effects of blood in urine on pregnancy testing performance, the assessment of specificity of examination procedures to samples with mixed viruses, effects of endogenous substances and metabolites on measurands, and the distribution of samples near the limit of detection of the assays.

In addition to interferences in samples, physiologically inhomogeneous measurands can also pose challenges for examination and, in the case of incorrect results, for root cause analysis and therefore they are a welcome challenge in EQA: Lp(a) levels may be under- or overestimated and individuals potentially misclassified for their cardiovascular disease risk, as

1. different immunoassay-based tests detect different proportions of the actual mass of the size-polymorphic measurand [12], 13],

2. conversion of the results from mass to molar units [14], especially in combination with

3. inferior calibration procedures such as serial dilutions of a single calibrator instead of employing multiple independent calibrators [15],

4. limited measuring ranges that require dilution of the sample, which in turn can result in a mismatch between sample and calibrator [16], and

5. another heterogeneity affecting the immunoassay result is attributed (but not proven) to the indirect measurement of partially glycosylated Lp(a) [17].

As long as no appropriate reference material or method has been recognized [18], but of course also afterwards, EQA providers can contribute to improving the situation by using targeted EQA samples, not least for educational purposes.

As “challenging samples” are often encountered in the laboratory, they must be included in selected cycles as part of the standard EQA and any conclusions should be fed back to the participants with the report. It is also important that the participant acts on the feedback from the EQA provider. However the performance on such samples may be excluded from the aggregated performance assessment of individual laboratories and/or methods.

Homogeneity and stability

Characterisation of homogeneity of EQA samples and stability of materials is important so that their contribution (as MU) to potential between-laboratory differences can be incorporated when achieving a consensus value and to ensure deviations are reflecting laboratory performance and are not due to differences in the samples. It is obvious that homogeneity and stability tests should only be carried out on the finished and prepared samples after they have been filled into final containers and, if applicable, lyophilised or subjected to a freeze-thaw cycle, in order to reduce the probability that influences during or after sampling for these tests may alter the characteristics of the samples but remain undetected. The EQA provider is responsible for defining homogeneity and stability of the materials that they distribute and ensuring they are within acceptable parameters. They may do this at two different steps, i) prior to distribution by analysis of a subset of samples (though for some measurands/samples homogeneity and stability do not need to be done for each production if the EQA providers can document that samples are produced in the same way as samples where homogeneity and stability has been proven) and/or ii) post distribution by comparison of imprecision data of current samples against that at a similar concentration from previous cycles, or even better from previous circulations of the same samples (under a different name or sample code to mask such repetition). The advantage of including all the participants’ results (usually many times the number of analyses in formal studies) is that it will include any impact of sample transportation.

Procedures applied in preparation of EQA samples may follow the requirements of ISO 33405:2024 on Reference materials – Approaches for characterization and assessment of homogeneity and stability [19]. Ideally, EQA samples should be characterised to the degree of inhomogeneity for each characteristic (measurand) of interest; in practice, the analysis may be limited to the determination of selected characteristics, provided their established chemical or physical relationships to measurands for which they serve as surrogate markers of homogeneity and stability. ISO 17043:2023 refers EQA organisers to ISO 13528:2022 for the statistical analysis of homogeneity and stability [20].

With regard to the stability of materials, a basic distinction must be made between long-term (shelf life), short-term (under “transport conditions”) and in-use stability (e.g. stability after reconstitution of lyophilised samples) stability. While “shelf-life” provides information about the period of time during which the properties of EQA materials remain within the tolerable limit under ideal storage conditions, extreme transport conditions such as delayed delivery and extreme weather, may need to be simulated in stability studies. Post-distribution assessment of between-laboratory agreement includes variations due to transport effects on stability.

Commutability

“Commutability is a property of a reference (or EQA) material that means results for a reference material […] and for clinical samples have the same numeric relationship, within specified limits, across the measurement procedures (MPs) for which the reference material is intended to be used. Consequently, a commutable reference material produces a measurement result that is equivalent to the measurement result that would be obtained for a clinical sample with the same concentration of the measurand” [21]. Commutability has emerged as a key property of i) Certified Reference Materials (CRMs) used in the calibration hierarchy of an end-user examination procedure and ii) EQA materials used as trueness verifiers [22].

All modifications of the matrix of an EQA sample material during its preparation can affect its commutability, like lyophilization, freezing, addition of preservatives, pooling, or spiking with exogenous substances [23]. This can lead to the inability of the EQA material to mimic the behaviour of clinical specimens and to conduct a proper between MPs evaluation. Behaviour in this context could reflect inter methodological biases that occur when comparing clinical samples not being mimicked by the EQA sample or the dynamic range differing between real samples and those used for EQA. At present, commutability in EQA is just starting to emerge and mostly applies for measurands in clinical chemistry. Work is being undertaken on how EQA providers can implement this routinely into their processes [21].

As some examination procedures are more affected than others by matrix effects, the commutability of an EQA material should be evaluated between each pair of examination procedures, which means that an EQA material may be commutable between some examination procedures but not between others [14], 24]. As a result, trueness assessment will be biased in a way that differs across IVD-MDs, leading to an inability to properly estimate both harmonisation and accuracy of the different assays [25]. It has been reported for some measurands that commutability of EQA sample materials proved to be highly heterogeneous, which means that commutability cannot be predicted [26].

The International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) Working Group on commutability has developed a series of recommendations for assessing commutability [27] according to the difference in bias approach [28] and the calibration effectiveness approach [29] and for correcting the bias caused by non-commutability [30]. In a recent IFCC guideline for commutability assessment in EQA, a practical online tool is presented to the commutability of EQA samples: The criterion for assessing commutability of an EQA sample material between two IVD-MDs is that its result should be within the prediction interval limits based on the statistical distribution of the clinical sample results from the two IVD-MDs being compared. A presupposition for this is that the differences in non-selectivity between the two IVD-MDs being compared are acceptable, the heterogeneity of the measurand is tolerable, and the quantity of the measurand is expressed in molar units [21].

Commutability of EQA materials is necessary in the context of EQA data aggregation. Combining results from various EQA providers may provide a powerful tool to monitor harmonisation of examination procedures in the medical laboratory. Therefore, the International Consortium for Harmonization of Clinical Laboratory Results (ICHCLR) [31] and the European Organization of External Quality Assessment Providers in Laboratory Medicine (EQALM) [32], have joined forces for an initiative called (HALMA) [33]. The acronym stands for HArmonization of measurands in Laboratory Medicine through data Aggregation and aims to collect and aggregate results from different EQA providers that use commutable samples. The purpose is to evaluate and assess the harmonisation of measurands through aggregated EQA data on an international basis. A working group on commutability works in parallel with the IFCC working group on commutability in metrological traceability on a definition of minimum criteria and evidence to accept that samples used in an EQA Program are, with a high probability, sufficiently commutable to represent examination procedure performance for authentic patient specimens.

Commutability of materials is not binary (i.e. commutable or not) but can range, with some materials being more commutable than others. Although a commutability assessment can have three types of outcomes (commutable, non-commutable and inconclusive), such an assessment provides a quantitative assessment of the non-commutability bias. This makes it possible for the EQA providers to evaluate the degree of non-commutability of a material and decide whether it is suitable or not for the intended purpose. Reaching the conclusion that a given material is commutable means that the non-commutability bias is sufficiently small compared to the APS and/or the clinical application of the MP so that the suitability of the material is not compromised [14]. As EQA materials may be used for different purposes, the level of commutability is not the same for all EQA programs. However, non-commutability between reagent lots may hamper this evaluation and reagent lots should therefore be registered in EQA [34].

Organization of a commutability study for EQA materials includes a number of practical challenges. Details on the organisation of a commutability study have been published [21], In brief: At least 30 clinical specimens are obtained, which should be as close as possible to unadulterated clinical specimens, ideally, fresh individual donations. In some cases, this requires specimens from sick donors (usually left over samples). Sourcing fresh single donations is preferable when logistically manageable. An alternative is to use frozen single donations that require demonstrating absence of pooling or freeze/thaw effects on the concentration of the measurand. The specimens as well as the EQA material should be analysed in multiple replicates within a short time interval and at the same time on all MP and together with the EQA sample(s) whose commutability is to be assessed. Details on how to assess the commutability is given in [21] and a user-friendly free online application on how to do this is available [35]. Be aware that since EQA materials are used for a different purpose than CRMs (Table 1 in [21]), the procedures for assessing commutability of CRMs and EQA sample materials are different [21], 36].

While challenging, commutability assessment of EQA materials is critical to appreciate how data analysis can be conducted. When commutability of EQA materials is unknown, results from different peer groups should only be compared with caution. Also, assigning RMP target values to materials of questionable commutability is not desirable as accuracy assessment could be wrong.

The challenge of preparing EQA materials

EQA samples should be such that conclusions can be drawn from the deviations of the measurement results in EQA about the consistency of the results obtained in routine diagnostics with the analytical method to be assessed. To judge the suitability of an EQA sample panel to assess the performance of its participants, the categorisation of Miller et al. is helpful and shown in Table 1 [37]. It should be noted that this categorization does not represent a hierarchical order of superior and inferior materials, but merely a classification according to their possible use.

An essential and defining characteristic of biology is the transience, the temporal change of biological systems and their parts, including humans and the specimens obtained from them. While the limited stability of some measurands in clinical specimens is known and considered in laboratory diagnostics, EQA samples must be designed to withstand even more challenging storage and transport times and conditions between production and analysis. Therefore, various measures are taken to give EQA samples appropriate properties. The first is cooling or freezing, which is also common for human specimens, but not all EQA samples can be cooled or frozen without destroying or at least impairing the measurands and/or the matrix. If cooling or freezing of a sample material is possible in principle, it must be decided whether to accept the disadvantage of an expensive cooling or freezing transport to the participant, or a possibly restrictive change in the samples during transport. These procedures are also used for patient specimens, but other methods for stabilisation are used for EQA samples. In the procedure of freeze-drying (lyophilization) to extend shelf life or make the material more convenient for transport, the water is removed from the samples, which stops biological decomposition processes. However, only materials that remain stable when freezing can be stabilised by lyophilization, so the advantages of this procedure are limited to those that could also be analysed in samples that are stored and shipped frozen. Stabilisers such as protease inhibitors like sodium dodecyl sulphate, and preservatives (i.e. antimicrobial biocides that kill or inhibit the growth of microorganisms) like sodium azide and guanidine isothiocyanate are also used to stabilise both measurands and matrices in samples.

Not all materials are available in the required concentrations in specimens of healthy donors. To produce materials with clinically relevant concentrations of certain measurands in sufficient quantities for EQA purposes, a matrix from a human donor plasma or serum can be spiked with purified substances. Depending on the substance, production of purified compounds in their pure form is often carried out by e.g., chemical-pharmaceutical processes, by animals, by bacteria in bioreactors, or virus culture methods. Synthetic substances often do not fully correspond to the endogenous compound of human origin. Depending on their specificity, different test systems already detect measurands of human origin to varying degrees. These differences can be even greater when samples are spiked with exogenous non-human materials. It should be considered that binding of some artificial substances to specific binding proteins in blood may not correspond to authentic substances of human origin. Therefore, spiking EQA materials with exogenous compounds should always be assumed as non-commutability unless proven otherwise and/or a (in)voluntary generation of challenging samples.

Spiking, cooling, freezing, lyophilizing, adding stabilisers or preservatives, but also simply the passage of time – all of these factors can affect commutability of EQA samples [21]. Therefore, the EQA provider must be well familiar with the evaluation of advantages and disadvantages and the selection of the procedures to be used to produce samples.

After all the fundamental obstacles mentioned so far in this chapter have been considered, producing EQA samples begins with selecting the starting material, i.e. the matrix. The starting material can be of human origin (e.g. whole blood, serum, plasma, urine) or produced synthetically. Starting material of human origin can potentially contain pathogens existing in donor blood and thus be infectious. Donor blood or blood components used for EQA purposes may, therefore, be tested like blood for transfusion purposes for the presence of pathogens. Yet laboratories are pointed out to the existing potential infectiousness. Of course, things are different with EQA samples for pathogen detection; In this case, samples with existing or added, but possibly also inactivated, pathogens are produced and labelled as “biohazardous” and sent for examination in EQA cycles. For some EQA samples, substances corresponding to measurands are added to the matrix. This is necessary so that the materials remain unaltered for the time between production, storage and shipping for analysis in the laboratory. The completed EQA sample material is filled in the required volume into suitable containers. These are then sealed or, if necessary, the filled samples are lyophilized before sealing. The stability of formulations has already been determined on their qualification as a sample material; nevertheless, stability tests of individual batches are sometimes performed to demonstrate the continued suitability of the formulations. For this purpose, relevant measurands are determined in several individual samples and the results are compared with those obtained from samples of the same batch later (e.g. after completing the EQA cycle or at the end of the accepted durability period). There is sufficient stability if the results differ by less than the acceptable maximum. Completed homogeneity tests confirm that it can be excluded with the highest statistical probability that divergent results obtained from different samples of the same batch are attributed to their different content (differences in concentration or absolute content). ISO 33405:2024 describes processes for assessing the homogeneity and stability of reference materials [19]. During and after conducting homogeneity and stability tests, the sample materials are stored under appropriate conditions, i.e. ultra-deep frozen, frozen, cooled or stored under environmental conditions. The samples are stored temporarily at least until the required proof of their homogeneity and, if necessary, their stability has been provided. Many EQA providers also examine the commutability of EQA samples at the time of the batch release (Figure 1).

Figure 1:

EQA sample preparation. The preparation begins with the determination of the intended use of the individual proficiency test samples, the selection of suitable starting materials and, if necessary, the addition of compounds, stabilisers and preservatives. After portioning and filling into suitable containers, the homogeneity and, if necessary, stability and commutability (EQA Categories 1–4) of the samples are tested; for samples intended to be used in EQA Categories 1 or 2, target values are determined by a RMP. If the results of the quality controls meet the requirements, the batch is released for use in EQA.

The production of EQA materials for several cycles, as well as the joint procurement of materials by several EQA providers, have economic advantages since the characterization of the material is associated with high costs, especially if reference methods are used. Production in bulk works well for highly stable substances such as Cortisol, Sodium, TSH, DNA derivatives of well-established cell lines, etc. Still it would not be possible for whole blood material without the addition of preservatives/stabilising agents. The advantage of having a repeat distribution of the same material is that it is possible to review performance over time, not only at the snapshot of a single EQA cycle. In addition to the economic benefits, another advantage of multiple EQA providers using the same material in their cycles is that it significantly increases the number of laboratories and test systems analysing the same samples at approximately the same time, allowing a more comprehensive comparison of laboratory and test method performance, even in different geographical regions, and thus also in different framework conditions. Such concerted EQA activities, their benefits and examples are presented in section “EQA providers’ networks” in Part V [4].