Analysis of a 6-year pilot external quality assurance survey of free light chain using Sigma metrics

Introduction

Free light chain (FLC) assays quantitatively measure light chains that are not bound to immunoglobulin heavy chains, and detect clonal restriction through a skewed ratio of free kappa (κ) to free lambda (λ) light chains [1]. The assay uses polyclonal antibodies that bind exclusively to the hidden epitopes of FLC molecules located at the junction of the immunoglobulin heavy and light chains in an intact immunoglobulin. These epitopes are only accessible when the light chain molecules are not bound to the heavy chain, which makes the assay highly specific. The use of latex enhancement further enhances the sensitivity of the assay to a few mg/L, and the assay can be performed by turbidimetry or nephelometry on a number of automated laboratory instruments.

The FLC assay has become standard practice and is recommended in various guidelines for diagnosis, prognostic assessment and evaluation of patient response to treatment in multiple myeloma and related plasma cell disorders [2], [3], [4], [5], [6]. However, technical concerns with the FLC assay include non-linearity and antigen excess, FLC polymerization, possibility of undetected “private” epitopes, lack of international standards, lot-to-lot variability of the reagent and inter-instrument variability.

In this regard, an external quality assurance (EQA) survey for the quantitative FLC assay was developed and implemented in collaboration with the Quality Assurance Committee of the Korean Society for Laboratory Medicine. We analyzed the 6-year EQA survey data and applied Sigma metrics to evaluate the observed analytical characteristics of the FLC assay and to assess the quality of the FLC assay that is being achieved.

Materials and methods

Results

Nineteen to 29 laboratories participated in the EQA survey. All participating laboratories used the FLC Freelite^® kits manufactured by Binding Site. Nephelometric (20%) and turbidimetric (80%) methods were used for the FLC assay, and different instruments including Siemens BNII (Siemens Healthineers, Deerfield, IL, USA), Siemens BN-Prospec (Siemens Healthineers), Roche Modular P (Roche, Mannheim, Germany), Hitachi analyzers (Hitachi, Tokyo, Japan), Toshiba (Toshiba, Tokyo, Japan), Siemens Advia 1650 (Siemens Healthineers) and Binding Site SPAPLUS (Binding Site) were used (Table 2).

Method-specific total error and Sigma metrics

The method-specific median TE values of κ FLC across 29 EQA specimens were 26% for nephelometry and 25% for turbidimetry. The method-specific median TE values of λ FLC across 29 EQA specimens were 24% and 19% for nephelometry and turbidimetry, respectively.

As the Sigma metric is inversely proportional to CV_{peer group} and EQA distributions with low FLC concentrations (less than the lower limit of the reference interval) are associated with a disproportionately large CV_{peer group}, three κ FLC and three λ FLC EQA distributions with low FLC concentrations were not included in the Sigma metrics analysis. The method-specific median Sigma metrics of κ FLC across 29 EQA specimens were 4.9 for nephelometric methods and 4.9 for turbidimetric methods. The method-specific median Sigma metrics of λ FLC across 29 EQA specimens were 4.9 for nephelometric methods and 5.4 for turbidimetric methods. There were no statistically significant differences in TE and Sigma metrics between turbidimetric methods and nephelometric methods for κ FLC; however, for λ FLC, nephelometric methods showed larger TE and lower Sigma metrics compared with turbidimetric methods (p<0.05) (Figure 1).

Figure 1:

Sigma Proficiency Assessment Chart for κ FLC (A) and λ FLC (B) with TEa=54.5%.

Diagonal lines from top to bottom represent 2σ and 3σ. The x-coordinate represents the interlaboratory precision observed for the method-specific peer group. The y-coordinate represents the bias compared to the laboratory consensus value used as the major performance goal of the EQA survey.

Method harmonization

We compared the median reported results of the nephelometric and turbidimetric methods for each EQA specimen. Due to the small number of laboratories reporting nephelometric results, the Bland-Altman plot was used. The Bland-Altman plot showed that the average of the difference between the nephelometric and turbidimetric methods for κ FLC was 7.2 mg/L and for λ FLC was 42.1 mg/L (Figure 2). While the average difference between the two methods for κ FLC can be regarded as insignificant, for λ FLC there seems to be a significant difference between the two methods, and the difference seems to be due to higher λ FLC concentrations that would require a different sample dilution from that of the initial measuring range.

Figure 2:

Bland-Altman plot of median reported results of nephelometry and turbidimetry for each EQA specimen.

Shown are absolute difference plots for κ FLC (A) and λ FLC (B).

Discussion

The FLC assay is extensively used for the diagnosis, prognostic assessment and management of patients with monoclonal gammopathy. The assay can be used on several types of automated analyzers and has been adopted for use in many laboratories nationwide in Korea as an indispensable resource. However, as with any clinical laboratory test, EQA (also known as proficiency testing) is considered a mandatory component of a laboratory quality system [15]. National EQA schemes for the FLC assay have been developed and implemented in France, United Kingdom and Czech [16], and a number of international schemes also exist. The lack of a national EQA scheme in Korea prompted the development and implementation of an EQA survey for the FLC assay in collaboration with the Quality Assurance Committee of the Korean Society for Laboratory Medicine. There have been more than 30 distributions over a 6-year period, and over 35 laboratories nationwide participated in the scheme.

In this retrospective analysis of the EQA scheme, we applied clinically relevant quality specifications using Sigma metrics. Sigma metrics combine bias, imprecision and the allowable TE and convert them into an overall assessment of the analytical quality of the test [15]. For that purpose, the previously described Sigma Proficiency Assessment Chart was used [9].

However, due to the lack of a well-established consensus definition of quality requirement regarding the FLC assay, multiple surrogate definitions of quality requirements, including those based on biological variations [12], [17], [18] and/or measurement uncertainties [1], were assessed. We chose a reported RCV value of 54.5% derived from the estimate of monoclonal FLC variability in patients with plasma cell disorders [12] and applied as tolerance limits in calculating the Sigma metrics for κ FLC and λ FLC. We acknowledge the limitations of using an RCV value as a quality specification as it does not have component for bias and therefore may not be an appropriate substitute for an allowable TE. Moreover, an RCV value of 54.5%, which is derived from the estimate of monoclonal FLC variability in patients with plasma cell disorders, is much larger than reported biological variation of FLC in healthy individuals. However, FLC is clinically a tumor marker and typically in myeloma the monoclonal FLC levels are variably increased above the upper limit of the reference interval. Therefore, a biological variation-based quality requirement may not be a clinically relevant quality specification for the FLC assay. In this regard, the RCV value of 54.5% was chosen because it was comparable to the reported interlaboratory reproducibility values achieved during the UKNEQAS FLC EQA scheme [14] and was similar to a clinical cut-point of 50% FLC reduction defining a clinically relevant FLC response reported in a previous clinical recommendation [10].

The method-specific Sigma metrics of κ and λ FLC achieved greater than Three-Sigma for 86–97% of all EQA distributions, and the higher quality specifications consistent with Five-Sigma, beyond which little improvement to quality can be achieved [11], was achieved in 48% of all κ FLC EQA distributions, for the nephelometric and turbidimetric methods, respectively, and 48% and 72% of all λ FLC EQA distributions for the nephelometric and turbidimetric methods, respectively. When only EQA distributions with consensus κ and λ FLC values in the vicinity of κ or λ medical decision points were considered, the method-specific Sigma metrics of κ and λ FLC achieved greater than Three-Sigma for all EQA distributions for both methods, and Five-Sigma was achieved in 67% and 100% of all κ and λ FLC EQA distributions, for the nephelometric and turbidimetric methods, respectively.

We also assessed the harmonization of results between the nephelometric and turbidimetric methods. While the average difference between the two methods is not significant for κ FLC, for λ FLC, the Bland-Altman plot showed a significant difference between the two methods at higher λ FLC concentrations that require a different sample dilution from that of the initial measuring range. This is not surprising as significant method-specific differences in FLC concentrations have been previously reported [14]. Hence, despite the use of the same source of commercial reagent, values were not completely harmonized between assay systems that use different method principles. Possible causes of discordant results between methods include different reaction times, differences in the antigen (sample)/antibody (reagent) ratio in the reaction mixture, different concentrations of polyethylene glycol, different sample dilution protocol and different linearity. The positive difference in nephelometry at higher λ FLC concentrations demonstrated in this study should be investigated further in a larger number of samples.

The method-specific median interlaboratory reproducibility (CV) of κ FLC and λ FLC across 32 EQA specimens ranged from 10 to 12% for both the nephelometric and turbidimetric methods. Interlaboratory reproducibility values reported in this survey were considered acceptable for clinical practice and were better than those achieved during the UK NEQAS external control cycles. Moreover, EQA specimens that had consensus values in the vicinity of κ or λ medical decision points achieved an all-method interlaboratory reproducibility value (CV) of 8–14%, which may be larger than, but not significantly different from, the manufacturer’s predetermined acceptance criteria of total precision (%CV<8.5%) and between-instrument precision (%CV<8.5%), for a relatively narrow FLC concentration range without the need for serum dilution [19].

EQA provides an essential tool for medical laboratories for evaluating assay performances and establishing quality standards [20]. During the 6-year EQA survey, we found that most of the results from participating laboratories meet clinically relevant quality specifications. We could also provide an estimate of the interlaboratory variability in the FLC assay that participants and physicians should be aware of. In addition, we have highlighted the method-specific differences for λ FLC and also how they tend to increase with increasing FLC concentrations. This may have the potential to mislead clinical assessment of the FLC assay results when serial FLC results involving different methods are used.

There are acknowledged limitations of this study. First, the small number of participating laboratories comprising the nephelometric method subgroup may have introduced bias in the estimation of method-specific performance, despite the use of a weighted average. Second, due to the lack of a well-established consensus definition of quality requirement regarding the FLC assay, a reported RCV value of 54.5% derived from the estimate of monoclonal FLC variability in patients with plasma cell disorders was applied as the tolerance limit in calculating the Sigma metrics. Lastly, as a reference measurement procedure or certified reference material is not available for FLC, we could not assess the reported results of the EQA scheme with reference to trueness.

Despite these limitations, the data collected by the first national EQA scheme for the FLC assay in Korea represent a unique resource for clinical laboratories. We used Sigma metrics and a clinically relevant tolerance limit in analyzing the FLC EQA scheme. The study also shows that between-method differences still exit, and further standardization and harmonization efforts are necessary.