Linearity assessment: deviation from linearity and residual of linear regression approaches

Introduction

Linearity assessment is an evaluation of the linear relationship between the measured (observed) values of a measurand and the true (assigned) values in a series of samples [1, 2]. There is guidance on the requirements to perform linearity assessment as part of pre-implementation method evaluation and periodic (e.g. six monthly or when analytical issue is suspected) reassessment of linearity assessment. Ideally, linearity assessments should be performed using commutable materials (e.g. residual patient samples) to ensure findings are translatable to results from patient samples. Non-linearity is generally sub-optimally detected or monitored by internal quality control procedures since they examines limited number of samples/concentrations. At the same time, the presence of significant non-linearity may be associated with poorer proficiency testing performance, likely depending on the concentration of the proficiency materials [3, 4].

Several approaches have been described to assess linearity in the clinical laboratory [1, 5]. Early assessment procedures for linearity were based on visual assessment of manually graphed points and straight line of best fit characterised. However, this approach is highly subjective in nature and not readily reproducible. More recently, software developments combined with formal statistical assessment have improved the objectivity and reproducibility of such assessments [6], [7], [8], [9].

In this computer simulation study, in recognition of the varying statistical approaches, study designs and laboratory conditions we examine four different statistical approaches of linearity assessment, including two variants of deviation from linearity, along with detection capabilities of residuals of linear regression. From this, we aim to provide the performance characteristics of these statistical approaches and study designs.

Methods

To evaluate linearity, samples with equidistant concentrations spanning an arbitrary analytical measurement range are simulated. The results of the simulated measurement are fitted with a straight line (linear regression), as well as quadratic and cubic lines (quadratic and cubic regressions) as defined below (Figure 1A):

Figure 1:

Illustration of statistical model fitting for measurement results in linearity assessment. (A) Fitting linear and nonlinear models to measurement results; (B) residuals of linear regression. Note curved nature of residuals indicating non-linearity.

Approaches to linearity assessment studied

Four approaches to linearity are assessed: (1) deviation from linearity – individual (IDL); (2) deviation from linearity – average (ADL); (3) regression residual – individual; and (4) regression residual – average.

Individual deviation from linearity (IDL)

Alternatively, instead of averaging the differences across all concentration points, the individual deviation from linearity, which is defined as the absolute difference between the linear and nonlinear fits (quadratic and cubic) at each concentration, it can be examined:

IDL i = |  linear fit − nonlinear fit  |

The IDL at each concentration was then compared against an a priori defined linearity acceptance limit using the average of the replicate measurements (if performed). When any of the IDLs exceeded the specified limit, non-linearity was considered detected.

Average deviation from linearity (ADL)

The average deviation (difference) between the fitted linear regression line with the quadratic or cubic regression lines is the average deviation from linearity (ADL), defined by the following formula [1, 8]:

ADL = ∑ n ( Linearfit − nonlinearfit ) 2 n

where n is the number of concentration levels.

In the presence of non-linearity, the detection capability of ADL was then compared against the pre-specified limit. In routine laboratory practice, the linearity acceptance limit can be set based on a priori determined analytical performance specifications such as those based on biological variation or state-of-the-art for the measurand.

Residual of regression

Non-linearity can also be assessed using the estimated residual from the regression models [9]. A residual is defined as the difference between the measured results with the fitted/predicted value by the regression model (Figure 1B). Two variations of residuals are explored here, namely the individual and average residual approaches.

As the name implies, the individual residual method compares the residual for each replicate at each concentration level against an a priori defined linearity acceptance limit. For the average residual methods, the residuals for each replicate at the same concentration level were first averaged before comparing it with the specified limit. If any residuals were larger than the limit, non-linearity is considered present.

Simulation parameters and statistical analysis

In this study, a simulated dataset with a range of laboratory parameters was generated to evaluate the false detection rate (false positive rate) and power of non-linearity detection (i.e. non-linearity detection rate) using the ADL, IDL, individual and average residual approaches. The concertation points or x-values are equidistant relative concentrations ranging from 0 to 1, while the y-values of the dataset are measurement results simulated based on the parameters in Table 1. The relative concentration is the relative position of the concentration along the analytical measurement range with 0 being the lowest concentration, 0.5 being the mid-point of the analytical measurement range and 1.0 being the highest concentration. The range ratio is the ratio between the lowest concentration and highest concentration of the analytical measurement range.

Table 1:

Parameters of the simulated dataset.

Parameter	Values
Number of equidistant concentrations, n	5, 6
Range ratio of measurement results	1:10, 1:1,000
Analytical coefficient of variation (imprecision), CV	3 %, 10 %
Replicates	1, 2, 3
Degree of non-linearity for quadratic and cubic modellinga	5 %, 10 %

1. ^aNot applicable for datasets with underlying true linear trends.

For the evaluation of the false positive rates, datasets representing measurement results with an underlying ‘true’ linear trend were simulated. In other words, these data do not contain any non-linear data sets. Under each scenario, 20,000 iterations of simulations were performed. The non-linearity detection approaches described above were then applied to these datasets to determine the proportion of (false) non-linearity detections, which is equivalent to the false detection rate (as the true underlying trend is linear).

Similarly, the power of non-linearity detection (i.e., non-linearity detection rate) can be evaluated when the same methods above are applied to simulated datasets with ‘true’ underlying quadratic or cubic trends. These data represents the non-linear data sets. The degree of non-linearity for the underlying data, Δ, is defined as the largest difference in the measurement values between the linear line and the quadratic or cubic line, divided by the difference between the lowest and the highest measurement concentration and expressed in percentage as displayed in Figure 2. This study arbitrarily sets the non-linearity acceptance limits at 5 and 10 %, respectively. For the quadratic dataset, Δ is located at the relative concentration of 0.5 while for the cubic dataset, Δ is defined at a relative concentrations of 0.2 or 0.8 due to the S-shaped nature of this trend (Figure 2). The simulation study was implemented in Python with libraries including statsmodels, numpy and pandas using a framework that has been previously described [10], [11], [12].

Figure 2:

The introduced degree of non-linearity is defined as the largest difference between the quadratic or cubic regression line with the linear regression line (Δ) divided by the maximum range of measurement. Δ is located at the relative concentration(s) of (A) 0.5 for quadratic, (B) 0.2 and 0.8 for the cubic dataset.

Results

Using data that represents ‘true’ linear trend (i.e. data sets without non-linearity), the ADL false detection rate was generally lower when imprecision was low (CV=3 %) but rose considerably approaching 35 % at higher CV (10 %) (Table 2). By increasing the number of equidistant concentration points and replicate measurements, there was an improvement (reduction) in the false detection rate, with the latter having the greatest impact. There was a small deterioration in ADL performance of no more than 3 % at a high range ratio (1:1,000) when compared to the low range ratio (1:10). A measurement procedure with a high range ratio has a high ratio of maximum to minimum concentration of the analytical measurement range, e.g. an analytical measurement range of 3 mmol/L–3,000 mmol/L would have a range ratio of 1:1,000. Conversely, a measurement procedure with an analytical measurement range of 100 mmol/L–1,000 mmol/L would have a low range ratio of 1:10.

The false positive rate of IDL is generally much greater than ADL since an assessment is performed at each concentration level (Table 2). Similar to ADL, higher assay imprecision (10 %) is associated with a higher false positive rate. In contrast to ADL, increasing the number of samples and replicate measurements generally increased the false positive rates of IDL. Of concern is the markedly elevated IDL false positive rate at the higher range ratio, particularly in the setting of replicate measurements.

In general, the false positive rate for the residual of linear regression approaches is greater than the deviation from the ADL and IDL approaches (Tables 2 and 3). The false positive rate of the average residual approach declines with increased replicates but increases with the increased number of concentration points. By contrast, the individual residual approach had an increased false positive rate when a higher number of samples or replicate measurements was used. At the higher imprecision (CV=10 %), the false positive rates of the residual of linear regression approaches are >50 % except when a high linearity acceptance limit (10 %) is used with multiple replicate measurements.

From the data sets that represents ‘true’ non-linear trends (i.e. data sets containing varying degrees of non-linearity), displayed in Table 4 is the power (detection capability) for underlying true non-linear quadratic and cubic simulated data for ADL and IDL when applied in quadratics or cubic detection strategies. In general, the power for detection of non-linearity is higher when the polynomial regression model matches the true underlying non-linear trend in simulated data. Subsequently, in this paper, results relating to ADL and IDL approaches are only shown in cases where the polynomial regression model and underlying data trend are matched, i.e. quadratic polynomial performance is benchmarked against underlying quadratic polynomial simulated data trends.

In contrast to IDL, the ADL approach generally had lower power for detection of non-linearity particularly when the non-linearity acceptance limit is wide (10 %), relative to the degree of introduced non-linearity (5 %) (Table 5). When the non-linearity acceptance limit (5 %) is set the same as the degree of non-linearity (5 %), the power for detection of non-linearity of the ADL method still remain sub-optimal (<80 %). Quadratic data was generally associated with lower non-linearity detection rates. A high range ratio (1:1,000 vs. 1:10) improves non-linearity detection, as does higher imprecision (10 vs. 3 %). Increasing the number of samples and replicates decreases the non-linearity detection rate with the ADL approach. The opposite is observed with IDL where a higher number of sample and replicate measurements is associated with increased power for detection of non-linearity. When a non-linearity of 10 % is introduced, it is generally well detected by a narrower non-linearity acceptance limit (5 %) (Table 6) and has a lower power for detection of non-linearity when the ADL approach is applied to data with underlying quadratic trends.

The power for detection of non-linearity for the residual of the linear regression approaches are generally higher than the deviation from linearity approaches (Tables 6 and 7), with the individual residual approach having higher power for detection of non-linearity than the average residual approach. However, these approaches are generally associated with high false detection rate. A smaller imprecision (3 %), a larger degree of non-linearity (relative to the acceptance limit), a higher number of samples, and replicate measurements, combined with data that has an underlying cubic trend, are associated with higher power for detection of non-linearity and low false positive rate.

Discussion

Linearity is an important characteristic of a measurement procedure that requires careful assessment for its fitness for purpose [13]. This study examined four different approaches for linearity assessment (1) deviation from linearity – individual (IDL); (2) deviation from linearity – average (ADL); (3) residual – individual; and (4) residual – average. A comparison of their false detection and power for detection of non-linearity was reported. This assessment provides evidence to guide laboratory practitioners in their performances under different scenarios.

Across the four different approaches (deviation from linearity – individual and average, residual – individual and average), a major factor associated with high false detection rates is analytical imprecision. The influence of high assay imprecision can be reduced using more replicate measurements and subsequently averaging observed results. Additionally, in the deviation from linearity approaches, higher false detection rates are observed when cubic regression models are applied, in comparison to quadratic regression models.

Power for detection of non-linearity can be improved by using the polynomial regression model that matches the underlying trend of the data. Given the true underlying trend of the data is generally unknown, in addition to linear modelling both quadratic and cubic regression models should be applied concurrently to the dataset. However, this may increase the overall false detection rate, which is the sum of the false detection rate of quadratic and cubic regression approaches. In scenarios where non-linearity reside mostly in one end of the measurement range (e.g. at the higher concentration end), the quadratic regression may better detect the deviation and reject the non-linearity.

From the results of the study, deviations from linearity both average and individual, are suboptimal for detecting relatively small departures (5 %) from linearity, as these approaches are either associated with a high false detection rate and/or marginal power for detection of non-linearity. However, a high power for detection of non-linearity (>80 %) combined with a low false positive rate (<10 %) can be achieved by using the IDL approach, following the conditions/constraints, low imprecision (3 %); low range ratio (1:10); replicate measurements; small non-linearity acceptance limit and regression model matched to the underlying trend in the data (cubic in this case).

For larger departures from linearity (10 %), the ADL approach achieves >95 % power for detection of non-linearity with <1 % false detection rates when assay imprecision is small, replicate measurements are performed and when a relatively small non-linearity acceptance limit (to increase the power of detection) is used.

For the residual of regression approaches, a small degree of non-linearity (5 %) with the underlying cubic trend can be detected with >95 % probability but at the expense of high false positive rates. On the other hand, <10 % false detection rate was observed if the average residual approach is used with a low imprecision measurement procedure and replicate measurements are performed. By contrast, a large degree of non-linearity (10 %) with an underlying quadratic or cubic trend can be detected with a power for detection of non-linearity of >95 %, albeit with high false positive rates. Similarly, a false positive rate of generally <10 % was reached for measurement procedures with low imprecision and with replicate analysis.

The following suggestions are provided to laboratory practitioners when performing linearity assessment:

1. A high imprecision can challenge linearity investigations by producing a high false positive rate or low power of detection. Therefore, the imprecision of the measurement procedure should be considered when interpreting linearity assessment results. In the presence of high imprecision, the results of linearity assessment should be interpreted with caution.

2. Different linearity assessment approaches examined in this study performed well under different analytical scenarios. For optimal outcomes, a considered and tailored study design should be implemented.

3. Providing assay imprecision is low:

4. Measurement procedures with a low range ratio (1:10) and a suspected small linearity departure, may be best detected by the IDL approach when triplicate measurements are undertaken (power >95 %; false detection rate <10 %) with a relatively large allowance for non-linearity (acceptance limit) of 10 % is budgeted.

5. Measurement procedures with a high range ratio (1:1,000) and a suspected small linearity departure, may be best detected by the IDL approach when a triplicate measurement is performed (power >95 %; false detection rate <5 %) and a small non-linearity acceptance limit (5 %) is adopted.

6. For a suspected large degree of non-linearity, ADL with at least duplicate measurements combined with a relatively small non-linearity acceptance limit (5 %) will produce a power of detection of >95 % and a false positive rate of <5 %.

7. Averaged residual of linear regression with triplicate measurements with a non-linearity acceptance limit of 5 % produces a low false positive rate of <5 % and a high power for detection of non-linearity of >70 % across different types and degrees of non-linearity.

Conclusions

This study successfully explored four different statistical approaches (deviation from linearity – individual and average, residual – individual and average) for the assessment of linearity. From the results of the study, with the exception of specific scenarios, both ADL and IDL methods were suboptimal for the assessment of linearity compared. Detection of departures from linearity are difficult to identify in practice and enhanced methods of detection need development.