Sparing the control arm using well-characterized diagnostic approaches – the Gart and Buck prevalence estimator for efficacy estimation in single-arm trials

Brief summary

Especially in situations where a double-arm trial is not feasible or is regarded as ethically unacceptable, options for obtaining reliable efficacy estimates from single-arm trials can be very valuable to provide evidence. Accordingly, we suggest a procedure that allows for obtaining efficacy estimates in single-arm trials using methods to adjust for imperfect predictive accuracy of diagnostic tests.

Efficacy estimations in clinical trials rely on diagnostic results that confirm the endpoint. The sensitivity and specificity of diagnostic tests used for endpoint definitions are usually lower than 1, which generates misclassification. An early discussion of the impact of sensitivity and specificity on the reliability of prevalence measures was published in 1966 [1] and 12 years later [2]. The authors of both studies [1], [2] presented an identical prevalence estimator adjusted for diagnostic accuracy. In 1998, Lachenbruch [3] published an adjusted estimator for intervention efficacy based on the adjusted prevalence estimator. This estimator is important for clinical trials as efficacy estimations in such trials are biased if the impact of diagnostic sensitivity and specificity on effect estimations is neglected. If the sum of sensitivity and specificity is greater than 1 but smaller than 2, the measured (unadjusted) effect underestimates the true effect of a medical intervention. If the sum of sensitivity and specificity is even smaller than 1, the measured (unadjusted) effect of a medical intervention is actually in the opposite direction to the real effect. Accordingly, the diagnostic accuracy has to be taken into account while planning clinical trials.

Our approach of estimating the efficacy of medical interventions in single-arm trials is based on the Gart and Buck prevalence estimator that is commonly known as the Rogan-Gladen estimator and which is adjusted for diagnostic accuracy. So far, a sensitivity- and specificity-adjusted estimator for treatment efficacy has been presented only for double-arm trials [3].

For estimation of the efficacy of medical interventions in single-arm trials, the predictive accuracy (sensitivity and specificity) of an additional endpoint-associated predictor (apart from the intervention under investigation) and its distribution in the trial population have to be known. Such predictors can be any information of relevance for which the predictive accuracy for the endpoint is known. Further, we adapted the concept of adjusting prevalence and efficacy estimates of medical interventions for diagnostic accuracy to single-arm trials using the prognostic accuracy of a predictor for the endpoint. In the course of this, we will use the following definitions:

With the binomial-distributed random variable T_E for the rate of positively tested individuals for an endpoint E given, an unbiased point estimator for the prevalence of E that is adjusted for diagnostic sensitivity and specificity was introduced in 1966 [1] as well as in 1978 [2]. It is known as the Rogan-Gladen estimator which is given as follows:

Note that the sum of the sensitivity and specificity has to be unequal to 1. For the accuracy of a prevalence estimation based on an adjustment for diagnostic accuracy, it is more important that the sum of sensitivity and specificity is considerably different from 1 than simply that it be larger than 1. So, a diagnostic test with a sum of sensitivity and specificity that is even smaller than 1 would lead to smaller confidence intervals than a diagnostic test with “better” accuracy but a sum of sensitivity and specificity that is close to 1.

The variance of Prev_E as published by Rogan and Gladen is given as follows:

Lachenbruch [3] used the aforementioned estimator and introduced an adjusted estimator for treatment efficacy based on identical diagnostic accuracy over the study arms. Lachenbruch’s diagnostic accuracy-adjusted efficacy estimator of a medical intervention M_E for varying diagnostic accuracy over the study arms can be given as follows:

with IncidenceE1 indicating the adjusted estimated incidence in the study arm with intervention and IncidenceE2 indicating the adjusted estimated incidence in the non-interventional study arm. The variance of M_E is given as follows:

with the variances as given by equation (2).

We demonstrate how to estimate the efficacy of medical interventions in single-arm trials by the following approach: (i) using the prevalence estimator adjusted for diagnostic inaccuracy using known sensitivity and specificity of the diagnostic test for the endpoint; (ii) using the knowledge of an endpoint predictor and its accuracy; (iii) using the prevalence estimator adjusted for diagnostic inaccuracy using known sensitivity and specificity of the diagnostic test for the predictor. Of course, the survey of the predictor distribution among the study population should be done before baseline and blinded.

In a population without medical intervention, for predictor P and incidence I holds (law of total probability):

With known predictive accuracy of the predictor for the endpoint, an unbiased and consistent estimator for incidence without medical intervention is given as follows:

where PPV_P is the positive predictive value for the predictor being present, PPVPC is the positive predictive value for the predictor being absent and Prev_P′ is the prevalence of the additional predictor. The variance of IMC is given as follows:

To estimate the efficacy of a medical intervention in a single-arm trial, all individuals initially tested for a predictor with known predictive accuracy for the endpoint will be subjected to the medical intervention and followed up for disease independently of the initial diagnostic test result for the predictor.

The estimator for the efficacy of the medical intervention M_E in a single-arm trial is now given as follows:

where I_M is the incidence of the population being subjected to the medical intervention and IMC is the incidence of the population without medical intervention, with

and

The variance of M_E is given by

with

It should be noted that the predictive accuracy (sensitivity and specificity) of the predictor for the endpoint has to be well known. Furthermore, the evaluated incidence period in the study should be the same as the predicted incidence period. An example is provided in Supplementary Material 1.

Single-arm trials are unlikely to replace the generally accepted standard of investigator-blinded, controlled double-arm trials but they can be used if circumstances make a double-arm trial very difficult or even impossible. Efficacy estimators can be considered if the impact of a predictor on the risk to reach the study endpoint is known. A decision analysis model supplemented by a Monte Carlo simulation could be such an approach to estimate the accuracy of possible predictors. Several methods for the estimation of diagnostic accuracy have been published [4] and these could also be adapted.

Also, precise knowledge of the sensitivity and specificity of all diagnostic tests conducted is essential for reliably estimating the efficacy of a medical intervention. This requirement is not limited to single-arm trials.