Meta-analysis comparing placebo responses in clinical trials of painful HIV-associated sensory neuropathy and diabetic polyneuropathy

2.1 Search strategy

All studies included in the 2015 comprehensive systematic review of pharmacotherapy for neuropathic pain in adults [13] were assessed for inclusion. This review consisted of a search of PubMed, Medline and EMBASE databases and the Cochrane Central Register of Controlled Trials to identify randomized, controlled, double-blind studies in neuropathic pain published between January 1966 and January 2014. Studies reporting results on ClinicalTrials.gov were also included (for full search strategy see Finnerup et al. [13]). The original search was conducted on 31 January 2014, and was updated for this analysis on 1 August 2017 using the same search strategy. An additional search performed on June 24 2019 confirmed that no new placebo-controlled trials in painful HIV-SN have been published since the last performed search.

2.2 Inclusion and exclusion criteria

All studies identified in the systematic search, referring specifically to painful DPN and HIV-SN (or both conditions), were assessed and only systemic interventions (intravenous, oral, sublingual and intra-nasal) were included. Topical interventions were not included due to the small number of studies identified (only two studies). Both parallel and cross-over designs with more than 10 patients per group and longer than 3 weeks duration of therapy were included. All articles were assessed for quality of methods reporting using the Oxford Quality scoring system [14] (or Jadad scale) and only those with a score of 2 or greater were included.

2.3 Data extraction

The original search and screening was led by NF as described in the publication Finnerup et al. [13] and the update by SH. Double data extraction was performed on each study (by HK, SH, SN, ABA, LV) and any discrepancies decided through group decision. The following data about trial design were extracted: class of drug intervention, final year of enrolment (or if not reported, year of publication as a proxy), maximum daily dose, dose frequency and flexibility, whether the study had reported a “positive” or “negative” outcome, number of patients in placebo and active cohorts, study design (parallel or crossover), duration of study arm, type of comparator (active or inert), the number of dropouts, who sponsored the study (industry vs. investigator/public), type of healthcare system (private, public or both), the country of lead study site, number of participating centres, the randomization ratio, whether there was a pre-randomization run-in period, whether blinding or adherence monitoring was reported, the cut-off pain intensity for inclusion and the number of contact with study staff (converted to the number of contacts per week of the study). The following participant characteristics were extracted: the mean age, gender ratio, body mass index, baseline duration of pain, baseline pain intensity and variability, baseline anxiety, depression and insomnia scales, proportion of placebo and active groups with a >50% pain reduction from baseline and >30% reduction from baseline or other measure of response determined by the study, percentage change in pain score from baseline (converted from Gracely scales [15], [16] if required; per cent change in Gracely pain score was calculated as the change in percentage of inverse log values of reported Gracely pain score from baseline to follow-up. Since there is no robust conversion from absolute change in Gracely pain scales to a 0–10 scale, any studies reporting only the absolute, not percentage change, could not be included in the magnitude analysis), mean reduction in pain score, number of adverse events and percentage of drop outs.

All pain intensity scales were converted to a 0–10 scale and only data from the first period of cross over studies (i.e. results prior to crossover) were included to reduce bias. The proportion of placebo responders were defined either as the proportion of responders with a response of >50% improvement from baseline or, if not described, a response of >30% to placebo or other primary outcome measure from the trial. The primary outcome was therefore ratio of responders for >50% improvement, if reported; if not reported, then ratio of responders for >30%, if reported; if neither reported, then any other primary outcome measure of the trial. The magnitude of placebo response was defined as the mean change in pain rating in response to placebo, or calculated from mean pre and post pain ratings.

2.4 Statistical analysis

The placebo response was defined firstly by the mean magnitude of the change in pain rating in the placebo arm. To calculate the effect size of the placebo response the standard deviation of the mean change in pain ratings was required. If not reported the standard deviation was calculated using reported confidence intervals, standard errors or t-test values. For those studies without sufficient data reported, standard deviations were imputed using techniques described by the Cochrane collaboration [17]. Means were calculated from studies reporting medians using the technique described by Hozo et al. [18].

Secondly, a meta-analysis was also performed on the proportion, or ratio, of placebo responders in the placebo group as defined by the criteria above (number meeting criteria of a response to placebo/total number of participants in placebo arm). The meta-analysis estimates for magnitude and proportion are reported as an estimate (95% confidence interval).

Heterogeneity, the total variation across studies, was assessed using I². A random-effects model was used to pool results of both the magnitude of placebo and proportion of responders to placebo effect size (Forest plots were used to show results).

Separate meta-regression analyses were performed on both magnitude and responder ratio outcomes, to identify whether the following hypothesized factors significantly influenced the effect size (reported as regression coefficient; 95% confidence interval; p-value). Factors tested where identified either as being the most frequently reported in the literature as having an effect on DPN or HIV-SN placebo responses or those identified from exploratory univariate analysis.

3.1 Study characteristics

A total of 12 trials of painful HIV-SN and 63 trials of DPN were included for analysis (Fig. 1).

Fig. 1:

Flow diagram for literature search for studies involving painful HIV-SN and DPN.

Studies involving HIV-SN patients included younger participants and had a higher male:female ratio in both active and placebo groups (Table 1). Although HIV-SN studies required a shorter duration of pain to be considered for inclusion, there was no difference in the baseline duration of painful symptoms between the DPN and HIV-SN studies. DPN studies involved more contact with the participants per week but less frequently used testing of adherence using the measurement of drug concentration in the blood. A full summary of the study characteristics extracted from each study is shown in Supplementary Table 1.

3.2 Reporting of outcome measures

A total of 36 studies reported the number of placebo responders showing a >50% reduction in pain (Table 2) and 28 studies reported a >30% reduction in pain. In 62 studies a proportion of responders to placebo value could be extracted using either the 50% or (if not reported) the 30% reduction or (if neither reported) another pre-defined cut off for an important response as defined by the trial. This outcome was therefore used for the main outcome meta-analysis.

In 28 DPN and 10 HIV-SN trials, either the mean change from baseline or standard deviation could not be calculated, therefore these trials could not be included in the meta-analysis of magnitude of placebo response.

The unadjusted mean reduction in pain intensity with placebo across all studies was 1.65 (0.89) on a 0–10 scale, and the ratio of responders to placebo was 0.29 (0.17).

3.3 Proportion of placebo responders

The overall adjusted (i.e. weighted to account for study size) proportion of placebo responders was 0.28 (0.25–0.32) for all studies. There was no difference between proportion of placebo responders in HIV-SN studies (0.35; 0.28–0.42) and DPN studies (0.27; 0.24–0.31), p=0.129 (Fig. 2). During the initial analysis, there was evidence of heterogeneity; after removing Stracke et al. (1992) then Gao et al. (2010) the effect size did not significantly change and there was ongoing evidence of heterogeneity therefore all studies were included in the next analyses.

Fig. 2:

Forest plot of proportion of placebo responders for painful diabetic polyneuropathy (DPN; trials n=54) and HIV associated neuropathy studies (HIV-SN; trials n=8). ES=effect size.

The comparison of placebo response rate between HIV-SN and DPN was also performed separately for trials reporting a 50% reduction and a 30% reduction in pain. For >50% pain reduction, the pooled ratio of placebo responders was 0.24 (0.21–0.27) in DPN, and 0.37 (0.31–0.42) in HIV-SN (Supplementary Fig. 1). For >30% pain reduction, the pooled ratio of placebo responders in was 0.41 (0.37–0.45) in DPN, and 0.49 (0.39–0.58) in HIV-SN (Supplementary Fig. 2). However, since there were so few HIV-SN studies that included these outcome measures (n=3 for >50% pain reduction, and n=3 for 30% pain reduction), a statistical comparison to determine the significance of the difference was not appropriate.

3.4 Magnitude of placebo response

The overall adjusted magnitude of placebo response (on meta-analysis) was a pain reduction of 1.72 (−1.93 to −1.51) (p<0.001) for all studies (Fig. 3). In painful DPN studies, the pooled reduction was 1.68 (1.47–1.88) compared to 2.38 (1.87–2.89) in HIV studies. Since only 2 studies could be included in the analysis we did not perform a statistical comparison between the magnitudes of placebo response between the conditions.

Fig. 3:

Forest plot magnitude of placebo response for painful diabetic polyneuropathy (DPN; trials n=33) and HIV associated neuropathy studies (HIV-SN; trials n=2). ES=effect size.

3.5 Characteristics influencing the placebo response

Since the effect size could be determined in only 8 HIV-SN studies for the proportion of placebo responders, and in 2 HIV-SN studies for the magnitude of placebo response, data from both DPN and HIV-SN studies were pooled in the regression analysis to identify factors influencing placebo response. A summary of the factors extracted for each study is presented in Supplementary Table 2.

Several patient factors were reported by too few studies to be included in the analysis. BMI was reported in 1 HIV-SN study and 12 DPN studies. Baseline depression, anxiety and sleep interference were measured by such a variety of methods that pooling the data was not possible. The most commonly used methods for measurement of mood included the Hospital Anxiety and Depression Scale, the Beck Depression Inventory, the Profile of Moods States and the Hamilton Depression Rating Scale.

Mean age at enrolment, gender ratio of patients, the duration of pain at enrolment, mean baseline intensity of pain at enrolment, the number of contacts with study staff per week, and the type of study sponsor were not associated with an effect on proportion of placebo responders or the magnitude of the placebo response when including both DPN and HIV-SN studies in the meta regression (Table 3). The final year of enrolment showed a weak association, with more recent studies showing a slight increase in magnitude of placebo response (standardised β coefficient −0.932; 95% CI −1.741 to −0.122; p=0.025). However, there was no association with the proportion of placebo responders (p=0.911).

A Meta-analysis regression model which adjusted for disease type (Table 4), baseline pain intensity, age, gender, and the final year of enrolment of the study showed no evidence that a single factor was associated with the magnitude of the change in pain rating or the ratio of placebo responders.

4.1 Magnitude of placebo response

The IMMPACT consensus suggests a change in pain intensity of at least one on an 11-point scale be considered “minimally important change” in response to an analgesic intervention [19]. The pooled pain reduction in response to placebo in the analysis was greater than this (1.72), similar to the reduction of 1.7 suggested as a definition for “much improved” [20]. The magnitude is similar to that reported by other meta-analyses of placebo response in DPN [5], osteoarthritis and low back pain [21] but is higher than associated with trials of central neuropathic pain [4], suggesting in certain instances that pain aetiology may be associated with placebo response. In this study, although the absolute value for the magnitude of placebo effect was larger in studies of painful HIV-SN, data were insufficient to perform statistical analysis to determine significance in the meta-analysis.

4.2 Proportion of responders to placebo

In the meta-analysis by Cepeda et al. [7] HIV-SN trials showed an increased proportion of responders to placebo, a finding that is not evident in our analysis. The difference in this study may be due to the use of a broader definition of a “placebo responder” to include those with >30% (compared to >50%) response which was used as data were too scarce when solely >50% was used as a definition. In our analysis 42% (26/62) of studies included in the responder analysis did not use >50% pain reduction as an outcome measure. Our analysis also included 25 studies that were published after the original Cepeda meta-analysis; it only used data from studies with a low “risk of bias score” and those with a treatment arm of at least 3 weeks. Cepeda et al. also included a wide range of neuropathic pain conditions, not just DPN and HIV-SN, which may have influenced their results.

4.3 Differences between trials of HIV-SN and DPN

When exploring differences in placebo responses between the two conditions, patient and study characteristics were also compared. Participants in HIV-SN studies were younger and more likely to be male, reflecting the incidence of HIV infection in countries where the majority of studies were conducted (USA [22] and Europe [23]). Age and gender have been suggested as factors that influence placebo response but, despite the difference in these between conditions, no effect was identified on placebo response. This may be due to the fact that studies of DPN and HIV-SN were pooled and the majority of studies were in DPN, where there was a preponderance of older age and a more equal gender distribution.

HIV-SN trials allowed for the inclusion of patients with symptoms for a shorter duration of time. This may reflect a concern about recruitment to HIV-SN trials or a perception of a different symptom profile, for example where those with HIV-SN are perceived to develop more severe pain sooner after diagnosis. The HIV-SN trials also appeared to have fewer patient-investigator contacts, potentially due to several of the HIV-SN studies being conducted by the same investigator groups.

4.4 Factors affecting placebo response

Although the increasing placebo response in neuropathic pain trials over the last 20 years has been well documented [3], [5], [8], this study showed a similar trend until underlying disease aetiology was accounted for, when the result became non-significant. This may indicate a complex interaction between disease type and the era when the majority of studies were conducted.

This systematic review did not reveal an association between the type of study sponsor, mirroring findings by Zhang et al. [24]. Neither the average number of contacts between participants and investigators, the mean age of participants, nor the baseline pain intensity were associated with placebo response which is in keeping with the inconsistent role that has been demonstrated for the influence of these factors on placebo [3], [4], [5], [6], [7], [9], [21], [24], [25].

4.5 Strengths and limitations

This systematic review and meta-analysis has several strengths. Only high quality trials were included, data describing a broad range of study and patient characteristics were extracted, and robust statistical techniques were used to impute unreported data.

Along with its strengths, the study had several limitations. Several characteristics that may influence placebo response, such as patient reported anxiety and sleep disturbance, could not be assessed, as they were not consistently reported. Although the Oxford “Jadad scale” [14] for assessing trial quality was used, this is limited by its focus on study reporting rather than conduct and does not include factors such as allocation concealment. It has been argued that the use of scales for quality assessment is inappropriate as it leads to “weighting” of characteristics with limited evidence for the justification of such weighting [26]. Many scales and methods have been developed for assessing trial quality [27] but the Oxford scale is simple and brief whilst covering essential, widely accepted components of quality.

Since so few HIV-SN trials could be included in pain intensity and responder rate analysis, it was not appropriate to analyse factors influencing placebo response in HIV-SN and DPN separately. These patient cohorts have many differences, such as gender ratio and socio-economic factors, which may suggest that differences between the two conditions could have been detected if more studies were included.

The majority of HIV-SN studies (7 studies) used the Gracely scale, and therefore could not be included in the meta-analysis. This is an example of how standardisation of reporting of clinical trials, such as the IMMPACT recommendations [28], could help to improve the validity and usefulness of meta-analyses.

No studies in this analysis included a “standard care” group to take into account a regression to the mean or natural improvement in pain therefore it was not possible to identify what proportion of the placebo response was due to a true placebo effect. As highlighted by Cragg et al. [4] this may play an important part in change in pain intensity over trials of longer duration.