Confirmatory factor analysis of 2 versions of the Brief Pain Inventory in an ambulatory population indicates that sleep interference should be interpreted separately

1 Introduction

Recent years have seen a rapid development in the number of Patient-Reported Outcome (PRO) instruments. Understanding the conceptual domain and integrity of PROs is fundamental to their validity. Some musculoskeletal PROs are designed to focus on symptoms and function related to health conditions affecting specific body regions, examples include the Neck Disability Index [1] or Roland Morris Low Back Disability Questionnaire [2]. Others are generic PROs that focus on a particular symptom regardless of the region or condition. Examples of generic scales include the Brief Pain Inventory [3] or Pain Disability Index [4]. It is generally assumed that region-specific scales offer better sensitivity to change in a particular region, but being only relevant to that specific body region prevents the ability to compare function or interference across regions.

The short-form Brief Pain Inventory (BPI-SF) was developed by the World Health Organization’s Cancer Pain Working Group [3].lt is a 9-item questionnaire that includes measures of self-perceived intensity of cancer-related pain symptoms (4 items), 4 qualitative questions including pain location, medication usage and magnitude of relief, and a 9th item divided into 7 sub-domains of resultant interference across 7 domains of function including normal work, sleep, and walking ability. Its simplicity and broad applicability has led to translation into several languages and application across a variety of conditions including neuromuscular [5,6] and musculoskeletal disorders [7,8]. Despite its ubiquity, some measurement properties of the BPl-SF remain unclear, primarily with respect to its factor structure. There is general consensus that the Severity subscale and the Interference subscales are unique domains. However, the unidimensional nature of the lnterference subscale remains debated, with some author groups endorsing a single Interference factor [9,10] while others endorse a 2-factor (Physical and Affective) Interference structure [9,11,12]. The axioms of quantitative measurement as proposed by Luce and Tukey [13] require that a tool measure a single construct in order to make scale scores interpretable. lf the BPl-SF does include a multifactor lnterference subscale than summative scoring may be inappropriate or may mask important changes in one domain when others are stable.

The majority of work on the properties of the BPI-SF has been conducted using the original 7-item version of the interference subscale in samples including palliative care patients with HIV/AIDS [14], cerebral palsy [15], and post-surgical populations [16. Tyler et al. [15] proposed an expansion that added 3 items to the interference subscale (social activities, recreational activities, self-care). The additional items were intended to prevent floor effects, enhance content validity, and provide a more holistic evaluation of pain-related interference in patients with a wide variety of pain conditions. The expanded 10-item version of the BPl-SF lnterference subscale has yet to undergo rigorous scrutiny of its psychometric properties despite its use in some large-scale population-based cohort studies such as the N = 5000 + Quebec Pain Registry (QPR, www.quebecpainregistry.com). This has offered a unique opportunity to explore the properties of the expanded tool, and to conduct independent analyses of the 7-item version in a large sample of ambulatory, community-dwelling patients with mixed chronic pain conditions. The purposes of this study were to explore and contrast the factor structure and concurrent validity of the original 7-item BPl-SF lnterference subscale and the 10-item version using the large, heterogeneous QPR patient database.

2 Methods

Data were obtained from the Quebec Pain Registry (QPR) database project. The QPR is a province-wide administrative and research database which provides standardize data on a large cohort of chronic pain patients referred to tertiary care pain clinics. Participants’ phenotypes are described using a set of common demographic and clinical measures based on uniform and validated measurement tools. Data were provided for 5654 participants recruited from 5 tertiary-care speciality pain clinics between October 2008 and December 2013. The QPR project was jointly funded by the Quebec Pain Research Network and the Quebec Ministry of Health. Patients were referred to the clinics from their primary care physician after failing conservative treatment such as standard pharmaceutical or non-medical rehabilitation approaches. All participants completed a battery of self-report questionnaires administered by a trained research nurse at intake. Participants in the database were at least eighteen years of age and could speak and understand conversational French or English. All data were de-identified prior to being stored in the database. Access to the database was granted following review by the Quebec Pain Research Network and the Institutional Review Board at Western University (London, Ontario).

The following tools were collected at intake and 6-month follow up. All tools were available in both English and French-Canadian versions using the appropriately validated versions of each:

1. A demographics questionnaire that captured data including sex, age, cause and duration of symptoms, medicolegal and compensation status, work status, ethnicity, educational attainment, marital status, medication use, and household income.

2. The 10-item Brief Pain Inventory Interference subscale [3], a self-report questionnaire that captures pain related interference across 10 generic domains including recreational activity, work and enjoyment of life.

3. The Medical outcomes survey short form-12 (SF-12), a 12- item generic health status measure that provides a variety of subscales including bodily pain, vitality, and general health. It provides two component summary health scales, a physical component (PCS) and a mental component (MCS) score. It is one of the most widely used health status measures and enjoys considerable support as a psychometrically sound measurement tool [17].

4. The Pain Catastrophizing Scale (PCat), a 13-item self-report tool that captures the construct of exaggerated negative cognitive orientation towards pain [18]. It has demonstrated consistent ability to predict pain related outcomes following medical or nonmedical intervention [19,20]. Rasch analysis has shown that it also functions well as a linear indication of pain catastrophizing in musculoskeletal pain [21].

5. The Beck Depression Inventory-Il (BDI-II), a revised version ofthe original BDl intended to be concordant with the new diagnostic criteria of major depression in the DSM-IV. It has been studied extensively, with Wang and Gorenstein recently synthesizing data from 70 unique evaluations [22]. Overall it demonstrates strong evidence of reliability and validity across a wide range of medical conditions.

6. Three numeric rating scales for pain intensity; one for current pain, one for average pain and one for worst pain over the past 7 days. The numeric rating scale has demonstrated adequate reliability and sensitivity to change in many different pain conditions [23].

Treatment decisions were made at the discretion of the clinician and were not influenced by the Quebec pain network.

3 Analysis

The data were checked for fidelity by a QPR-associated research nurse before being sent to our group. The database was subsequently explored for missing or extreme values that required checking. Missing values were replaced by the mean of the scale when ≤10% (1 item) of values were missing. Scale scores with >10% of missing values were removed. In pre-analysis data normality was checked using the Kolmogorov-Smirnov test and visual inspection of response frequency histograms. Three items showed stronger than expected bimodal response frequencies with >20% zero responses that could not be easily reconciled. To avoid spurious findings due to this unexpected response distribution that would not have otherwise lent itself to CFA we constructed regression equations for each of the 3 items, using the remaining items to mathematically predict their response. Cases were removed when the regression equation predicted a score at least 4/10 or greater on that item despite an observed score of 0. Doing so led to removal of 1658 cases and resolved the strong bimodality. Remaining responses showed statistically significant but not visually severe negative skew that deviated from normality on all BPl-SF items that could not be corrected through square-root or logit transformation. Non-parametric tests of association were therefore used as appropriate.

lnter-item correlations were evaluated using Spearman’s rho statistic. lnter-item correlations >0.9 were ear-marked for possible redundancy or multicollinearity. Response distribution to each item on the BPl-SF was calculated descriptively (mean, standard deviation, range). Randomly drawn subsamples of the full database were used to avoid incorrect attributions of statistical associations or differences by virtue of sample size. Where needed, random sub-samples were drawn using the SPSS random sample function. One random draw of 500 subjects was extracted for exploratory factor analysis (EFA). EFA was conducted using maximum likelihood estimation, first without rotation, and second allowing oblique rotation (assuming some covariance between factors). Horn’s parallel analysis technique [24] determined the number of factors extracted based on eigenvalues. Internal consistency for identified factors was calculated using Cronbach’s a, where a value >0.7 was considered acceptable [25]. This process was repeated in a second independent random sample of 500 responses for verification.

Confirmatory factor analysis was then undertaken using a third independent random sample of N =1000 responses using MPlus software (v6.12, Muthen & Muthen). Four separate models were tested: (1) The base model (10-item interference subscale with a single factor); (2) the 10-item interference subscale with 2 factors: Affective (items 2, 5, 6, 7, and 8) and Physical (items 1, 3, 4, 9 and 10) interference; (3) the original 7-item interference subscale with a single factor; (4) the 7-item interference subscale with 2 factors. Model fit was evaluated through interpretation of the following goodness-of-fit indicators (with thresholds of acceptability in parentheses): comparative fit index (CFl, ≥0.90), Tucker-Lewis index (TLl, ≥0.90), Root mean square error of approximation (RMSEA, < 0.10). The X² value and associated significance level was calculated but was a priori expected to be inflated by virtue of a large sample size. Change in X² by change in degrees of freedom (ΔX²/Δ) was used to determine the significance of change in model fit for each model in comparison to its base, as described by Atkinson et al. [11].

Concurrent validity (with hypotheses in parentheses) for the base model and subscales was tested through cross sectional correlations with the SF-12 physical (PCS) and mental (MCS) component scores (moderate negative correlation), the PCat (moderate positive correlation), the BDl-ll (weak positive correlation) and mean Pain intensity NRS (strong positive correlation). Bootstrapping (n = 1000 samples randomly drawn with replacement) with calculation of 95% confidence interval was conducted where required for statistical comparison of correlation coefficients with the criterion measures between the 10- and 7-item versions of the scale. For all analyses, a p value < 0.05 was considered statistically significant for purposes of this exploratory study, accepting an increased risk of alpha error in favour of avoiding beta error. Unless otherwise specified, analyses were conducted in SPSS v. 21 (lBM Corp.).

4 Results

The initial database for this analysis included 3996 subjects, of which 63 had 2 or more missing responses and were removed. A further 27 had a single missing response replaced by the mean. The remaining database therefore included 3933 subjects with complete baseline data and reasonably Gaussian response distributions. Table 1 provides the descriptive statistics of the sample as a whole, and the 2 random draws (1 × n = 1000 for EFA, 1 × n = 1000 for CFA) used for the factor analytic procedures. Table 2 provides the distribution of sample responses for all subjects. Inter-item correlations revealed no concerns with redundancy. The strongest correlation was between item 9 (recreational activities) and item 10 (social activities) at ρ = 0.79.

Both random draws of n = 500 provided strong confidence in sampling adequacy (Keiser-Meier-Olkin statistic = 0.93) and deviation from sphericity (Bartlett’s test p <0.01). In both random draws, a single factor was extracted with similar factor loadings for each item, all =0.32. The single factor explained 67.1% and 65.1% of variance, respectively. Cronbach’s alpha in both cases was α = 0.94.

Table 3 provides the goodness of fit indices for the CFA analyses. The base model (10 items, single factor) showed generally poor fit indicators, none of which met a priori criteria for acceptability. The 2-factor physical and affective interference model of the 10-item interference scale showed significant improvement in model fit, with CFI (0.94) and TLI (0.92) surpassing the acceptability threshold, but the RMSEA (0.12) remaining in the unacceptable range. As anticipated given the large sample, all x² values were statistically significant. Of the three additional items of the expanded version, two loaded on the Physical Interference subscale (recreational activities and social activities) and one loaded on the Affective Interference subscale (self-care).

Model fit of the expanded 10-item scale was compared against that of the original 7-item interference scale. The fit indicators of the base (single-factor) model were statistically similar (χ²/df=20.9 (21 df), p = 0.47). As with the 10-item subscale, the 2-factor model of the 7-item scale showed significant improvement over the base model (χ²/df= 324.6 (1 df), p <0.01) and was not significantly different from the same 2-factor model using all 10 items. The 2-factor structure of the 7-item subscale was the first model to satisfy all a priori criteria for goodness-of-fit (CFI = 0.99, TLI = 0.98, RMSEA = 0.07). Fig. 1 shows the path coefficients for the 2-factor solutions of the 10- and 7-item CFA models. In both models, the sleep interference item had the lowest path coefficient and highest error (residual) term. Using theoretical rationale (that sleep is conceptually distinct from waking activity), a third model was tested in which the sleep interference item was considered to represent a third latent construct unto itself. This required constraining the path coefficient to 1.0 and error variance to 0 for this single item to allow model identification to occur. In both the 10- and 7-item versions this 3-factor structure led to additional significant improvement in model fit (χ²/df (df) < 0.01 in both) over the respective 2-factor model, while not being significantly different between the two versions. In terms of absolute goodness-of-fit indices, only the 7-item 3-factor version satisfied our a priori criteria for acceptability (CFI = 0.99, TLI = 0.98, RMSEA=0.06).

Table 4 provides the results of the cross-sectional convergent and divergent validity testing using the full sample of 3933 responses including 95% confidence intervals through bootstrapping. The magnitude and direction of each association were congruent with our a priori hypotheses, and were congruent with the nature of each subscale. For example, the association between the BPI-SF Affective Interference subscale and the SF-12 Mental Component Score was moderate in magnitude and negative in direction (r = –0.61, 95%CI –0.59 to –0.63) and significantly stronger than the association between the same SF-12 Mental Component Score and the Physical Interference BPI-SF subscale (r = –0.50, 95%CI –0.48 to –0.53). As would be anticipated, the opposite association was found for the SF-12 Physical Component Score subscale. Similarly, the associations between the BPI-SF Affective Interference subscale and both of the cognitively-oriented PCat and the BDI- II were positive and moderate, stronger than the associations of the same criterion measures with the BPI-SF Physical Interference subscale.

Table 4 also provides the same estimates of association with each of the criterion measures and the BPI-SF-7 item version. In each case, the estimates of association between the criterion and the full 7-item scale are nearly identical to those ofthe 10-item scale and did not fall outside of the respective 95% confidence intervals of its 10-item analog. No criterion-related association was different by greater than r = 0.05 points for the 7-item scale compared to its 10-item analog.

Fig. 1

Path coefficients of 3 best-fit confirmatory factor analysis models. Top: 2- factor solution of 10-item Interference subscale: middle: 2-factor solution of 7-item Interference subscale: bottom: 3-factor solution of 7-item Interference subscale.

5 Discussion

The results of this study provide no statistical evidence for superiority of the expanded 10-item version over the original 7-item version of the Interference subscale of the BPI-SF. In contrast, the 7-item version appears to be more robust than that of the 10-item version based on the results of confirmatory factor analysis. The nearly identical cross-sectional correlations further indicate that the additional burden associated with answering 3 more items is not balanced by additional information gained.

There is little agreement in the clinical and empirical literature regarding the factor structure of the Interference subscale. Tan et al. [10] used exploratory factor analysis in a sample of 440 subjects with chronic noncancer pain to find support for a single factor. Using CFA, Lapane et al. [9] evaluated the factor structure of the 7-item version of the BPI-SF in 687 subjects with noncancer pain,and found adequate fit when the lnterference subscale was considered either a single factor or as separate Physical and Affective subscales. Atkinson et al. [11] similarly used CFA in a sample of 364 people with HlV/AlDS or cancer pain. They also found that the lnterference subscale of the BPl-SF provided adequate model fit when considered either a single factor or two factors. While Lapane et al. endorsed the single factor model for reasons of parsimony, Atkinson et al. endorsed the 2-factor model for statistical and conceptual reasons. Decisions regarding the most appropriate factor structure should be based on more than just statistics; conceptual and pragmatic reasons are salient considerations. Conceptually, it appears reasonable to consider mood, relations with other people, and enjoyment of life as distinct from general activity, walking ability, and normal work, with sleep interference representing a domain unto itself. As such we endorse a 3-factor model as the most statistically and conceptually sound.

The structural ambiguity highlights challenges inherent in psychometric analyses using approaches drawn from Classical Test Theory. Critics of CTT point to the necessary inclusion of an unobservable random error term that can be neither proven nor refuted as a notable weakness of such approaches [26,27]. From a pragmatic perspective, the structural ambiguity may have some benefit - users can arguably interpret either the full composite score when granularity is not necessary or can interpret the subscales individually for greater depth of understanding. However, it does pose statistical problems. While composite scores or ‘indices’ from multidimensional tools may be useful when purely descriptive statistics are desired, the application of inferential statistics may not be appropriate. Such conflation violates one of the primary axioms of quantitative measurement [13], that measurement tools should quantify only a single construct. While the degree to which this violation leads to spurious findings is debatable, it is problematic from a conceptual perspective. That is, inclusion of two or more constructs in a single scale risks masking improvement in one subscale if the other is stable or worsens. As a concrete example, it is conceivable that physical rehabilitation would improve Physical Interference scores at a faster rate than Affective Interference. By including both in the same summative score, the relatively greater improvement in one is buffered by stability in the other, potentially limiting overall scale responsiveness.

Detailed comparison of our results with those of previous findings from CFA reveals similarities and differences in path coefficients. Atkinson et al. [11] found identical path coefficients amongst their sample of ambulatory people for the items ‘interference with general activity’ (β = 0.85), ‘sleep’ (β = 0.66) and ‘enjoyment of life’ (β = 0.78), offering evidence of condition invariance forthose items. Lapane et al. [9] presented the path coefficients for the single factor lnterference subscale only, which were generally in agreement with our findings with the exception of the sleep item (β = 0.67 here vs. 0.85). This group also found evidence of variance across patient conditions for the sleep, relations with other people, and work items. ln the current analysis we did not consider clinical subgroups, but recognize the need for consideration of differential item functioning in future using Rasch-based modelling. This analysis has been reported in a companion paper [28] that similarly identified the sleep item as statistically misfitting while the remaining 6 items were best interpreted as 2 separate subscales. Similar results coming from two very different statistical approaches in independent samples strengthen our confidence in the validity of these findings.

While the BPl-SF is considered a generic pain interference scale, it does include items that may pose conceptual challenges in more specific contexts. Mendoza and colleagues [7] found that missing data on the interference scale was common, in particular the ‘enjoyment of life’ and ‘relations with others’, in their sample of people immediately following median sternotomy. Radbruch et al. [29] found that elderly cancer patients most frequently omitted questions about work and walking ability. Raichle and colleagues [5] substituted the word ‘mobility’ for ‘walking ability’ when evaluating pain interference in a sample of patients with spinal cord injury, as did Osborne et al. [5] when evaluating the tool in persons with multiple sclerosis and Tyler et al. [15] in persons with cerebral palsy. As a sensitivity analysis, we conducted a CFA with the walking item removed (not shown) that did not improve model fit. Item 4 ‘Normal Work’ has also been challenged for its universal applicability, having been removed by Darnall and Sazie [30] when exploring the nature of pain interference in incarcerated women, and Atkinson et al. [11] in their cohort of palliative patients with end-stage cancer or HIV/AIDS. Model fit worsened slightly in our sample of ambulatory outpatients when we conducted a sensitivity analysis without the Normal Work item (analysis not shown). Work is an important participation domain and, conceptually, interference in this domain is an important concern, particularly if unpaid work is included in the definition of work. Future users of the BPI-SF are encouraged to consider relevance of each item for their population, but be aware that ad hoc removal of items may adversely affect properties of the tool.

The magnitude of associations and score distributions for both the 7- and 10-item versions of the interference scale are also in keeping with previous reports, offering converging evidence for scale validity. Darnall and Sazie [30] found a correlation of r= 0.60 between the BPI-SF-7 Interference subscale and the Pain Catastrophizing Scale (cf. r =0.57 herein). Ferreira-Valente et al. [31] reported correlations of r = –0.63 and –0.61 between the full interference subscale and SF-12 MCS and PCS, respectively (cf. –0.58 and –0.49 herein), and r =0.57 with average NRS (cf. r = 0.69 herein). Tan et al. [10] reported overall higher mean response scores and similar standard deviations across all items in comparison to the QPR sample, while Ferreira-Valente et al. [31] reported overall lower mean scores in their sample of 214 respondents to the Portuguese version of the BPI-SF. The heterogeneity of conditions in our sample may explain the differences from Ferreira-Valente et al., but they do raise the possibility of cultural bias in responses.

Strengths of our study include a large sample size which allowed for replication, broad sampling that represents a diverse chronic pain population and the availability of multiple comparison measures. However, our findings should be interpreted considering some limitations. The diversity of clinical approaches and settings is likely to increase external validity (generalizability) but may have adversely affected internal validity if systematic biases in data collection existed. The non-normally distributed data open the possibility of biased findings, however the techniques we have used have demonstrated robustness to minor deviations from normality [32,33]. These data were not collected by our group, so we had little control over the procedures and were unable to go back to original data files to check for data errors. Our regression-based approach to data cleaning was the best option available, leading to removal of over 1600 subjects from an original database of 5654. We are confident that the data in our final database provide more accurate indications of psychometric properties than had we used the full database. The size of the database and findings consistent with previous author groups offer considerable protection against the likelihood that our findings are systematically biased, but the possibility remains. Finally, we wish to note that the optimal clinical measurement tool can only be identified through multiple types of analyses and considering different purposes (discrimination, prediction or evaluation of change) at multiple time points which will reveal important considerations not assessed in this study e.g. responsiveness. These are reasonable directions for future research to further explore measurement dimensions of the 10 or 7 item versions.

In summary, the purpose of this study was to revisit the factor structure of the interference subscale of the BPI-SF in a large cohort of adult patients with chronic pain, and to determine whether the 3 additional items proposed by Tyler et al. [15] offered any additional robustness in this analysis. The results suggest continued structural ambiguity but greater overall support for a 2-factor over single-factor structure, and for the first time to our knowledge, support for a novel 3-factor structure. This may suggest that the BPI-SF Interference scale possesses an uncommon property, that it can arguably be interpreted as either a full composite score or as independent subscores, the relative advantages and disadvantages of which have been discussed. Regardless, the additional 3 items offered no structural benefit over the original 7 items, suggesting they may add unnecessary burden for those wishing to use the BPI-SF. Additional analyses should be considered, specifically responsiveness and sensitivity to change, before definitive state-ments are made regarding the clinical utility of the 10- compared to the 7-item versions.

Author contributions

David Walton conceived of and coordinated the project, was responsible for acquiring the data and ensuring rigour of results, and completed the first draft of the manuscript.

Tyler Beattie was responsible for cleaning the data, conducting the analyses, and reviewing the final draft of the manuscript.

Joe Putos was responsible for reviewing the manuscript for accuracy and completeness, ensuring appropriate interpretation of results.

Joy MacDermid was co-lead and senior author on the paper, acquired funding to obtain the database, and reviewed the paper for accuracy and completeness.

Highlights

• A large confirmatory exploration of the BPI Interference scale was conducted.

• The 7- and 10-item versions of the BPI Interference scale are statistically similar.

• The Interference subscale is composed of at least 2 subscales.

• The sleep item should be interpreted separately.