Responsiveness and longitudinal validity of the Persian version of COMI to physiotherapy in patients with non-specific chronic low back pain

1 Introduction

Low back pain (LBP) is highly prevalent, costly and adversely affects people at all ages worldwide [1], [2]. LBP affects more than 70% of people in their lifetime [3], [4]. Nonspecific LBP where no specific pathology has been identified accounts for 80–90% of all LBPs [5], [6]. About 5–10% of patients with LBP develop chronic LBP which is defined as LBP lasting ≥12 weeks [7], [8].

In clinical practice and research, it is important that therapists and researchers utilize psychometric sound patient-reported outcome measures to assess the effectiveness of interventions. The patient-reported outcome measures used to evaluate efficacy of treatment in routine practice must consider the patients’ perspectives on different key dimensions. Further, patient-reported outcome measures used for assessment in practice and clinical trials must be brief and concise to keep patients’ compliance and importantly to detect changes after treatment.

The six-item Core Outcome Measures Index (COMI) is a short, self-reported, and multidimensional questionnaire for assessing patients with LBP [9]. Each item of the COMI asks about one dimension including: (1) pain intensity (for back and leg separately); (2) difficulty with daily life functioning; (3) symptom-specific well-being; (4) work disability; (5) social disability; and (6) quality of life. The COMI has been recently culturally adapted and validated with excellent psychometric properties in Persian-speaking patients with LBP [10]. However, the important psychometric characteristic of longitudinal validity and responsiveness (i.e. ability to detect changes due to the interventions) have not been examined for the Persian COMI (COMI-P). Furthermore, the COMI-P’s minimum clinically important change (MCIC) has not been determined. Therefore, the aim of this study was to examine the responsiveness and longitudinal validity of the COMI-P and establish the MCIC in a group of Persian-speaking patients with non-specific chronic LBP. A secondary aim was to perform analyses to explore whether the MCIC depends on the external criterion or anchor.

2 Methods

3 Statistics

The mean and standard deviation (SD) and the median (interquartile range) were calculated for continuous and ordinal variables, respectively. Kolmogorov-Smirnov test was used to examine for outcome data normality before statistical analyses. Responsiveness was determined using the internal (ability of an instrument to detect the changes after treatment within the clinical context) and external (ability of an instrument to detect the changes correspondent to a reference external measure) methods.

The internal method included the following: (1) a paired t-test was used to examine whether the changes from pre-treatment to post-treatment were significant; (2) the standardized effect size (SES) as a standardized measure of change over time was calculated by dividing the difference between the pre- and post-treatment scores by the pre-treatment SD; (3) the standardized response mean (SRM) was calculated by taking the mean change scores and dividing this by the SD of these change scores; and (4) the Guyatt responsiveness index (GRI) was calculated by dividing the mean of change scores by SD of change in patients whose clinical condition remained unchanged. Effect size values for SES, SRM and GRI of 0.20, 0.50, and 0.80 were interpreted as small, moderate, and large changes, respectively [21].

The receiving operating characteristic curve (ROC), the external method of assessing responsiveness, was used to examine the sensitivity and specificity of the COMI-P relative to the patient global perceived change outcome. Responsiveness was explained in terms of sensitivity and specificity that indicated the ability of the COMI-P to discriminate the patients who improved vs. the patients who did not improve according to the GRCS. The external criterion of change of the GRCS, is determined when the ROC curve is plotted with “true-positive rate” (sensitivity) on the “y axis” vs. “false positive rate” (1-specificity) on the “x axis” to determine the cut-off point in change score with the best sensitivity and specificity. The area under the curve (AUC) representing the probability the COMI-P correctly discriminates the patients as improved or unimproved ranges from 0.5 (no accuracy in discriminating) to 1.0 (perfect accuracy in discriminating) with a minimum 0.7 considered as adequate [11]. The AUC values were interpreted as excellent (0.97–1.00); very good (0.92–0.97); good (0.75–0.92); and fair (0.50–0.75) [16]. The MCIC was determined as the magnitude of change score with maximum sensitivity and 1-specificity located on the uppermost left corner point of the curve [22].

Additionally, further analyses were performed to calculate the MCIC based on the SDC score reported for the COMI-P [10] and to compare it with that calculated according to the GRCS. The real changes in patients after treatment were identified according to the SDC reported for COMI-P as being 2.1 [10]. Subsequently, the differences on COMI-P change score between the groups with mean change less or greater than SDC score was analyzed with independent sample t-test. Considering the SDC score of 2.1 as a threshold for success of treatment, patients with COMI-P change score >2.1 was defined as improved vs. those with COMI-P change score <2.1 defined as unimproved. This classification was dichotomized (0, 1) and was used as an ordinal measure for the correlation with the COMI-P change score and for the analysis of the ROC curve.

The external responsiveness of the COMI-P was further evaluated by means of correlation analyses with external criteria of GRCS, FRI-P, and VAS. Spearman’s rank order correlation coefficient was used to determine the strength of association between the COMI-P and the GRCS. Pearson’s correlation coefficients were calculated to determine the association between the COMI-P and the FRI-P and VAS. A correlation threshold of >0.30 is recommended as an acceptable association between change score in an outcome instrument and the external anchor [23]. The correlation values were interpreted as excellent (0.81–1.00); very good (0.61–0.80); good (0.41–0.60); fair (0.21–0.40); and poor (0.00–0.20) [24]. The SPSS statistic software version 17.0 for Windows was used to analyze the data with p<0.05 considered as statistically significant.

4 Results

The study population consisted of fifty subjects (44 females, 6 males) with a mean age of 50.6±13.8 years, BMI 28.9±4.0, education level 12.6±4.0 years, and median LBP duration of 60 months (interquartile range 16.5–120). The dichotomized GRCS scores showed that 43 subjects improved (GCRS>0, mean change 1.8±1.7) and seven subjects did not (GCRS≤0, mean change 0.7±1.5); the difference between the two groups for their GRCS scores was not significant (t=−1.55, df=48, p=0.13).

However, with the dichotomized SDC score of the COMI-P as the criterion (2.1), 32 patients (mean change 0.6±0.8) had not improved vs. 18 patients who had improved (mean change 3.5±1.2); the difference between the two groups was significant (t=−10.41, df=48, p<0.001).

The mean and standard deviation (SD) for COMI-P, FRI-P, and VAS before and after physiotherapy are presented in Table 1. Each of the disability scores (COMI-P and FRI-P) and the pain score (VAS) showed significant reductions after physiotherapy (p<0.001). All the effect sizes of internal responsiveness for COMI-P were large (SES=0.96, SRM=0.97, GRI=1.23).

The correlational analyses of COMI-P changes with external responsiveness measures showed that the COMI-P had fair correlation with the GRCS (r=0.34, p=0.02), and very good correlations with VAS (r=0.65, p<0.001), and FRI-P (r=0.67, p<0.001). The COMI-P changes had excellent, significant correlation with the dichotomized SDC criterion (r=0.83, p<0.001).

Using the subjects’ GRCS as an external anchor, the ROC analysis for the COMI-P showed the AUC=0.66 (95% CI: 0.46–0.86). The best COMI-P threshold of MCIC discriminating the improved and non-improved subjects was 1.03, which led to a sensitivity of 65% and a specificity of 86%. However, with the dichotomized SDC criterion as an external anchor, the AUC for the COMI-P was 1.00, and an MCIC of 2.15 points was subsequently established (sensitivity 94% and specificity 100%).

5 Discussion

The Persian version of COMI used in this study was developed and validated for psychometric characteristics and has been shown to be a reliable and valid questionnaire [10]. The reliability and validity are prerequisites for all measurement tools when they are to be used to distinguish between patients that have improved or not after applying interventions. The present study describes the responsiveness of the COMI-P in a population of Iranian subjects with non-specific chronic LBP demonstrating the COMI-P as a responsive instrument to distinguish patients who improved after physiotherapy from those who did not. The responsiveness of the COMI-P found in this study is consistent with previous reports on the responsiveness of the original English COMI [25] and COMI versions of German [26], Spanish [27], French [28], and Hungarian [29]. A study used the validated COMI versions of German, Spanish, and French to assess the responsiveness of the COMI in adult patients with spinal deformity (N=204) and reported that the COMI-back was responsive to detect important changes after treatment [30]. Other language versions of COMI have not been evaluated for responsiveness (Brazilian [31], Italian [32], Norwegian [33], Polish [34], Chinese [35], Turkish [36], Japanese [37], and Dutch [38]).

The t-test analysis showed significant mean changes in the COMI-P score after physiotherapy. This finding indicates the COMI-P as a responsive outcome measure to detect significant changes over time before and after physiotherapy treatment. In addition, the VAS and FRI-P also demonstrated significant mean changes pre-post physiotherapy intervention.

The statistics based on the effect sizes provide information on the magnitude of changes. The SES indicates the relationship between the mean differences between the baseline and the follow-up scores with regard to the SD of the baseline score [21]. The large SES of COMI-P indicates that the change relative to the baseline measurement is large. As shown in previous reports, the SESs of the COMI were between 0.4 and 2.3 for the Spanish [28] and 0.95 for the German [26] languages.

The SRM is the best statistic test for responsiveness evaluation of outcome measures in such designs [21]. The SRM is an effect size based statistical test that reflects the variability of the change scores. The large SRM shown for the COMI-P indicates the high responsiveness to change of COMI-P consistent with previous studies of the German COMI (SRM=0.98) [26] and the COMI version of Hungarian (SRM=1.16) [29]. In a study of patients with adult spinal deformity who completed the validated German, Spanish and French versions of the COMI questionnaire at baseline and at 12 months follow-up, the SRM reported for the change scores was 1.23 [30].

The GRI value in accordance with the other internal responsiveness statistics (i.e. SES, SRM) was large. This finding suggests physiotherapy improved the function of patients reflected in the significant changes observed in the COMI-P scores. The GRI is not reported for the COMI, original English and translated versions. Therefore, the GRI internal responsiveness results of our study can not be compared to any other studies within the literature.

We investigated the external responsiveness in terms of the correlations between the changes in the COMI-P and the related changes in FRI-P, VAS, and GRCS. The correlation coefficients reflect the extent of changes in a measure with relation to the corresponding changes in the reference external measure. The correlation between the COMI-P and the GRCS was fair compared to the very good correlation between the COMI-P and both the VAS and the FRI-P. The lower correlation between the COMI-P and the GRCS, though acceptable, could be the bias and error associated with self-reporting of changes in clinical status in the test situation. The patients answered the question positively when completing the GRCS. The acceptable correlations found for the COMI-P are in agreement with those reported for the COMI [30]. The correlation coefficients for the COMI-P with the FRI-P and the VAS were very good supporting the capacity of COMI-P to reflect changes in pain and disability as perceived by the patients.

Forty-three patients reported good outcomes based on the GRCS after treatment. However, the SDC reported for the COMI-P in agreement with the French version [28] is 2.1 points [10]. This implies that using the COMI-P as an outcome measure to determine the effects of interventions in patients with LBP needs change scores of >2.1 points between pre and post treatment to be considered as a real change. The change score for the COMI-P in 32 patients with LBP was below the SDC score. This may indicate bias of patients in completing the GRCS question about their improvement after physiotherapy. Hence, a practical and objective way to identify true changes in the clinical status of patients after interventions might be the use of SDC score of an outcome measure as an external criterion. The significant differences of COMI-P change score of groups dichotomized as improved/unimproved based on the SDC and excellent correlation between the COMI-P change score and the dichotomized SDC criterion implies that the SDC score might be an appropriate substitution for the GRCS in the clinic and research situations investigating the changes in functional status of patients due to an intervention. This is the first study that used and discussed the SDC score as a criterion for identifying real changes in terms of the COMI-P score in patients with LBP receiving physiotherapy. This approach allowed classifying patients who actually improved in their pain and function or not since there is an excellent correlation with COMI-P change scores.

The MCIC is used as a threshold to detect true meaningful changes in patients’ status beyond random error post treatment [39]. In the present study, we used the ROC curve approach to calculate the MCIC using two different criteria of GRCS and SDC score of COMI-P. The GRCS is commonly used for external responsiveness criterion as it takes the perspectives of patients into account. Based on the GRCS scores, the AUC value did not reach the threshold of 0.7 defined as adequate [11]. Further, the MCIC calculated (1.03) was less than the SDC of the COMI-P set at 2.1 [10] indicating that the GRCS-based calculation of MCIC was subjected to error not reflecting meaningful important and true changes in patients’ status. A caution then must be taken into account on the interpretation of the MCIC score smaller than the SDC as revealed in this study with GRCS as an external criterion, because it would falsely overestimate good treatment effect; and patients whose scores fall within the measurement error would be misclassified as clinically meaningfully improved. The SDCs exceeding the MCICs for other PROMs were also observed in previous studies with LBP patients [40].

A complementary analysis of the ROC using the SDC score as criterion, however, improved the AUC value in agreement with those reported for COMI [25], [26], [28], [30] and the MCIC value exceeded the SDC score. The perfect value of AUC implies that using the true change in the COMI-P scores results in the perfect ability to discriminate patients with non-specific chronic LBP that improved from those not improved after physiotherapy. The MCIC value (2.15 points) based on SDC score criterion found in this study for COMI-P is in agreement with those reported for COMI score in surgery patients (group MCIC 2.6 points; individual MCIC 2.2 points) [25], French version validated in non-surgical LBP patients (2.3) [28], and German COMI (1.85) [26]. The SDC-based determination of MCIC of COMI-P, which exceeded the SDC value, indicates clinical relevance, and a minimum decrease of 2.15 points in COMI-P score is required to be perceived as an important and beneficial change.

The external criterion used in studies of test responsiveness determines the success of treatment in improving patients’ condition. The correlation between the change scores in the outcome questionnaire and change scores on the external gold standard is one way to assess responsiveness of a new test. The ROC approach is used to determine how well the change scores for a questionnaire under investigation relate to the external criterion of treatment success. The external criterion to dichotomized patients as improved and unimproved could influence the ROC analyses and the interpretability of the optimal cut-off point. In the present study, the GRCS and the SDC score of COMI-P were used as criteria for ROC analyses and to estimate the best cut-off point to be considered the clinically important change score. According to the GRCS criterion, 86% of patients were classified as improved with good outcome. In contrast, the proportion of patients who were classified as improved, based on the SDC score, was much lower (36%). The discrepancy documented in higher percentage of patients reporting improved based on the GRCS might be due to the satisfaction of patients with physiotherapy care, not the treatment effects and real changes after treatment. This finding highlights the importance of distinguishing between satisfaction of patients with physiotherapy treatment/care which is influenced by factors such as the patient-physiotherapist interaction [41] from the treatment effects which is instead focused on the true improvement in symptoms, pain, and function [41], [42]. It follows that for assessing the responsiveness of questionnaires and ROC analyses that primarily focus on the treatment outcome, we recommend the SDC score as the external criterion to classify patients as improved and unimproved.

6 Strengths and weaknesses

The strength of the present study was that we compared the GRCS criterion with the SDC score for determination of MCIC and optimal cut-off point. The sample size in this study though adequate for responsiveness analyses, might be small for classifying patients into subgroups of improved and unimproved. A further study with larger sample size is required to determine the appropriateness of the SDC score against the GRCS as external criteria and to verify the SDC-based MCIC found in this study for the COMI-P in patients with non-specific chronic LBP.

7 Conclusion

In conclusion, our findings revealed that the COMI-P is a responsive instrument to physiotherapy in a sample of patients with non-specific LBP. The external criterion manner to classify patients influenced the optimal cutoff point and responsiveness. The best discriminative ability for the COMI-P to categorize patients as improved or not occurred with the SDC score as the criterion. The AUC was perfect, and the MCIC of 2.15 (sensitivity 94% and specificity 100%) represented a clinically meaningful change in level of pain and function exceeding the measurement error.