Reliability and responsiveness of the Norwegian version of the Neck Disability Index

2.1 Study design and study population

This prospective study was conducted at the neck and back outpatient clinic of Oslo University Hospital between December 2007 and December 2009 with follow-up in April 2011. Patients are referred to the neck and back specialty clinic from primary care and other hospitals in the Southeast Health Region of Norway. The study is a part of the larger project on patients with neck pain. For additional details of the study population and research questions, see two recently published articles [16,22].

Briefly, the inclusion criteria were as follows: 16 years of age or older, the presence of neck pain and the ability to communicate in Norwegian. The exclusion criteria were as follows: presence of neck fractures, inflammatory rheumatic disease and systemic disease causing neck pain. The Regional Committee for Medical Research Ethics in Health Region East approved the study. We obtained written informed consent from all of the participants.

During the recruitment period, the questionnaire was sent to 600 referred patients. Thirty percent were unable to complete the questionnaire due to language barriers, and approximately 10% did not meet the inclusion criteria. Of the 360 eligible patients, 30% refused to participate. A total of 255 patients were included in the study. As shown in the inclusion flow chart (Fig. 1), 42 patients completed a second NDI within a specific time period of 3–14 days and were thus included in the test–retest reliability assessment. The short time interval was chosen in order to reduce major real changes in the patients functioning to influence the results. The aim was to test all patients after one week; but due to weekends, and various other factors the real time interval for retesting was 3–14 days.

Fig. 1

Flow–chart of included patients

Furthermore, 113 participants completed the NDI and the Global Rating Scale of Change (GRS) at the 2-year follow-up and were included in the responsiveness assessment.

2.2 Measures

Demographic factors, which were recorded at inclusion, included gender, age, marital status, the number of years of formal education (dichotomized into ≤12 years vs. >12 years) and the duration of the current sick leave episode (in weeks). To further describe patients’ baseline clinical characteristics, we assessed pain intensity and emotional distress. We asked patients to rate their strongest pain intensity during the past week using a numeric rating scale (NRS) ranging from 1 (“no pain”) to 10 (“the worst pain imaginable”) as used in our hospital registry.

Emotional distress was assessed using the Norwegian version [23] of the Hopkins symptom checklist 25 (HSCL-25) [24]. The assessment is a 25-item questionnaire that includes items addressing depression (9 items), anxiety (9 items) and somatization (7 items). Each item is scored on a scale from 1 (“not at all”) to 4 (“very much”). The sum of the scores is divided by the number of answered items. A general score equal to or greater than 1.75 is consistent with an increased level of emotional distress.

The NDI is a self-rated disability questionnaire developed for patients with neck pain [20]. It consists of 10 items: pain intensity, personal care, lifting, reading, headaches, concentration, work, driving, sleeping and leisure activities. In the present study, the Norwegian translation of the NDI was used (http://www.mapi-institute.com/questionnaires-and-translation/ourcatalog/133-specific-questionnaires-rheumatology). Each item is scored from 0–5, (no disability to total disability), with the maximum score being 50. We report the total scores as percentages, with the maximum score corresponding to 100%.

To determine patients’ impression of improvement, we used the five-item Norwegian Global Rating Scale of Change (GRS), which includes the responses of “a lot worse, a bit worse, no change, somewhat better and much better” and is scored from —2 to +2.

3.1 Reliability

test–retest reliability assesses the extent to which scores for patients who have not changed remain the same for repeated measurements [6]. In this study, the test–retest reliability of the baseline scores and the second NDI scores (n = 42) were assessed using a 2-way random effects intraclass correlation coefficient (ICC). According to Rosner, ICC values from 0.40 to 0.75 indicate fair to good reproducibility, while an ICC value >0.75 shows excellent reproducibility [25].

In addition, Cronbach‘s alpha was used to estimate the internal consistency. The values >0.9 were regarded as excellent, 0.8-0.9 good and >0.7 acceptable [26].

3.2 Responsiveness

The anchor used in this analysis was the GRS scale, which was categorized into the following categories: worsened (response of a lot worse and a bit worse; n = 24), unchanged (response of no change; n = 27) and improved (response of somewhat better and much better; n = 62).

Similar to Pool et al. [27], we defined minimum detectable change (MDC) as the smallest difference in a score that can be detected, given the variation in changes on the NDI observed in patients who remained unchanged on the GRS (n = 27). The MDC was determined by first calculating the standard error of the mean (SEM) of the change in NDI scores. We used the formula SEM = SD √(1 – r), where SD is the standard deviation and r is the reliability coefficient, and then determined the 95% CI for MDC using the equation MDC = 1.96 × √2 × SEM [28,29]. Furthermore, we assessed responsiveness in patients who noted improvement on the GRS. Receiver Operating Characteristic curves (ROC curve) were plotted with the sensitivity values (true positive) on the y-axis against the 1-specificity values (false positive) on the x-axis for improved versus unchanged patient scores. The area under the curve (AUC) and 95% confidence intervals were calculated. The AUC value was considered acceptable if it was ≤0.70 (Hosmer and Lemeshow, 2000) [30]. A separate calculation was conducted for worsened versus unchanged patients scores. The minimum clinically important difference (MCID) was determined to be the magnitude of change associated with the uppermost left-hand corner of the curve, where both sensitivity and 1-specificity are maximized (Fig. 2) [28]. We performed the analyses using Predictive Analytics Soft Ware (PASW, version 18.0, SPSS, Chicago, IL, USA).

Fig. 2

Receiver operating characteristics curve on the Neck Disability Index (NDI) for those having improved according to the Global Rating Scale of Change (GRS). The arrow marks the data point nearest the uppermost left hand corner of the graph and represent the Minimum Clinically Important Difference (MCID) of the NDI.

4.1 Reliability

The mean NDI total score at baseline (n = 42) was 37.48 (SD 13.51), and the mean NDI total score after 3–14 days was 35.36 (SD 14.82). The mean difference between these 2 scores was 2.12%, which was not statistically significant.

The test–retest reliability between the baseline NDI total scores and the second NDI score was very good (ICC = 0.84; 95% CI 0.72–0.91). Table 2 shows the test–retest estimates of each of the 10 NDI items, with the ICC values in the range of good-to-excellent reproducibility (0.5–0.9).

The internal consistency of NDI assessed by Cronbach’s alpha was good-to-excellent in both the baseline and second administration (0.83 and 0.91, respectively).

4.2 Responsiveness

The mean change in NDI scores noted at follow-up for the responsiveness study (improved vs. unchanged patients) was 10.4 (SD 15.4). The SEM was calculated to be 4.44, while the MDC value was 12.3% (or 6.15 out of 50).

The difference in the ability of the NDI to discriminate between improved and unchanged patients (responsiveness) over time was significant based on the ROC curve analysis (AUC = 0.70; 95% CI 0.58-0.82) (see Fig. 2). Similar results were found when computing ROC curves for those reporting to have worsened, with an AUC value of 0.72 (95% CI 0.58–0.87).

In the present study, a change of 8.3 points (if referencing 50-point scale) or 16.6% was the estimated MCID for the Norwegian version of the NDI.

5.1 Reliability

The test–retest reliability of the total NDI scores and separate NDI items in this study may be regarded well, based on the ICC values. The values were within the range reported in other studies (0.50 and 0.98) [9,12,31]. Variance in different studies is expected and is dependent on the time intervals between the test–retest scores, the different anchors used to define the “stable” state, the different subgroup classifications used, the presence of acute versus chronic conditions and the origin of the pain (i.e., muscular versus neurologic) [9].

Previous studies of NDI have reported Cronbach’s alpha values in the range of 0.74 and 0.93 [31,32,33,34]. Our findings are consistent with other international studies in this respect, as we noted a Crohnbach’s alpha value of 0.83 and 0.91. In this context, it is important to note that it has been stated that a meaningful Cronbach’s alpha value demands a unidimensional scale [8]. However, previous studies questioned the unidimensionality of the NDI when the Rasch analyses were applied to test its unidimensionality and scaling [11,16]. Nevertheless, in the present study, the alpha value was very high, and we believe that it would not be significantly altered in a way that changes the test–retest reliability. Of note, our study population consisted of patients with chronic pain whose pain intensity did not fluctuate significantly from day to day.

5.2 Responsiveness

Responsiveness has been evaluated using 2 methods, the Minimum Detectable Change (MDC) and Receiver Operating Characteristics curve (ROC), both of which integrate an anchor-based and distribution-based approach. The instrument can be considered responsive when the MDC exceeds the SEM, which was the case in our study.

Based on the current AUC value, the Norwegian version of the NDI seems to be a responsive instrument for detecting changes in clinical state. However, the figures are not quite convincing, as the AUC value did not exceed 0.70. The ability to detect worsening was somewhat improved, although the difference was small and only suggestive. This result is consistent with findings from other studies, which have found that the instrument in question is somewhat better in detecting worsening than improvement [35].

Responsiveness can also be reported using the Minimum Clinically Important Difference (MCID), which is the minimal amount of change in an instrument score associated with subjective improvement. As mentioned before, this value is related to the sensitivity and specificity of the instrument. Furthermore, it is also dependent on the anchor being used and on the patients to whom it is applied (e.g., acute and chronic pain, with and without referred pain, patients seen in general practice versus specialized clinics). In our study, the MCID (8.3 points or 16.6%) was out of the bounds of measurement error as indicated by the MDC (6.15 points or 12.3%). Our MCID value was similar to that of Young et al. [28] which studied improved versus stable patient groups with mechanical neck pain, but their MCID value was within the bounds of measurement error. However, the MCID was lower than that noted by Cleland et al. [36] who reported an MCID value of 10 of 50 points and 19 of 50 points in two different studies of mechanical neck pain and radiculopathy, respectively. Stratford et al. [37] reported a MCID value of 5 of 50 points but they used an a priori assessment by the evaluating or treated therapist to assess meaningful change. Vos et al. [12] reported a value as low as 1.66 when including stable patients with acute neck pain from a general practice. Some of these differences may be explained by the different estimation methods and anchors used and the fact that different groups of patients may have different prognoses with respect to improvement. For example, patients with a higher baseline NDI score must experience a greater improvement to feel that it is relevant, which is reflected by a higher MCID value [20]. Furthermore, the MCID is an outcome measure that depends on the patient’s evaluation, the clinicians’ definition of important change values and the context in which it will be used. In addition, it should be interpreted in the context of empirical experience.

In the present study, chronic pain was defined as neck pain lasting 3 months or more. Thus, it is questionable to what extent these patients really can change or if the condition is an expression of a steady-state situation. However, three quarters of patients either improve or worsen, indicating that the use of the NDI as an outcome measure in these patients is reasonable.

In several other studies, the anchor-based method of evaluating the responsiveness of the NDI has been used [12,28,36,37,38]. However, there has been discussion as to whether it is appropriate to use the GRC as a comparison to the NDI and regarding the time delay for responses and the impossibility of recalling a subjective state condition at different times for comparison. In the current study, the patients had not been evaluated during or after a specified treatment session. Indeed, they had only been asked to evaluate their present neck pain status, and this assessment was compared to a created gold standard (i.e., the GRS), first presented by Jaeschke et al. [39]. The participants were presented with a set of questionnaires at the first and last distribution; thus, the ability to recall any exact answers on earlier occasions was diminished. We note that the patients completing the GRS had a significantly higher educational level and improved functioning as measured by the NDI.

The present results give further incentives to study the responsiveness of the NDI in different clinical settings, as well as with different patient groups and treatment options, e.g., studying patients in need of surgery versus patients seeking conservative treatment options. Furthermore, the NDI may be useful in other clinical settings as well due to the fact that it is a well-distributed instrument within the International Classification of Functioning (ICF) perspective [40] and the possibilities that this offers for the instrument as an outcome measurement to describe functioning and change in function, for example, with respect to work ability measurements.

Finally, it is worth mentioning the potential shortcomings of this study. In the test–retest reliability, the “practice effect” - respondents “learn” to answer the same questions in the first test and this affects their responses in the second test. Thus, the NDI scores may tend to be higher in the second test. However, the baseline NDI total score in this study was somewhat higher than the second scores. The other shortcoming is that only 44% of the patients participated in the responsiveness analysis. The external validity of the presented data may, therefore, be limited.