Meta-analysis: compared with anti-CCP and rheumatoid factor, could anti-MCV be the next biomarker in the rheumatoid arthritis classification criteria?

Search strategies

This meta-analysis was performed following the guidelines of the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy. Four databases were searched for this study: Medline, Embase, Cochrane Library and Web of Science, with publication time up to 25 August 2018. A combination of MeSH terms and keywords pertaining to RA (“Arthritis, Rheumatoid” OR “rheumatoid arthr*” OR “RA”), AND anti-MCV (“MCV” OR “mutated citrullinated vimentin”) were applied to the search strategies. The reference lists of all the included studies were searched to identify any additional eligible studies.

Study selection

Titles and abstracts were independently screened by two reviewers (Zhu and Nie). Full articles were retrieved and reviewed for the following inclusion criteria:

1. studies were prospective or retrospective or cross-sectional study of the diagnostic value of MCV, with or without CCP and/or RF in RA study with human subjects;

2. studies included a healthy and/or other diseases control groups for comparison;

3. RA patients fulfilled either the 1987 ACR classification criteria or the 2010 ACR/EULAR classification criteria, as which have been approved by the professional and scientific authorities;

4. studies evaluated MCV, with or without CCP and/or RF levels in serum, and were quantitative tests;

5. studies provided the necessary data, e.g. the number of true positive, true negative, false positive, and false negative, sensitivity, specificity, or likelihood ratios, to construct 2×2 contingency tables of levels of MCV, with or without CCP and/or RF for the diagnosis of RA.

Studies were excluded if they were not available in English or included duplicate data. No age restriction on participants was applied here.

Data extraction

Two reviewers (Zhu and Nie) individually extracted the data, with discrepancies resolved by discussion. When it was difficult to reach an agreement, a third reviewer (Lu) was consulted. When results of different cutoff values were reported on an index test, data from the recommended cutoff value according to the manufacturers’ instructions were extracted. Where there was more than one type of assay for the same index test, the more frequently used one among included studies was extracted. Where more than one subtype of the same kind of index test was available from the study, the more widely used one was included for the review, to be more specific, anti-CCP2 IgG and RF IgM were chosen as the representation.

Quality assessment

The quality of each included study was independently assessed by the two reviewers (Zhu and Nie), using the revised version of the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. Any disagreements were resolved through discussion.

Statistical analysis

Data analysis was performed using Stata 12 software (Midas commands) (StataCorp LP, College Station, TX, USA), Revman 5.2 (The Nordic Cochrane Centre, The Cochrane Collaboration, Copenhagen, Denmark) and R version 3.5.0 (R Foundation for Statistical Computing, Vienna, Austria).

The bivariate random effects model based on the binomial distribution for sensitivity and specificity was employed in primary data synthesis, with inferences made about summary estimates of sensitivity and specificity and their 95% confidence intervals (CIs). Derived logit estimates of sensitivity, specificity based on parameters estimated by the bivariate model were used to plot a hierarchical summary ROC curve, with a summary point estimate, 95% confidence region and 95% prediction region presented. The area under the curve (AUC) is the average TPR over the entire range of FPR values.

I² is viewed as the index to assess the heterogeneity, which calculates the percentage of total variation across studies caused by heterogeneity rather than chance. I² ranges from 0% to 100%, with 0% indicating no observed heterogeneity, and values greater than 50% suggesting substantial heterogeneity. Covariates regarding any test performance that may contribute to the heterogeneity were introduced into a regression as the dependent variable.

To investigate publication bias, the funnel plot with a superimposed regression line of diagnostic log odds ratio against 1/sqrt (effective sample size), weighting by effective sample size was conducted, and p<0.10 for the slope coefficient suggests significant asymmetry.

Search results

The initial search identified 704 articles, of which 300 were duplicates and 302 were excluded according to the title and abstract (Figure 1). One hundred and two full-text articles were reviewed, of which 25 were excluded as we were unable to extract sufficient information to construct 2×2 tables, 15 were excluded as they did not include a control group, seven were excluded due to a lack of a golden diagnostic tool, specifically, neither the 1987 ACR classification criteria nor the 2010 ACR/EULAR classification criteria for RA was adopted in these studies, 15 were conference abstracts, two were systematic reviews, two were duplicate reports and another two were not available in English. Therefore, the meta-analysis was conducted based on the remaining 33 studies [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41] (Table 1).

Figure 1:

Flowchart of selection of studies for inclusion in the meta-analysis.

Methodological quality of included studies

QUADAS-2 was selected as the guide for methodological quality evaluation for the included studies, this evaluates the risk of bias of each study cohort on four categories (patient selection, index test, reference standards, flow and timing), and the concerns regarding applicability on three categories (patient selection, index test, reference standards). The risk was graded as high, low or unclear, and an overall risk grades for each category mentioned were estimated (Figure 2). About half of the studies showed a high risk for patient selection because a case-control design was often difficult to avoid and a consecutive or random sample of cohorts was difficult to satisfy (see Supplementary material, Figure 1). Around 50% of the studies presented an unclear risk for flow and timing, based on the fact that except for the gold standard of diagnosis for RA, the reference standards for other rheumatic diseases were unclear, therefore making it challenging to decide whether the disease control groups received the same gold standard. Nearly all the studies demonstrated a low risk among all three categories of the concerns regarding applicability.

Figure 2:

Risk of bias and applicability concerns summary.

Each item in the graph was described with + (low risk of bias),−(high risk of bias) and ? (unclear).

Diagnostic value of anti-MCV with RA

The diagnostic value of MCV to RA vs. the disease control group mainly consisted of other rheumatic diseases, some infectious diseases with similar clinical features to RA, one study with unaffected first-degree relatives from RA families is also reported here. Among the 33 studies included in the present meta-analysis, eight of which provided information only on healthy controls rather than disease controls. According to the forest plot, the remaining 25 included studies [9], [10], [11], [12], [13], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], 32], [41], [42] showed a combined sensitivity of 0.71 [0.64–0.77] (95% CI), specificity of 0.89 [0.85–0.92] (Figure 3). The SROC curve showed an AUC of 0.89 [0.86–0.91] (Figure 4).

Figure 3:

Forest plot of coupled sensitivity and specificity for anti-MCV in the diagnosis of RA vs. DC.

The point estimate of sensitivity and specificity of each study is shown as a solid square on a line that represents the CI. The continuous vertical line represents the weighted average sensitivity and specificity and dotted vertical lines represent the CI.

Figure 4:

SROC curve for anti-MCV in diagnosis of RA vs. DC.

Each circle represents a study and the asterisk represents the global estimate.

The I² of combined sensitivity was 94.73 [93.42–96.03], with p=0.00 for Q test; similarly, the I² of combined specificity was 88.32 [84.66–91.98], with p=0.00 for Q test. A meta-regression was performed due to the obvious heterogeneity, with covariates including testing methods of MCV with its cutoff value, disease duration, diagnostic criteria and predesign of the primary study. As a result, the predesign of the primary study presents as a significant contributor to the heterogeneity for both sensitivity and specificity, with p=0.00 and p=0.01 respectively, while diagnostic criteria contribute to the heterogeneity for specificity (p=0.05).

Publication bias was examined using the Deeks’ funnel plot asymmetry test, symmetry in the data and a low likelihood of publication bias are observed in the present study (p=0.20).

Clinical subgroup analysis

The predesign of the study and the diagnostic criteria were observed to be statistically significant as sources of heterogeneity by meta-regression among the 25 studies where the information on the disease control group was available, therefore, these two covariates were investigated by subgroup analysis. Clinical subgroups of different testing methods of MCV with its cutoff value and disease duration were also analyzed, because of their clinical importance in RA diagnosis.

Seven studies [11], [16], [21], [24], [30], [31], [41] with a prospective study predesign were distributed into one subgroup. The results of this subgroup suggested that according to the forest plot, the combined sensitivity was 0.57 [0.46–0.68], the specificity was 0.90 [0.86–0.94] (see Supplementary material, Figure 2). Among those 25 studies, only Iwaszkiewicz et al. [12] used the ACR/EULAR 2010 criteria as its diagnostic criteria for RA, while the rest adopted ACR 1987. Nineteen studies that used an enzyme-linked immunosorbent assay (ELISA) as its testing method for MCV, with the cutoff set to 20 U/mL were assigned to one subgroup. The subgroup of uniform cutoff demonstrated a combined sensitivity of 0.70 [0.62–0.77], specificity of 0.90 [0.85–0.93] according to the forest plot (see Supplementary material, Figure 3).

Overall, the subgroup of the prospective study predesign demonstrated the lowest heterogeneity for both sensitivity and specificity and compared with all the other results in the present review, the value of I² can be reduced by up to 10% for sensitivity, and 30% for specificity. The heterogeneity of the combined sensitivity among these two subgroups ranged from 81.56% to 93.69%, represented by I²; the I² of specificity ranged from 57.47% to 89.92%, and Q test suggested p=0.00, except for the specificity of the prospective study predesign subgroup (p=0.04).

Comparison between MCV, CCP and RF

Among the 33 studies included for the present meta-analysis, the results of anti-CCP tests from the disease control group were not available for nine of them [11], [29], [31], [33], [34], [35], [36], [37], [38]. The remaining 24 included studies [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [30], [32], [39], [41], [42] which showed a combined sensitivity of 0.71 [0.64–0.77] and a specificity of 0.95 [0.94–0.97] (Figure 5) for anti-CCP according to the forest plot. The SROC curve showed an AUC of 0.95 [0.92–0.96] (see Supplementary material, Figure 4). With regard to RF, among the 33 originally included studies, 17 [10], [11], [12], [14], [16], [22], [24], [27], [29], [31], [34], [35], [36], [37], [38], [39], [40] of which lacked information of RF tests from the disease controls, only about half of the studies could be used to perform the following analysis. The result demonstrated a combined sensitivity of 0.77 [0.68–0.84], a specificity of 0.73 [0.63–0.82] for RF and an AUC of 0.82 [0.78–0.85] according to SROC curve (see Supplementary material, Figures 5 and 6). However, the results revealed substantial heterogeneity among the studies for each diagnostic marker. The I² of sensitivity for CCP was 92.70 [90.67–94.74], and that of specificity was 79.93 [72.44–87.42]. A lower heterogeneity of 89.94 [86.13–93.75] was observed in sensitivity for RF, with I² of specificity being 93.37 [91.55–95.59]. The p-value for the Q test was 0.00 for all the four heterogeneity results mentioned here.

Figure 5:

Forest plot of coupled sensitivity and specificity for anti-CCP in the diagnosis of RA vs. DC.

Forest plots (for sensitivity and specificity) and pooled estimates (95%CI) by using the I² test (quantified by p-value <0.01) were yielded to show heterogeneity between studies.

Combinations of markers in series and in parallel

Ten studies [13], [14], [15], [22], [23], [24], [25], 37], [38], [39], [40] were included to calculate the combined sensitivity of anti-MCV and anti-CCP when performed in parallel. The result suggested a sensitivity of 0.77 [0.72–0.82], with I²=80.1% [64.1%–88.9%]. Eleven studies [13], [15], [22], [24], [25], [30], [35], [37], [38], [39], [40] contained information on the performance of anti-MCV and anti-CCP in series and their combined sensitivity was 0.60 [0.54–0.67], with I²=84.7% [74.8%–90.7%]. Nevertheless, no information on the control groups can be abstracted from almost all the studies, so the combined specificity was not available. Three studies [32], [36], [37] were incorporated to evaluate the serial tests for anti-MCV and RF, with the combined sensitivity of 0.64 [0.40–0.83], with I²=95.7% [90.7%–98.0%]. All these calculations regarding the diagnostic values of combinations of the markers were done using random effects models, and all Q tests suggested p<0.0001.