Parallel testing of 241 clinical nasopharyngeal swabs for the detection of SARS-CoV-2 virus on the Cepheid Xpert Xpress SARS-CoV-2 and the Roche cobas SARS-CoV-2 assays

To the Editor,

The World Health Organization (WHO) has declared the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) outbreak to be a global pandemic. As of the time of this writing, several thousand new confirmed COVID-19 cases have been reported worldwide [1]. To cope with the surging demand for timely diagnosis of COVID-19 cases, many SARS-CoV-2 molecular assays have become commercially available [2]. At our institution (National University Health System, Singapore), the Cepheid Xpert Xpress SARS-CoV-2 (herein referred as the Xpert; Cepheid, CA, USA) and the Roche cobas SARS-CoV-2 (herein referred as the cobas; Roche Molecular Systems, NJ, USA) assays are deployed interchangeably to meet the national testing demand for SARS-CoV-2. The Xpert assay targets the N2 region of the nucleoprotein (N) gene for specific SARS-CoV-2 detection and a conserved region of the structural protein envelope (E) gene for pan-sarbecovirus detection. Likewise, the cobas also targets a conserved region of the E gene for pan-sarbecovirus detection but amplifies the open reading frame 1ab (ORF1ab) nonstructural region for specific SARS-CoV-2 detection. We have had concerns before the integration of two different platforms for SARS-CoV-2 testing. First, it was unclear whether the test performance of the two assays would be comparable in our hands. Second, existing comparative studies on the Xpert and cobas assays relied on retesting of residual positive samples that were initially detected by the cobas assay [3], [4], [5], [6]. This may lead to the exclusion of positive samples near the detection limit of the Xpert assay, which the cobas assay may fail to detect. Third, several reports performed their analyses using only a small number of low positive samples which might not be sufficient to interrogate the detection limit of the assays [3], [4], [5], [6], [7].

To address the concerns, we assayed 241 fresh clinical nasopharyngeal (NP) swab samples in parallel on both the Xpert and the cobas assays to allow for a head-to-head comparative analysis. Ethics approval was granted by the National Healthcare Group Domain specific Review Board (NHG ROAM Reference Number 2020/00337). Statistical analyses were performed using the R statistical language, version 3.6.0, with p values of <0.05 denoting statistical significance. All samples were collected between June and July 2020, transported in universal transport medium (UTM; Copan Diagnostics Inc., Brescia, Italy), and tested within 24 h of collection. A summary of the demographic characteristics of the study samples is shown in Table 1.

Following testing of 241 NP swabs, 58 tested positive and 172 tested negative with both assays, for an agreement of 95% (95% confidence interval (CI) 91.8–97.6%). The median cycle threshold (CT; Figure 1A) values for the E gene by the Xpert and cobas assays were 35.80 (interquartile range (IQR) 32.65–38.98) and 36.90 (IQR 35.57–37.86), respectively. The median CT values for the Xpert N and cobas ORF1ab gene targets are 38.30 (IQR 36.90–40.70) and 31.29 (IQR 23.85–33.22), respectively. For samples that were positive on both assays, 52/58 (90%) were detected with high E gene CT values ranging between 30.90 and 42.70 on either assay. Bland-Altman analysis (Figure 1B) showed a mean bias of −0.62 ± 2.87 (mean difference: −2.47; 95% limits of agreement: −6.26 to 5.01) on 45 samples that had E gene target detected on both assays. Interestingly, the analysis suggested that a significant proportion of our positive samples harbored SARS-CoV-2 at concentration near the assays’ detection limit, i.e., low viral titer, thus accounting for the large scattering observed exclusively at high mean E gene CT of >34.

Figure 1:

(A) Dot and box plots of the pan-sarbecovirus E gene cycle threshold (CT) values obtained with the Cepheid Xpert Xpress SARS-CoV-2 and the Roche cobas SARS-CoV-2 assays for all positive samples. The median CT values by the Xpert and cobas assays were 35.80 and 36.90, respectively. Wilcoxon test showed no significant difference in the CT values obtained with both assays (p=0.44). (B) Bland–Altman analysis on 45 samples that had E gene target detected on both the Cepheid Xpert Xpress SARS-CoV-2 assay (method 1) and the Roche cobas SARS-CoV-2 test (method 2). Bias: −0.62; standard deviation of bias: 2.87; mean of differences/means: −2.47; (95% confidence interval (CI): −1.49 to 0.24); upper limit of agreement: 5.01 (95% CI 3.52–6.50); lower limit of agreement: −6.26 (95% CI −7.75 to −4.78). (C) A dot plot showing the CT values obtained with the Xpert assay (N and E genes) and the cobas assay (ORF1ab and E genes) for 58 low positive samples. Data points above CT value 45 (horizontal dotted line) are negative. The detection rate for the respective gene targets is indicated below the data points. The highest sensitivity for the low positive samples was the Xpert SARS-CoV-2-specific N gene (57/58; 98%) followed by the cobas pan-sarbecovirus E gene (51/58; 88%) and the Xpert pan-sarbecovirus E gene (45/58; 78%). The cobas SARS-CoV-2-specific ORF1ab gene was the least sensitive (9/58; 16%).

A total of 11 discordant samples were noted where 10 were detected by the Xpert assay only (median E gene CT: 38.50 [IQR 38.03–39.38]; median N gene CT: 40.60 [IQR 39.20–41.60]) and one was detected by the cobas assay only (E gene CT: 38.66; ORF1ab CT: not detected). To adjudicate discrepant results, we retrieved each of the 11 samples for automated nucleic acid extraction on the KingFisher Flex instrument (Thermo Fisher Scientific, Massachusetts, USA) and retested them using the TaqPath COVID-19 RT-PCR assay (herein referred as the TaqPath; Thermo Fisher Scientific) on the Applied Biosystems 7500 Fast Real-Time PCR Instrument (Thermo Fisher Scientific), according to the manufacturer’s instructions. The TaqPath assay targets three different regions (ORF1ab, N, and Spike [S] genes) of the SARS-CoV-2 genome and the claimed limit of detection is 10 genomic copy equivalents per reaction with 40 cycles of amplification. The consensus result was defined as the result obtained by at least two of the three assays. After retesting the 11 discordant samples using the TaqPath assay, six of the 10 Xpert-only positive samples (median E gene CT: 38.40 [IQR 37.20–38.60]; median N gene CT: 39.50 [IQR: 38.45–40.40]) agreed with the TaqPath results and were deemed as positives based on consensus. On the other hand, four of the 10 Xpert-only positive samples (median E gene CT: 39.90 [IQR 39.10–41.55]; median N gene CT: 41.60 [IQR 41.60–42.10]) disagreed with the TaqPath results and were reclassified as negatives. Likewise, the single cobas-only positive sample also disagreed with the TaqPath result and was subsequently reclassified as negative. Compared to the consensus results, the Xpert assay had a kappa coefficient of 0.96 (95% CI 0.92–1.00; almost perfect agreement), positive percent agreement (PPA) of 100% (95% CI 94–100%), and negative percent agreement (NPA) of 98% (95% CI 94–99%). For the cobas assay, the kappa coefficient was 0.92 (95% CI 0.87–0.98; almost perfect agreement), PPA was 91% (95% CI 81–96%), and NPA was 99% (95% CI 97–100%).

In summary, we assayed 241 fresh NP swabs in parallel across the Cepheid Xpert Xpress SARS-CoV-2 and the Roche cobas SARS-CoV-2 assays. In our hands, there was 95% agreement between the assays for both negative and positive clinical samples, suggesting comparable diagnostic performance in the detection of SARS-CoV-2 virus from NP swabs. To the best of our knowledge, this is the third head-to-head comparison data generated via parallel testing of samples on the two automated platforms [6], [, 7]. The present work challenged the assays’ detection limit using 58 clinical samples with low viral titers. Figure 1C depicts the test performance of the different gene targets. The Xpert SARS-CoV-2-specific N gene is the most sensitive, detecting 98% (57/58) of the low positives while the Xpert pan-sarbecovirus E gene detects 78% (45/58) of the low positives. On the contrary, the cobas SARS-CoV-2-specific ORF1ab gene exhibits inferior sensitivity, detecting only 16% (9/58) of the low positives. However, the poor diagnostic yield is compensated by the cobas pan-sarbecovirus E gene which detects 88% (51/58) of the low positive samples. Our data corroborates with previous studies that the Xpert and cobas pan-sarbecovirus E gene targets performed similarly [3], [4], [5], [6], [7]. For the SARS-CoV-2-specific N and ORF1ab gene targets, a significant difference in the positive identification rate was observed using the McNemar’s test (p value < 0.05). Notably, the cobas ORF1ab gene has performed poorly when compared to the Xpert N gene which corroborated with Lowe’s data [3]. Altogether, our results show that the Xpert assay has higher sensitivity compared to the cobas assay, with the former detecting six additional cases among the low positives (6/58; 10%). It is unknown whether high CT values (low viral load) has significant diagnostic value due to the lack of correlation with culture results, variables associated with sampling site, and collection methodology. Further studies are required to determine the diagnostic value of using assays with higher analytic sensitivity. However, the authors suggest that retesting of the samples using the Xpert assay should be considered when the SARS-CoV-2 results do not correlate with the patients’ clinical presentations. Considering our findings, we believe that both assays can be deployed interchangeably in experienced molecular diagnostic laboratories for routine diagnostics of COVID-19 and the detection of low positive SARS-CoV-2 samples.