Meta-analysis of the diagnostic performance characteristics of three commercial and one in-house nucleic acid amplification tests for malaria screening

Brief summary: In non-endemic settings, microscopic skills in malaria microscopy are difficult to maintain. The study comparatively assessed three commercial and in-house platforms for the molecular identification of plasmodiae on the genus and species levels. It suggests high concordance of all tested assays for microscopically positive samples and only low to moderate discrepancies for microscopically negative ones, supporting the usefulness for screening purposes.

Nucleic acid amplification testing (NAT) approaches for the screening for malaria are useful options for laboratories in non-endemic settings, where training options to train and maintain skills in malaria microscopy are scarce, in particular for early infection stages due to their superior sensitivity [1], [2] or for the exclusion of mixed plasmodial infections [3], [4].

At the Bernhard Nocht Institute for Tropical Medicine, the German National Reference Center for Tropical Infectious Diseases, a variety of NAT assays for malaria screening and differentiation has been evaluated recently using identical samples [5], [6]. Here, a meta-analysis of the results is presented, allowing a direct comparison of the assays and particularly focusing on discordant results affecting sensitivity and specificity.

In previous studies, a collection of 1020 well-characterized nucleic acid extractions [5], [6] from ethylenediaminetetraacetic acid (EDTA) blood taken from patients with clinical suspicion of malaria and extracted using the EZ1 DNA Blood 200 μL Kit (Qiagen, Hilden, Germany) had been completely assessed by four different NAT approaches in addition to microscopy of Giemsa-stained thick and thin blood films:

1. genus-specific loop-mediated isothermal amplification (LAMP) using the Meridian illumigene/alethia Malaria platform (Meridian Bioscience Inc., Cincinnati, OH, USA) [5]

2. species-specific polymerase chain reaction (PCR) using the “FTD Malaria differentiation” real-time multiplex PCR kit (Fast Track Diagnostics [FTD], Sliema, Malta) for all samples, of which only the ones positively screened by RealStar Malaria PCR Kit 1.0 (altona Diagnostics, Hamburg, Germany) had been published so far [6]

3. genus-specific screening PCR using the RealStar Malaria PCR Kit 1.0 with subsequent discrimination using the RealStar Malaria S&T PCR Kit 1.0 (altona Diagnostics) in case of positive screening results [6]

4. species-specific in-house real-time PCR for all samples (unpublished data) using a previously described in-house SybrGreen real-time PCR assay [7], slightly modified exactly as described [8].

When comparing the NAT approaches, a NAT result was considered as truly positive, if it was either confirmed by microscopy, by a positive result of another NAT approach or by a matching result in another sample of the same patient during the same disease period. As the focus was on plasmodial nucleic acids, positive microscopic results were considered as sufficient but not necessary for confirmation.

In total, 1020 samples from 816 patients were assessed. This population was identical with the population described previously [6], with the exemption that the two samples with Plasmodium knowlesi could not be included due to insufficient sample volumes for all assessments. All four assessed molecular approaches led to concordant results for 988 out of 1020 samples (96.9%). These concordant results comprised 765 samples (75%) without any microscopic or molecular hint for the presence of plasmodiae. In 223 samples (21.9%), concordantly positive molecular test results were observed, comprising concordant identification on the species level by all three species-specific PCR tests. Within these concordantly positive samples, Plasmodium falciparum was identified in 199 instances, Plasmodium vivax in 12 instances, Plasmodium malariae in six instances and Plasmodium ovale in six instances. A total of 72.8% samples (172/223) were positive in microscopy as well, while the other samples comprised sub-microscopic parasite densities or circulating residual DNA during follow-up assessments in case of successful therapy. Observed parasitemia ranged from a few detected parasites in the whole thick blood film up to 14%, with a mean of 1760 parasites/μL in a left-shifted distribution and a median (±standard deviation [SD]) of 36225 (±101142)/μL.

The 32 samples with discordant results comprised 19 samples with P. falciparum, seven presumably falsely positive samples in line with the case definition as described above, two samples with mixed infections of P. falciparum and P. vivax, two samples with P. vivax and two samples with P. ovale. Missed samples were either microscopically negative (n=27) or at the borderline of the microscopic detection limit (2× P. ovale, 1× P. vivax) defined as 50 parasites/μL. In the samples with mixed infections of P. falciparum and P. vivax, the quantitative dominance of P. falciparum made P. vivax undetectable for melting curve-based identification with the SybrGreen-based in-house approach [7] despite high overall parasitemia of 4600/μL and 5240/μL, respectively. Although the low numbers of discordant results do not allow definite conclusions, the results suggest slightly better overall sensitivity of the molecular approaches with P. falciparum than with the non-P. falciparum species, for which high numbers of positive samples are usually more difficult to obtain for test evaluation purposes. Details of the discordant samples are shown in Table 1.

Focusing on genus-specific Meridian illumigene/alethia Malaria LAMP, the assessment confirmed three presumably falsely positive signals as it had already been described previously [5], confirming a specificity of 99.6% (769/772). None of these signals could be confirmed by any other applied diagnostic approach and no additional materials were available from these patients. However, the more extensive molecular testing in this study slightly decreased the calculated sensitivity to 96.4% (239/248) in comparison with 98.7% from the previous assessment [5]. The missed samples were from patients positive for P. falciparum (n=7) who had already become negative by microscopy under therapy as well as for P. ovale (n=1) and P. vivax (n=1) with only a few parasites within the whole thick blood film each.

When using the species-specific multiplex PCR by FTD for screening purposes, again a specificity of 99.6% (769/772) was identified with presumably false positive signals suggesting P. falciparum (n=2) and P. malariae (n=1) in patients, for whom no additional sample materials were sent for diagnostic assessment. Six samples, which were positive in line with the case definition, went undetected, resulting in a sensitivity of 97.6% (242/248). The falsely negative samples comprised patients with P. falciparum who had already become microscopically negative under therapy without exemption.

Focusing on the combined approach of genus-specific PCR by altona Diagnostics with subsequent species-specific PCR in case of positive results in the genus-specific PCR, genus-specific PCR provided one presumably false-positive signal, resulting in a specificity of 99.9% (771/772). With regard to genus-specific PCR alone, the sensitivity was 98.4% (244/248). The four missed samples were from patients with P. falciparum who had already become microscopically negative under therapy. However, when species differentiation was included, the sensitivity dropped to 94.0% (233/448). The samples missed by species-specific PCR comprised 12 samples with P. falciparum, two samples with P. vivax and one sample with P. ovale. While one of the samples with P. vivax and the one sample with P. ovale were microscopically positive with few parasites within the whole thick blood film, all other samples were from patients under therapy who had already become microscopically negative.

The species-specific SybrGreen-based in-house real-time PCR [7] did not show any falsely positive signals, resulting in a specificity of 100% (772/772). In 13 instances, however, the PCR produced falsely negative or incomplete results, reducing the sensitivity to 94.8% (235/248). As stated earlier, the incomplete results comprised two samples with co-infection with P. falciparum and P. vivax, for which the melting curve suggested mono-infection with P. falciparum only. The 11 falsely negative samples without any hint for malaria comprised nine samples with microscopically already cleared P. falciparum-malaria and two samples with few P. ovale parasites within the whole thick blood film each.

Details of individual test characteristics of the assessed assays at least for the detection on the genus level in comparison with the chosen composite gold standard are shown in Table 2.

All assessed molecular approaches proved to be reliable for identifying malaria in patients with microscopically positive samples. Parasitemia of the three samples with P. ovale and P. vivax parasites, which were missed by individual molecular tests, was so low that the likelihood of going microscopically undetected in laboratories other than highly experienced reference centers would have been high as well. The observed minor discrepancies between the assessed molecular assays may be of relevance in case of screenings for sub-microscopic parasitemia in the course of malaria eradication programs [9] but hardly for the diagnostic routine setting.

Nevertheless, there are a few challenges yet to be met by future NAT approaches. The ideal molecular screening test should detect and identify all Plasmodium spp. with relevance for human health. Among the species-specific tests compared here, only the altona Diagnostics approach was able to positively identify P. knowlesi [6] and none of the applied tests could discriminate Plasmodium ovale curtisi and Plasmodium ovale wallikeri, although respective real-time PCRs have been described [10], [11], [12] and recently evaluated [13]. Next to this, a reliable quantification of parasitemia by NAT approaches remains a challenge which is yet unmet by any commercially available, species-specific malaria NAT to the best of the authors’ knowledge.