Transformers meets neoantigen detection: a systematic literature review

3.1 Pre-trained transformer models

Additionally, there are pre-trained Transformer models like TAPE [27], ProtTrans family models (ProtBert-BFD, ProtT5-XL, and ProtT5-XXL) [28], ESM-1, with models with different sizes [29], and ESM2 [30]. All of them were trained for several proteomic tasks, like protein structure prediction, protein function prediction and more. Moreover, recent methods are fine-tuning these pre-trained models for some steps during the process of neoantigen detection like peptide-MHC binding prediction and pMHC-TCR interaction with good performance. In Table 2, we present a detailed comparison of these models.

Tasks Assessing Protein Embeddings (TAPE) [27] represents the initial effort to assess semi-supervised learning applied to protein sequences. TAPE comprises twelve layers, each containing 512 units and featuring eight attention heads, resulting in a total of 92 million parameters. The authors employed semi-supervised training using the Pfam dataset [31], which encompasses around thirty million protein domains. It is important to note that the Pfam dataset is a subset of the UniProt Knowledge Base (UniProtKB) [32]. Specifically, Pfam utilizes sequences exclusively from the Reference Proteomes [33] within UniProtKB, rather than incorporating the entire UniProtKB collection. Consequently, Pfam contains almost half the number of protein sequences compared to other datasets that are based on the entirety of UniProtKB.

ProtBert-BFD is a member of the ProtTrans family of models, as introduced by Elnaggar et al. [28]. In their study, the authors conducted evaluations using various deep learning architectures on three distinct datasets: BFD, UniRef50, and UniRef100, which encompass 2,122 million, 45 million, and 216 million sequences, respectively. BFD stands out as the most extensive collection of protein sequences, formed by merging data from UniProt [34] and proteins obtained from various metagenomics sequencing projects. On the other hand, UniRef [35] provides a curated set of protein sequences derived from UniProtKB. It is worth noting that the larger dataset, BFD, is known to contain more noise, including sequence errors [28]. Several models were proposed in this context, including ProtBert-BFD, ProtT5-XL, and ProtT5-XXL, boasting 420 million, 3 billion, and 11 billion parameters, respectively. ProtBert-BFD was trained exclusively on the BFD dataset. In contrast, the ProtT5 models underwent training using BFD initially, followed by further training with UniRef50, resulting in performance improvements of 2.8 % for ProtT5-XL and 1.4 % for ProtT5-XXL, respectively. Interestingly, despite the larger parameter size, ProtT5-XL outperformed both ProtBert-BFD and the larger model, ProtT5-XXL. The authors noted that while an increased number of samples did contribute to improved performance, it did not exhibit a consistent similarity to the model size. They proposed that larger models require access to larger datasets, as they tend to see fewer samples when processed within the same computing capacity.

ESM-2 [30] is part of the evolutionary scale of transformer models, ranging from 8 million to a staggering 15 billion parameters. This model is rooted in BERT [24] and manages to outperform its predecessor, ESM-1b [29], by eliminating dropout in both hidden and attention layers. Notably, the authors found that conventional absolute positional encoding methods do not generalize well. Consequently, they turned to Rotary Position Embedding (RoPE) for improved results. Although the use of RoPE slightly increases the training cost, it enhances the model’s quality, particularly for smaller models [30]. Furthermore, for their experiments, the authors utilized the non-redundant UniRef50 dataset from UniProt, which contains an impressive 60 million protein sequence.

4.1 Alignment

The initial stage involves the examination of DNA and RNA sequences obtained from tumor and normal cells. To ensure data quality, standard quality assurance tools such as FastQC or Trimmomatic are typically employed for both RNA-seq and DNA-seq samples. Following quality assessment, alignment tools are applied to align the sequences accurately. In this field, samples from tumor and normal cells are mapping to a reference genome. For this task, there are well-established tools, and Transformers methods are not used according to our systematic search. The most know tools are BWA [36], Bowtie2 [37], and Samtools [38]. Additionally, STAR is one of the most used because it aligns tumor samples more effectively [17]. The output of this stage consists of BAM, SAM alignment files.

4.2 Variant calling

Variant calling is the process by which we identify variants from sequence data. This stage, take as input the alignment files of the previous stage (see Figure 3). For variant calling, MuTect [39], Strelka [40], SommaticSniper [41], FreeBayes [42], VarScan [43], and BCFtools [38] are normally employed. Additionally, the information from both methods could be combined, following some approaches [17, 44], [45], [46]. Importantly, there is GATK [47], which integrated other tools and deliver best practices [48, 49] for identifying single nucleotide polymorphism (SNP), and indels in germline DNA and RNA data. Nevertheless, these tools doesn’t used Transformer methods.

Furthermore, there are ongoing efforts to integrate deep learning and Transformer methods. One of the first neural networks is DeepVariant [50], which utilizes a pictorial representation of the local alignment between reads mapping to the site and the segment of the reference sequence surrounding the site (referred to as a pileup); then, convolutional layers are applied. Additionally, there is Clairvoyante [51], which is a convolutional neural network designed to predict variant types. In this case, the authors encode alignment data into images of dimensions 33 × 4 × 4, where 33 corresponds to the position, 4 corresponds to the count of A, C, G, or T; and the third dimension of 4 corresponds to the method of counting. Furthermore, there is Hello [52], which also develops a method leveraging alignment data as images to utilize the Inception-v3 architecture. Moreover, another model has been applied to enhance somatic variant calling through the identification of somatic variants [53]. Another approach utilizes a machine learning classifier to discern between germline and somatic mutations [54]. These initiatives, in collaboration with variant calling tools, have the potential to enhance neoantigen detection; however, further research is imperative for a comprehensive understanding of their effectiveness.

Moreover, the majority of variant calling tools are primarily focused on single nucleotide polymorphisms (SNPs). However, a smaller number of tools, such as Manta [55], MetaSV [56], and Parliament2 [57], have been specifically developed for the detection of structural variants. Nevertheless, a recent thesis was published in which the authors proposed the utilization of Vision Transformers for structural variant identification and genotyping [58]. This innovative approach draws inspiration from the use of images as proposed by DeepVariant [50], thereby opening new possibilities for the application of Transformers in variant calling methodologies.

4.3 Variant annotation

In the subsequent step, variant annotation takes place, utilizing VCF-formatted files to derive peptides resulting from these variants. Various tools, such as Isovar [59], Annovar [60], Ensembl’s Variant Effect Predictor (VEP) tool [61], or SnpEff [62], are commonly employed for this task. Typically, these tools search for variants within databases to provide comprehensive annotation of the identified variants. Moreover, a comprehensive benchmarking study [63] was conducted comparing the performance of Ensembl’s Variant Effect Predictor (VEP) tool, Annovar, and Alamut Batch. The investigation utilized a meticulously curated ground-truth set comprising 298 variants. Notably, VEP exhibited the highest precision in variant annotations, attributed to its utilization of the latest gene transcript versions within its algorithm [63].

In order to complement the variant annotation task, fusion genes are promising candidates [64]. Fusion genes, formed by the merging of two or more independent genes, have implications in various cancer types, as evidenced by studies [18, 65], [66], [67], [68], [69], [70]. To enhance neoantigen detection outcomes, integrating annotation tools with fusion gene detection tools is a promising approach. Moreover, FusionGDB [71] serves as an annotation database for human fusion genes. Several tools, such as FusionCatcher [72], Arriba [73], and FusionQ [74], are capable of detecting both novel and known fusion genes. Additionally, there are workflows which include fusion genes detection methods: Integrate-neo [75], neoFusion [76], pVACfuse [77], NeoepitoPred [78], Epidisco [17] and Antigen.garnish [79]. Notably, the use of Transformers in fusion gene detection has not been explored as of now.

4.4 HLA typing

Human leukocyte antigen (HLA) serves as the major histocompatibility complex (MHC) for humans, and HLA typing involves identifying specific HLA types such as A03:01 or B07:02, etc. OptiType [80] is a tool designed for HLA typing using RNA-seq data, providing accurate and efficient results in this context. In addition, HLA MS [81] is another tool specifically tailored for HLA typing, utilizing mass spectrometry (MS) data to determine HLA types. This method offers an alternative approach for HLA typing and contributes to the diversity of available techniques in this field. Furthermore, until now, the application of Transformers in the domain of HLA typing has not been explored.

5.1 Databases

To prioritize neoantigens, researchers often collect samples from various sources, typically drawing from previous studies and similar resources. However, there are publicly available datasets, as listed in Table 3, that specifically focus on the interaction between peptides and MHC (peptide-MHC) [82–85], as well as the interaction between pMHC and TCR [86, 87]. Notably, a recent study provides 3D structures of peptides and HLA, introducing a novel avenue of investigation from a different perspective. Finally, the Immune Epitope Database (IEDB) [88] stands out as an exemplary resource in this domain.

5.2 pMHC binding prediction

The prediction of pMHC binding represents one of the final stages in the prioritization of neoantigens. Two fundamental approaches are employed to investigate pMHC bindings: (i) pMHC binding affinity (BA), which assesses the binding preferences of peptides and MHC [92]; and (ii) MHC eluted ligands (EL), generated through Liquid Chromatography Mass Spectrometry (LC-MS), enabling the identification of a large number of eluted ligands in a single experiment [93]. However, both methodologies are characterized by time-intensive and expensive processes, prompting the emergence of computational methods for predicting pMHC bindings. The methodology used is depicted in Figure 4, where two amino acid chains p = {a, w, d, r, a, b, …} and q = {b, w, c, x, a, r, …} are taken as inputs; then the amino acid are encoded or transform into a vector of real numbers (in Table 4, we described encoding methods); then, a prediction model is applied, yielding a binary output of 0 or 1 to signify binding or affinity based on real-number predictions.

Figure 4:

Picturing pMHC binding prediction: initially, the peptide and MHC undergo transformation or encoding into a vector of real numbers. Subsequently, a prediction model is applied, yielding a binary output of 0 or 1 to signify binding or affinity based on real-number predictions.

The approaches for predicting pMHC binding can be broadly classified into two categories: allele-specific and pan-specific methods. Allele-specific methods involve training a distinct model for each specific allele, while pan-specific methods entail the training of a universal model applicable across a range of alleles. Moreover, there two types of MHC related to immunology: MHC class I and MHC class II. Both classes of proteins serve the common function of presenting peptides on the cell surface for recognition by T cells. pMHC-I complexes are exhibited on nucleated cells and are identified by cytotoxic CD8+ T cells. Conversely, the presentation of pMHC-II by antigen-presenting cells, such as dendritic cells (DCs), macrophages, or B cells, can activate CD4+ T cells [116].

Furthermore, one of the most widely utilized metrics for evaluating pMHC binding predictions is the Area Under the ROC Curve (AUC). The Receiver Operating Characteristic (ROC) curve is a graphical representation illustrating the performance of a classification model at various classification thresholds. AUC serves as a comprehensive metric, offering an aggregated measure of performance across all conceivable classification thresholds. This metric is widely employed due to the dynamic nature of threshold selection in machine learning methods for predicting binding, which varies based on distinct peptide lengths and MHC types. Additionally, the Spearman’s rank correlation coefficient (SRCC) is utilized. This metric offers a comprehensive analysis of crucial factors influencing performance comparison across various positive and negative sample ratios [117].

An innovative pan-specific approach for pMHC binding prediction is presented by NetMHCpan [118]. This method employs a traditional feed-forward network with a single hidden layer and one output neuron. The authors conducted extensive testing by varying the hidden layer size, exploring configurations ranging from 22 to 86 neurons. Additionally, the authors introduced the concept of pseudo sequences for MHC proteins, which are composed of amino acid residues in direct contact with the peptide. This novel approach enhances the predictive capabilities of the model, providing a more comprehensive understanding of pMHC-I interactions.

Since then, subsequent iterations such as NetMHCpan2.0 [119], NetMHCpan3.0 [120], and NetMHCpan4.0 [121] have been developed. The version, NetMHCpan4.0, incorporates information from binding affinity (BA) and mass spectrometry (MS) ligands. Notably, the authors introduced the NNAlign training approach, enabling combined training on BA and MS EL data. In their methodology, an ensemble of 40 feed-forward networks was employed, with each network featuring a hidden layer comprising 60 to 70 neurons. Furthermore, the encoding of each amino acid was achieved using the BLOSUM matrix. This sophisticated approach enhances the predictive accuracy of the model by integrating diverse sources of information and employing an ensemble strategy for training.

In 2020, the latest iteration, NetMHCpan4.1, was introduced [16], maintaining the same network architecture as its predecessor. However, a significant enhancement was made by expanding the dataset to include eluted ligands multiple allelic (EL-MA) data. EL-MA data are originated from mass spectrometry (MS) experiments, where EL assays exhibit polyspecificity, associating one peptide with multiple alleles. The incorporation of EL-MA data introduces complexities in analysis and interpretation [122], requiring the use of algorithms know as deconvolution to transform these EL-MA data into individual pMHC pairs. NetMHCpan4.1 implemented NNAlign_MA, a modification of NNAlign, specifically designed to accommodate EL-MA data. This adaptation reflects the methodological evolution necessary to handle the unique challenges posed by EL-MA datasets. Presently, NetMHCpan4.1 stands as the benchmark method in the field, showcasing its adaptability and effectiveness in addressing the intricacies of pMHC-I prediction [16].

To summarize, the challenge of pMHC binding prediction has prompted the application of various machine learning methods like support vector machines (SVM), shallow neural networks (SNN), and random forest (RF). Moreover, this review specifically concentrates on approaches based on Transformers. In the subsequent section, we will delve into the details of these Transformer-based methods. In Table 5 we provide a comprehensive comparison of these Transformer models and deep learning methods that incorporate attention mechanisms.

5.3 pMHC interaction with TCR

TCRs consist of chains α and β, both chains contain information about the specificity of the TCR to determine the binding prediction with pMHC. Each chain has three loops called Complementarity Determining Regions (CDRs) are responsible for binding TCR and pMHC, there are three regions; region 1 and 2 are very likely to bind to pMHC, therefore region 3 is the determining factor for the prediction.

Models, in general, have as input TCR and pMHC sequences. Representing these data and feature extraction are essential for achieving better results. Many models use the BLOSUM MATRIX, while some models employ one-hot encoding, and others combine BLOSUM and one-hot encoding. However, some opt to used Granularity vectors [132]. The dimensionality of the input may vary depending on the chain or region, we must take into account the way it is encoded to extract the characteristics. There are several studies that use multi-head self-attention, recurrent layers and convolutional layers (see Table 6).

6.1 Bioinformatic pipelines

A bioinformatic pipeline in the neoantigen context is a software construct that assembles various command line tools. In the realm of neoantigen detection and prioritization, reliance on multiple tools is essential. For instance we can use tools such as (1) FastQC ensure sequence quality, (2) BWA handles alignment, (3) Samtools manipulates BAM files, (4) BCFtools is employed for variant calling, (5) Annovar provides variant annotation, and (6) netMHCpan4.1 predicts pMHC binding and pMHC-TCR binding affinity. However, the use of these diverse tools can introduce compatibility and dependency challenges. To address this issue, developers have created pipeline tools aimed at enhancing the usability of neoantigen detection software. These pipelines effectively manage the integration of these tools, mitigating potential conflicts and dependencies, thereby streamlining the overall neoantigen analysis process.

In Table 7, we present pipelines published since 2018. These pipelines use various types of information as input. For instance, the PGV Pipeline [17] and PEPPRMINT [44] use DNA-seq, while other tools such as PGNneo [145], NAP-CNB [146], NaoANT-HILL [147], ProGeo-neo [148], ScanNeo [149], and Neopepse [14] use RNA-seq because these sequences better capture information about mutations and non-coding regions of DNA [145].

To reduce the complexity of pipelines, some proposals have opted to use Variant Calling Format (VCF) as input. These files contain mutation information and are obtained through alignment and mutation calling methods (stages 2.1 and 2.2 in Figure 1b). Tools like HLA3D [150], Neoepiscope [18], pVACtools [77], and NeoPredPipe [151] thus reduce the number of tools used in neoantigen detection. However, the results obtained may be inferior compared to tools that use DNA-seq and RNA-seq.

Additionally, for accurate neoantigen detection, it is necessary to have the sequencing of Major Histocompatibility Complex (MHC) or Human Leukocyte Antigens (HLA) proteins. These proteins are necessary because they are used to predict the binding between potential neoantigens and MHC (pMHC: stage 3.1 in Figure 1b). These proteins are encoded by highly polymorphic genes, leading to substantial variation in peptide (neoantigen) binding, thereby influencing the set of peptides presented to T-cells [152]. In this context, the NeoPredPipe [151], and Neopepsee [14] pipelines request these HLA proteins as input, while others predict this information from DNA-seq. From a usability standpoint, obtaining the HLA types entails unnecessary effort for the user.

As we mention before, fusion genes are related to several types of cancer [18, 65], [66], [67], [68], [69], [70]. Thus, there are pipelines which include fusion genes detection methods: Integrate-neo [75], neoFusion [76], pVACfuse [77], NeoepitoPred [78], Epidisco [17], TrueNeo [153] and Antigen.garnish [79]. Gene fusions typically yield a higher number of neoantigens per mutation compared to single nucleotide variants (SNVs) and insertions/deletions (Indels). Furthermore, fusion-derived neoantigens exhibit heightened immunogenicity. Notably, neoantigens arising from frameshift fusions or passenger fusions are anticipated to possess the greatest immunogenic potential [64].

Furthermore, the type of variant is closely associated with specific cancer types [64]. For instance: (1) SNV are related to Melanoma, Lung cancer and Glioblastoma. (2) Indels are related to microsatellite instability-high tumors, Clear cell, papillary, and chromophobe renal cell carcinomas. (3) Fusion genes are related to Hematologic malignancies, sarcomas, prostate cancer, head cancer and neck cancer. Hence, depending on the cancer type, it is advisable to prioritize the selection of the appropriate analytical pipeline for genetic analysis and interpretation.

Moreover, neoantigen pipelines deliver its results and demonstrated its performance in several ways. For instance: pVACtools, reported 8-fold increased in the number of strong binding compared to Integrate-Neo. TrueNeo evaluated its performance detecting 19 identified neoantigens from 1,599 non-redundant SNVs from 134 patients. Neverthleses, TrueNeo, compared its performance varying the pMHC tools used (NetMHCpan, MHCflurry, deepHLA, etc.). On the other hand ProGeo-neo, evaluated its performance by applying similarity analysis between 746 validated neoantigens from dbPepNeo2.0 and 6,400 random peptides. These methods of evaluation difficults a comparative analysis of neoantigen pipelines.

6.2 Clinical trials

Before applying any type of treatment, there are several stages that any drug, product, or even a specific technique being considered for therapy must undergo. The most basic stage involves conducting preclinical trials. Preclinical trials are studies conducted in laboratories and on animals to assess the safety and efficacy of new treatments before testing them on humans. These trials provide crucial data about potential side effects and determine the appropriate dosage [157, 158]. As for clinical trials, they are investigations carried out in humans to evaluate the safety and efficacy of new treatments or therapies. They are divided into four phases: (1) Phase I: Evaluates safety and determines the initial dosage in a small group of volunteers. (2) Phase II: Focuses on efficacy and continues to assess safety in a larger group of participants. (3) Phase III: Confirms efficacy and monitors side effects in a large population. Compares the new treatment with existing standards. (4) Phase IV: Conducted after approval, continues monitoring safety and long-term effectiveness in real-world conditions. These trials are essential to ensure that new treatments are safe and effective before being applied on a larger scale. In Table 8, a list of recent clinical trials are presented.

Regarding clinical trials of immunotherapy with vaccines based on tumor neoantigens, the subject of this work, comparing results to draw conclusions based on overall survival and disease-free time across different types of neoplasms is not possible due to the heterogeneity of their characteristics, which confer distinct evolution and more or less aggressive behavior. Even comparisons within the same types of neoplasms, referring to the macroscopic aspect (those affecting the same organ), become complicated due to histological and, more importantly, molecular characteristics defined by the types of mutations, leading to different outcomes [159]. This is precisely why current trends in treatment are more focused on the molecular aspect rather than the organ affected by cancer. It is observed today that applying the same drug for cancers affecting very different organs but sharing common mutations can be an effective strategy. This highlights the importance of targeting treatments to molecular targets present in mutations, which is the foundation of immunotherapy based on tumor neoantigens [160]. This type of intervention was applied in the clinical trials analyzed in our work, from which we have derived specific conclusions mentioned below after the analysis.

In all the reviewed clinical trials that have been concluded to date, a common point found is that adoptive cell therapy is safe, resulting in manageable side effects, and generates an immune response effective enough to assist in combating different types of cancer. It also positively contributes to other types of treatments, especially the use of checkpoint inhibitors [161–166]. This is reflected in the increase in disease-free time or the overall survival rate of the patients who participated in these studies compared to conventional treatments. This, of course, depends on the type of neoplasm, the stage it is in, and the more or less aggressive nature of each neoplasm. Particularly relevant is the result of a randomized clinical trial showing that personalized vaccines based on dendritic cells loaded in vivo with tumor neoantigens demonstrated generating stronger immune responses with fewer side effects than other types of adoptive cell therapy, specifically tumor cell vaccines exposed to antigens [167].

Furthermore, in all clinical trials, personalized vaccines based on neoantigens were administered to patients with solid tumors, and in most studies, to patients in advanced stages of the disease [100, 161], [162], [163, 165, 167], [168], [169], [170], [171], [172], [173], [174] considering an advanced stage as one in which there is metastasis or extension of the disease. This does not rule out the possibility that this type of treatment could also be applied to hematological cancers.

Lamentably only two of the reviewed studies were randomized [164, 167], all the studies are interventional, demonstrating the researchers’ intent to establish the safety, efficacy, or both parameters for personalized vaccines based on neoantigens, either as individual therapy or in combination with other types of treatments, as previously discussed.

Moreover, close to half of the studies are phase II trials [100, 163, 167, 168, 170], [171], [172, 174], [175], [176], [177], [178] documenting the immunogenic efficacy of personalized vaccines based on tumor neoantigens in the treatment of various types of cancer. It is interesting to note that neoantigens could have other applications not only therapeutically against cancer but also as predictors of the response to immunotherapy treatments such as checkpoint inhibitors, which could be defined by analyzing their interaction with CD8 T lymphocytes [163].

A clinical peculiarity of pancreatic cancer is that it is generally diagnosed in advanced stages, leading to very limited survival post-diagnosis. This is due to its histological properties, allowing it to form a fibrotic barrier that prevents the entry of drugs or their active components into the tumor. Hence, clinical trials for this type of cancer seemingly involved patients who could be diagnosed at earlier stages [166, 179, 180]. A common exclusion criterion in many trials with this type of immunotherapy is the overall poor health of patients, a constant in advanced pancreatic cancer. These pancreatic cancer characteristics limit the progression of clinical studies for various therapeutic approaches, particularly neoantigen-based vaccines. While establishing the safety of the treatment, further studies are needed to determine efficacy or strategies to achieve it. Similar challenges are observed in cancers affecting the central nervous system, where the complexity of the affected organ likely determines a sluggish clinical behavior and evolution in most cases. This impedes conducting clinical trials in advanced stages, and the inherent limitations result in scarce trials, making it challenging to conclusively establish efficacy [181, 182].

In gastrointestinal and hepatic cancers, neoantigen vaccines have been tested in both advanced disease stages and earlier stages [168], [169], [170, 175]. This variation may be attributed to the disease progression depending on the affected organ, as well as the histological and molecular characteristics of each tumor. A preventive vaccine trial was conducted in patients with Lynch syndrome, a hereditary syndrome increasing the likelihood of stomach or colorectal cancer. The trials not only established safety but also demonstrated the efficacy of the treatment [176].

For lung cancers, clinical trials were conducted in advanced disease stages. Safety and efficacy were established, likely linked to the availability of new targeted therapies, contributing to increased patient survival [100, 161, 162, 169].

Concerning cancers affecting the male and female genitourinary system, the clinical behavior, evolution, and the types of trials conducted are likely determined by variables similar to those observed in previously mentioned cancers. Additionally, anatomical disposition and hormonal influences on gender-specific organs could impact disease progression and evolution [162], [163], [164, 169, 178].

On the other hand, Melanoma, a type of skin cancer, has been a focus of numerous trials, providing more information on the safety and efficacy of neoantigen-based immunotherapy compared to other cancers described in our study [162, 165, 167, 172, 177, 183].

7.1 Neoantigen candidates detection

Neoantigen candidates detection delivers neoantigens taking inputs like RNA-seq, and DNA-seq (see Figure 3). Despite, this process is complex, actually they don’t commonly relies on Transformers or deep learning methods.

The alignment process involves utilizing DNA-seq or RNA-seq data, wherein these samples are mapped to a reference genome to know the specific location of reads. This step stands as a cornerstone in contemporary genomic data analysis. Additionally, in this alignment phase, machine learning methods are not commonly applied due to the inherent nature of the problem, which revolves around identifying similarity regions among sequences of bases. There are proposals which focus on DNA-seq clustering or classification. Moreover, a RNA-seq fasta file typically encompasses approximately 1.7 gigabytes and 5.5 G bases, and it is advisable to include a minimum of 12 samples for a robust RNA experiment [1]. Thus, it is impractical to have this amount of information like input for a deep learning technique.

In variant calling methodologies, several tools described in Section 4.2 and associated algorithms are available. Many of these methods focus on identifying regions of the genome where variants have been called [185]. The most widely used tool does not rely on machine learning methods; however, some proposals involve transforming alignment data into images to facilitate the use of convolutional neural networks. Moreover, the concept of representing alignments as images presents new avenues for utilizing Vision Transformers. Currently, there is a limited number of published works proposing the use of Vision Transformers for variant calling.

Furthermore, the majority of neoantigen detection methods have primarily been employed for the identification of single nucleotide variants (SNV) and small insertions or deletions (indels) [6]. However, it is noteworthy that neoantigens derived from SNVs often exhibit substantial similarity to their normal counterparts. Consequently, only a limited proportion of these putative neoantigens are deemed immunogenic [6]. Moreover, several cancer types are related to alternative splicing events, structural variants and fusion genes [18]. Thus, there is a need for further research that integrates SNPs and structural variants, including fusion genes and alternative splicing events, into variant calling methodologies.

The challenges in neoantigen candidate detection can be summarized as follows: (1) Complex Data: Both RNA-seq and DNA-seq data entail vast amounts of bases, posing difficulties for the application of deep learning techniques. Furthermore, inherent error rates in Next-generation sequencing technologies can adversely affect neoantigen detection. (2) Dependency on Variant Caller Tools: Neoantigen detection heavily relies on variant caller tools. However, the variants identified can vary significantly depending on the alignment tools employed, as well as the sequence accessions and versions [63, 186, 187]. (3) Focus on SNPs: While most variant callers are geared towards single nucleotide polymorphisms (SNPs), numerous cancer diseases are associated with structural variants and fusion genes. Despite this, there remains a scarcity of tools that encompass these events, necessitating further research and methodologies for integrating variant call format (VCF) files from multiple tools. On a positive note, efforts to integrate and establish best practices for variant calling tools in cancer research have been advanced by tools such as the Genome Analysis Toolkit (GATK).

7.2 Neoantigen prioritization

Neoantigen prioritization is intricately linked to the task of predicting peptide-MHC binding and pMHC-TCR binding affinity, with the peptide serving as the candidate neoantigen. While the formulation of this problem may seem straightforward, its complexity is undeniable. Consequently, extensive research has been conducted in this area, spanning from classical machine learning approaches to cutting-edge deep learning techniques such as Transformers.

However, these methods have limitations, including their dependence on training datasets that overlook posttranslational modifications (PTMs) like phosphorylation, glycosylation, and deamidation, which influence MHC binding specificity. Additionally, several aspects of pMHC biology are still poorly understood. To improve neoantigen detection accuracy, integration with pMHC-TCR studies is essential.

The baseline methods for this task include netMHCpan4.1 and MHCflurry2.0. A recent comparative study of these tools [117] highlighted the superior performance of MHCflurry. However, it’s worth noting that this comparison did not take into account proposals involving Transformer models. Moreover, recent publications have demonstrated that Transformer-based models, such as ESM-GAT [123], CapsNet-MHC [124], STMHCpan [125], and HLAB [127], exhibit superior performance compared to netMHCpan4.1 and MHCflurry. Nevertheless, there has been no benchmarking comparison conducted on these Transformer models, and as of now, there is no universally acknowledged best tool for pMHC binding prediction.

Another significant challenge in pMHC binding prediction is associated with the MHC class. While a considerable portion of research publications has focused on MHC class I, addressing MHC class II presents a greater challenge. MHC class II peptides tend to be longer and exhibit more variability in length compared to MHC class I peptides. Furthermore, datasets for MHC class II are typically smaller in scale than those available for MHC class I, further complicating the development and validation of predictive models for this class. As a result, overcoming these obstacles and improving prediction accuracy for MHC class II binding remains a pressing area of research in immunoinformatics.

Furthermore, the data within the EIDB continues to expand, furnishing us with an ever-expanding wealth of information crucial for training extensive transformer models. Within this domain, notable examples include ProtTrans [28] and ESM-2 [30], advanced protein language models trained on extensive protein datasets. These models hold promise for transfer learning applications aimed at resolving the challenge of pMHC binding prediction. However, Transformers suffer from instability, where variations in the random seed can lead to significant variance in task performance. Moreover, this instability is exacerbated by optimization difficulties that result in vanishing gradients [188]. Another challenge related to transformers, is the high cost of training huge models, it is less accessible to smaller institution or students with limited resources.

Future directions in pMHC binding prediction may involve leveraging the pHLA3D dataset, which provides 3D structures of alpha/beta chains and peptides of MHC-I proteins. This invaluable resource opens avenues for research in pMHC prediction by incorporating insights from 3D protein structure. Additionally, the emergence of 3D protein structure prediction methods like AlphaFold introduces a new perspective for studying pMHC binding prediction. By integrating information from 3D protein structures, researchers can gain deeper insights into the molecular interactions underlying pMHC binding.

In the case of pMHC-TCR binding affinity prediction, a lot of studies focused only on TCR binding prediction to class I pMHC [133, 135], [136], [137], [138, 140], [141], [142], [143], [144, 189], others studies to class II pMHC [134], and some in both classes [132, 139]. TCR-CD4 binds to pMHC class II, while TCR-CD8 binds to pMHC class I. TCR-CD8 contains information to destroy infected cells along with its potential for recognition and direct elimination, causing many to focus on pMHC class I. However, TCR-CD4 plays a critical role in the initiation and maintenance of an immune response, pMHC class II can incorporate many of the same features as pMHC class I, such as TCR expression, processing, binding and recognition [4]. MHC class II has been shown to play a key role in binding, therefore it should be taken into account [190]. Nevertheless, there are some limitations such as the variation of the input size of the sequences, and the limited training samples. This latter influenced the model, AttnTAP [132] had to reduce the complexity of the model to avoid overfitting. Although there are studies that focus on pMHC class II, which improves binding prediction, it is important to conduct more research.

The database is important to train the models, there are public databases, both for pMHC class I and II, the best known are VDJdb [86], Immune Epitope Database (IEDB) [88] and McPAS-TCR [191]. The recent models are very large and need large training databases, some achieve this by creating their database based on samples from other studies, in some cases they make it public and others do not, making it public facilitates the development of new models that improve the prediction. Furthermore, each author obtains its results based on comparisons with other tools and testing its model with different databases, this makes it difficult to know which proposed model is better. In addition, the databases are constantly being updated, benchmarking is necessary.

In the same way as in pMHC binding prediction, the pHLA3D data set will be used to have new research focuses. The extraction of TCR and pMHC specificity is important. It has been shown, based on experiments, that multi-head self-care extract has better characteristics [136, 137]. Transformers have made many advances in natural language processing and have been shown to perform well in predicting pMHC-TCR binding, however they require a large training database, but still achieve good results. Some choose to use pre-trained, such as transfer learning to improve their results, especially when they have access to limited data or to reduce the computational cost of training.

7.3 Pipelines and clinical trials

Despite extensive efforts to develop pipelines and algorithms, less than 5 % of the identified neoantigens have been found to activate the immune system [7, 10], [11], [12], [13]. According to the authors of these pipelines, this limitation may be attributed to the omission of critical data integration, such as DNA-seq, RNA-seq, and Mass Spectrometry (MS) data [14]. Notably, many proposed solutions do not incorporate MS data, despite the increasing availability and its application across various facets of bioinformatics.

Furthermore, outdated tools for predicting peptide-MHC (pMHC) binding (at stage 3.1 in Figure 1b) are commonly employed. Most applications still rely on MHCFlurry [15] and NetMHCpan4.1 [16], although newer, high-performance tools based on transformers are now accessible. Moreover, in the context of stage 3.2 (Figure 1b), authors frequently neglect the prediction of pMHC-TCR binding, though many researchers intend to incorporate this aspect in their future work [17].

Perhaps most crucially, the absence of information related to alternative splicing events, structural DNA variations, and gene fusion mutations represents a significant oversight, despite their strong association with various cancer types [18].

Moreover, we included revision of clinical trial studies in this review. In concluded clinical trials, adoptive cell therapy has demonstrated safety, the ability to elicit an effective immune response against cancer, and an improvement in patient outcomes, including extended disease-free intervals and overall survival, especially when used in combination with checkpoint inhibitors. However, it is worth noting that certain limitations and challenges may have affected the trial outcomes. These include conflicts of interest in the majority of the studies [162, 163, 165, 167, 176], [177], [178, 180, 183], a small sample size in some trials [165, 169, 170, 175, 177, 179], and technical difficulties encountered in a subset of the research. Furthermore, it is essential to note that two trials are still ongoing, and their results are not yet available [166, 184].

Furthermore, it is evident that cancer vaccines utilizing tumor neoantigens offer clinical advantages to cancer patients and hold promise as potential therapies. Hence, dedicating time and resources to their research and development is of utmost importance, with the goal of enhancing current techniques or discovering novel approaches.

Although the focus of immunotherapy is based on the molecular characteristics of tumors, which is the most novel and promising trend for cancer treatment, it is precisely these characteristics that give certain types of cancer peculiarities, making them more or less resistant to different types of treatment, including vaccines based on neoantigens. However, therapeutic strategies that consider both aspects – molecular, based on technological advancements, and clinical, through increasingly extensive, frequent, well-designed therapeutic trials with a larger number of participants – would allow for a comprehensive patient treatment with the aim of achieving better results.

The continuous evolution of vaccine development methods, application techniques, sample acquisition, and the analysis of the molecular profile of tumors, accompanied by timely diagnosis and close monitoring, always prioritizing the human aspect alongside technology, will enable the refinement of these treatments and pave the way for the future in the pursuit of increasingly effective therapies.

7.4 Final remarks

In general, the utilization of Transformers in neoantigen detection is exceptionally well-suited for variant calling, pMHC binding prediction, and pMHC-TCR interaction. New proposals continue to evolve, consistently demonstrating strong performance; however, the efficacy of these tools can be compromised due to the insufficient volume of data available in databases. Fortunately, the ongoing expansion of data resources, coupled with advancements in Transformers, paves the way for innovative research in these domains. Furthermore, there are numerous pipelines designed for neoantigen detection. Ensuring the updating of these pipelines to incorporate the benefits of new transformer-based methodologies is of paramount importance. This initiative serves as a driving force behind the development of more robust and current software pipelines. Remarkably, despite the relative immaturity of these technologies, clinical trials have already been conducted, yielding generally positive results and offering promise for potential therapeutic interventions.