Air fare sentiment via Backtranslation-CNN-BiLSTM and BERTopic

2.1 Research framework

According to the characteristics of the problem, the research idea is shown in Figure 1, and the specific steps are as follows.

1. Data fetching. First, civil aviation passenger comment data are collected from the platform as sentiment analysis text. Then, a large number of Chinese-dominated civil aviation-related texts are obtained from Wikipedia, Baidu, and Zhihu as word vector training data.

2. Data processing. The two text data are processed by unifying the data format, removing the duplication of the text, specifying the characters, and unifying the language into simplified Chinese. Then, based on the basic corpus of stuttering word segmentation, the civil aviation terms are expanded, and the text segmentation is carried out, and then, the stop words are removed.

3. Annotate the data. The data were unlabeled, and the samples were selected for transfer learning, unsupervised training, and manual annotation to label the data. Therefore, a good representation is learned from the unlabeled data, and the sentiment of civil aviation passenger comment data is divided into negative, positive, and neutral and labeled to form a text database for civil aviation passenger comment sentiment analysis. Then, TF-IDF analysis was performed on different sentiment polarity datasets of the data.

4. Word vectorization. On the basis of text preprocessing and word segmentation, Word2Vec technology is used to train word vectors based on civil aviation-related texts; to train custom word embeddings, we use the aerial dataset to train the word embedding model. During training, we tune the parameters of the model to optimize domain-specific performance, such as setting a smaller window size to capture tighter context, or increasing the number of iterations to learn more in-depth domain-specific features. Then, the trained word vectors are used to vectorize the sentiment analysis text to form vectorized data.

5. Model training. According to the characteristics of the data, in order to reduce the generalization error as the goal, the Backtranslation, CNN, dynamic learning rate correction, and callback function are combined to establish the Backtranslation-CNN-BiLSTM model, and the corresponding model parameters are set to train the vectorized data.

6. Thematic analysis. On the basis of the aforementioned research, the BERTopic topic technology, which is the optimal number of topics determined based on perplexity, is used for topic analysis of sentiment research data.

Figure 1

Research idea chart.

3.1 Dataset processing

The data come from Weibo, TripAdvisor, Yelp, travel blog, travel aggregator platform, and flight review website. Through integrating a wider range of data sources, passengers’ emotions can be understood more comprehensively and diversified. Data processing includes data preprocessing and transfer learning techniques combined with manual labeling.

The main steps of data preprocessing are as follows: first, data set sorting, data format specification and deletion of blank lines, confusion lines, repeated lines, and meaningless special characters, resulting in 12,352 pieces of data. The second is data preprocessing, using regular expression matching to further remove invalid data such as characters, and then stuttering word segmentation, and then according to the stop word list based on the characteristics of the data to remove stop words.

3.2 Analysis of the TF-IDF dataset

Through feature extraction, several features of sentiment polarity can be obtained, which is of great significance for subsequent sentiment analysis. TF-IDF is a method of calculating the importance of a term in a document set. The importance of a word is directly proportional to its frequency in a particular document, but inversely proportional to its frequency in the entire document set. The more frequently a term appears in a given document, the higher the TF-IDF value will be, indicating that the term is more important in that document. Conversely, if a term appears in many documents, its TF-IDF value will decrease, indicating that it is not very important in the entire document set. Based on the constructed dataset of airline passenger reviews, the TF-IDF method [27] is used to extract the features of sentiment analysis text after preprocessing.

The TF-IDF method was used to extract data features, and the data were divided into positive, neutral and negative data sets according to the sentiment polarity. Extracting the positive dataset, we can see that the positive words are thank you, nice, and like for the plane, the airport, the stewardess, and the meal. In the neutral dataset, it can be seen that the neutral feature words are mostly neutral words, and there are also a small number of positive words and negative words. The overall performance of the neutral feature words is related to flights, airports, and airlines. The negative feature words in the negative data set include cancellation, delay, and refund, involving baggage, tickets, and passengers. In order to intuitively show the theme of the data, the top eight meaningful words in the proportion of positive dataset, neutral dataset and negative dataset are shown in Table 1, considering the rule of word frequency and adopting the descending order of TF-IDF value. We can know that the categories corresponding to these keywords have conformity. The larger the TF-IDF value, the more frequently a word appears in the document. If the word is a sentiment word, it also means that the word can better represent the topic and content of the sentiment in the document. The results of TF-IDF analysis will be used as a general understanding of the dataset, a reference to build or extend the stop words list of the sentiment dictionary, and decide whether to add it to the sentiment dictionary based on its TF-IDF value and sentiment tag.

3.3 Backtranslation-CNN-BiLSTM sentiment analysis model

CNN has the characteristics of focusing on the most obvious features in the line of sight, and BiLSTM uses a combination of forward and backward hidden layers, which has the rich ability to access history and future context and is proven to be better for task order modeling. Adding the past feature states of time series data to the output results has important neural computing and application value. After BiLSTM, it is more widely used to tune the prediction results. According to the characteristics of Backtranslation, CNN, and BiLSTM, a sentiment analysis model based on Backtranslation-CNN-BiLSTM is established.

The Backtranslation-CNN-BiLSTM model is established, which consists of BackTranslation layer, input layer, Word2Vec word embedding layer, CNN layer, BiLSTM layer, and output layer. Sentiment analysis architecture is shown in Figure 2.

Figure 2

Diagram of the Backtranslation-CNN-BiLSTM architecture.

Bcktranslation layer, the machine translation model used, namely, youdao commercial API translator, is a system of neural networks, especially seq2seq model and attention mechanism. These models are able to capture the semantic information of the input text and generate natural and fluent translations at the time of output. In addition, API also employs terminological base and linguistic knowledge, combining human-produced computational grammar and monolingual and bilingual vocabulary to ensure the accuracy and professionalism of translation. When the same translation model is used to train translations in different directions between two languages, the same training corpus dataset is used, which may result in the backtranslated sentences that are similar to the original ones. To solve this problem, a variety of intermediate languages were also selected.

3.3.3 BiLSTM

BiLSTM is a bidirectional LSTM network, which is composed of forward LSTM and backward LSTM. The LSTM recurrent network is a more commonly used gated RNN, which is a creative improvement of the RNN and uses the “three-threshold design.” In addition to the external RNN cycle, the LSTM recurrent network also has an internal LSTM cell cycle, in which the gate includes input gate, forget gate, and output gate, which can effectively retain long sequence information [30]. The state of each LSTM cell is calculated as shown in equations (1)–(4).

(1) f t = σ ( W i ⋅ [ H t − 1 , X t ] + b i ) ,

(2) i t = σ ( W i ⋅ [ H t − 1 , X t ] + b i ) ,

(3) C t = f t * C t − 1 + i t * C ˜ t ,

(4) o t = σ ( W 0 ⋅ [ H t − 1 , X t ] + b 0 ) ,

where σ is the sigmoid function, and f t , i t , C t , and o t are the forget gate, input gate, cell state, and output gate, respectively. W is the weight matrix; H t represents the output at time t . b is the bias term. The LSTM structure [31] is shown in Figure 3.

Figure 3

LSTM structure diagram.

In the BiLSTM network, forward LSTM and backward LSTM are independent of each other, and BiLSTM can be regarded as bidirectional LSTM [32]. It is very suitable for modeling time series data, which can better capture long-distance dependencies, and is good at modeling context information, encoding information from back to front, and can better capture bidirectional semantic dependencies through bidirectional LSTM network.

3.4 Performance evaluation

3.5 Backtranslation-CNN-BiLSTM model verification

According to the dataset of, the Backtranslation-CNN-BiLSTM model and evaluation indicators are combined, and Softmax is used as the activation function. The results of the analysis are shown in Table 3.

It can be seen that the test set accuracy of the Backtranslation-CNN-BiLSTM model is high, which can better capture the bidirectional semantic dependence and local features. The large AUC value indicates the high stability and robustness of the classifier, and the high F value indicates that the results used to test the sample can represent the overall truth [37], and the effect is good.

From Table 4, it can be seen that the MacroAvg of Backtranslation-CNN-BiLSTM is the highest, indicating that its performance on various categories is relatively balanced, and there is no excessive preference or neglect of a certain category. It has good generalization ability and robustness when dealing with this multi-category problem. The WeightAvg weighted average of the model is also the highest, indicating that the model performs well on the overall dataset and can comprehensively consider the contribution of each category, which further confirms the excellent performance of the model in the global scope. At the same time, the Kappa value of the model is also high, which indicates that the predicted results of the model are highly consistent with the real situation. The classification performance is very stable and reliable.

In order to show the superiority of the model more intuitively, the accuracy and running speed of the model in the training process are compared in Figures 4 and 5.

Figure 4

Plot of model training accuracy change.

Figure 5

Comparison chart of model running speed.

Among them, the number of positive samples before the current value of S Pos , S Neg refers to the current number of negative samples before, The sum of TP and FN is the total number of positive samples, while the sum of FP and FN is the total number of negative samples. Obtain a series of (FPR, TPR) coordinate points, connect these coordinate points to form an ROC curve, and AUC is the area under this curve. The larger the AUC, the better the classification performance of the classifier.

It can be seen that, first, the accuracy of each model shows an increasing trend as a whole, and the TextCNN model shows a slight decrease in the fifth epoch, which may be because it fails to effectively learn the before and after features of the text. The accuracy of the proposed Backtranslation-CNN-BiLSTM model is significantly improved compared with other models as a whole. Second, the TextCNN model runs faster than other models, which may be due to its small size, and the CNN-BiLSTM model runs at a reasonable speed. In summary, the proposed Backtranslation-CNN-BiLSTM model has higher accuracy than other models, but the running speed needs to be improved. The structure and parameters of the BackTranslmCNNbilSTM model are maintained. The model is based on zhihu text word vector sgns.zhihu, Baidu text word vector sgns.baidubaike, public word vector sgns. The specific results are shown in Table 5.

It can be seen that the accuracy based on CAReview is higher. Among them, the dynamic learning rate correction started with l r of 1 e − 3 and m i n l r of 1 e − 8 , and the callback function was written. At the same time, the confusion degree is calculated, and Cohen’s Kappa statistical coefficient [38] for positive, neutral, and negative reaches 0.8824, and the result of sentiment analysis is better.

3.6 BERTopic topic analysis

In order to better analyze and mine passenger review data, this study conducted a theme analysis on the basis of sentiment analysis.

BERTopic is a powerful tool that combines the pre-trained model BERT with topic modeling [39]. The BERTopic model is used to analyze the topic of the dataset, and the topic distribution of the dataset is obtained. The method of calculating perplexity first to measure the quality of the number of selected topics concluded that the optimal number of topics was 5. The number of topics identified here is five. Topic analysis was then performed to obtain the vocabulary distribution of each topic, and the top ten words of each topic were sorted by their posterior probability of occurrence in descending order, as shown in Table 6.

BERTopic topic model [40] is the extraction of topic words from documents. The five themes focus on different aspects. The subject words “flight” and “route” in topic 1 have a large weight, which is the content highlighting airline operations and international flights. Topic 2, which contains words such as “apologies,” “apologies,” and “aircraft,” is the content that highlights airplane-related incidents. Topic 3 contains words such as “aircraft,” “student,” and “baggage” and is content that highlights the travel experience or the relevant services of the airline. Theme 4 contains words such as “flight,” “company,” and “stewardess,” which highlights the overall operation and service content of the airline. Theme 5, which contains words such as “company,” “aircraft,” “flight,” and “enthusiasts,” is the content that highlights the service of airline activities. The analysis shows that the data mainly focus on these five themes.