A Bi-GRU-DSA-based social network rumor detection approach

3.1 Data source of Weibo

Weibo provides extensive global connectivity, and the existence of unprecedented massive data is like a double-edged sword. It is easy for people to obtain unreliable information from these sources, and the speed of error information dissemination is greatly improved. How to control malicious or unintentional rumor information is a challenge. In view of the large amount of rumor information spread on Weibo, Weibo has set up rumor refutation accounts such as “catch rumor” and the official platform “Weibo community management center.” The reporting hall of the Weibo community management center contains various types of reporting information, such as reporting of false information and impersonating others. For each type of report, there are four states: default, proof stage, judgment stage, and result publicity. Among them, the result publicity is the confirmed rumor information. Therefore, a large number of links to rumor information can be obtained from the result publicity of untrue information.

In addition, the data of Weibo headlines are released by some relatively reliable official media, which are real and credible non-rumor data. Therefore, the rumor data in this article mainly use the existing large-scale official rumor data source of Weibo, and non-rumor data mainly use the data source of the headline section of the Weibo home page.

3.2 Weibo data collection

There are now two ways to gather Weibo data: one relies on a web crawler, and the other is the API made available through the Weibo platform. A web crawler is a program or script. It refers to the collection of web pages through hyperlinks starting from a small number of seed pages. The web crawler is used to download the web pages connected with one or more given seed Uniform Resource Locators (URLs). The web crawler extracts the hyperlinks existing therein and constantly downloads the web pages recursively identified by the extracted links for further processing until a sufficient number of pages are found. The web crawler’s flowchart is displayed in Figure 1.

Figure 1

Flow chart of web crawler.

The work of a web crawler starts with a list of starting URLs provided by the user, and the crawler accesses the web page through an HTTP request and then passes the content of the page to a parser, which extracts the required information and also discovers new links. The new links pass through a page filter, uninteresting links are excluded, and then the storage processor interacts with the database and stores the new links. This cycle continues until all accessible links have been processed. Web crawlers are tools used for information gathering, but they need to be used with caution to comply with legal and ethical codes.

Ma et al. [27] released a dataset on rumors and non-rumors on Sina Weibo in 2016. Although the publicly available dataset by Ma et al. is very large and widely used, their dataset is too outdated for now. Therefore, in addition to using the microblog rumor dataset published by Ma et al. in 2016, this article also uses web crawler technology to crawl rumor and non-rumor event data from Sina Weibo for nearly 2 years from 2019 to the end of February 2021. The link information of the rumor data that needs to be crawled comes from the result publicity of false information in the Weibo community management center. For non-rumor data, this article crawls from the headline section of the Weibo homepage. This section covers a wide range of topics, including various information in people’s lives. Similarly, after crawling the data, delete text files with too few comments. Finally, 1,000 rumor events and 1,000 non-rumor events were obtained. And the dataset of Ma et al. was processed into a file with the same format as the dataset crawled in this article and then merged with the dataset in this article to form a new dataset for experimentation. Table 1 provides a detailed information on the final dataset.

3.3 Dataset preprocessing

Due to the openness of the Weibo platform, users generally release information freely and informally, and the credibility of the information cannot be guaranteed. At the same time, in order to increase the interaction of the Weibo platform, users can freely publish their views and comments on the information without strict review. Certain users will purchase a substantial quantity of followers with the intention of bolstering their influence and notoriety. An extensive quantity of spam comments can be found within the Weibo comment corpus as a consequence. The presence of such “noise” data frequently significantly affects the accuracy of the experimental findings.

Whether a comment is a spam comment is related to the similarity between the comment text and the original Weibo text, the number of URLs contained in the comment, the repeated comments’ number, the replies’ number, the comments’ length, and the nouns’ proportion. Therefore, these are studied as the features of judging whether a comment is a spam comment.

3.4 Similarity with the original Weibo

Similarity with the original text of Weibo is an important standard to judge whether the comment is a spam comment. Map the original Weibo and comment to two vector spaces, and calculate their vector cosine similarity. The lower the similarity, the greater the irrelevance between the comment and the original text, and the greater the possibility of being a spam comment. Supposing that x n and y n are the vector spaces of Weibo original text and comments, respectively, the similarity calculation is shown in the following formula:

3.5 Noun proportion

By studying the text characteristics of normal comments and spam comments, it can be found that normal comments are mostly the expression of users’ happy and angry emotions or likes and dislikes toward Weibo information [28]. However, spam comments are mostly declarative expressions, and the proportion of nouns is higher than that of normal comments. Therefore, calculating the proportion of nouns in the comment information can effectively judge whether the comment is spam. Set N t as the total words’ number in the comment information, N n as the number of nouns contained in the comment information, and the calculation of noun proportion is as follows:

3.6 Other features

The number of URLs included in comments, the repeated comments’ number, the replies’ number to comments, and the length of comment text can be calculated statistically. If the number of URLs included in the comment is more, the number of repeated comments is more, and the comment text is longer, the more likely it is to prove that the comment contains other information such as marketing and “flooded comment,” and the more likely it is that the comment is a spam comment [29]. On the contrary, the higher the number of comment responses, it proves that the comments have aroused extensive discussion and communication, and the probability of such comments being normal comments is greater.

The logistic regression approach is used in this study to forecast the likelihood that a comment will be spam and the labeled data are employed to train the model, so as to build a classifier to classify and filter spam comments.

Set X = { x 1 , x 2 , … , x n } as the vector space of comments, y i ∈ Y , Y = { 0 , 1 } , Y as the classification label, 0 means useful comments, 1 means spam comments, and e ( w , b ) as the loss function. The closer the computed value of the predicted sample is to its true labeled value (0 or 1), the smaller the value of the loss function and the better the match between the model and the data. Therefore, the goal of the problem is shown in the following formula:

where w represents the weight, b denotes the bias, and y i denotes the predicted sample. The probability that y i is predicted as a spam comment among all comment texts is denoted as:

The probability that y i is predicted to be a useful comment is denoted as:

Without loss of generality, the threshold of probability is set to 0.5. The predicted probability P ( Y = 1 | X ) > 0.5 indicates that the user’s comments are spam comments. All comments with probability P ( Y = 1 | X ) > 0.5 should be filtered out and leave useful comments.

4.1 Proposed model

The proposed model architecture comprises four network layers in its primary form, as illustrated in Figure 2. The input of the embedded fusion layer consists of heterogeneous temporal feature data, which is utilized to assemble these features through structured splicing. After extracting and obtaining the implicit data of the ternary features: text content of the information, the propagation track of the event, and the user’s feedback signal. These three types of feature aggregation data are used to capture the timing information of the event through the encoder layer. Then, the attention layer is used to learn the hidden-layer expression of the event period sequence from the time level and obtain its weight in different periods of the event propagation cycle. In this way, when learning the representation of the event information sequence, the model can focus on using all kinds of potential propagation mode information in the event period. Finally, the input of the fully connected layer is the implicit expression of the event output from the attention layer, which is utilized to predict and evaluate the event’s category.

Figure 2

Framework of depth attention model based on ternary elements.

The feature aggregation model of hierarchical attention not only captures the change trend of the ternary features of information text content, event propagation trace, and user feedback signal in the whole propagation cycle through the attention mechanism, but also learns the three heterogeneous features, respectively. This aggregation model makes full use of the feature perturbation, so that the ternary heterogeneous features can fully integrate the knowledge of learning on the basis of their original time series data. Moreover, the feature aggregation model combines unsupervised learning and external knowledge acquisition of user opinion signals to fully obtain the surface information of event propagation mode. Its generated implicit state has strong directivity and expression ability.

In essence, the objective of the rumor identification task on social networks is to classify event information that is disseminated on these platforms as rumor information or not. The provided mission is defined formally as follows:

Given an event set E = { e 1 , e 2 , … , e n } and a label set L = { l 1 , l 2 } , e i represents a subject event composed of multiple pieces of information on social media, which contains m i j pieces of information. Given timestamp t i j and other related attributes f i j , i.e., the event is composed of the temporal sequence of information e i = { ( m i j , t i j , f i j ) } ( j = 1 , 2 , … ) . l 1 and l 2 represent the two category labels of rumor and non-rumor, respectively. The task of rumor identification is to map each event e i into the corresponding category label by learning and training the classification model f , i.e., f : e i → l . The input of the model is an event composed of several pieces of information, and the output is the rumor identification label for judging the event.

4.2 Model method

5.1 Evaluation index

Confusion matrix type of evaluation metric used to measure the accuracy of the classifier is a matrix of NXN, where N represents the number of classes in the data. The confusion matrix provides a comparison of the number of correctly categorized samples with the number of incorrectly categorized samples.

Each element in the confusion matrix is defined as:

True positive (TP): samples whose data are predicted to be “true” by the model and whose deeds are labeled as “true.”

False positive (FP): samples whose data are predicted to be true by the model and whose deeds are labeled as “false.”

False negatives (FN): samples whose data are predicted to be false by the model and whose events are labeled as “true.”

True negatives (TN): samples whose data are predicted to be false by the model and whose events are labeled as “false.”

Tables 3 and 4 show the description and data for the confusion matrix.

A number of assessment metrics can be derived from the confusion matrix, for instance, accuracy, precision, recall, and F1 value. The performance indexes of the method are evaluated by accuracy, precision, recall, and F1 value. The calculation methods of these four indexes are depicted in the following formulas:

5.2 Adjustment of experimental parameters

Model training is a critical component of deep learning in particular. Numerous parameters must be specified for a neural network. Critical to the process of model training is the modification of the super parameters. The grade of the parameter adjustment will have a direct impact on the model’s classification performance. Several parameters are briefly introduced below. (1) Sentence vector dimension: the dimension of sentence vector is obtained by doc2vec. (2) The learning rate is a determining factor in the pace at which the model converges. A significant increase in the learning rate will result in a highly unstable learning curve. The training process will be significantly slowed down if the learning rate is inadequate. (3) Batch size: the data in a batch jointly determine the direction of gradient update. When the batch size is too large, the calculation cost is high. When the batch size is too small, it is difficult to converge or obtain the local optimal solution. (4) Number of iterations: the more iterations, the more stable the model is, but it is easy to lead to over-fitting. (5) Optimizer: the optimizer significantly impacts the efficacy of the model. The settings information for experimental parameters is presented in Table 5.

Figure 3 shows the different sentence vector dimensions’ effect on the accuracy and F1. It can be seen that when the dimension of sentence vector is 200, the accuracy and F1 are the highest. Figure 4 shows the different learning rates’ effect on accuracy and F1. Figure 5 depicts the effect of different number of epochs on model’s loss function. It can be seen that when the initial learning rate is about 0.001, the accuracy is the highest. In the model training’s process, Adam optimization algorithm can automatically adjust the learning rate to make the model better. In addition, the number of epochs is set to 30 to lead model’s loss function become minimum.

Figure 3

Effect of sentence vector dimensions on accuracy.

Figure 4

Effect of learning rate on accuracy.

Figure 5

Effect of different number of epochs on model’s loss function.

5.3 Comparison with other methods

Patel et al. [22] proposed that Event2vec module combined with co-attentiion helps to learn a good representation of events, after which a convolutional neural network can extract the key features of the input sequence, thus effectively capturing the error messages to achieve excellent detection capability. Amma and Selvakumar [25] used a convolutional neural network to reduce tweet features and learning using fully connected neural network to achieve faster and more promising detection results. Reference [26] method based on deep learning has shown excellent detection ability in early rumor detection tasks. To validate the efficacy of the proposed method, it is compared to the methods referenced in previous studies ([22,25] and [26]) under identical experimental conditions. The results of the comparison are presented in Table 6. Through observation of the experimental outcomes, it is possible to ascertain that the proposed method yields the highest index. The accuracy, precision, recall, and F1 value of the proposed method are 0.962, 0.960, 0.904, and 0.931, respectively. By comparison, the accuracy, precision, recall, and F1 value of the method proposed in Yu et al. [22] are only 0.848, 0.852, 0.840, and 0.846, respectively. The proposed method improves at least 0.114, 0.108, 0.064, and 0.085 in accuracy, precision, recall and F1-scores over the existing methods.

The proposed method aggregates the text content of the information in the event cycle, the propagation trace of the event, and the feedback signal of the user and uses the temporal rumor identification modeling method based on deep learning to detect the Weibo data. The DSA mechanism is implemented in the model through the utilization of sentence-level and word-level self-attention mechanisms. Consequently, the proposed method effectively addresses the issue of redundant data in both the sentence and word dimensions. The comparison methodologies fail to address the issue of redundant data. Hence, in contrast to the comparison method, the proposed method yields considerably superior indexes. It demonstrates that the DSA mechanism has significantly enhanced the capability of determining the veracity of rumors.

The variations in the accuracy of the proposed method, as referenced in previous studies [22,25,26], in the validation set throughout the training process are illustrated in Figure 6. The learning capability of the Bi-GRU method with DSA is evidently the quickest. Following 200 iterations of training, the model has achieved a stable accuracy of approximately 0.96. While the reference models in previous studies [22,25,26] exhibit considerable initial learning capability even in the absence of word-level attention, their final learning outcome does not surpass that of the proposed model. As a result, the incorporation of the DSA mechanism may enhance the model’s capacity for learning.

Figure 6

Comparison of accuracy change.

5.4 Validity of unbalanced datasets

To demonstrate the efficacy of the model on an unbalanced dataset, this study takes the dataset with different ratios to assign rumor event and non-rumor event, and Figure 7 represents the detection results under different ratios between rumor and non-rumor.

Figure 7

Detection results under different ratios between rumors and non-rumors.

The results in Figure 7 demonstrate that the model works best at a 1:1 ratio. At 1:4 and 4:1 ratio, the model results are the worst, but it is still within an acceptable range. Analyzing the reasons, it becomes evident that the proposed model did not explicitly implement pertinent technologies to address the issue of data imbalance. Rather, its emphasis was placed on the extraction of deep-level features and contextual semantic information. Consequently, the proposed model is capable of producing more accurate detection outcomes when data are balanced.