Attention community discovery model applied to complex network information analysis

2.1 NI analysis model construction based on AMs

With the explosive growth of Internet information, how to extract valuable content accurately from massive information has become the focus of research. The traditional method is based on simple statistics and rules and cannot adapt to the complex and changeable network environment [12,13]. The AM is a technique in deep learning that assigns weights by calculating similarities between queries and key values, thereby enabling dynamic focus on input data [14]. The AM is a core component of this study, which allows the model to dynamically focus on important nodes and relationships when processing NI [15]. A dot-product AM is used to determine the attention weight by calculating the dot-product between the query vector and the key vector, and then the weighted summation of the value vector is performed. The mechanism of attention is shown in Figure 1.

Figure 1

Basic structure of AM.

AM is an important concept in deep learning, and its core principle lies in assigning different attentional weights by calculating the mapping relationship between queries and key values. The input data X is represented by

where K in Eq. (1) is the distribution σ i for computing attention and V is used to compute the aggregation information. This mechanism can be decomposed into three main steps: first, the weights are calculated based on the proximity of the query to each key value, and the formula for the scoring model is shown as follows:

where s i represents the calculated attention score, Q is the query, and k i represents the input information. Next, these weights are normalized by the softmax function, whose normalization formula is shown as follows:

Finally, the weighted summation of the corresponding values using the normalized weights yields the final attention value, whose expression is shown as follows:

In a physical sense, AM is designed to selectively focus on specific parts of the information with limited resources, which can be regarded as a similarity metric [16,17]. Combined with the existing model structure, AM not only improves the model, but also dynamically enhances the model performance by adjusting the threshold value. In the context of data explosion and limited computational resources, AM is important for the future development of deep learning. NI has diversity and complexity, containing various forms such as text, image, video, etc., each of which has its own unique structure and characteristics. In order to extract and analyze NI efficiently, an NI analysis model based on the AM is proposed in this article. The model converts information into vectors through the embedding layer, calculates the weight by using the AM, and fuses information from different sources to obtain a comprehensive representation. The model consists of an information embedding layer, an attention computing layer, and an information fusion layer. These three parts collaborate with each other to realize the effective analysis of NI. First, the information embedding layer is responsible for converting different forms of information into machine-processable vector forms. For textual information, advanced word embedding techniques are used to convert the text into a representation in vector space. In this study, Word2Vec is used as the word embedding method, and the expression of the embedding vector is shown as follows:

where e i is the embedding vector of word i , W emb denotes the word embedding matrix, and v i denotes the unique heat encoding of word i . Next is the attention computation layer, which is the core part of the model. This layer is responsible for calculating the attentional weights between different information to achieve focusing on key information. Then, the similarity is converted into weights by means of the softmax function, so that the information related to the current task is given a larger weight, while the irrelevant information is suppressed. In calculating the attentional weights between different information, this study uses dot product AM, and the expression for calculating the attentional weight α is shown in Eq. (6), given the query vector q , key vector k , and value vector v

where the dimension of the key vector is denoted by v i . Then, the similarity is converted into weights by the softmax function, whose expression is shown as follows:

Finally, the output vector o is a weighted sum of the value vectors with the expression shown as follows:

where α i is the attention weight corresponding to the value vector v i . This AM enables the model to be more efficient and accurate in processing NI. Finally, there is the information fusion layer. According to the weights obtained from the attention calculation layer, the information fusion layer weights and fuses the information from different sources to obtain a comprehensive information representation. In the information fusion layer, information from different sources can be weighted and fused according to the weights obtained from the attention computation layer. Assuming that there are n information sources, the information representation of each source is x i and the corresponding attention weight is α i , the expression of the fused information y is shown as follows:

In this way, the fused message y contains both the richness of the original message and highlights the task-relevant information. This representation contains both the richness of the original information and highlights the information relevant to the task, providing strong support for subsequent tasks.

2.2 ACDM construction incorporating CNN and SCAs

In order to process complex network information more efficiently and to analyze the community structure better, the study proposes the ACDM, which incorporates CNN and SCA, based on the AM-based network information analysis model. The model combines the advantages of CNN in feature extraction and the flexibility of AM in information processing and utilizes SCA to identify the community structure in the network. The main process of the ACDM proposed by the study is shown in Figure 2.

Figure 2

Main architecture of the ACDM proposed in the study.

As shown in Figure 2, the ACDM proposed in this study is mainly divided into three parts: convolutional layer, AM layer, and community discovery layer. The model first needs to input the pre-processed data into the convolutional layer and use CNN to extract the features of the NI. One-dimensional convolution operation is used for text information to capture the local dependency of text. For image and video information, a two-dimensional or three-dimensional convolution operation is used to extract visual features. The extracted features are then input into the AM layer for weighted processing, so that the model can focus on more important information and ignore irrelevant or noisy information. Through the AM, the model can dynamically adjust the attention to different features and improve the accuracy of community discovery. Then, the weighted information is input to the community discovery layer. In this step, the similarity matrix is constructed based on the feature representation of the output of the AM layer, and the spectral clustering operation is carried out to divide the network nodes into different communities. Finally, the community discovery classification results are output. The specific structure of the convolutional layer used in the CDM proposed in the study is shown in Figure 3.

Figure 3

Convolutional neural network structure diagram.

It can be seen from Figure 3 that the CNN structure used in the study includes multiple convolution cores, each of which slides on the input data and performs convolution operations to generate feature graphs. Through multi-layer convolution operation, the model can automatically learn the feature representation of NI from low level to high level. In addition, the second module in the CDM is the AM layer, whose role is to introduce AMs to assign weights to different nodes or edges when calculating community structure. The second module is the AM layer, which serves to introduce AM to assign weights to different nodes or edges when computing the community structure. By calculating the similarity or importance between nodes or edges, the model is able to focus on elements that play a key role in community formation while suppressing irrelevant and noisy information. The last module is the community discovery layer, which serves to identify the community structure in the network using SCA based on the extracted features and attention weights. SCA is based on the graph cut theory, where the data set samples are considered as graph vertices, and the similarity between the samples is the inter-vertex weights to construct the undirected graph [18,19]. Using methods such as connectivity and segmentation, the undirected graph is partitioned into multiple connected subgraphs, each representing a cluster [20]. The optimal graph cut algorithm needs to satisfy the two conditions of minimum and maximum connectivity weights between subgraphs, but the graph cut partitioning may lead to an imbalance phenomenon; the graph cut partitioning imbalance phenomenon is shown in Figure 4.

Figure 4

Unbalanced phenomenon of graph segmentation.

For the graph cut imbalance problem, there are three graph-cut approaches, which are the proportional cut approach, normalized graph cut approach (NCut), and min–max graph cut approach. The graph cut approach used in this study is Ncut, and its computational expression is shown as follows:

where vol ( A i ) denotes the sum of connection weights between all vertices and vertices in the subgraph. A 1 , A 2 , … A k denote all vertices in the subgraph. W is denoted as the similarity matrix. Then, a degree matrix is built, and the element values on its diagonal are used to calculate the sum of the element values in the similarity matrix. Eq. (11) provides the formula for determining the element values on the diagonal of the degree matrix

where W denotes the similarity matrix. Subsequently, using the similarity matrix and the degree matrix, the Laplace moments are computed, and the Laplace matrix formula is shown as follows:

where Lsym denotes the Laplace matrix and D denotes the degree matrix. The optimization objective of the Ncut method is shown as follows:

where T denotes the maximum iterations. Through the application of SCA, the proposed ACDM of the study is able to effectively identify the community structure in the network and provide a powerful tool for complex network analysis.

3.1 Comparative analysis of ACDM incorporating CNN and SCAs

To evaluate the effectiveness of the suggested ACDM using fused CNN and SCA, Table 1 displays the fundamental experimental setup used in this study.

Table 1 demonstrates the basic experimental environment configuration used to test the performance of ACDM with fused CNN and SCA in this study. With an NVIDIA GeForce GTX 1080 Ti graphics card, 32 GB DDR4 RAM, and an Intel Core i7-8700K processor running Windows 10, the trials were performed with robust computing capability and processing speed. Python 3.7 was used as the development language, and tools such as PyTorch 1.6 and scikit-learn 0.23 were combined for model development and training. To guarantee the accuracy and impartiality of the findings, the model’s performance was evaluated using the assessment criteria of accuracy, NMI, and modularity. The data set chosen for this study is the LinkedIn social network, because, as the world’s largest professional social networking platform, its data set contains rich user information, social relationships, career background, and other multidimensional data. These data can fully reflect the structure and characteristics of complex networks and provide rich experimental materials for community discovery tasks. At the same time, the community structure in the LinkedIn social network is relatively obvious, such as industry groups, occupational groups, etc., which is conducive to verifying the effectiveness of the CDM. The training set and test set, which each contain 40 samples and are split into four groups, are used to validate the accuracy of the research proposed ACDM during the testing process. Figure 5 displays the test results of the model in the training set and test set.

Figure 5

Test results of the model on the training and testing sets. (a) Training set test results. (b) Test set test results.

As can be seen from Figure 5(b), the test accuracy of the research-presented model is 95.0% for the 40 samples in the test set, while Figure 5(a) shows that the model’s test accuracy is 92.5% for the 40 samples in the training set. The aforementioned findings demonstrate the research-proposed model’s high degree of accuracy in both the test and training sets, which is useful. Afterwards, the study, in order to further highlight the superiority of the research-proposed model (Model 1), conducts a comparison test with the advanced deep learning-based CDM (Model 2), statistical inference-based CDM (Model 3), and modularity-optimized CDM (Model 4) and uses the standardized mutual information and modularity as the comparison metrics. Model 2, which uses deep learning technology to extract features and divide communities from network data, has strong adaptive ability and generalization ability. Based on the statistical inference theory, Model 3 finds the community structure by calculating the similarity and connection probability between nodes, which is suitable for large-scale network data. Model 4 is one of the classical methods in the field of community discovery, aiming to maximize the modularity and divide the network community by the optimization algorithm. The standardized mutual information and modularity comparison results of the four models in the LinkedIn social network data set are shown in Figure 6.

Figure 6

Standardized mutual information and modularity comparison results of the four models. (a) Comparison results of the modularity of the four models. (b) Comparison results of the normalized mutual information of the four models.

From Figure 6(a), Model 1 has the highest average modularity level in the six experiments, and its average modularity is 0.95, which is significantly higher than Model 2’s 0.88, Model 3’s 0.82, and Model 4’s 0.71. In Figure 6(b), Model 1 also has the highest average normalized mutual information (NMI) level in the six experiments, and its average NMI value is 0.94, which is significantly higher than Model 2’s 0.87, Model 3’s 0.79, and Model 4’s 0.69. The above results illustrate that the proposed CNN- and SCA-based ACDM outperforms the comparative models in terms of both modularity and standardized mutual information level dimensions. Finally, in order to compare and analyze the specific community discovery performance of the four algorithms, the study presents the specific classification performance of the four models on the data in the data set in a three-dimensional diagram, and the specific results are shown in Figure 7.

Figure 7

Comparison of classification results of the four algorithms in community network discovery. (a) Classification results of model 1. (b) Classification results of model 2. (c) Classification results of model 1. (d) Classification results of model 1.

In Figure 7, the clustering effect of the four recognition models in this data set is consistent. From the clustering results in Figure 7(a)–(d), the proposed model 1 has fewer discrete points in this data set and has a better clustering effect. In contrast, the other three models have obvious individual data ambiguities when clustering. The above results indicate that the overall performance of the proposed ACDM fusing CNN and SCA is better. Finally, in order to compare and analyze the practical application effects of the four models, the research applies the four models to the same group of community relationship recognition, which contains 20 fields. The comparison of the test results of the four models on community relationships and the actual relationship is shown in Figure 8.

Figure 8

Test results of the four models on community relations and comparison of actual relationships. (a) Actual classification. (b) Test results for model 1. (c) Test results for model 2. (d) Test results for model 3. (e) Test results for model 4.

The red lines in Figure 8 indicate domain relationships that do not match the actual relationships. It can be found from Figure 8(b) that there are two relationships obtained from model 1 proposed by the study that are inconsistent with the actual situation, while it can be found from Figure 8(c)–(e) that there are 5, 4, and 6 relationships, respectively, obtained from Models B, C, and D that are inconsistent with the actual situation. The above results show that the CDM proposed in this study is significantly better than the comparison model in identifying community relations.

3.2 Effectiveness of the practical application of ACDM

The study uses a genuine community network analysis to examine the proposed ACDM’s impact on practical implementation. The study chooses an online community with rich social activities as the research object, which has millions of registered users, and the topics in the community cover popular fields such as artificial intelligence, machine learning, blockchain, etc., attracting a large number of professionals and enthusiasts to participate. In this study, 17 areas of the community were selected for relationship testing, and the experimental results of the proposed ACDM were compared with the traditional CDM. Among them, the traditional CDM is based on graph theory, which has wide application and deep research foundations in the field of community discovery. Using them as a baseline comparison, it is possible to assess the degree of improvement of the proposed ACDM relative to existing techniques. Although the traditional model is limited in some ways, it still performs well in many practical applications. By comparing these models, we can better understand the strengths and weaknesses of the ACDM in practical applications. The specific comparison results are shown in Figure 9.

Figure 9

Comparison between the experimental results of the proposed ACDM and the traditional CDM and the actual results. (a) Actual classification. (b) Study the classification of proposed models. (c) Classification of traditional models.

The red lines in Figure 9(b) and (c) indicate the domain relationships that do not match the actual relationships. Two relations total that were acquired using the research-proposed model do not match the real scenario, as shown in Figure 9(b). On the contrary, Figure 9(c) shows that five relations total that were obtained using the traditional model do not match the actual situation. The aforementioned findings show that the suggested CDM has superior practical application than the conventional model and a very high degree of recognition accuracy. In order to compare the recognition accuracy of the two CDMs in a more detailed way, the prediction errors of the two models for each domain are normalized and the data are statistically obtained in Figure 10.

Figure 10

Comparison of prediction errors of the two models.

Figure 10 shows that the research-proposed CDM’s overall prediction error curve outperforms the classic model’s overall prediction curve. The proposed CDM has a minimum average prediction error of −0.019, which is better than the traditional model’s −0.037, and a maximum average prediction error of 0.018, which is better than the traditional model’s 0.039. These results further demonstrate that the proposed CDM performs better in real-world applications than the traditional model. Finally, Table 2 is used to provide statistics on the standardized mutual information and modularity results of the two models in the process of practical application.

In Table 2, the average standardized mutual information level of the proposed ACDM in the seven domains is 0.917, which is better than that of the traditional model of 0.777. In addition, it can also be found from Table 2 that the average modularity level of the proposed ACDM in the seven domains is 0.920, which is better than that of the traditional model of 0.793. The aforementioned findings show that, when compared to the old model, the suggested ACDM performs better in terms of practical applications due to its modularity and standardized mutual information level. Level two results are compared, and the performance of the research-proposed ACDM based on CNN and SCA is better than the traditional model and more practical.