Intelligent accounting question-answering robot based on a large language model and knowledge graph

2.1 Knowledge graph completion based on self-attention mechanisms and convolutional neural network

Knowledge graph presents the real world in the form of graphs, which can process and display heterogeneous data from multiple sources and their complex relationships. It is one of the core cognitive intelligence technologies driven by both data and knowledge. In the daily service of intelligent accounting question-answering robots, it is necessary to match the relevant knowledge points of the questions raised in a timely manner and then query to obtain appropriate answer content. A complete and accurate accounting knowledge graph is required as data support in this process. However, things themselves have diversity and variability. The knowledge obtained through automatic or semi-automatic knowledge extraction techniques is usually incomplete. If the knowledge graph is incomplete, it will further affect the learning effectiveness of the neural network and reduce the question-answering intelligence of the question-answering robot [16]. Therefore, the study considered introducing attention mechanisms and convolutional neural networks to predict unknown information and complete the knowledge graph. Convolutional neural networks capture the interaction information between entities and relationships through simple and efficient convolution operations, mine and extract the potential semantic information of triplets, and obtain more accurate representations of entity and relationship vectors. This type of method usually seeks rich interaction information between entities and relationships within triplets to enhance the expressive power of the model. To further enhance the ability of convolutional neural network models to obtain rich interactive information, a self-attention mechanism (SAM) was introduced to improve them while retaining the translation features of triplets and the global features of the same dimension. The vector representation process of SAM is shown in Figure 1.

Figure 1

Vector representation process of SAM.

In Figure 1, the SAM models the interdependence of different positions in the input sequence by creating three vectors: Query, Key, and Value for each input vector, and then calculating the attention score using these three vectors. The calculation method of attention value is denoted in the following equations:

In Eq. (1), Q , K , V are the three vector matrices of Query, Key, and Value, and d k is the input Query dimension. To enhance the effectiveness of attention mechanisms, the study combined individual SAMs to obtain multi-head self-attention, performed self-attention in parallel, concatenated multi-self-attention, and then transformed them into output vector dimensions. The triplets in the knowledge graph contain rich semantic information. To obtain the rich information of the triplet content, a knowledge graph linked unknown information prediction model was established. By using SAM on the concatenation vector of triplets through this model, feature representations that fuse information from different dimensions were calculated and obtained. By focusing on the varying degrees of importance of information from different dimensions and increasing the interaction between them, the model could capture potential information within the triplet through subsequent convolution operations, thereby enhancing its feature extraction ability and achieving simple and efficient link prediction. The structure of the knowledge graph link prediction model proposed in the study is shown in Figure 2.

Figure 2

The structure of the knowledge graph link prediction model.

In Figure 2, the study concatenated the initial embeddings of entities and relationships and generated Query, Key, and Value matrices using a mapping matrix of three vectors. Afterward, the transpose of the Query matrix and Key matrix were used to perform matrix multiplication operation. Afterward, the attention distribution matrix was obtained through normalization processing. The obtained attention distribution matrix and Value matrix were multiplied to obtain the Output matrix. Finally, the initial Embedding matrix was merged and the Output matrix obtained using SAM was concatenated to obtain a new matrix, which was used as the input of the convolutional layer. Finally, the convolutional layer was applied to extract semantic information and overall features from the triplet, concatenate all feature matrices, and calculate the score of the triplet. Based on the predicted scoring results of each triplet link, it selected the most suitable completion content to establish a knowledge graph. The scoring function is specifically denoted in the following equation:

In Eq. (2), concat ( ⋅ ) represents concatenation, Ω is the convolution set, w is the weight set, h , r , t are the triplets of the knowledge graph, and X ′ is the Output matrix obtained through the attention layer. Based on the above content, the study utilized convolutional neural networks and SAMs to extract a complete accounting knowledge graph.

2.2 Text information named entity classification and recognition based on improved GPN

After establishing a knowledge graph, textual information was extracted to provide a foundation for subsequent IQA. Named entity recognition is an important subtask in information extraction, which can effectively enable computers to understand professional accounting terminology and provide intelligent answers to accounting-related questions. In professional terminology texts, long nesting and complex structures often occur, which makes it difficult for traditional named entity recognition methods to effectively address. Therefore, the study considered introducing GPN for named entity recognition. The BERT model, as a high-performance model in LLMs, provides a solid foundation for named entity recognition with its powerful pre-training and context-understanding capabilities. Therefore, the study combined it with GPN and achieved named entity recognition by classifying multi-label text. The overall process of the named entity recognition model raised in the study is shown in Figure 3.

Figure 3

The whole process of the named entity recognition model.

In Figure 3, the study inputs text questions raised by users into the LLM BERT, which consists of two components: text convolution and GPN. Text convolution classifies the text, while GPN annotates the text to obtain a labeled matrix. GPN, as the core component, was designed to process the vector sequence output by BERT models. GPN is a decoding method that utilizes global normalization and span classification to annotate different types and entities by constructing a matrix form. The global-to-local memory pointer network (GLMP) combined the GPN and memory mechanism to strengthen the model’s ability to capture long-distance dependencies and nested structures in text by constructing a global-to-local memory mapping. Based on the vector sequence output by the BERT model, GLMP further explored the contextual information and associations between entities in the text, raising the accuracy and efficiency of named entity recognition. GPN captured and marked potential entity boundaries and types in vector sequences through a global pointer mechanism. On this basis, GLMP utilized memory networks to store and integrate long-distance information, helping the model better understand complex structures and nested relationships in text. The structure of GLMP is shown in Figure 4.

Figure 4

Structure of GLMP.

In Figure 4, GLMP first constructed a global memory matrix to store all possible entities and their related information in the text. Then, during the decoding process, GLMP used the preliminary entity labels generated by the GPN as guidance to retrieve and integrate relevant information from the memory matrix to generate more accurate and complete entity recognition results. This global-to-local memory mapping mechanism enabled the model to maintain high recognition accuracy and efficiency when dealing with long texts and complex structures. Based on the above content, a long nested named entity recognition model was established using GLMP and BERT, which combine GPN and memory mechanisms.

2.3 Intelligent accounting question-answering method based on Bi-GRU and knowledge graph

After establishing a knowledge graph and extracting information, research was conducted to improve the BERT LLM using Bi-GRU, further enhancing the ability of IQA robots to extract correlated information from sequences. The research decomposed the intelligent accounting question-answering method into three tasks: named entity recognition, relationship vector representation, and entity vector representation. As mentioned earlier, the study utilized GLMP and BERT to identify named entities. The BERT model’s structure is indicated in Figure 5.

Figure 5

Structure of large language model BERT.

However, traditional BERT models have not fully explored the correlation information in named exercise encoding and vector representation. To improve the accuracy of the three tasks and answer search tasks, Bi-GRU was introduced to improve the traditional BERT model. Research further processed the vector sequence output by the BERT model using Bi-GRU to strengthen the model’s ability to capture correlated information in text. Bi-GRU is a recurrent neural network that combines bidirectional structure and gating mechanism, which can simultaneously consider the contextual information of text, thus performing well in sequence modeling. After adding the Bi-GRU layer to the output of BERT, the BERT model first encoded the input text through a multi-layer transformer structure to generate a vector sequence containing rich semantic information. Subsequently, these vector sequences were fed into the Bi-GRU layer, which utilized its bidirectional structure and gating mechanism to further mine the associated information in the text and generate more refined vector representations. The specific structure and process of the integration of Bi-GRU and BERT are as follows. In this study, Bi-GRU was taken as the next layer structure of the BERT output layer, in which context information was captured and output, and text association information was further mined. The task of named entity recognition is a crucial step in intelligent accounting question-answering systems, which directly affects the accuracy of subsequent relationship vector representations and entity vector representations. The study applied named entity recognition results to answer retrieval tasks, calculated the degree of match between the retrieved answer and the recognition result, scored them, and finally selected the best answer for response. The calculation process is shown in the following equation:

In Eq. (3), α 1 , α 2 , α 3 are constants, Score ( ⋅ ) is the answer matching score, Rate ( e h , e i ) is the matching score of characters e h and e r , e ˜ is the entity representation, r ˜ is the relationship representation, and ( h i , r i , t i ) is the triplet vector representation in the candidate set. The research utilized Bi-GRU to improve the LLM BERT and applied it to named entity recognition, relationship vector representation, and entity vector representation. The model structure of the improved model in single-task and multi-task situations is shown in Figure 6.

Figure 6

The model structure of the improved BERT model in single-task and multi-task cases: (a) single task and (b) multi-task.

In Figure 6(a), in a single-task scenario, the improved BERT model received text as input and first encoded it through BERT’s multi-layer transformer structure. By encoding, semantic information in the text was captured, and vector representations of each word were generated. Subsequently, these vectors were fed into the Bi-GRU layer to capture the associations and dependencies between words. As shown in Figure 6(b), in a multi-task scenario, the model added a delimiter (SEP) on top of the single task to distinguish different task inputs. Each task had its specific context and objectives, and the model could use this delimiter to distinguish and process different tasks. In multi-task learning, the model was able to learn multiple tasks simultaneously. By sharing the underlying BERT and Bi-GRU structures, different tasks could share semantic information and features, thereby improving overall learning performance and generalization ability.

3.1 Performance analysis of knowledge graph completion method based on SAM convolutional neural network

To prove the rationality of the raised knowledge graph completion method, the model was experimentally assessed on datasets FB15k-237 and WN18RR. The study introduced mean reciprocal ranking (MRR), mean ranking (MR), and hyperlink-induced topic search (Hits) as evaluation metrics for knowledge graphs. The knowledge graph completion method (Method 1) of the application link prediction method was compared with the traditional knowledge graph extraction method (Method 2), which does not apply the completion method. The comparison results are shown in Figure 7.

Figure 7

Comparison of the effect of knowledge graph establishment before and after applying knowledge graph completion method: (a) FB15k-237 and (b) WN18RR.

In Figure 7(a) and (b), in both datasets, the MRR, MR indices, and Hits index values of Method 1 were significantly better than those of Method 2. On the FB15k-237 dataset, the MRR of Method 1 improved by about 15% compared to Method 2, while the MR decreased by about 30%, Hits@10 increased by nearly 10%. On the WN18RR dataset, the improvement of various indices was also significant, further verifying the performance of the method.

To test the rationality of introducing SAM in the study, the training of IQA models under knowledge graph extraction results before and after introducing SAM was compared, as shown in Figure 8.

Figure 8

The training situation of the IQA model under the knowledge graph extraction results before and after the introduction of the attention mechanism: (a) loss and (b) recall.

In Figure 8(a), after introducing the attention mechanism, the model had better training iteration convergence and could reach the optimal loss value faster. After the introduction, the number of iterations to convergence of the model decreased by 6 compared to before the introduction. As shown in Figure 8(b), the model achieved the optimal recall rate after only 15 iterations and converged faster compared to the model before introduction.

To further validate the superiority of the raised knowledge graph-building method (Method A), the study compared it with the knowledge graph-building methods (Method B) in ref. [17], (Method C) in ref. [18], and (Method D) in ref. [19]. The comparison findings are indicated in Table 1.

In Table 1, the MRR value of Method A was 0.539, which was significantly improved compared to Method B’s 0.300 and Method D’s 0.350. Meanwhile, the MR value of Method A was only 144.0, far lower than other methods, indicating that its predicted results ranked higher, that is, it was more likely to find the correct answer. On Hits@1, Hits@3, and Hits@10 indicators, Method A also performed well and outperformed other comparative methods.

3.2 Performance analysis of the IQA method based on GPN and Bi-GRU

Research aimed to address the deficiencies of the convolutional LLM BERT and improve it using Bi-GRU. To test the improvement effect of the language model, the comparison results of the model before and after improvement were studied in the named entity recognition task (Mission 1), relationship vector representation task (Mission 2), entity vector representation task (Mission 3), and answer retrieval task (Mission 4), as shown in Figure 9.

Figure 9

The application effect of the model before and after improvement in three tasks: (a) named entity recognition task accuracy and (b) relational vector representation and entity vector representation, accuracy of answer retrieval task.

In Figure 9(a), in Mission 1, the improved model achieved higher recognition accuracy, reaching 93.48%, an increase of 5.24% compared to before the improvement. As shown in Figure 9(b), the Mission 2, Mission 3, and Mission 4 exceeded 90%. Compared to the previous model, the improvements were 2.54, 4.58, and 3.15%, respectively.

When annotating text tags, research improved GPN by combining it with memory mechanisms to further enhance the recognition of named entities. To validate the performance of the raised improvement method, experiments were organized to compare the changes in recognition accuracy before and after the improvement. The specific results are shown in Figure 10.

Figure 10

The change of recognition accuracy of the named entity recognition method before and after improvement.

In Figure 10, as the number of named entities to be recognized increased, the recognition accuracy of both methods decreased. The improved method GLMP proposed in the study had the smallest decrease in accuracy and an average accuracy of over 90%. Compared to GPN, the average recognition accuracy increased by 8.47%.

To further prove the efficacy of the raised IQA model (Model 1), the study compared it with the IQA models: Model 2 in ref. [20], Model 3 in ref. [21], and Model 4 in ref. [22]. The study applied these four models to the customer service departments of three different financial institutions in Place A to learn and apply them in business consulting, transaction problems, loan consulting, and other work tasks. The application period was 3 months, and the answer accuracy, coverage percentage, response time, and user satisfaction of the four models were recorded during the experiment. Among them, user satisfaction was the result of the questionnaire completed by the customers who came to the financial institution for consultation each time during the application period. During the experiment, a total of 684 customers evaluated the intelligent question-and-answer system after consulting, and the evaluation results were obtained in the form of average scores, as shown in Table 2.

In Table 2, the answer accuracy of Model 1 was 94.6%, significantly higher than Model 2, Model 3, and Model 4, indicating that the model was more effective in accurately answering user queries. In addition, Model 1 covered 95.2% of the coverage area, demonstrating its comprehensive ability to handle a wide range of problems. In terms of response time, Model 1 had a response time of only 120 ms, which was faster than other models and enhanced the user experience. Finally, the user satisfaction score for Model 1 was 9.2 points.

To further emphasize the excellent performance of the model designed in the study in practical financial applications, the study compared it with the most advanced question-and-answer model published this year. The experimental scenario was the customer service department of three different financial institutions in Place A. The experiment included a variety of customer consultation scenarios, including account inquiry, transaction questions, loan consultation, etc. The experiment lasted for 10 months. The comparison models included the question-and-answer models in ref. [23], ref. [24], and ref. [25]. The comparison indicators included the model’s answer accuracy, business processing time, MRR, and F1 score. The specific results are shown in Table 3. The comparison results are shown in Table 3.

In Table 3, Model 1 outperformed other comparison models on all evaluation indicators, especially in terms of response accuracy and MRR. With the progress of the experiment, the indicators of the four models have been significantly improved, of which Model 1 had the largest improvement. This showed that it could continuously improve its function in the process of continuous learning. Model 1 showed significant advantages at every stage of the experiment. This showed that Model 1 not only provided more accurate answers when handling question-and-answer tasks in the financial sector but also performed better in terms of user satisfaction. In addition, Model 1 also excelled in business processing time, with average processing time significantly lower than other models. This showed that Model 1 could respond to customer needs faster in practical applications and improve customer service efficiency. The comparison results of F1 score further confirmed the superiority of Model 1 in considering accuracy and recall rate comprehensively.