E-commerce big data processing based on an improved RBF model

3.1 Construction of the RBF model improved by the Lasso regression algorithm

In the field of e-commerce, the diversity, dynamics, and large-scale characteristics of customer data can significantly affect the performance and accuracy of the model. To better predict CC in e-commerce, this study builds an e-commerce CC prediction model using an improved RBF network. The impact of data diversity on model performance is first considered. E-commerce customer data cover a wide range of consumption behaviors, preferences, and transaction histories. This data diversity requires the model to be able to effectively process different types of data and extract valuable information from them. The proposed L-RBF model aims to better adapt to this diversity through a flexible network structure and optimization algorithms, thus improving the accuracy of prediction. Next, the impact of data dynamics on model performance is considered. E-commerce data change dynamically rapidly and their customer behavior and market trends evolve continuously [17]. The L-RBF model is optimized using Lasso regression, which aims to enhance the model’s adaptability to new data and learning speed, so as to respond to the dynamic changes in the data more effectively. Finally, the large-scale characterization of data is considered. In e-commerce, various types of models are often required to handle large-scale customer data, which challenges the computational efficiency and processing capability of the models. The L-RBF model optimizes the computational process and reduces unnecessary data processing with a view of effectively reducing time complexity while maintaining high prediction accuracy.

The RBF, as an efficient feed-forward neural network, has a strong global approximation capability and can accurately represent any nonlinear function. Therefore, this neural network is suitable for e-commerce domains with large amount of data [18]. In addition, RBF has strong generalization ability and can map higher features, so it can also be used for the dynamic prediction of e-commerce CC. The standard RBF model has issues with overfitting, high computational complexity, parameter selection, and sensitivity to noise. The study introduced least absolute reduction and Lasso regression to optimize it. First, the RBF model often overfits with high-dimensional data due to its complexity, leading to poor test data generalization. Second, calculating distances for each sample point to all RBF centers is computationally heavy and time-consuming. Third, selecting key parameters is difficult and typically requires multiple cross-validation trials, adding complexity. Finally, RBF models are sensitive to noisy data, common in e-commerce, which reduces the prediction accuracy. The general structure of RBF is shown in Figure 1.

Figure 1

Structure diagram of the RBF neural network.

Figure 1 shows the structural diagram of the RBF. It is divided into three layers: input, hidden, and output layers. The amount of nodes in the input layer depends on the dimensionality of the input sample features. The function of the hidden layer is to perform radial transformation on the input feature data, transform the spatial dimension of the features, and transform them into linearly separable states in high-dimensional space. In the input layer, it randomly inputs data X n into the input layer, then randomly selects m center points, and map the input data to the hidden layer. Finally, it calculates the output calculation result through the hidden layer function and outputs the output layer output result y . In the hidden layer, the RBF of neurons is a Gaussian kernel function, and its expression is shown in formula (1) [19].

In formula (1), C is the central value of RBF. σ means the expansion coefficient. G ( ⁎ ) denotes a Gaussian kernel function. X indicates the input characteristic value. Since the operation process of RBF neural network is actually the weighted operation of hidden layer neurons, to avoid the extreme phenomenon of RBF, the definition expression of the expansion coefficient σ is shown in formula (2).

In formula (2), d max is the maximum distance of the center point of the RBF. c k expresses the amount of center points. The final layer of the RBF neural network is the output layer, which is used to weigh and calculate the neurons in the hidden layer. Its weight values are generally within the range of [0,1]. The output value of the RBF network is indicated in formula (3).

In formula (3), y i is the i th neuron of the output layer’s output feature. k is the number of input features. w i j is the weight between the i th and j th neurons. x p is the p th sample feature of the input. c j is the j th center point. From formula (3), the operational efficiency of RBF is influenced by factors such as input feature numbers, center points, and expansion coefficients.

The data characteristics of e-commerce customers are relatively complex, including much data not related to CC prediction, coupled with the fact that the traditional RBF network faces the challenges of complex operation and difficult parameter selection when dealing with large-scale e-commerce data. To solve this problem, this study introduces the Lasso regression algorithm to improve the traditional RBF network and finally designs the L-RBF model. Lasso regression is able to effectively reduce the complexity of the model by introducing a regularization term and helps to prevent overfitting. In the L-RBF model, the centroid selection and weight allocation of the RBF network are optimized by the Lasso algorithm, which improves the model’s ability to process big data and its prediction accuracy. In addition, the L-RBF model utilizes the sparse nature of Lasso regression to automatically select the most informative features, which further improves the efficiency and accuracy of the model in large-scale e-commerce data processing.

The Lasso regression algorithm is based on least squares and solves the problem of linear overfitting by adding a norm. This algorithm can simplify excess feature items to 0 based on the obtained feature coefficients, reducing the computational complexity of data operations. It assumes that there is a dataset called ( X i , y c ) , where the expression for X i is shown in formula (4).

Here, N stands for the amount of samples. y c in ( X i , y c ) refers to the output value of different category features, where c has a value range of [ 1 , M ] . M means the amount of output categories of sample features. The linear regression model expression of y is shown in formula (5).

In formula (5), β 0 expresses a constant. β 1 , β 2 , … , β q means the linear regression coefficient. ε refers to the penalty term. It performs the Lasso algorithm estimation operation on it, and the expression is shown in formula (6).

In formula (6), β stands for the average of all linear regression coefficients. Extracting non-zero coefficients with high stability in formula (6) can obtain feature points related to the target data, achieving the effect of dimensionality reduction on their features. At the same time, the regression formula of Lasso can be obtained as shown in formula (7).

In formula (7), β 0 , β 1 , … , β k is a non-zero constant. According to the Lasso regression algorithm mentioned above, target features can be extracted from the original e-commerce customer dataset to obtain the dimensionality reduced dataset. The dimensionality reduced dataset can also optimize the parameter selection process of RBF, obtain suitable center points, and improve the operational accuracy of RBF models.

Figure 2 shows the operation steps of the RBF model under the Lasso regression algorithm. Lasso regression enhances the model performance by using L1 regularization to select important features and reduce redundancy. In e-commerce, customer behavior data are complex with many features, but only some significantly impact churn prediction. Lasso regression filters critical features like purchase frequency, spending amount, and browsing time, which more accurately reflect the customer behavior and potential churn risk. This improves the model’s predictive accuracy and robustness. There are multiple features in the original dataset. They are input into the Lasso regression algorithm as input values, and their dimensions are reduced to obtain a dataset containing only the target features. By substituting this dataset into the RBF for operation, the operational parameters of the RBF, including the amount of center points in the hidden layer, can be obtained. The proximity to a predicted value of 1 can be calculated based on the center point of the hidden layer. The higher the proximity, the higher the fitting accuracy of the Gaussian kernel function at the center point, and the more accurate the operation results of the RBF. The expression for the proximity of the center point is shown in formula (8).

Figure 2

RBF model operation steps under the Lasso regression algorithm.

In formula (8), p is the proximity of the center point to the predicted value 1. After obtaining the appropriate operating parameters of the RBF, the RBF model is updated. It trains and calculates the dimensionality reduced training set based on the updated RBF model to obtain feature output values. The output value is compared with the expected value, and if the output value is less than the expected value, the predicted result is loss. If the output value is greater than the predicted value, it is predicted as non-loss.

3.2 L-RBF e-commerce CC prediction model based on lifecycle

The purchasing needs of customers in e-commerce enterprises vary over different periods, resulting in dynamic changes in their feature datasets. To improve the accuracy of L-RBF prediction, this study combines the concept of customer lifecycle with the L-RBF prediction model. The combined model can dynamically predict CC rates in different periods based on changing customer feature datasets [20,21]. The customer lifecycle refers to the development process from the beginning of consumption to the end of consumption by customers on e-commerce platforms.

Figure 3 shows a schematic diagram of the customer lifecycle of an e-commerce platform. On e-commerce platforms, the customer’s lifecycle shows a trend of segmented changes. The first stage is the exploration stage, where the customer type is potential customers who have not yet collaborated but are willing to consume. The second stage is the advanced stage, where potential customers are successfully discovered and become new customers for e-commerce enterprises. The third stage is the stable stage, which refers to the customer’s consumption level and willingness in the e-commerce enterprise being relatively stable. The fourth stage is the recession stage, which refers to customers losing their willingness to consume on the platform. The fifth stage is the churn stage, where customers lose interest in the platform’s products and no longer pay attention to the company’s products. So, to reduce the proportion of lost customers, the risk of customer loss should be predicted before the loss stage.

Figure 3

Customer lifecycle diagram of the e-commerce platform.

Figure 4 shows a schematic diagram of L-RBF e-commerce CC prediction based on lifecycle. First, it obtains the customer’s dataset from the data management department of the e-commerce enterprise platform and then divides the dataset into cycles based on its lifecycle characteristics. Datasets from different periods are utilized as input values for the L-RBF model prediction model and then it determines whether customers have a tendency to churn based on the output results. As customers have already undergone periodic classification, when analyzing the reasons for churn, classification analysis can also be conducted based on the characteristics of customers in different periods, to develop the corresponding strategies to retain customers at risk of churn. It assumes that the original data set of customers in the exploration, advanced, stable, and declining stages is as shown in formula (9) [22,23].

Figure 4

Schematic diagram of L-RBF e-commerce CC prediction based on lifecycle.

In formula (9), n is the data feature dimension, and the original feature stages of the four stages are the same. However, the customer demand characteristics in the four stages are inconsistent, so there are certain differences in the regression formulas constructed by using the Lasso algorithm to extract features. The Lasso regression formula for the four lifecycles is denoted in formula (10).

In formula (10), y A , y B , y C , and y D are customer dimensionality reduction datasets for the mining, advanced, stable, and declining stages, respectively. i , j , J, and h are the number of features, with values less than or equal to the data feature dimension n . a i , b j , c m , and d h are non-zero regression coefficients. The above content can extract customer features from different lifecycles, and different L-RBF prediction models can be constructed based on the feature set.

Figure 5 shows the schematic diagram of the L-RBF model training process based on the lifecycle. The Lasso regression algorithm can extract features based on the characteristics of different lifecycles. The L-RBF model can obtain the center points of different models based on different feature sets and construct different prediction models based on the obtained center points. According to the L-RBF prediction process, customers with loss risk can be predicted. To deeply analyze the reasons for CC, this study introduces the correlation coefficient R to represent the correlation between the features and churn degree, which is expressed in formula (11).

Figure 5

Schematic diagram of the L-RBF model training process based on lifecycle.

In formula (11), x ¯ and y ¯ are the average values of the loss rates of the original feature set and the feature set, respectively. The criteria for determining correlations are indicated in formula (12).

In formula (12), ∣ R ∣ means extremely weak, weak, moderate, strong and extremely strong correlations between the ranges of [0, 0.2], [0.2, 0.4], [0.4, 0.6], [0.6, 0.8], and [0.8, 1.0], respectively. E-commerce platforms can analyze the reasons for CC based on their characteristics and the degree of correlation with CC and develop retention strategies for customers at risk of churn.

When making predictions about e-commerce data, attention should also be paid to customer privacy and the handling of sensitive personal data. This study employs four measures to ensure security and compliance when handling sensitive personal data in e-commerce. The first is data anonymization and de-identification. When processing customer data, data anonymization and de-identification are performed first to ensure that personal information cannot be identified directly or indirectly, a step that helps reduce the risk of personal privacy breaches. This is followed by compliance with data protection regulations. When conducting predictive analytics on CC data, there is a need to strictly comply with relevant data protection laws and regulations to ensure that all data processing activities comply with legal requirements. Next is the need to restrict data access. It should strictly limit access to sensitive data to authorized researchers and necessary system administrators. In addition, encryption is used to protect stored and transmitted data. Finally, regular security audits are conducted. Regular security audits and privacy impact assessments are conducted to identify and address any potential privacy and security issues.

In this study, ethical and legal issues also need to be given high priority when using AI and machine learning techniques for e-commerce customer data analysis. First, strict compliance with data protection laws and regulations, such as the EU General Data Protection Regulation, during data collection and processing, ensures the protection of customer privacy and sensitive information. Second, considering the issues of bias and discrimination that may arise from machine learning algorithms, this study takes appropriate measures to identify and mitigate these potential risks during the model design and implementation stages. In addition, this study focuses on the transparency and interpretability of the models so that users and regulators can understand the decision-making process of the models. Finally, this study emphasizes the continuous monitoring and evaluation of model performance in real-world applications to ensure that it meets ethical standards and legal requirements. Through the above measures, the study aims to ensure that the application of AI and machine learning in e-commerce customer data analytics is both efficient and responsible while complying with ethical and legal requirements.

4.1 Performance analysis of the L-RBF CC prediction model

To improve the overall performance of e-commerce customer prediction models, this study proposed using the Lasso regression algorithm to optimize the RBF model. To address the model bias and discrepancy, several steps can be taken. First, comprehensive pre-processing, including data cleaning, outlier detection, and normalization, reduces noise and improves model robustness and accuracy. Second, Lasso regression for feature selection filters out the most influential features, excluding redundant and irrelevant ones. Finally, cross-validation divides the dataset into training and validation sets, allowing effective model evaluation and avoiding bias from uneven data distribution.

To verify the performance of the optimized L-RBF model, the traditional RBF model, the original logistic regression algorithm, and the gradient boosting decision tree (GBDT) were used as comparison algorithms. Among them, the original logistic regression algorithm is an iterative classification algorithm used to solve the optimal value, and the GBDT is an integrated learning algorithm used to reduce the deviation, which can classify and predict the data according to the weight. The above three methods have varying degrees of application in the field of loss prediction. The data set used in this experiment was the customer information set of a clothing store in an e-commerce platform, which contained 100,000 pieces of data. The above four algorithms were used to analyze the data set, and the performance of the algorithm was compared and analyzed according to the accuracy, recall rate, receiver operating characteristic (ROC) area, and iteration error value of the algorithm.

Table 1 shows the experimental environment details. The experimental computing environment was 64 bit Windows 10. The customer prediction model was constructed using the L-RBF, RBF, original logistic regression algorithm, and GBDT algorithm. The performance of the algorithm was analyzed according to the performance of the computing process and the results.

Figure 6 shows the PR curves under different algorithms. From Figure 6, it can be observed that RBF, original logistic regression, and GBDT algorithms had different cross positions, which showed that at different recall rates, the accuracy of the three algorithms had their own advantages. The L-RBF model proposed in the study outperformed the other three algorithms in accuracy at any recall rate, achieving a maximum accuracy of 95%. The comprehensive performance of the L-RBF algorithm was superior to the other three algorithms. Therefore, the L-RBF algorithm proposed in the study had better comprehensive performance.

Figure 6

PR curves under different algorithms.

Figure 7 shows the ROC under different algorithms. The models used in Figure 7(a)–(d) were L-RBF, GBDT, original logistic regression, and RBF models. From Figure 7, the ROC of the L-RBF model was closer to the upper left corner, and its AUC was larger, followed by the GBDT model, then the original logistic regression model, and finally the RBF model. It showed that the L-RBF model could better classify the features related to CC from the original data set, which was conducive to the subsequent prediction of CC. In summary, the L-RBF model had better performance in classifying CC features.

Figure 7

ROC under different algorithms. (a) ROC of L-RBF algorithm, (b) ROC of GBDT algorithm, (c) ROC of logistic regression algorithm, and (d) ROC of RBF algorithm.

Figure 8 shows the loss rate curves of prediction models under different algorithms. From Figure 8, as the amount of customer data processed by the four algorithms increased, the overall loss rate of the algorithms was on a downward trend, and finally tended to stabilize. Among them, the overall loss rate of the L-RBF algorithm was smaller and tended to stabilize the fastest. When the amount of customer data reached 100,000, the loss rate of L-RBF algorithm started to converge. The final loss rate was 3%, lower than 4% of GBDT algorithm, 5% of original logistic regression algorithm, and 6% of RBF algorithm. In summary, the L-RBF algorithm had a higher algorithm loss rate and could more effectively predict CC.

Figure 8

Loss rate of prediction models under different algorithms.

Figure 9 shows the time complexity of prediction models under different algorithms. As the number of processed samples increased, the time complexity of all four models increased. Among them, under the same sample size, the time complexity of the L-RBF model was the lowest among the four models. When the number of processed samples was 20,000, the L-RBF model’s O ( 2 n ) , O ( n 3 ) , O ( n 2 ) , O ( n log 2 n ) , O ( n ) , O ( log 2 n ) , and O ( 1 ) were 142, 86, 47, 34, 30, 17, and 3, respectively, followed by the original logistic regression model, RBF model, and GBDT model. Because the Lasso regression algorithm in the L-RBF model improved the RBF model by simplifying the feature coefficients and reducing the computational complexity of data operations, the time complexity during the operation process would also be correspondingly reduced. Therefore, the improved L-RBF model in this study had better time complexity.

Figure 9

Time complexity of prediction models under different algorithms. (a) L-RBF, (b) logistic regression, (c) GBDT, and (d) RBF.

In order to demonstrate the statistical significance of the L-RBF model in benchmarking performance tests, the study statistically analyzed the results of several repeated experiments, which are shown in Table 2.

The statistical test results of the four models under multiple experiments are given in Table 2. As can be seen from Table 2, L-RBF has the best performance in average accuracy, average AUC, and average F1 score compared to the other three models, which are as high as 95.0%, 0.96, and 0.94, respectively. In addition, paired t-tests were also conducted to test the performance metrics of the L-RBF model against the other models, and the results show that the L-RBF model is statistically significantly better than the other models (P < 0.05). Through the above statistical tests, the validity and robustness of the L-RBF model under multiple repetitive experiments were finally verified.

4.2 Performance analysis of improved L-RBF’s e-commerce CC model application

To make the L-RBF prediction model applicable to the field of e-commerce, this study further improved the L-RBF model by combining customer lifecycle concepts. To verify the application performance of the L-RBF model combined with the concept of customer lifecycle in e-commerce CC, the research would still use the dataset in Section 3.1, with the feature dimensions shown in Table 3.

Table 3 shows the feature dimensions of the experimental customer dataset for clothing shops, which include the physiological index, daily consumption level, dressing style, platform activities, and shop purchase. Sensitivity analysis reveals that daily consumption level has the highest importance (0.89), followed by shop purchase (0.85), platform activities (0.78), physiological index (0.72), and dressing style (0.64). Physiological index refers to the match between customers’ height and weight with clothing style. Daily consumption level assesses the similarity to shop prices. Dress style consistency, platform usage frequency, and shop purchase frequency also impact customer stickiness to the shop, influencing their life cycle characteristics. Because the number of features was 5, which means that the dimension of the data set was 5, the number of input neurons of the L-RBF model was determined to be 5. There were 1,524 missing data in the data set, and 70,000 data were randomly selected from the remaining non-missing data for detection. Because the number of core points was usually 10% of the sample data, there were 14,000 core points in this L-RBF model. The study would use an L-RBF model that combined the customer lifecycle to construct a corresponding cycle churn prediction model for analysis. The application performance of the model would be analyzed based on the prediction accuracy and error rate of the four models, as well as the correlation between features and CC at different stages.

In order to realize the conversion processing between different life stages, the L-RBF model incorporates the following improvements into the life cycle features. First, the dynamic feature update, the improved L-RBF model will periodically update the customer feature data to identify changes in customer behavior. By monitoring customer behavior in real time, the model can dynamically adjust the feature weights to ensure that the model can adapt to new data patterns during the life cycle transition. Next is adaptive learning. The improved L-RBF model adopts an adaptive learning mechanism, which is able to adjust itself during the customer life cycle transition. Through continuous learning and updating, the model is able to quickly adapt to new customer behavioral characteristics, ensuring that the prediction accuracy is not affected by the life cycle transition. Through the above methods, the improved L-RBF model can effectively handle the transitions between customer life cycle stages and ensure the prediction accuracy at different stages, and the specific results are shown in Figure 10.

Figure 10

Improved L-RBF algorithm for predicting the accuracy and loss rate curves in different customer lifecycles. (a) Improving the prediction accuracy and error rate of L-RBF model in the excavation stage, (b) improving the prediction accuracy and error rate of L-RBF model in the advanced stage, (c) improving the prediction accuracy and error rate of L-RBF model in the stable stage, and (d) improving the prediction accuracy and error rate of L-RBF model in the decline stage.

Figure 10 shows the prediction accuracy and loss rate curves of the improved L-RBF algorithm in different customer lifecycles. Figure 10(a)–(d) represents the stages of customer lifecycle excavation, advanced, stable, and decline, respectively. The improved L-RBF model had high prediction accuracy at different customer lifecycle stages, and both accuracy and loss rates tended to stabilize with the increase of customer data quantity. The improved L-RBF model could achieve stable accuracy rates of 97.6, 93.1, 92.7, and 91.8% in the excavation, advanced, stable, and decline stages, respectively, with stable loss rates of 0.5, 1, 1.3, and 1.3%, respectively. Therefore, the improved L-RBF model could accurately predict the loss of customers at different stages of their lifecycle.

Figure 11 shows the correlation between different feature types and CC. From Figure 11, different types of features had different correlations with CC at different stages of the customer lifecycle. In the stages of excavation, advanced, stable, and decline, the characteristic indicators with a higher correlation with CC were physiological indicators, dressing style, platform activity, and store purchase frequency, with correlation indicators of 0.75, 0.58, 0.64, and 0.83, respectively. According to formula (12), the correlation degrees of the above correlation indicators were strong, moderate, strong, and extremely strong correlations, respectively. Therefore, based on the improved L-RPF model, features related to CC at different stages could be obtained, which helped platform stores develop corresponding strategies based on the obtained correlation features.

Figure 11

Correlation between different feature types and CC.

To further demonstrate the effectiveness of the L-RBF model incorporating the concept of customer lifecycle in e-commerce CC prediction, this study applied L-RBF, RBF, logistic regression, and GBDT to the actual e-commerce CC prediction task, respectively. A home-made large and diverse real-time e-commerce dataset was used for testing, and the prediction results of the four models for CC were further obtained, as shown in Table 4.

The performance of the four e-commerce CC prediction models in real applications is given in Table 4. As shown in Table 4, the average prediction accuracy, average prediction time, and mean square error values of L-RBF were 0.93, 0.13 s, and 1.87 in the actual prediction task, those of RBF were 0.84, 1.06 s, and 8.98, those of Logistic regression were 0.82, 0.87 s, and 11.15, and those of GBDT were 0.82, 0.87 s, and 11.15 time, respectively. By combining the three indexes, it can be found that the L-RBF model proposed in this study showed a better performance of the practical application.