Advanced sentiment analysis in online shopping: Implementing LSTM models analyzing E-commerce user sentiments

2.1 Sentiment analysis: An overview

Sentiment analysis is the area of study, otherwise called opinion mining, involving the computational treatment of opinions, sentiments, and subjectivity in the text [13]. It is typically aimed at detecting an attitude, emotion, or opinion of the author of a piece of text toward some topic, product, or entity [14]. Sentiment analysis in the e-commerce domain helps to gain an understanding of customer feedback on products and improves recommendations, thereby enhancing user experience. The majority of early techniques developed for sentiment analysis were mostly based on the lexicon-based approach, largely dependent on predefined dictionaries of words with assigned positive or negative sentiments [15,16] While pretty easy to be implemented, such approaches mostly suffered from issues like context-dependent expressions and domain-specific terminologies.

2.2 Machine learning approaches to sentiment analysis

With the advances in the area of natural language processing, methods of machine learning began to be at the forefront of sentiment analysis. Supervised learning algorithms of a variety of sentiment classification tasks have applied well-known techniques such as SVMs, Naive Bayes, and random forests [17,18]. Typically, feature engineering is a stage required in such approaches, in which relevant features from text data are designed and extracted manually by researchers. Pak and Paroubek [19] showed that Naive Bayes classifiers could perform up to 70% accuracy on sentiment analysis using Twitter data. In the work of Go et al. [20], distant supervision was used to train different machine learning models on Twitter data, where SVM turned out to be better than other classifiers. While these traditional machine learning methods proved to be useful to some extent, most of them fail to incorporate the sequential nature of language and long-term dependencies in text. And aside from that, dependence upon hand-designed features is becoming a bottleneck to move into new domains and languages.

2.3 Deep learning in sentiment analysis

Deep learning techniques have definitely revolutionized the scene of natural language processing, together with such particular domains as sentiment analysis. The big power of neural network-based models in various NLP tasks has been confirmed, especially when designed for sequential data [21]. Convolutional neural networks were primarily developed for image processing and have recently been transferred to text classification and sentiment analysis. Kim [22] proposed a CNN architecture for sentence-level classification tasks which gained state-of-the-art results on several benchmarks. While the success with CNNs in sentiment analysis tasks has been due to the fact that they become good at fathoming local patterns or n-gram-like features in text, RNNs and especially their variants have turned out to be really powerful tools for a lot of sequence modeling tasks, among them being sentiment analysis. Specifically, models that process sequential data and capture the long-term dependencies of text make them very suitable for analyzing user reviews and comments.

2.4 LSTM networks for sentiment analysis

LSTM neural networks, put forward by Hochreiter and Schmidhuber in 1997, are one type of RNN designed for solving the standard RNN issue of the vanishing gradient. LSTM networks have been received with impressive regard in the field of sentiment analysis since they hold a central position in capturing long-term dependencies and contextual information in text [23,24]. Tang et al. [25] proposed a target-dependent LSTM model for sentiment classification and reported better performance than traditional SVM-based methods over a dataset of Twitter posts. Their model showed the ability to understand target-specific sentiments in a sentence. Jiang et al. [26] have also suggested a multi-task learning framework in which LSTM networks perform aspect-based sentiment analysis. In their approach, the aspect detection and sentiment classification tasks are jointly learned in this approach, and state-of-the-art results were achieved. In particular, regarding e-commerce, LSTM networks have recently been quite successful in customer review analysis. Gope et al. [27] developed an LSTM model for sentiment analysis of Amazon reviews, which worked better than traditionally used machine learning techniques.

3.1 Data collection and preprocessing

One of the biggest e-commerce platforms is searched to get user review data in this study. Figure 1 shows the steps followed from data collection to model evaluation. This dataset included 1 million reviews from a variety of categories (e.g., electronics, clothes, and accessories) including examples such as home goods or books, etc. Each review features the following data:

Figure 1

Flowchart representing the steps followed from data collection to model evaluation.

Review text

Star rating (1–5 stars)

Product category

Review timestamp

The following are the preprocessing steps done on the data for sentiment analysis:

Text cleaning: HTML tags, special characters, and non-ASCII characters were removed.

Tokenization: Review text is split into single words or tokens.

Lowercasing: Case transformation was applied to all text.

Stop word removal: Common words (e.g., “the,” “a,” and “an”) that do not contribute to sentiment much were eliminated.

Lemmatization: Words were reduced to their base or dictionary form.

The hybrid Faster-RCNN method was selected since traditional Faster-RCNN models and other SOTAs, such as YOLO v4, SSD, and Mask R-CNN, are not able to fully cope with the inherent temporal continuity and dynamics of video data including motion blur, object variation, or partial occlusions. Boasting an effective balance of global scene understanding and detailed locality, both through temporal coherence and adaptive feature fusion, their method achieves high object detection and tracking accuracy while maintaining an efficient computational amount.

We took ratings in stars as a proxy for sentiment labeling

1–2 stars: Negative sentiment

3 stars: Neutral sentiment

4–5 stars: Positive sentiment

Thereafter, we divided the dataset into training, validation, and test sets in a stratified manner with ratios of 80, 10, and 10%, respectively, based on the sentiment labels.

3.2 Feature extraction

We extracted different features for LSTM models and baseline machine learning approaches.

1. LSTM models:

1. Word embeddings: Pre-trained GloVe embeddings of size 300 were used for experiments in this article.

2. Sequence padding: Reviews are padded or truncated to a fixed length of 200 words.

1. Baseline models:

1. TF-IDF vectorization.

2. N-gram features: unigrams and bigrams.

3.3 Model architecture

For our research, the LSTM models were implemented in Python and utilized Keras with a TensorFlow backend using scikitlearn. Indeed, this has proven to be a very powerful and flexible framework to build up and train deep learning models. Table 1 represents the models that are implemented and compared.

The flowchart of using an LSTM model in sentiment analysis on an e-commerce platform is shown in Figure 2. This ranges from data collection to pre-processing, handling of the model (training or loading), prediction and aggregation, and finally storage and report on the results. In the process, each step and decision point is very important in ensuring accurate and insightful sentiment analysis.

Figure 2

Flowchart representing the working of the LSTM model for sentiment analysis.

First of all, the collection of user reviews is the fundamental step through which the raw data comes. These reviews give insight into customer experiences and form the input for sentiment analysis. After this, preprocessing is necessary for the preparation of text data cleaning and normalization. This makes the data suitable for processing by the LSTM model effectively. Apart from that, load a pretrained LSTM model from storage.

This model is trained on similar datasets to understand and predict sentiments. In the next stage, there would be a decision point where it would be determined whether the current LSTM model is sufficient or if it needs to be re-trained. This can be attributed to the drift in the patterns of the data, user behavior, or even architecture improvement; this is followed by splitting the data into training and validation sets, where training involves feeding the training data into the LSTM network to learn the patterns associated with different sentiments. After training, save the newly tuned model to disk for use next time. This will ensure the model is not trained every time sentiment analysis is done. In case no new training is required, this step would involve loading the existing model directly for making predictions. Make a prediction of the sentiment of every review using the loaded or trained LSTM model. The sentiment of every review is classified or scored. Following this are the individual sentiment predictions combined to return a single score or distribution of sentiments over the dataset. At this point, see if there is any need for an analysis deeper than simple predictions; this could be for detailed reports, identification of trends, etc. In case of need, detailed reports of sentiment analysis should then be created in summary form. Store the results of the sentiment analysis for future reference and use in the final stage. This includes raw predictions, aggregation of results, and any generated reports.

3.4 Hyperparameter tuning

In the present study, we performed a large number of experiments with multiple hyperparameter configurations to evaluate their effect on model accuracy and reduce overfitting. In particular, we experimented with 64, 128, and 256 LSTM units to minimize the trade-off between model complexity and error propagation generalization capability. The dataset was shuffled and split into mini-batches during training, controlled by the batch size hyperparameter. A performance evaluation yardstick was also run on the entire validation set at the end of each epoch step, in addition to the mini-batch updates. Once we identified the best hyperparameter configuration we tested the model theoretically being an independent test set, which through it we can provide a fair estimation of the models’ real performance in the live mode.

Further, the dropout rate is changed to 0.2, 0.3, and 0.4 to regularize the amount and hence avoid overfitting of the network. The batch size was varied to 32, 64, and 128 to see the effectiveness of the size over data while training. Different learning rates were tried, such as 0.001 and 0.0001, to further tune the magnitude of the weight updates of this model. In these LSTM models, we performed grid search optimization for the following hyper-parameters. The best configuration was chosen based on the performance on the validation set.

Number of LSTM units: [64, 128, 256]

Dropout rate: [0.2, 0.3, 0.4]

Batch size: [32, 64, 128]

Learning rate: [0.001, 0.0001]

3.5 Training and evaluation

The LSTM models were trained on the Adam optimizer, using categorical cross-entropy loss. In efforts to avoid overfitting, the early stopping was implemented; patience was even set to 5 epochs. To a maximum, models were trained for 50 epochs.

We used the following measures of evaluation:

1. Accuracy,

2. Precision, recall, and F1 score: macro-averaged, and

3. Confusion matrix.

We have also performed a qualitative analysis of the predictions made by both models in a subset of challenging cases, including reviews implying mixed sentiment and sarcastic comments.

3.6 Statistical analysis

By using statistical methods, we can analyze the performance difference of models experimented on E-commerce platforms for user sentiment analysis. We performed paired t-tests on the accuracy score results of 10-fold cross-validation to validate our findings. Here, the paired t-test has been adopted since it can directly and properly compare the performance metrics between two models over the same dataset by accounting for the paired nature of the data through the comparison of the models’ results on identical samples. This procedure helps us to identify whether the difference that we observe in model performance is statistically accurate than it just happened due to random variation. We chose the paired t-test since it is especially well-suited when we have two similar samples, in this context represented by performance metrics of different models on one dataset. With the above-mentioned test, we can find out whether is there a significant mean difference between pair observations from zero.

Our process involved the following steps:

1. Run cross-validation on each model with 10-fold for a series of accuracy scores.

2. For each model, compute the average and standard deviation of these scores.

3. Running paired t-tests between accuracy scores for pairs of models.

4. State a significance level, usually α = 0.05, with which the result is to be compared.

We performed statistical test and paired t-test on accuracy scores derived from 10-fold cross validation to analyze the performance difference of models. We used the paired t-test as it allowed a direct comparison of two models over the same folds while accounting for data being a pair.

The paired t-test formula used was

where t is the value of the t-statistic. d̄ is the mean difference between paired observations the parameter μ0 is the hypothesized mean difference; in our case, it is set to 0. sd represents the standard deviation of differences. n would be the number of pairs, which would be 10 in our setup with 10-fold cross-validation. Such tests return p-values, which are the probabilities of getting such differences in performance when the null hypothesis – no significant difference between models – is true. A p-value of less than 0.01 means that there is fairly great evidence against the null hypothesis; thus, evidence of the fact that the difference between performances in the bidirectional LSTM model and the other models compared – LSTM, SVM, Naive Bayes, and random forest – is statistically significant. This said the probability of getting this difference in performance just by chance is less than 1%. p-Values less than the selected level of significance indicate that performance differences between models are statistically significant, so we can make more confident conclusions about the relative effectiveness of the LSTM models for analyzing user sentiment about e-commerce platforms.

4.1 Model performance comparison

This section compares the performance of different models in sentiment analysis of user reviews from e-commerce. We compare the performance of five types of models: bidirectional LSTM, LSTM, SVM, Naive Bayes, and random forest. These metrics are summarized in Table 2, including accuracy, precision, recall, and the F1-score. For all performance metrics, the best results on the test dataset still appear in the bidirectional LSTM model. It has an accuracy as high as 92.1%. This was closely followed by the one-way LSTM with an accuracy of 91.3%. Compared with the deep learning model, traditional machine learning models performed pretty poorly: SVM, random forest, and Naive Bayes gave an accuracy of 86.5, 85.1, and 83.2%, respectively. In addition to accuracy, the bidirectional LSTM model performed very well in terms of precision, recall, and the F1 score. It was far ahead of other models in every respect. The LSTM model came second in ranking for all metrics.

In general, SVM did a little better than random forest and Naive Bayes. Confusion matrices also help us further into the models’ performance in capturing positive, negative, and neutral sentiments. These matrices show that LSTM-based models, especially bidirectional LSTM, are pretty good at capturing subtle variations in sentiment across e-commerce reviews as opposed to traditional machine learning models. These results underline the good performance of deep learning approaches, and more exactly of bidirectional LSTM, on the task of sentiment analysis of reviews in e-commerce platforms. This high performance of such models could be interpreted by their capacity to trace long-term dependencies and context in sequential data, which is very relevant for extracting sentiment from user reviews. The performance metrics that are normally used for comparing the different models of sentiment analysis for e-commerce user reviews are depicted in Figure 3. From the analysis, it can be claimed that the bidirectional LSTM models enhance the accuracy of sentiment analysis in e-commerce by 92.1%. This postulates that in future research, these models need more tuning for complexities such as sarcasm, mixture of sentiments, and implicit expressions. These in turn would ensure that the models were better placed to interpret customer sentiments correctly and therefore give more subtle and dependable analyses.

Figure 3

Bar chart representing accuracy, precision, recall, and F1 score values.

Such models, in practical applications, will yield better performances in recommendation systems by much finer laying out of customer preferences and sentiments. For instance, the better detection of sarcasm or mixed sentiments will avoid the misinterpretation of customer opinions that otherwise will result in unsatisfactory recommendations. Also, it will be very efficient to have automation in the analysis of customer feedback with much greater accuracy in detecting sentiments. This will allow businesses to identify the problems raised by customers and take rectification measures, which will allow businesses to provide greater satisfaction and improvement in service quality. Overall, these could very well lead to more personalized and responsive e-commerce platforms that would assure better customer engagement and satisfaction.

4.2 Confusion matrices and error analysis

Confusion matrices for the bidirectional LSTM and SVM models provide a comprehensive representation of their performance on binary sentiment classification tasks, offering valuable insights into each model’s strengths and areas for improvement. Results from the confusion matrices show that LSTM-based models are of high performance in the classification of positive and negative sentiments. Noted herein is that the bidirectional LSTM model was better at classifying neutral sentiment, which is normally elusive and highly evasive to classify.

The results can be qualitatively analyzed to show that while the bidirectional LSTM model generally works quite decently on clear sentiment expressions, it fails in very sarcastic comments and reviews about two completely different and contrasting views of a product or service. In the error analysis, several main sources of misclassification were categorized, including the following: (1) sarcasm and irony detection, (2) mixed sentiments, (3) implicit sentiments, (4) domain-specific terminology, and (5) negation handling.

These findings underscore the strength of bidirectional LSTM in catching subtle sentiments against traditional machine learning approaches like SVMs. Figure 4 shows confusion matrices for the bidirectional LSTM model and the baseline model with the best performance among the others, which was the SVM model.

Figure 4

Confusion matrix comparison of bidirectional LSTM and SVM models for sentiment classification.

The confusion matrices show that both models are pretty good at classifying positive and negative sentiments but bidirectional LSTM performs much better in correctly classifying the neutral sentiment, which is usually a challenging sentiment to identify.

4.3 Learning curves

The graph shows the learning curves for two different models during training: bidirectional LSTM and unidirectional LSTM (refer to Figure 5). The epochs run on the x-axis, whereas on the y-axis, there are performances, which could be, for example, accuracy, loss, or any other evaluation metric in consideration. The performance of the bidirectional LSTM improves steadily over time, starting at about 0.8 and then converging at about 0.95 after 30 epochs. Its curve is rather smooth, which means it has learned stably without many ups and downs. During training, the former performed better than the latter all the time.

Figure 5

Learning curves of bidirectional LSTM and unidirectional LSTM models during training.

Whereas the unidirectional LSTM model improves over time but does so at a pace that is slower than the bidirectional LSTM. The performance starts from 0.75 and then converges to 0.85 by 30 epochs. Its learning curve is smoother than that of the bidirectional LSTM, yet it shows lagging performance all the time. Both models seem to converge around the 30-epoch mark, indicating additional training beyond that point doesn’t move performance much. This is an indication that both models have learned trends in training data and are giving their best results, based on parameters and architecture used. These learning curves suggest that while both models perform better with increased training, the bidirectional LSTM has been performing better than the unidirectional LSTM in the course of training; hence, it is always the better choice where high accuracy and context awareness are required on sequential data.

4.4 Hyperparameter optimization results

Table 3 shows the best settings for the hyperparameters of the LSTM and bidirectional LSTM models. The process of tuning allowed the establishments of settings that improved the model performance by larger ends. The best configuration for the LSTM model is an architecture consisting of 128 LSTM units of dropout 0.3, a batch size of 64, and a learning rate of 0.001. This struck a balance in the degree of complexity of the model and generalization – to allow effective feature extraction but without overfitting.

The bidirectional LSTM model showed very similar preferences to its LSTM units and dropout rate, performing best with 128 units at a 0.3 dropout rate. It did, however, benefit from a reduced learning rate of 0.0001, which indicates that the bidirectional architecture requires more careful parameter updates to converge effectively. The best performance by both models with a moderate number of 128 LSTM units thus shows that it is in this range that enough capacity is provided to capture relevant patterns in the e-commerce sentiment data without unnecessary complexity. On the other hand, the optimal dropout rate of 0.3 across both models could be an indication of the right level of regularization that does not allow overfitting and yet stays very expressive. It outperformed all other models, including the unidirectional LSTM. Paired t-tests proved that the performance differences are statistically significant at p < 0.01, thus implying that bidirectional processing is particularly suited for sentiment analysis tasks within e-commerce contexts. Indeed, the findings stipulate that LSTM-based models have huge potential for sentiment analysis if hyper-parameters are carefully tuned. These optimal configurations provide important clues to researchers and practitioners working on such problems within the e-commerce domain.

4.5 Statistical significance and qualitative analysis

We used paired samples t-tests to compare the bidirectional LSTM model with every other model: LSTM, SVM, Naive Bayes, and random forest. The null hypothesis in all cases was that between the bidirectional LSTM and the compared model, there is no relevant difference in performance. The statistically significant differences (p < 0.01) in the performance of the bidirectional LSTM against all the other models have been brought out by the results of our statistical analysis. This allows us to confirm that the differences between the bidirectional LSTM model and the others are not, in fact, random fluctuations or variations in the data. Both the previous findings have confirmed that the proposed bidirectional LSTM architecture for a sentiment analysis task might indeed capture quite complex patterns and dependencies in text data that classical machine learning methods or even usual LSTM models would have ignored. The statistical significance of the differences in performances highlights the practical importance of the choice of the bidirectional LSTM in applications of sentiment analysis where high performance is a key factor.

To help understand the performance of the models on the challenging cases, we conducted a qualitative analysis of predictions on a subset of reviews. Table 4 shows examples of correctly classified and misclassified reviews for the bidirectional LSTM model.

The qualitative analysis explains that the bidirectional LSTM performs well in reviews with clearly indicated sentiment expressions and it is able to capture some cases of mixed sentiment. It misclassifies highly sarcastic comments, and it also misclassifies reviews that express conflicting opinions about different aspects of a product or service.

More specifically, the authors identify a few limitations with the described hybrid Faster-RCNN technique. However, its performance decreases in crowded scene (complex overlaps and occlusions), with a large number of false negatives (missed object) and reduced recall, which is one of the important drawbacks. And even worse, misclassifications take place in visually similar object categories, reducing the precision of the models like with animal species or vehicle types. This points to future improvements needed with techniques that could enhance differentiation such as attention mechanisms and multi-scale feature extraction.

Nonetheless, the methodology has big advantages compared to other ones. It performs better in terms of object detection accuracy than baseline Faster-RCNN and the previous models: YOLO v4, SSD, and Mask R-CNN with a mAP improvement of 18.7, as well as achieves better temporal coherence through adaptive feature fusion. The computational efficiency is still relatively high, which leads to a 32.4% processing time reduction over Faster-RCNN, turning it into a candidate for real-time video analysis. Similarly, the hybrid model is also very solid on object tracking performance, significantly boosting both tracking accuracy and search precision with its modular design.

4.6 Error analysis

To better understand the limitations of our models, we conducted an error analysis on misclassified reviews. The main sources of errors were:

Sarcasm and irony (22% of errors),

Mixed sentiments (18% of errors),

Implicit sentiments (15% of errors),

Domain-specific terminology (12% of errors),

Negation handling (10% of errors), and

Others (23% of errors).

Figure 6 illustrates the pie chart representation of the sources of errors in sentiment analysis. According to the pie chart, the largest single category of errors was sarcasm and irony at 22%, very closely followed by “others” at 23%. This means that a large bulk of errors in the text come from the detection and interpretation of sarcastic or ironic statements correctly.

Figure 6

Pie chart illustrating the distribution of sources of errors in sentiment analysis.

Mixed sentiments account for 18% of the errors, hence the difficulty in clearly identifying texts containing multiple or even opposite feelings. Implicit sentiments have 15% errors, so it is hard to identify those sentiments that are not stated but are implied in the context. Domain-specific terminology has 12% of the errors; hence, the need to tune sentiment analysis models to specific industries or fields of discourse. Negation handling, at 10%, shows that this remains one of the main challenges in the proper deciphering of statements containing negation, which may turn around the sentiment of a sentence very drastically. The “others” category, which forms 23% of errors, indicates some other less frequent or unclassified sources of error that together add up to form a substantial share of challenges in sentiment analysis.

In light of the fact that the bidirectional LSTM model has reached an accuracy as high as 92.1%, it goes without saying that it could be put to work in real-world e-commerce applications, such as the following:

1. Automated filtering and moderation of reviews,

2. Improved recommendation systems for products,

3. Real-time sentiment tracking for customer service, and

4. Trend analysis with improvement insights for products.

However, while applying these applications, one should not forget the limitations of the model on handling sarcasm and mixed sentiments. Probably in the most critical decision-making processes, a hybrid approach shall be needed, wherein the machine learning models are accompanied by human supervision.

The preprocessing steps involved included the removal of HTML tags, special characters, non-ASCII characters, and stop words. Besides, the researchers had lowercased and lemmatized the input data to normalize it. For specific challenges like sarcasm and mixed sentiments, the researchers, while identifying these complexities, did focus on the effectiveness of the bidirectional LSTM model in capturing subtle variations in sentiment. This article mentioned that the model did not do well in the case of sarcasm or mixed sentiments, hence misclassifying them. On top of that, the errors analyzed included 22% from sarcasm or irony and 18% from mixed sentiment. This points out that research and development regarding this linguistic phenomenon is still required.

Including several models into one ensemble may significantly enhance the robustness of sentiment classification. For instance, ensemble bidirectional LSTM with CNN can take advantage of both temporal and spatial features in the input data. The integration of the models specializing in either sarcasm detection or sentiment analysis in a different domain can help to mitigate some of the identified errors.

As shown in the error analysis technique of the hybrid Faster-RCNN model, there were two key errors – false negatives and misclassifications. In complex scenes with many overlaps and heavy occlusion, many objects cannot be correctly classified leading to a significant drop in recall, manifesting itself as false negatives predominately in crowded scenes. The same goes for visually similar objects (e.g., to differentiate between certain animal species or vehicle types) which leads to misclassifications and lower precision of the model. These errors were explained via representations given in Figure 9. Utilization of more advanced techniques like attention mechanism, multi-scale feature extraction, and fine-grained classification on the additional datasets is a prospect in the future works with respect to this work.

Feeding the hybrid Faster-RCNN model with enhanced video analysis capability could provide a better customer experience along with enhanced customer decision-making within e-commerce via superior product discovery and personalized recommendations. For example, this model has translated into a 41.2% increase in video search accuracy at a 32.4% lower computational cost, where businesses could use this model to deliver a more accurate and faster search over video-based product catalogs to increase customer satisfaction. It manages to achieve impressive F1 scores (0.937 for object detection and 0.912 for tracking), enabling accurate tracking of products in a video stream, which can help in real-time inventory management or product live showcase. Additionally, the scalability and generalization abilities of the model facilitate the use of video content on e-commerce platforms over a wide range of video resolutions and network conditions, enhancing accessibility and engagement [9,10,20]. These innovations enable businesses to drive intelligent decision-making, improve operational performance, and offer more personalized shopping experiences.