Machine learning-assisted spectrometric method for pulp extractive analysis based on model pulps

2.1 Reagents and materials

Acetone (≥99.8 %), heneicosanoic acid (99 %), cholesteryl heptadecanoate (≥95 %), 1,3-dipalmitoyl-2-oleyl glycerol (99 %), cholesterol (≥99 %) were purchased from Sigma-Aldrich (St. Louis, MO, USA) and stearic acid (99 %) from Fluka Chemie GmbH, Sigma-Aldrich (Buchs, Switzerland), while glacial acetic acid (≥99.7 %) and pyridine (≥99.5 %) from Fisher Scientific (Loughborough, UK). Chlorotrimethylsilane (TMCS, 98 %) and N,O-Bis-(trimethylsilyl)trifluoroacetamide (BSTFA, ≥99 %) were obtained from Acros Organics (Geel, Belgium) and Apollo Scientific (Bredbury, UK) respectively. Distilled and deionized water (resistivity 18.2 MΩm) was prepared using ELGA Purelab Ultra purification system (ELGA LabWater, Lane End, UK). Betulinol (≥99.9 %) was isolated from outer birch bark and purified through several re-crystallizations. The bases used for preparing the model pulps were: Machery-Nagel 640d quantitative filter paper (<0.01 % ash content) and cellulose sheets made of softwood dissolving pulp provided by the forest industry.

2.2 Methods

2.2.1 Preparation of model pulps

The model pulps contained only cellulose and either one or both of the chosen extractive model compounds (betulinol and stearic acid). The concentrations of the extractives in model pulps were higher by several orders of magnitude compared to extractives concentrations in industrially sourced pulps.

Thirty five model pulp samples were prepared, where either a Machery-Nagel quantitative filter paper or cellulose sheets were used as bases. The fabrication included cutting the base into approximately 5 × 5 cm pieces, placing them in beakers and spiking with solution of one or both extractive model compounds in acetone (concentration range of the stearic acid solutions: 0.04–10 mg/mL; betulinol solutions: 0.025–2.5 mg/mL). The solvent was then removed through evaporation. The composition of used samples is illustrated in Figure 1.

Figure 1:

Types of model pulp samples.

2.3 Spectroscopic measurements

2.4 Spectra preprocessing

It was necessary to perform preprocessing of both NIR and Raman spectra to effectively extract the chemical data from them. In NIR, the main issues are baseline shift, scattering and non-linearities and the data needs to be transformed, so that it follows Lambert–Beer’s law. One of the most suitable methods for pretreatment of NIR spectra is Savitzky–Golay filter, which smooths the spectra and performs differentiation of the data to reduce the multiplicative and additive effects without reducing the signal-to-noise ratio. Several normalization methods such as standard normal variate (SNV), multiplicative scatter correction, mean-centering or Euclidean distances are used to remove the baseline shifts and variability caused by samples (Peris-Díaz and Krężel 2021).

In Raman spectroscopy it is necessary to mitigate the effects caused by Raileigh scattering, fluorescence and the experimental conditions, in particular the presence of cosmic radiation. These result in baseline drift, peak shifting, random noise, cosmic spikes and poor signal-to-noise ratio. The main strategies of reducing these effects in Raman spectra include intensity and wavenumber axis calibrations, removal of baseline and cosmic peaks, smoothing through Savitzky–Golay filter and various normalization techniques (SNV, vector, area or min–max normalization) (Mostafapour et al. 2023). Peaks in Raman spectroscopy caused by cosmic rays were removed and baseline correction was applied using WiRE software (Renishaw, Wotton-under-Edge, UK). The spectra of reference samples were subtracted ahead of the data analysis.

The following spectra preprocessing methods were investigated: Savitzky–Golay filter (0, 1st, 2nd and 3rd derivatives; window 5) and normalization (SNV, vector, min-max, area). The most suitable method for each dataset was chosen based on the lowest relative standard error of cross-validation and prediction (RSE, Equation (1)) (Goodarzi et al. 2009) and highest coefficient of determination for cross-validation and prediction (R², Equation (2)). (Renaud and Victoria-Feser 2010) in predictive modelling of extractive compounds.

where: n - dataset size; y_i - observed value; y ˆ i - predicted value; y ‾ - mean observed value.

Orange Data Mining software version 3.36.2 (University of Ljubljana, Ljubljana, Slovenia) was used to apply these preprocessing methods to the NIR and Raman spectra.

The spectra used for building the classification models were not normalized or treated with Savitzky–Golay filter, as spectral pretreatment is not particularly essential for classification (Geladi and Dåbakk 2017).

2.5 Machine learning

Orange Data Mining software version 3.36.2 (University of Ljubljana, Ljubljana, Slovenia) was used for machine learning.

The spectral data was divided into subsets displayed in Table 1, to investigate how the following factors influence the predictive models: (a) using two different bases to fabricate the model pulps and (b) adding either one or both extractive compounds to the bases.

Reference samples were not used in the calibrations or classifications based on the Raman spectra, because all model pulp spectra were pretreated by subtracting the spectra of reference bases.

Additionally, two data subsets were used for semi-quantitative prediction of betulinol and stearic acid contents in the cellulose sheet-based model pulps (Table 2). In this case the Raman spectra were not pretreated by subtracting the spectra of reference samples – the reference samples were included in the classification.

The following classification models were used and compared: artificial neural network (ANN), logistic regression (LR), random forest (RF), naïve Bayes (NB) and k-nearest neighbors (k-NN). The models were subjected to a threefold stratified cross-validation. The best model for each dataset was chosen according to the highest classification accuracy (CA) score, which is the ratio of correct predictions (Opitz 2024).

When building a calibration model, partial least squares (PLS), linear regression (LnR) and supporting vector machines (SVM) algorithms were examined. After performing the threefold cross-validation, the best model for each dataset was selected on the basis of lowest root standard error of cross-validation (RSECV) and highest coefficient of determination for cross-validation (R_CV²). The spectroscopic data was also split into calibration set and test set in 3:2 proportion and the training and testing sequence was repeated 10 times to examine the predictive ability of the models. The best predictive ability was determined by the highest coefficient of determination of prediction (R _P ²) and lowest root standard error of prediction (RSEP).

3.1 GC-FID extractive analysis

The GC-FID analysis of filter paper and cellulose sheet reference samples revealed that they contained 63.04 and 67.74 mg/kg of impurities, respectively. The content ranges and mean content of stearic acid and betulinol in the model pulp samples are shown in Table 3.

3.2 Spectroscopic measurements

3.2.1 Near infrared spectroscopy

The bands in spectra of betulinol and stearic acid considerably differ from the other spectra (Figure 2). Both have absorption bands around 1,200 nm (C–H), 1,390 nm (O–H and C–H) and 1720 (C–H). Additionally, in the stearic acid spectrum the bands at 930 nm and 1,030 nm caused by C–H are observed and the band from O–H and C–H at 2,100 nm appears in the betulinol spectrum (Streamer 2013).

Figure 2:

Averaged raw NIR absorption spectra of model compound powders (betulinol and stearic acid), filter paper, cellulose sheet and different model pulp types (F - filter paper base; C - cellulose sheet base; S - stearic acid added; B - betulinol added).

There are no significant differences between the spectra of filter paper, cellulose sheet and all model pulps. The absorption band at 1,220 nm can be attributed to C–H stretching, while the band at 1,490 is most probably related to the O–H stretching in cellulose. In the 1st overtone region, there are two absorption bands: the weak one at 1780 is caused by C–H stretch in cellulose and H–O–H deformation in water, while the one at 1930 by O–H in water and by C=O. In the combination region 1900–2,500 nm the bands are a result of overlapping – the band at 2,100 nm is caused by O–H and C–H stretching in cellulose and polysaccharides (Fardim et al. 2002; Gierlinger et al. 2002; Hayes 2011; Li et al. 2015). It is impossible to directly observe the absorption bands in the model pulp spectra which are directly caused by the addition of betulinol and stearic acid, because the spectra of all model types are very similar to the reference samples.

3.2.2 Raman spectroscopy

Figure 3 shows that the spectrum of betulinol yields much weaker Raman effect than the model pulps, filter paper or cellulose sheets. Many small peaks between 1,640 cm⁻¹ and 150 cm⁻¹ were observed in the betulinol spectrum. The spectrum of stearic acid is typical for long-chain saturated fatty acids with bands at around 1,050 cm⁻¹ and 1,150 cm⁻¹ related to C–C stretching, a very intense band at 1,300 cm⁻¹ caused by CH₂ twisting and a cluster of peaks between 1,400 and 1,500 cm⁻¹ resulting from CH₂ wagging, deformation and scissoring or CH₃ deformation (Saggu et al. 2015). The broad peak at 2,900 cm⁻¹ appears in every spectrum and is related to C–H or C–H₂ stretching in cellulose, alkyl chain of stearic acid or in the rings of betulinol. The same peak groups can be observed in all spectra of model pulps as well as in the filter paper and in the cellulose reference spectra. The peak group around 380 cm⁻¹ can be seen in every model pulp and base spectrum, which suggests that it originates from cellulose. The group at 1,380 cm⁻¹ is a result of C–H₂ or C–H₃ deformation or C–O–H bending. The peaks observed at 1,100 cm⁻¹ can be caused by C–C skeletal mode vibrations, C–O–C glycosidic vibration, C–O stretching or C–C stretching from either cellulose or polysaccharides (Belt et al. 2017; Dou et al. 2023; Ibrahim et al. 1997; Ji et al. 2013).

Figure 3:

Averaged Raman spectra of model compound powders (betulinol and stearic acid), filter paper, cellulose sheet and different model pulp types (F - filter paper base; C - cellulose sheet base; S - stearic acid added; B - betulinol added). The spectra pretreatment included baseline correction and cosmic peak removal.

3.3 Machine learning

3.3.1 Classification

The qualitative classification was performed on data subsets as shown in Table 1, while the samples used for semi-quantitative classification are listed in Table 2.

3.3.1.1 Qualitative classification

Table 4 depicts the results of classification of samples in five subsets depending on the classification algorithm used. The aim was to classify the samples into the following categories based on the spectra: model pulps containing betulinol, model pulps containing stearic acid, model pulps containing both extractive compounds, references (and whether it was a filter paper or a cellulose sheet in subsets 4 and 5; only applies to NIR).

Table 4:

Classification accuracy scores of five different algorithms used to classify model pulp samples within five subsets based on NIR and Raman spectra.

	Sample subset	Classification accuracy (%)
		NIR					Raman
		k-NN	LR	NB	ANN	RF	k-NN	LR	NB	ANN	RF
1.	Filter paper-based model pulps – samples containing one extractive compound	92.4	92.4	42.4	91.3	91.3	86.0	86.0	76.0	80.0	92.0
2.	Filter paper-based model pulps – mix of samples containing one extractive compound or two extractive compounds	87.8	89.2	43.9	86.5	84.5	71.4	80.5	62.3	68.8	74.0
3.	Cellulose sheet-based model pulps – samples containing one extractive compound	63.9	73.2	37.1	81.4	70.0	62.9	68.6	68.6	65.7	62.9
4.	All model pulps – samples containing one extractive compound	80.4	82.5	22.8	83.1	78.8	74.1	75.3	61.2	76.5	71.8
5.	All model pulps – mix of samples containing one extractive compound or two extractive compounds	85.7	86.9	27.3	84.9	83.3	59.8	70.5	53.6	64.3	64.3

1. k-NN - k-nearest neighbors; LR - logistic regression; NB - naïve Bayes; ANN - artificial neural network; RF - random forest.

According to Table 4, naïve Bayes was generally the worst performing method both in classification based on NIR spectra and Raman spectra. The algorithm which most successfully classified the samples based on either type of spectra was usually logistic regression.

In Figure 4, the details on performance of the best algorithm in each subset can be seen for classifications based on both types of spectroscopies. According to Table 4 and Figure 4, on average, the first sample subset had the highest scores of CA for both types of spectroscopies, which can be attributed to the fact that the first subset contained one type of reference and did not contain any samples which were spiked with both extractive compounds. Interestingly, the third subset also contained only one type of reference and no model pulps spiked with both extractives, but on average the CA in this subset was the lowest out of all subsets both based on NIR and Raman spectra. The reason might be that the cellulose sheets contained more impurities than the filter paper, which could negatively affect the outcome of classification. This data subset also had the most variation, according to the mean absolute deviation values in Table 3. Another interesting tendency was that for NIR the classification in the last subset was more successful than in the fourth subset, even though the last subset contained one more category (the model pulps spiked with both betulinol and stearic acid). Overall, the classifications based on NIR were always more accurate than those based on Raman, which could be related to the fact that fewer spectra were used for the Raman classifications. On the other hand, the NIR classifications always contained one or two more categories for reference samples.

Figure 4:

Graphs illustrating the details of most successful classification in each sample subset for both spectroscopic methods. F - filter paper base; C - cellulose sheet base; S - stearic acid added; B - betulinol added.

3.3.1.2 Semi-quantitative classification

To develop a semi quantitative model, the model pulps produced with the cellulose sheet bases were divided into two subsets: model pulps, where different amounts of betulinol were added to the samples and model pulps containing different amounts of stearic acid. Additionally, reference samples were included in each subset (see Table 2). The goal was to check if the algorithms are capable of distinguishing between samples containing different quantities of the same model extractive compound. The results of the classification are shown in Table 5.

Table 5:

Comparison of results of semi-quantitative classification of model pulps and reference samples within two data subsets using five different algorithms and two spectroscopic methods.

	Sample subset	Classification accuracy (%)
		NIR					Raman
		k-NN	LR	NB	ANN	RF	k-NN	LR	NB	ANN	RF
1.	Cellulose sheet-base model pulps containing betulinol	63.0	77.8	81.5	70.4	66.7	9.1	9.1	36.4	18.2	0.0
2.	Cellulose sheet-base model pulps containing stearic acid	76.3	73.7	86.8	89.5	84.2	23.5	23.5	29.4	23.5	5.9

1. k-NN - k-nearest neighbors; LR - logistic regression; NB - naïve Bayes; ANN - artificial neural network; RF - random forest.

The CA for the classification based on NIR spectra was significantly higher than based on Raman spectra, but it can be partially caused by the lower number of Raman spectra used for building the models. Interestingly, the best CA for the samples containing different quantities of betulinol was obtained when applying NB for both types of spectroscopies, even though this algorithm performed the worst when used for qualitative classification. Meanwhile, the classification of samples containing stearic acid was the most successful when ANN and NB were used on NIR and Raman spectral data, respectively. The performance of models was usually better when classifying the samples in the second subset.

In Figure 5a, it can be seen that the NB algorithm was always correct when classifying the reference and the sample containing the highest quantity of betulinol. The sample B1 was misclassified half of the time – as either B2 or B3, and the sample B2 was misclassified twice as B3.

Figure 5:

Details of classification by the best performing model for each sample subset and concentrations of extractive compounds added to the model pulps based on NIR spectra. Classification of betulinol-containing model pulps using naïve Bayes (a) and stearic acid-containing model pulps using artificial neural network (b).

As displayed by Figure 5b, it is noticeable that misclassification happened only four times – S4 was misclassified twice: as S1 and S2, while S3 was misclassified as S4 once and S1 was misclassified as S2 once. Surprisingly, the CA of samples containing stearic acid is higher than CA of samples containing betulinol (89.5 % and 81.5 % respectively), despite the fact that there are two more categories in SA-containing sample subset and that there is no large difference in concentrations of the extractive compound between S4 and S5.

3.4 Calibration

In order to choose the optimal combination of pretreatment methods and algorithm for building the model for prediction of extractive content in model pulps, all possible combinations for every sample subset and analyte were tested. The best approach was chosen based on the highest R_CV² and lowest RSECV.

Based on Table 6, it can be noticed that the optimal pretreatment method almost always included normalization combined with Savitzky–Golay filter. In case of NIR, each normalization method was optimal for at least one of the calibrations, while in Raman SNV performed the best in most calibrations. Using Savitzky–Golay filter to smoothen and derivate the spectra was beneficial for all calibration sets and, in the calibrations, based on NIR spectra usually the 2nd or 3rd derivative led to the best results, while 1st derivative was the optimal choice in three calibrations. In Raman-based calibrations S1D is clearly the most favorable smoothing and derivatization method.

PLS was the best-performing algorithm in all calibrations based on NIR spectra, while LnR rather than PLS led to the most accurate prediction of extractives based on Raman spectra most of the time.

The only case, in which it was not possible at all to correlate the spectral data with extractive content was in the third set in the attempt of calibrating betulinol content in the cellulose sheet-based model pulps based on Raman spectroscopy.

Table 6 also summarizes the results of predictive modelling of extractives in model pulp sample subsets. The coefficient of determination (R²) and root standard error (RSE) of cross-validation and prediction are depicted only for the best performing model in each calibration. Generally, high R² and low RSE indicate that the model can accurately predict the extractive content in the model pulp samples.

As seen in Table 6, all values of R² for all calibrations are between 0 and 0.78, while RMSE is between 0.35 and 0.86. The highest R² and lowest RSE values were obtained in the NIR-based prediction of total extractive content in the 2nd subset, which contained filter-paper models spiked with either betulinol, stearic acid or both compounds. Meanwhile, the calibration of betulinol in the 2nd sample subset based on Raman yielded the lowest R² (0.11 and 0.01 for cross-validation and prediction, respectively). The biggest errors of prediction and of cross-validation, 0.86 and 0.85 respectively, were a result of calibrating the stearic acid content in the 4th sample subset based on Raman.

NIR performed better in the 1st, 2nd and 3rd subsets, while the calibrations in 4th and 5th subset were more accurate when based on Raman. This indicates that Raman could be less affected by the base used for fabricating the model pulps and more selective towards the analytes. The predictions of betulinol content and total extractives in the sample were usually more accurate if based on NIR, but the predictions of stearic acid were more accurate if the model was built using Raman spectra. The reason for that could be that NIR is more sensitive to the two hydroxyl groups in betulinol, while Raman detects the long aliphatic chain of stearic acid better. It also seems that selectivity of NIR towards total extractive content is significantly better than towards betulinol or stearic acid separately.