Deep learning application for stellar parameter determination: III-denoising procedure

1 Introduction

Observations in astronomy have always been associated with noise. Trying to minimize the noise is one of the needs of astronomers. Several observation techniques have been suggested to reduce the noise in spectra; however, once the observation is performed, the only way to proceed is to apply mathematical algorithms that can improve the signal-to-noise ratio (SNR) of the data. These techniques involve but are not limited to Gaussian smoothing (Chung 2020), median filtering (Kumar and Sodhi 2020), wavelet denoising (Halidou et al., 2023), and principal component analysis (PCA) (Bacchelli and Papi 2006, Zhang et al., 2010, Murali et al., 2012, Li 2018). More recently, and with the advancement of computational power, deep learning algorithms started to be used for that purpose. Gheller and Vazza (2022) used a convolutional denoising autoencoder to decrease the noise of synthetic images of state-of-the-art radio telescopes like LOFAR (Offringa et al., 2013), MeerKAT (Jonas 2009), and MWA (Tingay et al., 2013). The technique was applied to different kinds of corrupted input images. The autoencoder was able to effectively denoise images identifying and extracting faint objects at the limits of the instrumental sensitivity. The authors state that their autoencoder was capable of removing noise while preserving the properties of the regions of the sources with SNR as low as 1. Scourfield et al., (2023) used a variational autoencoder to denoise optical SDSS spectra of galaxies (York et al., 2000). Their main goal was to denoise the spectra while keeping the important information they can retrieve from low SNR galaxy spectra and avoiding the use of sample averaging methods (smoothing or spectral stacking). They tested the method in the context of large optical spectroscopy surveys by simulating a population of spectra with noise to mimic the ones at galaxies at a redshift of z = 0.1 . Their results showed that the technique can recover the shape and scatter of the mass-metallicity relation in this sample.

In this work, we introduce two types of spectral denoising techniques, autoencoders (Ballard 1987, Baldi 2011) and PCA (Wold et al., 1987, Makiewicz and Ratajczak 1993). We test the need of the denoising technique on the derived stellar parameters: effective temperature T eff , surface gravity log g , equatorial projected rotational velocity v e sin i , microturbulence velocity ξ t , and the overall metallicity [M/H]. These stellar parameters are derived using the neural network introduced our previous work (Gebran et al., 2022, Gebran et al., 2023, Gebran 2024). The article is organized as follows: Section 2 introduces the calculation of both datasets and noisy spectra, Section 3 explains the autoencoder construction used in the denoising procedure, and Section 4 describes the denoising technique using PCA. Section 5 shows the results of the denoising technique using both procedures and the effect on the derived stellar parameter accuracy values. Finally, we conclude in Section 6.

2 Datasets

Two datasets were used in the context of the present study. The one analyzed in Gebran et al., (2023), and the one of Gebran (2024). The characteristics of these two datasets are described in Table 1. The reason for selecting these diverse datasets is to check the procedure over different wavelength ranges and different resolving power.

The steps of calculating the datasets are detailed in Gebran et al., (2022), Gebran et al., (2023), and Gebran (2024). In summary, line-blanketed model atmospheres are calculated using ATLAS9 (Kurucz 1992). The models are plane parallel and in local thermodynamic equilibrium (LTE). They are in hydrostatic and radiative equilibrium. We have calculated the models using the opacity distribution function (ODF) of Castelli and Kurucz (2003). Convection was included according to Smalley’s (2004) prescriptions. Convection is included in the atmospheres of stars cooler than 8,500 K using the mixing length theory. A mixing length parameter of 0.5 was used for 7,000 K ≤ T eff ≤ 8,500 K , and 1.25 for T eff ≤ 7,000 K .

We have used the radiative transfer code SYNSPEC (Hubeny and Lanz 2017) to calculate the synthetic spectra. As mentioned previously, two datasets were calculated with each one containing around 200,000 spectra. In both datasets, metal abundances were scaled with respect to the Grevesse and Sauval (1998) solar value from − 1.5 dex up to +1.5 dex. The effective temperature, surface gravity, projected equatorial velocity, and microturbulence velocity were also modified according to the values displayed in Table 1. The first dataset consists of spectra having a resolution of 42,000 and a wavelength range between 4,450 and 5,400 Å. As explained in Gebran et al., (2022, 2023), this wavelength range is sensitive to all stellar parameters in the spectral range of AFGK stars. The second dataset has spectra computed between 8,400 and 8,800Å at a resolution of 11,500. This region includes the Gaia radial velocity spectrometer (RVS, Cropper et al., 2018). The RVS spectra contain lines sensitive to the stellar parameters and to the chemical abundance of many metals (Mg, Si, Ca, Ti, Cr, Fe, Ni, and Zr, among others) at different ionization stages. The linelist used in this work is the one used in Gebran et al., (2022) and Gebran et al., (2023). It contains updated values for the atomic parameters such as the wavelength of the transitions, the oscillator strengths, the damping constants, and others.

In summary, we ended up with two datasets of around 200,000 synthetic spectra each, with T eff , log g , v e sin i , [M/H], and ξ t randomly chosen from Table 1. Figure 1 shows a color map of a sub-sample of the datasets. The Balmer line is detected in the left color map for dataset 1, and the absorption lines of the calcium triplet ( λ = 8,498, 8,542, 8,662 Å) are also shown in the color map of dataset 2 in the bottom part of the figure.

Figure 1

Color map representing the fluxes for a sample spectra of the two training datasets. The left color map represents dataset 1 and the right one represents dataset 2. The y -label represents the number of spectra, N.

For each dataset, a set of spectra were calculated with random Gaussian noise between 5 and 300. This SNR is used to mimic the noisy observations that we will be denoising later on as they represent the average SNR encountered in real stellar spectra. An example of a spectrum calculated with and without noise in the parameter range of dataset 2 is shown in Figure 2.

Figure 2

Example of a spectrum of dataset 2 calculated using a random selection of stellar parameters from Table 1. The black spectrum represents the synthetic spectrum calculated without noise, and the blue one corresponds to the same parameters but with an SNR of 19. The stellar parameters of the spectrum are 14,550 K, 3.05 dex, 44 km s ‒ 1 , ‒ 1.15 dex, and 3 km s ‒ 1 for T eff , log g , v e sin i , [M/H], and ξ t , respectively.

3 Auto-encoders

Autoencoders, usually used in denoising and dimensionality reduction techniques (Lecun 1987, Fogelman Soulie et al., 1987, Ballard 1987, Baldi 2011, Schmidhuber 2014, Einig et al., 2023, Scourfield et al., 2023), are a type of neural networks that work in an unsupervised way. They consist of two distinct yet similar algorithms, an encoder and a decoder. The encoder’s role is to transform the spectra from a dimension of N λ flux point to a smaller size of N latent inside a latent space. The decoder re-transform the N latent to the original spectrum of N λ flux point. The choice of N latent depends on the characteristics of the dataset. However, using the two datasets in this work, we found that the optimal size for the latent space is N latent = 10 . This is found by minimizing the difference between the output spectra and the input one during the training process. It is true that different values of N latent could be used, but our choice of N latent was based on the smallest value that gives a reconstruction error less than 0.5% as will be explained in the next steps.

The classical architecture of an autoencoder is shown in Figure 3 where the initial spectrum is introduced having 19,000 or 4,000 data points depending on the dataset and is then reduced to N latent points through successive hidden layers. This first step defines the encoder part of the autoencoder. Then, the N latent points are transformed to 19,000 or 4,000 data points while passing through different hidden layers. This second step defines the Decoder part of the autoencoder. The hidden layers are usually symmetrical in the encoder and decoder parts.

Figure 3

Sketch of an autoencoder that transforms an input spectrum of N λ data point to a lower dimension using a series of hidden layers (encoder). The middle layer is the latent space. The decoder reconstructs the spectrum to its original dimension.

Two autoencoders were used in this work, one for each dataset. In both cases, the spectra are reduced to ten parameters in the latent space. The architecture of the used autoencoders is displayed in Table 2. We have used an Adam optimizer with a mean squared error (MSE) loss function.

Calculations were performed using TensorFlow ^[1] with the Keras ^[2] interface and were written in Python.

The training of the autoencoders was performed using the two datasets containing the synthetic spectra with no noise. The convergence is achieved when the difference between the output and the input spectra is minimized through the MSE. Convergence usually occurs after around 500 epochs. For both datasets, we achieved an R 2 score larger than 0.995, meaning that the reconstruction of the spectra is performed with an error < 0.5 % . Once the training is done, the denoising is performed when the trained autoencoders are applied to the noisy spectra.

4 PCA

PCA is a non-parametric mathematical transformation that extracts relevant information from a dataset (Wold et al., 1987, Makiewicz and Ratajczak 1993). Its goal is to compute the most meaningful basis to represent a noisy dataset. The new basis usually reveals hidden structure and filters out the noise (Shlens 2014). PCA has been used for denoising (Bacchelli and Papi 2006, Zhang et al., 2010, Murali et al., 2012, Li 2018) or spectral dimension reduction (Makiewicz and Ratajczak 1993, Paletou et al., 2015a, Gebran et al., 2016, Gebran et al., 2022, Gebran et al., 2023). The main power of PCA is that it can reduce the dimension of the data while maintaining significant patterns and trends.

The basic idea behind the use of PCA is to derive a small number of eigenvectors and use them to recover the information in the spectra. The steps of PCA calculation are

1. The matrix containing the training dataset has N λ flux points per spectrum; therefore, the dataset can then be represented by a matrix M of size N spectra × N λ , where N spectra represents the number of spectra in the dataset.

2. The matrix M is then averaged along the N spectra -axis and this average is stored in a vector M ¯ .

3. The variance–covariance matrix C is calculated as

   (1) C = ( M − M ¯ ) T ⋅ ( M − M ¯ ) ,

   where the superscript “T” stands for the transpose operator.

4. The eigenvectors e k ( λ ) of C are then calculated. C has a dimension of N λ × N λ . The principal components (PC) correspond to the eigenvectors sorted in decreasing magnitude.

5. Each spectrum of M is then projected on these PCs in order to find its corresponding coefficient p j k defined as

   (2) p j k = ( M j − M ¯ ) ⋅ e k .

6. The original “denoised spectrum” can be calculated using

   (3) S j = M ¯ + ∑ k = 1 n k p j k e k .

The PCA can reduce the size of each spectrum from N λ to n k . The choice of n k depends on the many parameters, the size of the dataset, the wavelength range, and the shape of the spectra lines. We have opted for a value for n k that reduces the mean reconstructed error to a value < 0.5 % according to the following equation:

We have opted to a value for n k that reduces the mean reconstructed error to a value < 0.5 % . This value is found to be n k = 50 . A detailed description of all steps of the PCA can be found in Paletou et al., (2015a), Paletou et al., (2015b), Gebran et al., (2016, 2022, 2023), and Gebran (2024). For both datasets, we achieved an R 2 score larger than 0.996.

5 Denoising and parameter determination

The datasets that contain the synthetic spectra without any added noise are used to train the autoencoder and to find the eigenvectors of the PCA procedure. These two techniques are then used on the set of noisy spectra that are calculated in Section 2. The evaluation of the denoising procedure is tested in two ways. First, we checked the similarity of the denoised spectra with the original one with no noise added. Second, we checked the accuracy of the derived stellar parameters when we applied the procedures of Gebran et al., (2022, 2023) on the denoised spectra from the autoencoder and PCA.

Autoencoders usually replace PCA because of their non-linear properties; however, both techniques showed a good reconstruction power, as shown by the R 2 score in Sections 3 and 4. A way to visualize the denoising of spectra is shown in Figure 4. The figure is divided into two parts, the upper one displays a spectrum having the parameters of dataset 1, and the bottom one has the parameters of dataset 2. In each part, the noisy spectrum is in black, the original one without noise is in dashed blue, the denoised spectrum using the autoencoder (left panel) or PCA (right panel) technique is in red, and the difference between the denoised spectrum and the original one without noise is in dashed–dotted green.

Figure 4

Denoising example of spectra having stellar parameters in both datasets. The upper plot displays a spectrum having the parameters of dataset 1, and the bottom one has the parameters of dataset 2. The noisy spectrum is in black, the original one without noise is in dashed blue, the denoised spectrum using the autoencoder (left panel) or PCA (right panel) technique is in red, and the difference between the denoised spectrum and the original one with no noise is in dashed-dotted green.

In Gebran et al., (2022, 2023), we have introduced a technique to derive the stellar parameters of spectra using a neural network. We have used the same procedure to derive the accuracy of the stellar parameters once we apply the same technique to the denoised spectra. The main purpose of this step is not to evaluate if the derivation technique is accurate or not, but it is to check how similar are the derived stellar parameters of the noisy spectra to the ones derived from the original spectra with no noise added.

The networks that we used are made of several fully dense layers and are trained to derive each parameter separately. The layers are described in Table 3. The first step of the analysis is to reduce the dimension of the spectra using a PCA procedure. This PCA is not related to the one used for denoising, it is just a step for optimizing the network and making the training faster (see Gebran et al., 2022 for more details).

Two different trainings are performed for each dataset. The first one is done using a dataset of only synthetic spectra with no noise added, and the second one consists of applying data augmentation with spectra having a range of SNR between 3 and 300.

Because we already know the stellar parameters of the spectra, the evaluation is performed by calculating the difference between the predicted parameter and the original one using the equation

where N is the total number of noisy spectra used in the evaluation. This is done for T eff , log g , v e sin i , ξ t , and [M/H]. Tables 4 and 5 display the accuracy values for the parameters for the two datasets when deriving the stellar labels of ∼ 25,000 with no noise added (Col. 2), with random noise (Col. 3), with random noise then denoised using autoencoder of Section 3 (Col. 4) and using PCA of Section 4 (Col. 5). Each table is divided into two, one part when data augmentation is performed and one without it.

A detailed analysis of Tables 4 and 5 show that:

1. Data augmentation is an important step to be applied if we need to derive the stellar parameters of noisy spectra. Without it, the model will only learn to derive the parameters of synthetic spectra without any noise added. A similar conclusion was also found in Gebran et al., (2023).

2. PCA denoising is capable of recovering the line profile and the details in the spectra. This is reflected by comparing the accuracy values of the derived parameters using the denoised spectra from the autoencoders and PCA (i.e. comparing Cols 4 and 5).

3. The parameters derived using the PCA denoising technique are more accurate than the ones derived using the autoencoder denoising.

4. No denoising technique is capable of improving the accuracy of the stellar parameters for the one directly derived from noisy spectra (displayed in Col. 3).

5. The stellar parameter algorithm is capable of deriving the stellar labels without the need for a denoising technique.

Figure 5

Predicted T eff as a function of the true values for the data with noise (left panel) and the denoised data using autoencoder (right panel). The spectra are from the augmented dataset 2. The data are color coded to the SNR values.

6 Conclusion

In this work, we have applied two different denoising techniques, an autoencoder, and a PCA, on spectra with random Gaussian noise added to derive the stellar parameters using neural networks of Gebran et al., (2022, 2023). The method was applied to two different spectra ranges, one in 4,450–5,400 Å and one in the Gaia RVS range from 8,400 to 8,800 Å. In this study, we do not constrain the stellar parameter derivation technique, and this was done previously in Gebran et al., (2022, 2023). Interestingly, when applying the model to denoised spectra, there was no noticeable improvement in the accuracy of the derived fundamental parameters, such as T eff , log g , v e sin i , ξ t , and [M/H]. This outcome was unexpected, as denoising is typically thought to enhance the precision of predictions. However, the results indicate that data augmentation plays a more crucial role. When the model is trained on datasets that include noise, the accuracy of predictions for noisy spectra improves significantly, suggesting that the network becomes better equipped to handle real observed spectra. This highlights the importance of incorporating noisy data into training rather than relying on post-processing techniques like denoising to improve accuracy. To further validate these findings, it would be valuable to explore other denoising techniques and assess their impact on prediction accuracy. Techniques such as those presented in Alsberg et al., (1997), Koziol et al., (2018), and Zhao et al., (2021) could be tested to see if they yield better results in reducing noise while maintaining or enhancing the precision of derived parameters. These additional experiments would help solidify the conclusion that data augmentation is more effective than denoising in improving the accuracy of noisy spectra predictions, offering deeper insights into how best to model real observational spectra.