Bayesian optimization of single-pulse laser drilling using advanced image processing

2.1 Experimental setup

Figure 2 shows the experimental setup of the single-pulse laser drilling process. In this study, a continuous wave (cw) single mode fiber laser (TRUMPF TruFiber 2000) was used to perform the single-pulse laser drilling experiments. The emission wavelength of the unpolarized laser was specified as 1,075 nm in conjunction with a beam propagation factor of M² < 1.2. The laser beam was positioned onto the stainless steel sample with a galvanometer scanner. A telecentric F-Theta lens with a focal length of 163 mm was used, resulting in a focus diameter (1/e²) of d_f = 20 µm. The pulsed operation mode of the laser source enables the generation of pulses with a peak power P_P up to 1,400 W. This allows for the adjustment of the pulse length between values from 1 to 25 µs. The setup was equipped with linear stages (x,y) for the sample and a linear drive (z) for the process optics to adjust the focus position. The focus position can thus be positioned with an accuracy of 1 μm.

Figure 2:

Left: Experimental setup. Right: Borehole cross section with images of an (a) inlet with diameter d _I and (b) outlet with diameter d _O .

The materials used for the experiments are stainless steel (1.4310) substrates. The substrates, with a thickness h of 0.3 mm, were cut to a size of 100 mm × 50 mm. An optical microscope (Zeiss Axio Imager) was used to evaluate the borehole criteria, such as inlet (Figure 2a) and outlet (Figure 2b). A 20× magnification was used for optical microscopic observation, where one pixel is equivalent to 0.172 × 0.172 µm². The evaluation criteria include the diameter of the inlet d_I and of the outlet d_O, as well as the roundness R of the outlet. We drilled and analyzed i = 3 holes per parameter set to reduce the influence of side effects from the inherent noise of laser processing and other uncertainties.

2.2 Feature extraction

In the context of manual measurement, the borehole diameter is determined through the use of the Menger Curvature [12]. The Menger Curvature (MC) quantifies the curvature of a triplet of points in an n-dimensional Euclidean space. It is defined as the reciprocal of the radius of the circle that passes through three points p₁, p₂, and p₃, as illustrated in Figure 3(a).

Figure 3:

Manual measurement of the borehole diameter (a) The Menger Curvature of a triple of data points on a 2-D space, based on Zuo et al. [13]. (b) Example of radius determination using the Menger Curvature on a borehole.

For the given borehole image, the problem is treated as two-dimensional, with p₁, p₂, and p₃ representing three non-collinear points (as shown in Figure 3(b)) in the 2D Euclidean space E². As shown in Figure 3(a), MC on p₂ is calculated as:

where r denotes the radius of the circumcircle, ‖p₁, p₃‖ represents the Euclidian distance between points p₁ and p₃, and φ is the angle at vertex p₂ formed by p₁, p₂, and p₃ [14]. The angle φ can be determined using the Law of Cosines.

The three points p₁, p₂, and p₃, which are suitable to determine the Menger Curvature that assigns a reasonable radius to the borehole, are currently determined manually by process experts by inspecting calibrated images of the boreholes. This method is not only time-consuming but also prone to variability, both among different experts and within repeated measurements by the same expert, due to inconsistencies in the placement of the measurement points.

The objective is to automatically extract the features that are required for the parameter optimization directly from the microscope images. The features include the borehole’s inlet diameter d_I and outlet diameter d_O, the borehole’s roundness R, the area of the melt deposits around the borehole, and a classification of whether a breakthrough has occurred. Initial attempts to perform feature extraction based solely on conventional image processing methods have not delivered satisfactory results. Due to the divergent surface properties of the materials to be processed, there is a high degree of variance in the captured images, for example, due to reflections and mirroring. This variance requires great efforts to manually adjust the algorithm parameters of conventional image processing methods. Deep learning methods represent another viable approach to address natural deviations in images like reflections and mirroring. However, a method based exclusively on deep learning that directly determines quality characteristics can be intricate and challenging for operators to understand. A combined approach, comprising semantic segmentation models and conventional image processing methods, enables a more robust and understandable extraction of features. In our study, we employ two semantic segmentation models, each with a neural network architecture modified from the SDU-Net [15]. Our approach builds directly on principles established in the foundational U-Net architecture by Ronneberger et al., which demonstrated successful segmentation results with only 30 annotated training samples [16]. The SDU-Net architecture enhances the original U-Net design by incorporating stacked dilated convolutions, which expand the receptive field without increasing the number of parameters [15]. Additionally, we employ Categorical Focal Loss [17] to mitigate overfitting in class-imbalanced scenarios by focusing learning on hard-to-classify examples, ensuring robust performance even with limited data. These models are used to segment images from the top (inlet) and bottom (outlet) of the borehole. The inlet model classifies the image into the following classes, as partly shown in Figure 4(b): burr, melt deposits, and one of the classes borehole with breakthrough or borehole without breakthrough. The outlet model segments the image into: background, melt deposits, borehole with breakthrough, and borehole without breakthrough. To train the inlet model, 68 labeled images were used, while the outlet model was trained with 44 images. The discrepancy in the number of training images is because only through boreholes are included in the outlet dataset. The models are initialized randomly without any pre-training. The classes segmented by the models are further analyzed using conventional image processing methods. Figure 4(c) shows, how the borehole diameter d_I was calculated using the contour (green) of the segmented borehole class borehole with breakthrough. This calculation involves averaging the diameters of two specific circles: the minimum enclosing circle, d_min,enc (shown in yellow), determined using the method proposed by Welzl et al. [18], and the maximum inscribing circle, d_max,ins (shown in red), as described by Xia et al. [19].

Figure 4:

Feature extraction approach. (a) Inlet of a borehole with breakthrough. (b) Segmented classes by the neural network: melt deposits, burr, and borehole with breakthrough. (c) Contour of the borehole (green) from which the diameter of the maximum inscribing circle d_max,ins (yellow) and the diameter of the minimum enclosing circle d_min,enc (blue) were derived.

The roundness R of the borehole is defined by the ratio of the borehole area A_borehole (Figure 4(b) green) to the area of the minimum enclosing circle A_min,enc [20]:

This equation provides a quantitative measure of roundness, enabling straightforward comparisons between different boreholes. The method is applicable to boreholes that are shaped irregularly, as shown in Figure 5, ensuring that deviations from perfect circularity are effectively captured by employing the minimum enclosing circle. Furthermore, the use of the minimum enclosing circle ensures that the largest possible deviation from roundness is considered, offering a conservative assessment of circularity. The method’s sensitivity to outliers, such as melt deposits as shown in Figure 5(c), is a notable characteristic. These irregularities can disproportionately influence the area of the minimum enclosing circle, thereby distorting the roundness value. However, this high sensitivity to outliers is not inherently disadvantageous, as such irregularities directly impact the quality of the borehole. Consequently, high sensitivity is a desirable characteristic in this context, as it accentuates deviations that could compromise the functionality or structural integrity of the well.

Figure 5:

Illustrations of the minimum enclosing area (A_min,enc, blue) for various borehole geometries (A_borehole, green) with results for roundness values R.

The melt deposition area is calculated as the sum of the segmented burr and melt deposits area classes. In order to ascertain whether breakthrough is present, the areas belonging to the borehole with breakthrough and borehole without breakthrough classes are compared. A breakthrough is classified if the area of the borehole with breakthrough class is larger than that of the borehole without breakthrough class. Otherwise, no breakthrough is classified.

2.3 Bayesian optimization

To optimize the single-pulse laser drilling process, we used extracted process and quality data in a Bayesian optimization framework to identify optimal laser parameters. The AX Service API [21] was used, with a Gaussian Process as a surrogate model [22], and the default Matern 5/2 kernel for the optimization. This approach efficiently explored the parameter space, aiming to optimize the drilling process with minimal experimental effort. As acquisition function Expected Improvement [23] was used. The choice of these Bayesian Optimization parameters is derived from the comprehensive research results by Menold et al. [8], who demonstrated their effectiveness across various other laser processes.

3.1 Results of the feature extraction

The effectiveness of the feature extraction was evaluated by measuring 75 inlets and outlets, which were then compared with the results of a manual measurement conducted by experts. These images were not included in the training data set. We employ the Intersection over Union

introduced by Jaccard et al. [24] as an evaluation metric to assess the predictions of the model. Here A is the segmentation mask used for training and B is the prediction of the segmentation network. During the evaluation, the inlet model achieved an IoU value of 0.97 for the borehole classes, while the outlet model achieved an IoU value of 0.95 for this class. However, the melt deposits and burr classes exhibit a decline in performance, each reaching an IoU value of 0.75. This can be attributed primarily to the distinctive characteristics of the melt deposits, which also leads as a maximum IoU value of 0.78 for these two classes during training.

After image segmentation, diameters are calculated based on the prediction of the borehole models. To assess the precision of the measurement techniques with respect to representative data, the inlets and outlets of 75 additional boreholes, drilled in identical experimental conditions as illustrated in Figure 2, were evaluated. The measurements were performed using two methods: automatic feature extraction and manual measurements based on the Menger Curvature method. Each of the 75 boreholes was measured manually by two experts (Person A and Person B), with Person A performing two independent measurement trails for all 75 boreholes. Table 2 presents the mean values of inlet and outlet diameters for each measurement method. The distribution of inlet and outlet diameters, along with respective deviations between measurements, is shown in Figure 6.

Figure 6:

Comparison of measurement methods for 75 measured borehole inlets and outlets. Distribution of inlet (left) and outlet (right) diameters and their mean value by using automatic feature extraction and manual measurements.

The mean deviation between Person A’s first trial and the automated measurement trial is 0.36 µm for inlet diameters and 0.61 µm for outlet diameters, which is within the expected accuracy and tolerance limits for borehole measurements. The mean deviation between manual measurements conducted by Person A (first trial) and Person B is 1.39 µm for inlet diameters and 0.43 µm for outlet diameters. These low deviations across all automated and manual measurement techniques validate the effectiveness of feature extraction in determining inlet and outlet diameters. The methods outlined enable automated borehole measurement, facilitating parameter optimization while significantly reducing manual effort.

3.2 Results of the Bayesian optimization

Figure 7 shows the evolution of the process parameters (left) and quality variables (right) during the optimization process. During the initialization process (orange) with parameters chosen by the Sobol sequence, a wide range of process parameters is covered, resulting in a high cost value (red curve). In the start sequence, three parameter sets n = 3, 4, 5 did not lead to through holes, because the pulse length was too short. In the following optimization steps the BO suggested only one more parameter set at n = 26, where no breakthroughs were achieved.

Figure 7:

Results of Bayesian optimization process. Left: Evolution of process parameters (focal position, pulse power, pulse length) over 30 iterations. Right: Corresponding quality variables (inlet diameter, outlet diameter, roundness) and cost function value. Orange points indicate initialization phase. Insets (a) and (b) show inlet and outlet of optimal borehole at iteration n = 12.

In the bottom right of Figure 7 the inlet and outlet of the borehole with the minimum cost value C₁₂ = 86.00 after n = 12 iterations with process parameters z_f = −79.0 µm, P_L = 898 W and t_P = 17 µs is shown. This led to an inlet diameter of d_I = 68.0 µm, an outlet diameter of d_O = 18.03 µm and a roundness of R = 0.84 which are close to the targeted values.