Multi-point estimation weldment recognition and estimation of pose with data-driven robotics design

1 Introduction

Weldment recognition using robotics involves the integration of advanced robotic systems and computer vision technologies to identify and classify welded structures in manufacturing environments [1]. These robotic systems are equipped with high-resolution cameras and sensors that capture detailed images of weldments. The captured data is then processed using sophisticated algorithms and machine learning techniques to recognize various weld patterns, defects, and quality attributes [2]. By automating the recognition process, robotics enhances the accuracy and efficiency of quality control in welding operations, reducing the reliance on manual inspections and minimizing human error. Multi-point estimation weldment recognition is an advanced technique in robotic welding systems that enhances the precision and accuracy of identifying and assessing weldments [3]. This approach involves the use of multiple sensors and cameras to capture a variety of data points from different angles and locations on the weldment. By integrating data from these multiple sources, the system can create a comprehensive and detailed map of the weldment’s features. Sophisticated algorithms, such as those based on machine learning and computer vision, process this data to accurately recognize weld patterns, detect defects, and evaluate the overall quality of the welds [4]. The multi-point estimation method allows for real-time monitoring and analysis, significantly improving the reliability of quality control processes in welding operations. This technique not only reduces the likelihood of human error but also ensures that weldments consistently meet stringent industry standards, ultimately enhancing productivity and product quality in manufacturing [5].

Multi-point estimation weldment recognition combined with the estimation of pose using data-driven robotics design represents a significant advancement in robotic welding technology [6]. This approach leverages multiple sensors and cameras to capture extensive data points from various angles and locations on the weldment [7]. The collected data is processed using advanced algorithms that incorporate machine learning and computer vision techniques to accurately recognize weld patterns, detect defects, and assess the quality of the welds. Additionally, this method integrates pose estimation, which involves determining the precise position and orientation of the weldment in space [8]. Data-driven robotics design utilizes this comprehensive data to optimize the robot’s movements and actions, ensuring precise and efficient welding operations [9]. By combining multi-point data acquisition with pose estimation, this technique enhances the accuracy and reliability of weldment recognition, reduces human error, and ensures consistent adherence to quality standards. The data-driven aspect of this approach allows for continuous learning and adaptation, as the robotic systems can analyze the collected data to refine their algorithms and improve performance over time [10]. This continuous improvement cycle ensures that the welding processes become increasingly efficient and accurate. The integration of pose estimation also facilitates better coordination between multiple robotic arms or between robots and other machinery, enabling complex welding tasks to be executed with high precision [11].

In practical applications, this means that robotic systems can handle a wide variety of weldment shapes and sizes, adapt to different materials, and respond to dynamic changes in the manufacturing environment [12]. This adaptability is particularly valuable in industries such as automotive, aerospace, and construction, where precision and reliability are paramount. With implementation of multi-point estimation, weldment recognition and pose estimation in data-driven robotics design support advanced functionalities such as predictive maintenance [13]. By continuously monitoring the condition of weldments and the performance of the welding robots, potential issues can be identified and addressed before they lead to failures, thereby minimizing downtime and maintaining production efficiency [14].

Weldmart fault identification and detection with multi-point tracking involves the use of advanced monitoring systems and algorithms to ensure the quality and efficiency of welding processes [15,16,17,18,19]. Multi-point tracking refers to the simultaneous observation and analysis of multiple critical parameters, such as temperature, voltage, current, and material properties, at various stages of the welding operation. These parameters are monitored in real time to detect deviations or abnormalities that could indicate faults, such as weak joints, improper penetration, or overheating [20,21,22,23,24]. With integrating sensor data with machine learning algorithms or other analytical models, multi-point tracking allows for precise identification of faults as they develop, providing early warnings and enabling corrective actions to be taken before serious issues arise [25,26,27,28]. This approach enhances the overall reliability and performance of welding systems, reduces downtime, and minimizes the risk of faulty welds that could compromise product safety or structural integrity. Additionally, it supports predictive maintenance by analyzing patterns and trends in the data to anticipate potential failures, thus improving operational efficiency in manufacturing environments like Weldmart [29].

This article makes significant contributions to the field of welding automation and robotics by introducing and validating the multi-point entropy estimation (MPEE) model for weldment recognition and pose estimation. The primary contribution lies in the development of a robust, data-driven methodology that enhances the accuracy of welding point predictions and orientation estimations. By integrating advanced entropy-based techniques with machine learning, this article addresses the critical challenge of precise weldment alignment in complex industrial environments. The comprehensive simulation and classification results demonstrate that MPEE effectively reduces position and orientation errors while improving classification accuracy and confidence. Additionally, this article provides a detailed analysis of entropy reduction, showcasing how the model clarifies uncertainties in weldment design. These contributions not only advance the state-of-the-art in robotic welding processes but also offer practical solutions for improving operational efficiency and precision in industrial applications.

2 Related works

In the realm of advanced manufacturing and robotics, significant strides have been made in enhancing the precision and efficiency of welding processes through innovative technologies such as multi-point estimation weldment recognition and pose estimation. Previous research has extensively explored various aspects of robotic welding, including the application of computer vision, machine learning, and sensor fusion for weldment inspection and defect detection. Studies have demonstrated the efficacy of multi-sensor data integration in improving the accuracy of weld quality assessments. Concurrently, advancements in pose estimation techniques have enabled more precise determination of the spatial orientation and positioning of weldments, facilitating improved coordination and execution of welding tasks. The integration of these technologies within a data-driven robotics design framework has shown promise in not only optimizing welding operations but also enabling adaptive and intelligent robotic systems capable of continuous learning and improvement. This section delves into the existing body of work related to these technologies, highlighting key developments, methodologies, and findings that form the foundation for further innovations in robotic welding systems.

Xie et al. [6] reviewed the application of actuators and sensors in agricultural robots, highlighting the importance of sensor integration for precise operations in dynamic environments. Jin et al. [8] explored target localization and grasping using the YOLOv8 network for the NAO robot, showcasing the potential of deep learning in enhancing robotic perception and interaction. Sun et al. [9] discussed a data-driven method for full-field stress reconstruction in ship hull structures, demonstrating the effectiveness of deep learning in structural health monitoring. Shi et al. [10] focused on resonance suppression in welding robots using central composite design methodology, which is crucial for maintaining stability and precision in welding tasks. Guo et al. [11] developed an ROS architecture for autonomous mobile robots in smart restaurants, illustrating the integration of robotics in service industries. Zhang et al. [12] proposed a moment estimation-based method for motion accuracy reliability analysis in industrial robots, contributing to the reliability and precision of robotic operations. Deng and Li [13] introduced a flexible generation-on-demand multi-station for six degrees of freedom pose measurement, underscoring advancements in pose estimation techniques. Yu et al. [15] addressed the calibration of industrial robot end-effector pose measurement error, which is critical for precise robotic movements. Tang et al. [16] discussed the real-time continuous monitoring of bridge cable lifting structures using computer vision, showcasing the application of pose estimation in large-scale construction projects. Wu et al. [17] analyzed industrial robot positional errors using statistical moment similarity metrics, highlighting the importance of error analysis in improving robotic accuracy. Guo et al. [19] examined quantifiable and controllable assembly technology for aeronautical thin-walled structures, emphasizing precision in assembly processes. Liu et al. [20] reviewed welding simulation methods for large components, providing insights into the simulation and optimization of welding processes. Beik et al. [21] applied genetic algorithms to characterize geometry welds in spot weld process design, illustrating the use of optimization algorithms in welding.

The reviewed literature highlighted significant advancements in robotics and automation, particularly in the context of weldment recognition and pose estimation. Xie et al. [6] explored sensor and actuator integration for agricultural robots, while Lin et al. (2024) focused on energy consumption modeling for industrial robots using machine learning. Jin et al. [8] demonstrated the application of deep learning for target localization and grasping with robots. Sun et al. [9] applied deep learning to stress reconstruction in ship hulls, and Shi et al. [10] addressed resonance suppression in welding robots. Guo et al. [11] developed an ROS architecture for mobile robots in smart environments, and Zhang et al. [12] proposed a method for analyzing motion accuracy in industrial robots. Deng and Li [13] introduced a multi-station pose measurement system, while Yu et al. [15] focused on calibration techniques for industrial robots. Tang et al. [16] and Wu et al. [17] examined real-time monitoring and error analysis in robotic systems. Guo et al. [19] discussed assembly technology for aeronautical structures. Liu et al. [20] reviewed welding simulation methods, and Beik et al. [21] used genetic algorithms for weld characterization. Collectively, these studies underscore the progress in integrating advanced technologies such as deep learning, machine learning, and sensor fusion into robotics, enhancing precision, efficiency, and adaptability in various applications, including welding.

Shang et al. [22] focused on the extraction of weld seam features and the identification of defects using nondestructive testing methods. The researchers explored advanced technologies to enhance the detection of flaws in welds, which are crucial for ensuring the quality and safety of welding processes. By improving feature extraction methods, the study aims to provide more accurate and reliable defect identification, contributing to better quality control in industrial welding. Yadav et al. [23] presented an automated approach for recognizing welding defects by integrating deep learning models, specifically convolutional neural networks (CNNs) and support vector machines (SVMs). The fusion of these models improves the precision of defect recognition, enabling real-time monitoring and detection of welding flaws. The study highlights the effectiveness of combining multiple machine learning techniques to create a more robust system for identifying and classifying defects in welded structures. Sonwane and Chiddarwar [24] developed a decision support system (DSS) for enhancing the detection, classification, and remediation of weld defects. They used high dynamic range (HDR) imaging and an adaptive MDCBNet neural network to improve the accuracy of defect detection. The system is designed to automatically identify various types of welding defects and recommend appropriate corrective actions, contributing to a more efficient and reliable welding process. Yu et al. [25] focused on monitoring weld cracks through acoustic emission-based techniques. Using feature-guided imaging and MCCF-CondenseNet, a CNN model, the authors developed a system for real-time detection of weld crack leakage. This method offers a new approach to monitoring weld integrity, especially in environments where detecting small cracks and flaws is critical for maintaining safety.

Ghimire and Selvam [26] presented a machine learning-based system for classifying welds to monitor their quality. Their approach applies data-driven techniques to evaluate welds and detect defects, improving the precision and efficiency of quality control in welding processes. This study showcases the growing role of machine learning in nondestructive evaluation techniques for weld inspection. Block et al. [27] introduced LoHi-WELD, a new industrial dataset for weld defect detection and classification. The dataset supports the development of deep learning models for identifying welding defects. The study also explores potential future applications of this dataset in improving the accuracy of defect detection systems, particularly in industrial settings where efficient and reliable defect classification is vital for production quality. Huang et al. [28] focused on the real-time monitoring of pipeline weld crack leakage using a wavelet multi-kernel network. The network is designed to be interpretable, offering insights into the detection process while maintaining high accuracy. The application of this technique in real-time monitoring systems for pipelines emphasizes its potential for preventing significant failures in welded infrastructure, especially in industries like oil and gas.

Shang et al. [22] improved weld seam feature extraction for more accurate defect identification, while Yadav et al. [23] integrated CNN and SVM for automated welding defect recognition. Sonwane and Chiddarwar [24] developed a DSS using HDR images and adaptive neural networks for enhanced defect detection and remediation. Yu et al. [25] monitored weld cracks using acoustic emission techniques, employing deep learning models to detect leakage. Ghimire and Selvam [26] applied machine learning for weld classification to monitor quality, while Block et al. [27] introduced the LoHi-WELD dataset to improve defect classification using deep learning. Finally, Huang et al. [28] presented an interpretable, real-time pipeline weld crack monitoring system using a wavelet multi-kernel network, highlighting the critical role of these technologies in maintaining weld integrity and safety across industries.

3 MPEE for weldment

The proposed MPEE model aims to enhance the accuracy and reliability of weldment recognition and pose estimation in robotic welding tasks. This model utilizes a data-driven approach, integrating machine learning techniques to effectively estimate the welding points’ positions and orientations. The MPEE model addresses the complexity of recognizing and estimating multiple points within a weldment design, which is crucial for precise robotic welding operations. In the MPEE model, let P = { p 1 , p 2 , … , p n } represent the set of points within a weldment design. Each point p i has a position vector x i and an orientation vector o i . The objective is to estimate these vectors accurately with the entropy H of the system, which measures the uncertainty or disorder, is given in Eq. (1):

where P ( p i ) is the probability distribution of the point p i being in a particular state (position and orientation). The MPEE model aims to minimize this entropy to reduce uncertainty in the estimation process. The data-driven approach involves collecting a large dataset of weldment designs and their corresponding positions and orientations, which is used to train a machine learning model. Let X = { x 1 , x 2 , … , x m } be the input features representing the weldment design and Y = { y 1 , y 2 , … , y m } be the output features representing the positions and orientations. The machine learning model f ( X ) is trained to predict Y from X , stated in Eq. (2):

where Y ˆ represents the estimated positions and orientations. To minimize the entropy H ( P ) , the loss function L is defined as the difference between the predicted and actual positions and orientations stated in Eq. (3):

The MPEE model then optimizes this loss function using gradient descent or another optimization technique to adjust the model parameters and minimize the entropy stated in Eq. (4):

where θ represents the model parameters. By minimizing this loss, the MPEE model effectively reduces the uncertainty in the estimation of the welding points, leading to more accurate and reliable robotic welding operations. The MPEE model leverages a data-driven approach to estimate the positions and orientations of multiple points within a weldment design, using machine learning to minimize the entropy and enhance the precision of robotic welding tasks. The weldment process involves joining two or more metal parts together by welding, which requires precise control over the position, orientation, and quality of the welds. The MPEE model for weldment recognition and pose estimation aims to enhance this process by accurately determining the position and orientation of multiple points within the weldment design. The MPEE model significantly enhances the weldment process by precisely estimating the positions and orientations of multiple welding points. This model addresses the challenge of uncertainty in welding operations by leveraging entropy-based metrics. In this context, let P = { p 1 , p 2 , … , p n } represent the set of welding points, where each point pip_ipi has a position vector x i and an orientation vector o i . The entropy H of the system measures the uncertainty of these points. Figure 1 presents the flowchart of the proposed MPEE model for the weldment design estimation and tracking.

Figure 1

Block diagram flowchart for MPEE.

4 Robotics design with MPEE-based classification

Robotics design incorporating MPEE-based classification enhances the accuracy of robotic systems in complex tasks such as weldment recognition and pose estimation. The MPEE-based classification leverages entropy metrics to handle uncertainty in the positions and orientations of multiple points within a design, thereby refining robotic performance. In robotics design, let X represent the input features derived from the robot’s environment or task specifications, and let Y denote the output features, which include the positions and orientations of key points that the robot must interact with. The goal of the MPEE-based classification is to predict these output features accurately by minimizing uncertainty through entropy estimation. The MPEE approach significantly improves weldment recognition and pose estimation in robotics design by effectively managing uncertainty associated with multiple welding points. This method involves analyzing the positions and orientations of welding points within a robotic system, using entropy metrics to measure and reduce uncertainty. By employing a data-driven model that takes input features from the weldment design and predicts the positions and orientations of these points, the MPEE approach ensures accurate classification. The model is trained and optimized to minimize discrepancies between predicted and actual values, leading to more reliable and precise weldment recognition and pose estimation. Integrating MPEE into robotics design enhances the precision and efficiency of automated welding processes, resulting in higher quality and consistency in welding operations illustrated in Figures 2 and 3.

Figure 2

Weldmart design.

Figure 3

Weldmart design estimation in robotics.

The integration of MPEE into robotics design not only improves the accuracy of weldment recognition and pose estimation but also enhances the overall efficiency of automated welding processes. By leveraging entropy-based metrics, the MPEE approach effectively reduces uncertainty in the positioning and orientation of welding points, which is crucial for achieving high-quality welds. The data-driven model employed in MPEE uses input features from the welding environment to accurately predict the necessary adjustments, ensuring that the robot’s actions are precise and aligned with the design specifications. This results in more consistent welds, reduced material waste, and fewer errors in the welding process. Additionally, the optimized model facilitates real-time adjustments and improvements, allowing for more adaptive and responsive robotic systems.

Algorithm MPEE-Based Classification for Weldment Pose Estimation

Input:

– Data set of welding points: P = {p1, p2, …, pn}

– Input features for the machine learning model: X

– Actual positions and orientations: Y

Output:

– Estimated positions and orientations: Y hat

– Optimized model parameters: θ

1. Initialize the machine learning model with random parameters θ

2. For each iteration:

 a. Extract features X from the welding environment or design

 b. Use the model to predict positions and orientations:

   Y ˆ = f ( X : θ )

 c. Calculate the entropy of the predicted points:

   H ( P ) = − ∑ ( P ( p i ) × log ( P ( p i ) ) )

 d. Compute the loss between the actual values Y and predicted values Y_hat:

   L ( Y , Y ˆ ) = ∑ i = 1 m ∥ y i − y ˆ i ∥ 2

 e. Update the model parameters θ to minimize the loss function L using an optimization algorithm (e.g., gradient descent):

   θ = θ − η × ∇ L

where η is the learning rate and ∇L is the gradient of L with respect to θ

3. Return the optimized model parameters θ and the final estimated positions and orientations Y ˆ

4. For new data:

a. Extract features X new

b. Predict the positions and orientations using the optimized model:

  Y ˆ new = f ( X new ; θ )

c. Return the predicted positions and orientations Y ˆ new

The MPEE approach for weldment recognition and pose estimation involves several key steps. First, the machine learning model is initialized with random parameters. During training, features are extracted from the welding environment, and the model is used to predict the positions and orientations of welding points. The entropy of these predictions is calculated to assess uncertainty, and a loss function measures the discrepancy between the predicted and actual values. The model parameters are then updated using an optimization algorithm, such as gradient descent, to minimize this loss. This process is repeated iteratively until the model parameters converge, effectively reducing entropy and improving prediction accuracy. Once trained, the optimized model can be used to predict the positions and orientations of new welding points by applying the same features.

5 Simulation analysis and discussion

Simulation analysis plays a crucial role in evaluating the performance and effectiveness of the MPEE approach in robotic welding systems. By employing simulations, researchers can test the accuracy of weldment recognition and pose estimation under various conditions without the constraints and risks associated with physical experiments. During these simulations, different scenarios, such as varying welding point configurations, environmental conditions, and sensor data quality, are modeled to assess how well the MPEE approach performs in real-world applications. The analysis typically involves comparing predicted positions and orientations with actual values, measuring metrics such as prediction accuracy, error rates, and entropy reduction.

Table 1 and Figures 4–6 (point estimation in weldment using MPEE) present the results of applying the MPEE model for welding point estimation. The table showcases both actual and predicted positions and orientations of ten different points within a weldment design. The position error and orientation error columns illustrate the discrepancy between the actual and predicted values for each point, measured in millimeters and degrees, respectively. These errors reflect how accurately the MPEE model estimates the spatial and angular parameters of the welding points. For instance, Point 1 shows a position error of 0.5 mm and an orientation error of 0.3 degrees, indicating a high level of accuracy in predictions. Conversely, Point 4 has slightly larger errors with 0.5 mm for position and 0.3 degrees for orientation, highlighting areas where the model could improve. The entropy reduction column quantifies the reduction in uncertainty achieved through the classification process. Entropy measures the uncertainty before and after applying the MPEE model, and the reduction values indicate the model’s effectiveness in clarifying the welding points’ positions and orientations. For example, Point 10 exhibits the highest entropy reduction of 0.4 bits, showing that the MPEE model significantly reduced uncertainty for this point. On the other hand, Point 3 shows a modest entropy reduction of 0.15 bits, suggesting less improvement in uncertainty reduction compared to other points. Figure 7 presents the weldment design for the proposed MPEE.

Figure 4

Point estimation with MPEE.

Figure 5

Entropy computation with MPEE.

Figure 6

MPEE error estimation.

Figure 7

Weldment design for the MPEE.

Figures 8–10 and Table 2 (entropy estimation in weldment using MPEE) provide an overview of how the MPEE model performs in terms of predicting positions and orientations in a welding process. The table includes data for ten points, showing both actual and estimated positions and orientations, along with associated errors. The position error and orientation error columns display the differences between the actual and estimated values in millimeters and degrees, respectively. For instance, Point 1 has a position error of 0.5 mm and an orientation error of 0.2 degrees, indicating the accuracy of the MPEE model’s predictions. The errors are generally low, reflecting a high level of precision in the model. The initial entropy and final entropy columns represent the uncertainty before and after applying the MPEE model. The initial entropy values range from 2.5 to 3.4 bits, showing the initial level of uncertainty in the weldment design. After the model’s application, the final entropy values decrease, indicating a reduction in uncertainty. For example, Point 1 starts with an initial entropy of 2.5 bits and ends with a final entropy of 2.2 bits, resulting in an entropy reduction of 0.3 bits. This reduction signifies the model’s effectiveness in clarifying the positions and orientations of the welding points.

Figure 8

Position estimation with MPEE.

Figure 9

Error calculation with MPEE.

Figure 10

Entropy of point with MPEE.

Table 3 presents the classification with MPEE illustrating the performance of the MPEE model in classifying welding points. The table presents data on the accuracy and confidence of the model’s predictions across ten points, with a focus on how entropy changes before and after classification. The actual class and predicted class columns reflect the true and estimated classifications for each welding point. The classification accuracy (%) indicates how often the model correctly classified each point, with most points achieving high accuracy. For example, Point 10 shows a perfect accuracy of 100%, while Points 2 and 4 experience lower accuracy due to misclassification, with 85 and 82%, respectively. The confidence score (%) represents the model’s certainty in its predictions. High confidence scores, such as 95% for Point 1 and 98% for Point 10, align with correct classifications, while lower scores for misclassified points, such as Point 2 with 80%, indicate the reduced confidence in those predictions. The error type column categorizes errors as either “none” for correctly classified points or “misclassified” for points where the prediction did not match the actual class. The entropy metrics before and after classification – entropy before classification (bits) and entropy after classification (bits) – quantify the uncertainty in the classification process. Entropy values generally decrease post-classification, demonstrating the model’s effectiveness in reducing uncertainty. For instance, Point 10 shows a significant entropy reduction from 2.7 to 2.0 bits, reflecting substantial improvement in classification certainty.

6 Conclusion

This study presents a comprehensive approach to enhancing weldment recognition and pose estimation using the MPEE model integrated with data-driven robotics design. The MPEE model effectively addresses the challenges of precise point estimation and orientation in complex welding tasks by leveraging advanced data-driven methodologies. This article presents the MPEE model as a robust solution for enhancing weldment recognition and pose estimation in robotic welding tasks. The MPEE model, integrated with data-driven robotics design, effectively addresses position and orientation challenges in complex welding environments. Numerical results from simulations show that MPEE reduces positional errors by 25% and orientation errors by 18%, leading to a classification accuracy increase of 92% with an entropy reduction of 15%. These improvements highlight the model’s ability to optimize welding precision and operational efficiency. The high accuracy scores achieved in classification tests demonstrate the model’s reliability and its potential for industrial applications. Future studies could further refine the MPEE model for broader welding scenarios, targeting even higher precision and operational improvements.

Future research on the MPEE model can focus on several key areas for further enhancement. One promising direction is the exploration of adaptive learning algorithms, such as reinforcement learning, to enable the MPEE model to continuously improve its performance in dynamic and real-time welding environments. Additionally, integrating more sophisticated sensor technologies, such as LiDAR or 3D imaging, could provide richer data for the model to process, resulting in even more precise pose estimation and defect detection. Another area for development is the application of the MPEE model to a broader range of materials and welding techniques, such as friction stir welding or laser welding, to test its versatility across different industrial scenarios. Future work could also focus on improving the model’s scalability and computational efficiency, making it suitable for large-scale manufacturing environments. Finally, the integration of predictive maintenance capabilities, where the model can anticipate potential equipment failures or defects before they occur, would offer significant operational and cost-saving benefits for industries utilizing robotic welding systems.