Extended Intelligence for Rapid Cognitive Reconfiguration

Introduction and Background

In today‘s manufacturing landscape, the need for rapid reconfiguration is growing due to factors such as shorter product lifecycles and increased customization. Reconfigurable manufacturing systems offer a solution by allowing the structure and control of manufacturing systems to adapt to new requirements quickly. However, challenges such as time-consuming processes, errors, and lack of methodological support hinder the efficiency of these reconfigurations [1] .

Recent advances in Artificial Intelligence (AI), Digital Twins (DTs), and eX-tended Reality (XR) provide practical tools to address these challenges [2, 3]. For example, DTs enable real-time simulation of physical systems, enabling predictive maintenance and reducing reconfiguration times by up to 58 percent [4]. XR technologies, when integrated with DTs, provide immersive interfaces for operator training, real-time feedback, and system control, improving the effectiveness of human-machine interaction.

Challenges in Reconfigurable Manufacturing Systems

Reconfigurable manufacturing systems are designed to support the flexibility and adaptability required in modern manufacturing. However, current reconfiguration practices face long downtimes, complex decision-making processes, and operator reliance on intuition, leading to inefficiencies. As manufacturing systems become more automated, they become more complicated, making traditional reconfiguration methods less effective. This complexity stems from managing large amounts of data, coordinating complex machine behaviors, and ensuring minimal downtime during reconfiguration. AI, DTs, and XR address these challenges by providing powerful tools to streamline processes: AI supports decision-making by analyzing data patterns and optimizing configurations; DTs simulate real-time changes, enabling predictive analysis and error prevention; and XR provides intuitive interfaces for operators to interact more efficiently with complex systems. Together, these technologies reduce human operators‘ cognitive and operational burden while significantly improving the reconfiguration process.

Despite the growing interest in each technology individually, few industrial projects fully integrate them. Most applications focus on individual optimizations, leaving a gap in realizing their combined potential.

Extended Intelligence for Rapid Cognitive Reconfiguration

eXtended Intelligence for Rapid Cognitive Reconfiguration (XIRCON) aims to address gaps in current reconfiguration processes by integrating AI, DTs, and XR into a cohesive system, as shown in Figure 1. The architecture is divided into components that are part of the virtual world, components that are part of the physical world, and components that connect both worlds. This results in a cyber-physical system where changes in the physical world affect the virtual world and vice versa. First is the group of reconfigurable manufacturing systems, actuators and sensors, and DT.

Figure 1

System Architecture – Integrating AI, DT, and XR to a holistic system

This group mirrors the reconfigurable manufacturing system in the virtual world as DT, as shown in Figure 2. Changes made to the manufacturing system update the DT through the sensors. Changes made to the DT update the manufacturing system through the actuators. The second group comprises the human operator, the head-mounted display, and the XR. The human operator wears the head-mounted device while interacting with the manufacturing system. The head-mounted device shows the operator the physical world as a virtual world representation. The advantage of this is that information, such as a recommendation for the next reconfiguration steps, can be visualized in the operator‘s field of view. The XR component provides helpful information and recommendations to the operator. It can also store and replay recordings of the operator‘s motion and voice through the head-mounted device. DT and XT exchange information such as possible trajectories. The trajectories can be displayed to the operator or used to interact with the manufacturing system. In addition to trajectories, all data generated by DT and XR can be fed into the AI component. The AI learns from the data and uses it as the basis for recommendations and information that is rendered to the operator. It can also interact directly with the manufacturing system via the DT.

Figure 2

Implementation of the Value Stream Kinematics vision (left) and its DT (right)

The XIRCON architecture results in a cyber-physical system where navigation in the physical and virtual worlds becomes one. This allows virtual components such as the AI to interact directly with the physical world. Physical tasks such as reconfiguration can also be virtually recorded and replayed. Applied to the reconfiguration use case, an expert operator can record while reconfiguring the manufacturing system. A novice operator can replay the recordings and get assistance from the AI to learn the reconfiguration task. This is shown in Figure 3. The left side shows virtual hands performing a reconfiguration task as visual feedback for the operator. The right side shows the recorded task with the operator visualized as an avatar.

Figure 3

View from the XR headset: Human operator explaining the pneumatic quality assurance step (left) and virtual replica of the human operator demonstrating the same step (right)

Robotic Manufacturing: Value Stream Kinematics

One example of a reconfigurable manufacturing system is a manufacturing system based on the Value Stream Kinematic vision. Value Stream Kinematics represents a different approach to the challenges of mass customization. Universal kinematics, such as jointed-arm robots combined with intelligent software, implement the entire value stream of a product. All manufacturing steps become feasible using universal kinematics with a standard set of tools. The intelligent software plans and executes all processes autonomously, from the customer order to the necessary manufacturing steps. The result is highly flexible yet productive manufacturing [5, 6].

Three layers show the dimension of the Value Stream Kinematics: the machine, production planning, and human interaction layers. The machine layer challenges how to perform all manufacturing steps using only joint-arm robots as universal kinematic representants. Machine tools perform most manufacturing steps to achieve high precision. Joint-arm robots are inherently less rigid due to their open-chain kinematics. And the loss of stiffness results in lower precision. Stiffness must be increased to perform typical machine tool tasks such as milling. This is achieved by coupling the robots, which results in closed-loop kinematics. For pick-and-place tasks, each robot can operate independently or must cooperate with another. This variety of possible configurations creates challenges for the next layer, production planning. Production planning analyzes product specifications from customer orders. It makes a task plan, decides which robots will interact and which will be coupled, selects the right tools, executes the manufacturing steps, and sends feedback to the customer. The final layer is the human interaction with the Value Stream Kinematics. Even though the system works autonomously, human work is still part of it. In addition to the customer sending his order, the machines and software must be maintained. Such a complex machine is error-prone and difficult to learn [5, 6].

Key Results and Approaches

The XIRCON project has made significant progress in developing an architecture that integrates AI, DT, and XR to optimize reconfiguration processes in manufacturing. Key results achieved so far include:

1. Integrated system architecture: The project has established a basic software architecture that integrates these components and enables real-time feedback between the XR interface, DT, and AI, as shown in Figure 1. This architecture lays the groundwork for future enhancements, such as fully automated reconfiguration processes and further AI-driven operator assistance and reconfiguration optimization.

2. XR component for human operations: The XR interface shown in Figure 3. has been developed. It captures and replicates human operations within the manufacturing cell. This provides a detailed record of human activities during reconfiguration that can be analyzed and used for future improvements. By capturing these operations, the system also helps preserve tacit operator knowledge, which is critical for training and ensuring consistency in reconfiguration tasks.

3. DT component for the robotic manufacturing system located at Karlsruhe Research Factory: The project has developed a DT of the robotic manufacturing cell. Figure 2 shows a kinematic model of the robots that can partially read and interpret robot programs. In particular, the DT can extract trajectory information from the robot code, allowing real-time simulation and validation of reconfiguration steps. This helps reduce the time and errors associated with manual robot programming.

These advances demonstrate how the integration of XR, DT, and AI technologies can streamline reconfiguration processes, reduce errors, and improve the overall adaptability of manufacturing systems.

Benefits for the Industry

Integrating AI, DT, and XR brings significant benefits to industrial applications. These technologies can reduce reconfiguration time by more than 50 percent, reduce errors during the reconfiguration process, and improve operator interaction with increasingly complex systems [4] . They also enable predictive maintenance, ensuring systems operate at peak efficiency with minimal downtime [1].

In practice, manufacturers can expect:

1. Shorter reconfiguration cycles, enabling faster response to market changes.

2. Improved human-machine collaboration, where operators are supported by real-time feedback from DTs and intuitive XR interfaces.

3. More adaptive systems that can be reconfigured without significant manual intervention .

In particular, AI plays a critical role in achieving these benefits by:

1. Optimizing decision making: AI analyzes data from DTs to recommend the most efficient reconfiguration strategies, improving the accuracy of decisions and reducing downtime during system changes.

2. Learning from operator input: AI models in XIRCON are designed to elicit operator knowledge during reconfiguration and use this data to train machine learning models that suggest optimal reconfiguration actions for future tasks.

3. Enable predictive maintenance: AI algorithms continuously monitor system performance and predict when maintenance or reconfiguration is needed, ensuring that systems operate at peak efficiency with minimal downtime.

4. Reduce human error: By automating reconfiguration tasks and learning from previous configurations, AI minimizes the likelihood of human error and ensures a more reliable and resilient system.

Synergies between AI, XR, and DT

The combined use of AI, DT, and XR allows for more synergistic and adaptive reconfiguration systems. AI and DT already enable predictive maintenance and real-time monitoring, while XR provides an immersive interface that facilitates natural human-machine interaction. The next step in this evolution is fully integrating these technologies to create systems that can autonomously optimize reconfiguration while maintaining human oversight.

Future Work and Outlook

As AI, DT, and XR technologies continue to evolve, future work will focus on improving their interoperability and scalability. Emerging trends include the development of bidirectional DTs that not only mirror but also control physical systems and automated reconfiguration driven by AI algorithms.

In summary, integrating AI, DT, and XR will redefine how manufacturers approach system reconfiguration. These technologies will play a critical role in the next generation of reconfigurable manufacturing systems by reducing downtime, increasing flexibility, and improving decision-making.