Industry goes metaverse – the fusion of real and virtual industrial worlds exemplified by the wastewater industry

1 Introduction

By introducing Meta, Facebook founder Mark Zuckerberg revealed the vision of a shared virtual world where you can meet your friends, participate in events, play sports and even shop. A concept that was mentioned as early as 1992 in a science fiction novel and is now regarded by experts as the next stage in the development of the Internet [1]; the concept being 3D interaction on the Web. With its announcement, Meta has given the technologies around “extended reality” (generic term for VR/AR/Mixed Reality (MR)) a new dynamic. A strategic move that also aims to turn the subsidiary Oculus into a profitable business. At the turn of the year 2021/22, the number of downloads of the Oculus app increased to approximately 2 million. This indicates a surge in the sales numbers of VR headsets [2].

In Germany, the use of VR headsets has also increased according to a survey by the trade association Bitkom. In addition to the 17 percent of people surveyed who already use VR headsets, a further 23 percent could imagine using them in the future to explore virtual realities. 18 percent of people surveyed are certain that they will work with VR headsets in the near future [3]. Although the potential of the metaverse seems very visionary, there are already a large number of virtual reality applications today. Indeed, the majority of these applications are used for video games (77 percent), virtual applications are also already being used in other areas of entertainment (traveling, movies, music concerts), sports (37 percent) and education (16 percent).

The aforementioned scenarios show that currently private users and the creative industry are primarily benefiting from virtual reality and its possibilities. On the other hand, the use of VR in a professional context or in the manufacturing industry is still very uncommon (7 percent). At this point, however, it should be noted that the possible applications of VR for collaboration, construction and training purposes, were not included in this assessment. Instead, augmented reality, meaning the perception of reality supplemented by digital elements, is proving to be beneficial for the manufacturing industry. This technology can be used in order picking, on the assembly line or to remotely support maintenance work. Although closed vision (VR) headsets are predominantly used in the context of artificially created worlds, for example in video games, they can also be used for real-world activities. This is demonstrated by the solution for virtual inspections of decentralized wastewater plants, developed by the Entsorgungsverband Saar in cooperation with the August-Wilhelm Scheer Institute.

2 The Challenges at the Entsorgungsverband Saar (EVS)

The Entsorgungsverband Saar is responsible for approximately 140 wastewater treatment plants, 293 pumping stations and 625 stormwater overflow basins. One to two inspections are carried out per plant every week, adding up to 104 routine on-site inspections over the course of a year. While central treatment plants are staffed, decentralized treatment plants are not. A staff member has to drive to the plant and conduct an inspection on site. The procedure essentially includes a visual inspection designed to ensure the operational reliability of the pumping stations and treatment plants. This process takes up a considerable amount of the operating staff’s time.

3 The solution: a VR headset

To make the assessment process more efficient, a virtual inspection solution was implemented. The developed solution is composed of a VR headset, sensors and a 360° camera as well as a data interface. The following scenario describes the interaction and function of these components: the operating staff of the main wastewater treatment plant must perform a routine inspection of a remote wastewater treatment plant. Usually, an employee would drive to the site and inspect the plant with their own eyes, checking the conditions and the state of the plant. However, instead of getting in the car and making the trip, the virtual inspection solution can now be used to carry out a location-independent inspection.

To do this, the staff use a VR headset and connect to a 360° camera mounted at the inspection site. This set-up allows for a real-time view of the facility while simultaneously providing the user with an immersive experience. While wearing the VR headset, the staff member has the feeling of being at the wastewater treatment plant. In order to fully assess the operating status of a plant, it is also necessary to be able to view the operational data. The approach of a digital twin allows the staff to interact with the facility in the VR application. When the employee uses the VR controllers and clicks on a component of the plant, the operational data is displayed in real time. Analysis of the data is enhanced with visual dashboards that simplify monitoring. The image, video, and operational data are accessed through a central database (cloud). Figure 1 depicts the operational data of a sewage sludge plant, an image taken from our first pilot project.

Figure 1:

Using a digital twin to view the operational data of a sewage sludge plant in real-time.

The solution is currently being integrated into two facilities: a pumping station and a wastewater treatment plant. Previously, a Proof of Concept (PoC) was already carried out on a sewage sludge plant. In total, there are three sites on which the solution has been integrated. Depending on the location, a different number of cameras is used.

In the initial PoC, a 360° camera was deployed, focusing on investigating the functionality of the solution. In the current follow-up projects, multiple cameras are being deployed at the sites. For example, one 360° camera was installed in the outdoor area and one in the indoor area of the pumping station. For the current integration at the wastewater treatment plant, three cameras are planned (one 360° camera and two 180° cameras).

4 Customers benefit from operational reliability and reduction of operating expenses

The Entsorgungsverband Saar has recognized the added value the pilot project provided and is now transferring the developed solution to a wastewater treatment plant and a pumping station. Essentially, the EVS benefits from ensuring the operational safety of its wastewater plants while reducing operating expenses. In the case of wastewater plants, operational safety aims to rule out the risk of endangering the environment. If, for example, a throttling device for the transfer of wastewater fails, environmentally harmful discharges could flow into surrounding bodies of water. While previously a staff member went out once a week to check on these sources of danger, this can now be done regularly and regardless of location via VR headset. Inspections can also be conducted more frequently, which is a huge advantage. In the case of severe weather events, a quick look through the headset can be particularly advantageous. Nevertheless, trips to the wastewater plants will continue to be necessary for maintenance work or to take samples for laboratory analysis. However, the location-independent inspections will reduce operating expenses.

Using different parameters such as distance and the monthly frequency of inspections, the operating costs for 100 plants were calculated. The results were 86,400 km of travel, 1760 h of travel time and 18,352 kg of CO2. Each trip that can be saved simultaneously reduces the time required and the CO2 emissions that are generated by the inspection.

5 New potentials of collaborative and hybrid working

Although the virtual inspection of unmanned facilities was the initial focus of the project, challenges such as the shortage of skilled workers, mean that collaborative work and the utilization of centralized knowledge are increasingly becoming more important. What this can mean can be illustrated using AR glasses during remote assistance. For example, if a maintenance worker with data glasses is at a facility that requires to be repaired and needs support from a colleague in the operations center, they can help remotely with their knowledge. Everything that the data glasses are pointed at can be perceived visually via a screen. At the same time, instructions can be given using the microphone or visual signals, such as arrows that appear on the data glasses. With its three-dimensional interaction, the Metaverse opens new possibilities for collaborative or hybrid working.

If we take the remote support scenario and equip the person providing support in the control center with a VR headset, then the following situation is conceivable: While the maintenance worker on site continues to be on duty with AR glasses, the expert can connect to the facility remotely with a VR headset and walk through it in three dimensions. This makes it possible for the person on site to perceive the back-office colleague as a hologram via the data glasses. Although they are physically separated from each other, they still work together on the system. A vision that certainly sounds like science fiction but seems quite imaginable in the context of the Metaverse.

Digital solutions at wastewater plants have a high security relevance (cybersecurity), as these facilities are classified as part of the critical infrastructure. The use of this solution is particularly suitable for regular, obligatory visual inspections of plants that are unmanned and have a high operating cost, but at the same time require data-based inspection. This is especially true for a high number of decentralized and modern plants. For a virtual inspection to be equivalent to an on-site inspection, good Internet access and high data transfer rates are necessary to obtain high-quality images and videos. If the sole focus of the solution is to visually monitor the site; proven video technologies are sufficient. The Internet of Things solution is more purposeful if a holistic system is desired that combines optical and data-based components.

In Short: The metaverse has triggered a new dynamic around the technologies of virtual and augmented reality. While private users and the creative industry already know how to use the potential of virtual reality, the manufacturing industry is currently still acting quite cautiously and sees the added value primarily in augmented reality. A joint innovation project of the Entsorgungsverband Saar and the August-Wilhelm Scheer Institute illustrates the potential of augmented reality and VR headsets for the virtual inspection of wastewater plants.