Matschoss 2.0 - Virtual machine collections as the missing link between museums and historic monuments

1 Introduction

The history of technology is one of the small subjects that try to bridge the gap between technology studies and humanities. Material objects have played a certain role in the history of technology from the beginning. Many objects have therefore been digitized in recent years, especially in museums and university collections. Therefore, the advantage of digitalization that has so far mainly made use of is to heighten the focus of both research and knowledge transfer on (technical) objects in cultural monuments or landscapes. In this article, we want to demonstrate the possibility to tie in with old traditions and potential of new technologies opening opportunities for efficient way to solve this problem (to put it bluntly, “Matschoss 2.0”). The article, however, focuses less on the motives for such projects rather than on examples providing theses-like answers on how to approach them.

We have already discussed digitization methods, processing and benefits of digital models as part of the 3D project [1], [2], [3], [4], [5]. We regard digital twins as a supplement and aid to the real object and for this reason consider their use to be sensible. The aim of their digitization project is to promote research on the object and to support it with retro-digitized material. The former is only possible as long as the objects exist. No historical buildings are as endangered as the technical ones. This chapter therefore aims to address the problem of how retro-digitized copies of historical machines can be made available in a compatible way for research into the history of technology. Reusability must be considered from the outset. This could be applied to all areas of digitization: digital recording, documentation and indexing, publication, use and archiving [6].

In the knowledge that the problems mentioned in the field of digitization are not unknown, that solutions are being worked on and that the results will still take time or are not yet ideal due to external conditions, the authors would like to explain the needs for the field of the history of technology.

Our examples are historic machines as a central field of investigation in the history of technology. The purpose of a machine is its function, which is accompanied by mechanical motion sequences. A technical device that is still in operation or only partially in operation is more meaningful than one that has been shut down. If it is no longer functional, an important part of understanding the mechanics is missing. Without this knowledge, answers to questions are less precise and often no longer reproducible, especially for non-measurable variables [7]. However, this is a basic prerequisite for a scientific approach to a technical object.

Our article places first the collecting of virtual models of machines in a historical context (2), gives then an overview on the appropriate methods of digitalisation (3), treats the use of the virtual 3D models in general (4) and finally especially the aspect of animation and simulation (5).

2 Collecting machines in history

Conrad Matschoss (1871–1942) was one of the most important players in establishing the history of technology as a scientific discipline. With regard to power engines, he stands on the one hand for the tradition of researching them, and on the other hand for the establishment of machine collections of various kinds: generations of technology historians have dealt with his “History of the Steam Engine,” on which Matschoss first published in 1901. By elaborating this topic into the two-volume, scientifically oriented “Development of the Steam Engine” (1908), Matschoss broadened his knowledge of exemplary objects. It is therefore not surprising that he, who was director of the Association of German Engineers (VDI) from 1916, wrote the contribution on “Power Machines” in 1925 in the catalog published by the VDI for the opening of the permanent exhibition of the Deutsches Museum. The reference here was to a material collection that included original power machines, replicas and models of power machines. This already contained a certain amount of immateriality, in that reference was made to originals that no longer existed or were preserved elsewhere [8].

Seven years later, Matschoß had finally arrived at a quasi “virtual” collection: In the volume “Technische Kulturdenkmale” (Technical Cultural Monuments), which he published together with Werner Lindner in 1932, he again wrote the contribution on power machines. The individual machines, which were scattered all over the empire, appear there only as pictures and are described less than was usual for other monuments at that time. Nevertheless, this volume can be seen today as the beginning of a series of inventories along which industrial monument preservation developed [9].

Conrad Matschoss thus also had material sources in mind, both those in the museum and those in the cultural landscape. With regard to the question of the digital networking of collections, his role, sharpened as “Matschoss 1.0,” can be classified as follows in the development of collecting machines:

3 Digitalizing machines

Machines as a part of the Industrial heritage represent a big variety of objects, which differs in their dimensions, constructional complexity, disposition (“built-in” or detached), material property and functionality.

So it is a challenging task, to find a suitable approach to digitize such samples in a necessary manner. For each individual case should be decided, what parameters and characteristics, like geometry, colour and texture. Material properties (e.g. reflectivity, roughness) or internal invisible structures (built-in parts or mechanisms) are relevant for documentation and which imaging sensor can capture this information.

3.1 Methods in overview: laserscanning

In general, the optical imaging methods can be represented through two principal approaches – active and passive measurements. The active imaging sensor emit a signal to objects capturing mostly geometrical information in form of 3D coordinates or distances relatively to a sensor position. The passive imaging use ambient lightning (natural or artificial) to capture light waves reflected from objects (no additional signal will be sent from a sensor).

The active imaging sensors use different light emission techniques. The laser based sensors send a laser signal (Light Amplification by Stimulated Emission of Radiation) to identify pointwise a distance from sensor to object surface calculating time needed for laser beam running to object and back (ToF/Time-of-Flight technique) or wavelength phase difference of emitted and reflected laser signal (Phase-shift technique)

Such sensors build a big methodology group named LiDAR (Light Detection And Ranging) represented through a number of different 3D laserscanners, which differs in their range (from close- and medium-range with laser emission capacity till 150 m to long-range till 1 km), recording location (terrestrial/TLS, aerial/ALS) and scanner positioning (static or dynamic).

Horizontal and vertical directions are measured using angle encoder devices, resulting in spatial polar coordinates for each scanned point. Then XYZ coordinates related to the device coordinate system can be derived, which can be transformed into a parent object coordinate system via coordinate transformation. The result is a three-dimensional point cloud with 3D coordinates, intensity values representing laser beam reflectivity from object materials and colours, if the digital camera has been applied in parallel (built in scanner) (Figures 1 and 2).

Figure 1:

Samples of heritage objects captured with a 3D structured light scanner Artec Spider ©University Bamberg, Digital Technologies in Cultural Heritage.

Figure 2:

Samples of photogrammetric captured objects in different sizes (point clouds and meshes) ©University Bamberg, Digital Technologies in Cultural Heritage.

Terrestrial close- and medium-range scanners are often applied for capturing relatively large objects like architecture (castles, palaces) and archaeology sites (caves, headstones, graveyards) [15], [16], [17], [18].

3.2 Scanning smaller objects like machines

For out contribution, we focus on power machines as Matschoss had it in focus. With regard to the mentioned parameters and characteristitcs, one can arrange this power machines in the following order:

1. Its size range from about 2 m to about 25 m of the longest part

2. The geometry is often very complex, e.g. concerning objects with open gear wheels.

3. In the objects of the 20th century, this wheels and other structures are often internal

4. As metal is a dominant material (besides some wood in the older machines), there is often the problem of reflectivity.

In relation to the System of Luhmann concerning the relationship between measurement methods and object size and accuracy [19], only a part of the methods is relevant (Figure 3).

Figure 3:

Measurement methods relatively object size and accuracy clearly presented in Luhmann et al. [19].

The method named light sectioning approach, which is often applied for accurate surface inspection in industrial production and monitoring captures the surface with μm-details, however, without colour information and is not relevant for our purpose.

Meanwhile the Near light sectioning approach, easy-to-use sensor technology with high recording speed, precise geometry with high detailing and texture mapping is widely used in industry as well as in cultural heritage documentation [20], [21], [22], [23]. Here, 3D close-range scanners use structured light technique, where device projects a light pattern (net, strips, points, stochastic pattern, etc.) on object surface and calculate its 3D form observing with built-in cameras a pattern deformation on object surface. In this case, object geometry can be captured in sub-mm range with colour and texture information.

The passive technologies are based mostly on photogrammetric approach including different camera sensors like professional and amateur cameras, video cameras, drones and smartphones. 2D and 3D object geometries as well as colour with texture can be captured using digital images and photos. For a 3D reconstruction, series of photos should be done around an object from different positions and directions. Using Structure-from-Motion algorithm, 3D object points be calculated from the set of overlapping images. After orientation and alignment of images, a photorealistic 3D model will be generated with high accuracy [19].

A 3D model includes a point cloud or a polygon mesh to be coloured or textured. Up to objective capacities, a detail reproduction namely GSD (Ground Sampling Distance) varies between sub-millimetres to several metres, which allows to capture different sized objects.

Photogrammetric systems using camera are assessed for accuracy according to the guideline VDI 2634 [24]. Imaging approach can achieve maximum length measurement deviations of less than 0.03 mm in 1 m³ according to VDI 2634.

The photogrammetric method “Structure from Motion” originally comes from computer vision [25] and is widely applied in the documentation of cultural objects. The combination of sensors (e.g. laserscanning and photogrammetry) is also becoming increasingly important 3D reconstruction [19], [26], [27]

However, optical methods come with certain limitations, primarily concerning the accessibility of objects (or their components), the complexity of their geometric shapes, the interrelationship between parts and the surface textures. Figure 4 effectively illustrates the challenges involved in digitizing cultural industrial artifacts:

1. some components are embedded within a wall, making them difficult to record or accurately reconstruct (Figure 5);

2. larger and smaller details cannot be captured using the same sensor, necessitating the use of a combination of recording techniques or sensors (e.g. laserscanning for room and large parts as well as photogrammetry for fine details and realistic textures);

3. object textures may be rough, dusty and lacking distinct features, which can significantly impact photogrammetric reconstruction and structured light scanning.

Figure 4:

Virtual reconstruction of the Lower Waterworks, Schloss Schwetzingen using CAD by C. and M. Steffen ©LAD, RPS (https://bit.ly/3WrpCsK).

Figure 5:

Virtual reconstruction of the Lower Waterworks, Schloss Schwetzingen using CAD by C. Kopanitsak ©ITZ, KIT (http://bit.ly/4aw7pQv).

Especially for recording of complex construction and mechanisms with internal structures methods like computer tomography (ICT) or ultrasonic approach, are more suitable for object structure exploration.

ICT is an active imaging method, where the X-ray beam serves as the emitter (transmitted signal). Due to the X-ray beam properties, it is efficiently used for the internal structure capturing and suitable especially for inhomogeneous objects [28], [29], [30]. In addition to common medical applications, the method is used for the digitisation of glass objects, natural objects in combination with photogrammetry, as well as for the recording and inspection of heritage and industrial objects [31], [32], [33], [34], [35].

4 Using virtual 3D models

Creating virtual 3D models of complex objects such as historical machines, known as retrodigitized models, is no easy task. If you want the geometry of the original, including deformations, the object must be digitally captured. A sufficient data basis for alternative approaches is usually not available.

The digitally captured machines are seen as a single coherent object, as a surface model. This means that all the individual parts are fused together.

If you want to illustrate more than the outer shell, such as the materiality, condition, etc., in order to also show the internal processes, the parts of the object must also be recreated virtually. This is because the individual parts that were captured externally in the surface model cannot be easily removed. The result would be fragmented parts of the surface with significant holes, which would not be of much use for further observation or investigation. The question is whether to dismantle the machine completely and capture the components individually, or whether to reconstruct the individual components. The virtual reconstruction is a less complex option for larger machines but currently has to be carried out manually. Automated conversion processes are not mature enough for complex objects. After the components of the machine have been reconstructed, the object can be animated with a replica (see chapter 5).

The function of machines consists of mechanical processes. The representation of these movements is a fundamental prerequisite for understanding the objects.

Retrodigitals are already in use in the field of education: in university teaching, in museums, in the field of cultural heritage sites, etc. Depending on the requirements, they are prepared in a more or less simplified form in order to convey complex processes in a supportive manner. Above all, however, they represent an end product that is usually presented passively to the user. Although 3D viewers allow the user to be more active, such as rotating, zooming, viewing by driving in, etc., access to the model itself with all its data is usually not possible in these applications. Only appropriate software allows complete access. A prerequisite is knowledge of how to use them.

If you want to create virtual 3D models as a source for research, the models and data must be made available for use accordingly. Also the changeability of the data must not be ignored. With the advent of new technology, new calculation methods allow the 3D model to be displayed differently from the previous one. The existing (research) databases cannot cope with the work with virtual 3D models.

Some research work in the field of digital collections does not yet reflect the concrete use of digital models as a knowledge repository. Models are still being analyzed and compared with the original [36], [37]. The approach is sometimes superficial and does not exploit all the possibilities. Just as hyperspectral photography reveals things in paintings that you would otherwise not see, 3D objects can also expand the possibilities for researching the technical artefact. For example, by adding missing components, calculating forces with physical simulations (see chapter 5), viewing the machine from a bird’s eye view – all without invasive intervention. However, one reason why these possibilities cannot be fully utilised is due to the fact that 3D tools for scientific research are not yet fully developed. They require a high level of computing power, as the operation and reading of the models into the corresponding software is more complex. And most good programmes are either expensive and proprietary or are still at a stage of development where they are not yet fully functional.

Extensive work with digitized material takes place primarily when digital methods are tested or an object is digitally recorded in connection with a research question [23]. The recording and thus also the processing is then usually focused on a specific problem and takes place within a working framework that is generally not available to researchers, such as technical equipment, digital know-how and tools. The digitized material is primarily used to find a solution. The application is therefore predetermined, but also limited. A specific question would not be sustainable when creating a digital collection of historical machinery in situ for teaching and research. The data collected should contain answers to as many questions as possible and be able to answer further questions with the digitised objects that cannot be answered with the originals.

In archaeology, for example, whose sources mostly consist of three-dimensional finds and investigations on the object, such as surveying, are common – whereby digital techniques have long since been introduced. In the history of technology work on analog objects is rarer. Here the step to the digital model is even further. Although virtual models are slowly finding their way into teaching, they are still a rarity, possibly because many people are not yet aware of the possibilities offered by digitised objects. Apart from the fact that there are still too few digital collections of machines or that they are still being developed, most of them have not been created specifically for research purposes or do not meet the requirements. How these could or should be designed has not yet been determined and requires discussion. Detailed standards exist primarily in the 2D area. For digital recording in 3D, workflows from related areas have so far been adapted and customized.

5 Animation and simulation

Although imaging digitization processes can record the machine geometry, dynamic components such as kinematics can only be recorded in the configuration available at the time of recording. This greatly reduces the added value of digitized data for further research.

In the case of decommissioned systems, the virtual animation of the digital copies counteracts the loss of vividness. With animation technology, individual sequences are positioned manually. Movement is generated by calculating the interpolation between the individual images.

The process is suitable for understanding the mechanics, which are not readily apparent in complex systems. However, the representation of functions using animation has its limits. Although the general process is simple, its complexity grows according to this of the object. Above a certain point, this is not to manage with an appropriate effort. Although it is possible to create dynamic models based on scan data, this is hardly practicable for large collections, complex systems or extensive facilities. In addition, physical motion sequences cannot be represented correctly. Changes to the animation can only be made holistically, as the animation technology cannot independently transfer individual changes to the remaining processes and has to be carried out manually for the most part. There is no connection between the various components and therefore no knowledge.

Further and more in-depth analyses and research questions regarding the mechanics are not possible, as the animation process does not produce any data. Without data, no new insights can be gained and problems remain unsolved. Systems that are no longer in operation and whose modes of operation can no longer be fully deduced from sources require other solutions.

A simulation is the closest possible reproduction of events in digital form using a model. It provides realistic predictions of actual behavior. Virtual simulation is used in plant engineering to validate the functioning of mechanical components. It can realistically reproduce physical motion sequences of mechanics by calculating the development of the dynamic model components in real time on the basis of physical rules and regulations.

The advantage over animation is that the simulated behavior of machines is more realistic and comprehensible, as every parameter of the simulation model can be read out and displayed at any time. This also allows further questions to be answered that depend on pressure, temperatures or forces, for example. Another advantage is that the dynamic simulation model is interactive, allowing users to control a machine or vary external parameters depending on the programme used [43].

Methods already exist for application to CAD models. The mathematically defined surfaces are an advantage here. The algorithm developed for the geometric analysis of shafts and gears calculates the simulation parameters for the required components of a kinematic simulation, the physics and mechanics simulation. Thanks to the geometric analysis, the kinematic simulation can be parameterized fully automatically. This means that a complex gearbox model can be converted into a kinematic simulation model and simulated in real time.

Photogrammetric models are covered with a dense mesh. For this reason the algorithm does not recognize geometric features without errors and requires manual support. The model has to be segmented, classified and structured manually. The effort involved is great and the results are not reliable.

The procedure was tested on the model in Figures 4 and 5 and realised in PolyVR. When the model is imported into the program it ideally detects the corresponding gears, their axes, and the number of teeth. Based on these parameters and the spatial positions of the other gears, the program can already perform a real-time kinematic and mechanical simulation applied to the individual components. In this process, all components respond to the mechanical translation, which can also be modified. This represents a significant difference compared to animation, where the speed of each individual element must be set manually and does not respond to other components or the environment. The calculated values of the gears are displayed by the program, as shown for instance in Figure 6, as wavy lines. In the future it should be possible to transfer these and other parameters into additional simulation programs to analyze, for example, material weaknesses or heat development, enabling the investigation of even more complex research questions.

Figure 6:

Simulation of 3D-photogrammetry-model with PolyVR by V. Häfner ©IMI, KIT (https://bit.ly/40qdVna).

Despite some problems, such as the overshadowing of frequencies by the mesh, it was possible to parameterize the simulation and determine the axes of rotation and the periodic properties of the toothed and comb gears. However, the simulation of the mechanics using virtualization methods from the CAD model to the photogrammetry model still requires more in-depth analysis and application to various mechanics. There are not only unanswered questions about further application, but also about how the data can be made available to users. What is certain is that the simulation is only semi-automated. The process is still too complex for the use of AI. Answers can therefore initially only be used for pre-defined questions [43], [44], [45]

Online 3D viewers are currently being developed for the representation and accessibility of animations or simulations, including the 3D viewer Kompakkt, which now enables animation within the tool itself [46]. Annotations on the models, which can be placed at individual points and explain them with specific information, are also very useful. This makes it possible to record and understand correlations more quickly. It is also advantageous if users can add their own annotations to support their work with the model. In the future, the use of AI would be conceivable and helpful here, which could provide data on further questions on a selective basis. These functionalities are essential for them to be useful in research. However, these viewers are still under development. On the one hand, the difficulty lies in making the use of 3D objects as user-friendly as possible, which is associated with technical challenges such as keeping the loading time of the objects as short as possible, as well as optimising ease of use such as intuitive navigation in space or the use of interactive elements. If these elements are missing, they only represent preview images for inaccessible machines.

Another way to represent these technical functionalities would be to bring the 3D models from their digitized form into the analogue world using 3D printing. However, this would require a CAD model, which would need to be reconstructed with the intention of being subsequently 3D-printed and assembled. However, it would make it possible to test the functionalities on a smaller scale in the real world if necessary and would probably come closest to a reconstruction in the traditional sense. That being said, the physical data would then have to be obtained again using measurement instruments, as it would not be directly available through simulations in the digital form.

6 Conclusions

Compared to other disciplines, research into historical machines can still be quite time-consuming these days. Most of the information is still on paper and has to be collected from different places and areas (libraries, building research, monument preservation, museums, companies). It becomes even more difficult when comparisons have to be made. Each comparison doubles the effort. Written sources are not enough either. The facilities should be inspected on site, but these are not always accessible. Some technical installations have been moved to museums and thus taken out of context. The associated buildings, which also tell a lot about working conditions, have often been demolished. Here you have to make do with photos. Documentation with plans and descriptions are very helpful here, but these were not always produced. In general, there are still many gaps in the information available, as objects were not indexed or researched in detail for a long time.

Digitization enables broad indexing and easy access to all technical objects – at any time, in any place. A digital collection enables a uniform, standardized description and indexing that promotes and advances research on the object. Objects can not only be compared with each other, but also linked and reassembled. Musealized installations can be completed virtually by recreating their original location as a virtual supplement. This is how Matschoß’ goal could be achieved.

All that needs to be done is to promote the digitization of such facilities and to set up digital research infrastructures to make them accessible.