3D concept creation of permanent geodetic monitoring installations and the a priori assessment of systematic effects using Virtual Reality

3.1 Software environment

A broad spectrum of geodetic software products is available for the handling of 3D models and spatial analysis. There are point cloud additions for CAD software solutions [14], 3D modelling software from laser scanner distributors [15], BIM analysis software [16] and 3D GIS [17]. The functionality of these products is aimed at the analysis of measurement data, geometrical topology and spatial queries and visualisation. Although most of these software products provide the possibility to implement user customizable functionalities by coding interfaces, the dynamic simulation of virtual surveying instruments with mechanical functionalities is hardly possible in real time.

The required functionality for the simulation of the instrument behaviour and sensoric effects in a three dimensional, dynamic and object orientated environment can be found in game engines from the entertainment sector. These software developing environments, which are originally designed for the production of video games, like Unity, the Unreal Engine or the CRYENGINE offer a suitable object oriented 3D coding environment and provide also rudimental physics and light engines [18].

For the development of the experimental software, described in this paper, the game engine Unity (2019.3.3f1) was used. It provides the required degree of freedom with an easy-to-use C# coding interface and supports the usage of Virtual Reality (VR) gear (HTC Vive Pro [20]) to provide a suitable access point to the interactive 3D environment for a user (figure 3). The viewpoint with VR from the ego perspective on the task enables a better estimation of dimensional relations and provides a comparable experience of the scene as a user would get on site. The improved 3D data perception of geodetic datasets with VR gear has already been demonstrated by several other geodetic institutions (see [21] and [22]). In general, the adaption of 3D modelling and data interaction approaches from the entertainment sector is a common process throughout many industries, because this sector has a long tradition in human-computer interaction. The improvement of technical applications with well established concepts from the film industry or video games is widely known as gamification or gameful design [19].

Figure 5

Collision Control with a Autodesk Revit model in the VR application in overview mode (a) and ego perspective with VR (b).

3.2 The virtual surveying equipment

The central element of the 3D simulation is the virtual surveying equipment. The user is provided with a catalogue of items which can be selected and placed directly in an imported 3D model, as it can be seen in figure 3. The current available surveying gear is orientated at commercially available products and not limited to any specific manufacturers. All used items have been modelled in 3D construction software according to public available schemes and available open source meshes. The attributes have been extracted from the corresponding data sheets and laboratory investigations.

Therefore, all items are separate objects that consist of detailed 3D meshes (figure 4), attributes and functions. The attributes and functions are implemented in C# scripts which are attached to the objects in Unity.

These functions handle the interaction between objects and enable basic mechanical functionality. For instance, the total station is able to turn and lock onto prisms, the tripod legs automatically adjust to the ground for a horizontal setup, the joints of the console are moving according to the orientation of the mounting point and the optimal orientation of a prism is automatically calculated.

With this virtual surveying equipment another synergy is created with BIM. For future applications an exchangeable format for surveying equipment would be beneficial to increase the interoperability. Approaches for modelling surveying equipment with the open IFC format [23] do exist, but can still be seen as experimental [24] and are not supported as data templates by any manufacturer yet.

4.1 Collision control

Basic requirements for a monitoring installation are an obstacle free mounting position and an undisturbed line of sight to the target points. For the computation of the intersection of objects the physics engine in Unity provides colliders. A collider is an abstracted 3D polygon of the 3D model with a lower level of detail which is used in mathematical operations for performance reasons. When placing 3D objects into the scene the intersections with all adjacent colliders are computed. If the object interferes with a collider which is restricted (e. g. instrument and building facade), the placement is aborted.

A similar procedure is applied for the validation of the line of sight. Here the intersection of a vector and a collider polygon is computed. Only if the vector intersects the collider of the corresponding prism, the line of sight is set as valid. The operation of vector intersection is referred to as raycast in Unity. Because the method of raytracing [25] is widely used for the computation of shadows and lightning conditions in media productions the raycast in Unity can be carried out in nearly real time. In figure 5 the raycast is performed on a building model, which was designed in Autodesk Revit.

Figure 6

Schematic monitoring setup with two total stations in an urban environment.

According to the result of the raycast the line of sight is coloured. The colour red is applied when an obstacle is detected. If it is valid, it is coloured blue (if the hierarchy of the prism was set as a reference point) or green (if the hierarchy of the prism was set as a monitoring point). This way the user gets direct visual feedback about the possibility of sightings and can directly readjust the placement if needed.

Figure 7

Results of the rotation tests at 7 m (a) and results of the rotation tests at 25 m (b).

As seen in the figure 5 only the geometry information is used for quality assessment at the current stage of development. If a scenario requires a distinction between different materials during the raytracing, for instance, glas facades and concrete walls, the implementation of a loading algorithm for more complex models (with metadata) has to be implemented. This would also effect the placement of the instrumentation, because further restrictions could be implemented. An example would be the stability of the mounting point where the placement of the object is aborted when the required drilling for the rawlplugs is prohibited due to the material of the wall.

4.2 Incidence angles on prisms

It is known that the achievable accuracy of angle and distance measurements onto a surveying prism is directly correlated to the incidence angle of the line of sight [26]. This systematic behaviour has to be taken into account especially when multiple total stations are measuring at the same prism from different setup points. In such a scenario, a situation can occur as it is displayed in figure 6. If not all observation points (green dots in figure 6) are visible form one setup point, multiple total stations have to be operated in a common coordinate system and overlapping measurements have to be carried out onto stable reference points (blue dots in figure 6).

To highlight this issue, laboratory investigations have been performed where a Leica TM50 carried out measurements to different target types (Leica GPR121, Leica GMP101 and a Goecke spherical-prism) mounted on a rotation platform. The deviations of the horizontal angle of the total station measurements are displayed for different rotation angles in figure 7a for a distance of 7 m and in figure 7b for a distance of 25 m.

The Goecke spherical prism shows a more asymmetric behaviour in comparison to the GMP101. Only the standard GPR121 round prim shows at both distances the same systematic behaviour which enables the user to model this effect and apply it to actual measurements. However, the numerical corrections for the GPR121 could be also very high and therefore, a detailed knowledge of the actual orientation of the prism on site would be crucial to apply them accordingly.

Figure 8

(a) Underground monitoring situation with multiple targets (b) Usage of the light engine in Unity onto prisms.

It is better to avoid this effect already in the network design and to operate all target points under safe conditions without applying any corrections in the post processing. Therefore, it is necessary to define an accuracy level for the monitoring installation which represents the safe operation. In figure 7 an accuracy level of 1 mm is assumed for demonstration purposes which leads to an capable working range of ±25° for the GPR101. For the GMP101 an orientation below ±35° is still sufficient and because of the inhomogeneous effect and the increased standard deviation the acceptable orientation of the Goecke sphere was set to only ±20°.

By defining a certain accuracy level in the VR application, the working ranges for the available prism types are automatically set. Based on the implemented mechanical functionality of the prisms and the known normal vector of the 3D surface of the model, also the optimal prism orientation for the setup is computed. The prism’s centreline is oriented towards the normal vector of the surface or it is matched to an existing mounting point. Afterwards the prism is virtually turned towards the centroid of all contributing total stations. At this stage all incidence angles are calculated with vector geometry and marked in red colour if the angle exceeds the defined limit. The final prism orientations (with respect to the rotation axis of the prism) can then be exported along with the mounting coordinates in ASCII format to support the installation on site.

4.3 Proximity of adjacent prisms in the field of view

A monitoring system based on total station measurements delivers only deformation data at discrete points which are stabilised with a reproducible target sign. To increase the spatial resolution of the monitoring setup, a high number of prisms is usually placed into the scene as seen in figure 8a for an underground tunnelling scenario. Due to the high dynamic of a tunnel excavation site (trucks, blasts) and the fast progress of the drilling these convergence measurements are carried out manually on a movable tripod. Therefore, distinguishing between prisms is done by the human operator and is not the major problem in the displayed scenario. If this scenario takes place in a geotechnical challenging area where long term observations have to take place with a high temporal resolution, the automatic operation of a permanent installation is more feasible. This applies even more when the site is inaccessible to humans because the effort of reaching it with the required temporal resolution is unreasonable or if the site is exposed to life threatening conditions such as unstable geology or radioactivity.

In this case the aiming of the total station has to be carried out by automatic aiming technologies which are implemented in all modern robotic total stations. The majority of the used aiming technologies base on the emission of infrared light, its reflection at the prism and the evaluation of the backscattered light with an infrared detector. Depending on the used approach the total station can detect multiple prisms in the field of view of an infrared camera [27] or aims at the intensity centre with a quadrant detector [28].

Each technology has a different realisation in the instruments. Therefore, the field of view of the aiming mechanism differs between instruments (table 1). It can be seen that the visual fields of view of most available instruments are the same with an opening angle of 1.5°. Regarding the automatic aiming technologies the Leica TM50 has a standard field of view of only 1/3 of its visual field of view (28’). Contrarily the Trimble S-Series has a bigger field of view of the Autolock technology (2.1°) compared to its visual field of view (1.5°). This means that prisms can contribute to the aiming process of Trimble S-Series instruments even if they can not be seen through the telescope. For long range applications the field of views of the automatic aiming technologies can be additionally reduced with narrow aiming with specific Leica and Trimble instruments.

Depending on the technology, failures in the automatic aiming process show different behaviour. On the one hand, a quadrant detector is more likely to aim at the common centre of multiple prisms on the other hand, camera-based approaches are more likely to snap to a false prism. Either way, when having multiple prisms in the search cone the likelihood of a system failure is dramatically increased.

To avoid any distractions, a monitoring configuration should foresee only one prism in the search cone for a safe operation of the system. Therefore, an abstracted automatic aiming process was integrated in the VR application to make the user aware of this problem while placing the targets. In a first step the search cone is implemented as a collider that detects all prism colliders within it. This reduces the amount of prisms that have to be further investigated. In a second step, the light engine in Unity is used to detect if a prism is in the search cone, but is not hit by the search light due to obstructions. A virtual camera which is placed in front of the prism detects the colour shift of the prism when it is hit by emitted red light. Therefore, it is also possible to detect if the prism is only partly hit by the red light as seen in figure 8b.

Figure 9

Underground scenario with confidence ellipsoids.

The working principle of emitting a red light and detecting it with a virtual camera has worked out stable with the physics engine that is provided in Unity. This opens up the possibility of a more detailed modelling of the automatic aiming technology in future applications by using reflective surfaces at the prisms and by placing the virtual camera directly in the optical context of the telescope of the simulated instrument.

Figure 10

3D model of the tunnel interior and measurement concept.

4.4 3D simulation of the error distribution and confidence ellipsoids

The line of sights of the network configuration are stored in a database, which is updated with every user input. If a feasible configuration is found by the user a traditional network simulation with a Gauß-Markov model can be carried out to obtain confidence information for the planned setup points. Based on the line of sights between instruments and reference points the design matrix is automatically derived. Also the variance matrix of the observations ( Σ l l ) is filled with the corresponding stochastic attributes of the implemented surveying items. In figure 9 the 3D representation of the resulting covariance matrix in form of confidence ellipsoids can be seen. This 3D representation is a clear improvement compared to traditional 2D network sketches with confidence ellipses. Moreover, the 3D simulation of the confidence ellipsoids can be easily adjusted by adding and deleting connections in the database in real time.

5.1 Task description and requirements

The installation of a 3D permanent monitoring in a geologically challenging area in a railway tunnel has been considered by the tunnel operator. The concept foresees six cross sections which are approximately 30 m apart from each other, see figure 10. The distance between the cross sections is related to the common measurement setups in tunnelling defined in the monitoring handbook by the Austrian Society for Geomechanics (OeGG) [33]. Also based on the OeGG handbook in this measurement setup, each section should be equipped with at least five well distributed observation points. Due to the remote location of the deformation area the automatic operation of a total station was intended because the train operation of the tunnel should not be disturbed by unscheduled transportation of surveying personal into and out of the tunnel.

The monitoring concept had to be investigated at a time when the excavation of the tunnel was still in progress, therefore an on site visit was not possible and the concept creation had to be carried out virtually. The complexity of the task resulted from the curvature of the tunnel walls at the cross sections but also from the curvature of the tunnel itself. Moreover many inventory objects for the train operation like marker boards, railroad signals, power lines may obstruct the line of site in the final building. Furthermore, safety regulations had to be considered, e. g. the pedestrian walkway should be kept free of obstacles and any instrumentation must not interfere with the safety space of the trains or the power lines.

To cover all requirements in a single environment and to sustain a reliable concept creation the object orientated approach inside the VR application was preferred to a conventional approach of manual construction inside a CAD environment.

The design of the monitoring system required a 3D data foundation which was not available for the required tunnel section. Instead, construction schemata of representative tunnel cross sections, inventory plans and a georeferenced centreline of the tunnel were provided by the tunnel operator. The 3D model that can be seen in figure 10, which was then used for the concept creation, was modelled according to the given information within the conventional 3DReshaper software. Every inventory object has been modelled separately and placed into the scene according to the construction plans.

5.2 Console design and placement of the total station

The first step of the design process was to define the mounting method of the total station. The stabilization should not interfere with the safety area of the pedestrian area, nor should the stabilization affect the handrails of the sidewalk. The placement should also take into account that the total station should be accessed without the use of a ladder for maintenance purposes and should be easily operable if needed. To decide which of the available console solutions fit for this scenario several types have been modelled in 3D according to the construction plans of the distributors. Two commonly used types can be seen in figure 11.

The safety spaces were introduced as separate objects with colliders which have been taken into account while placing the consoles and the instrument.

Both consoles have been placed in an user operable height into the scene without intersecting any relevant colliders, therefore, both displayed console types would fit according to the formal requirements. Because of stability reasons (console 14-TK3 has a long lever) the usage of the 14-TK400 is preferred in this tunnel setup. Moreover, the VR application showed clearly that the standard model of the console can be used without any customizations.

Figure 11

Placement of the total station on (a) console 14-TK3 and (b) console 14-TK400 in the scene.

Figure 12

(a) Number of prisms in the field of view of 28’ (b) Tunnel cross section with target mounting points and restricted space (highlighted in red).

5.3 Prism configuration and automatic target aiming

The required cross sections have been imported as polylines into the VR application to highlight the area of interest to the user. During the placement of the prisms several obstructions have been indicated by the collision detection at the top position near the power line. To avoid any interference with the power line, these top points have been moved to the right-hand side of the power line afterwards, as seen in figure 12b. Obstructions have also occurred for several prisms at the same height as the total station. Therefore, these prisms have been moved in height to avoid an alignment of the line of sights.

To cover a wide selection of possible instruments for the simulation of the automatic aiming process the most common field of views (2.1°, 1.5°, 28’ and 9’) have been used, according to the instruments in table 1. The simulation of 2.1° and 1.5° revealed a very limited operability of the configuration. A slightly better result has been obtained with an opening angle of 28’. The result of this investigation can be seen in 12a, where the table displays the number of prisms in the search cone when aiming at specific positions. Barely, the fist two cross sections can be operated remotely in an automatic setup under the assumption that multiple prisms have to be avoided. Especially, the prisms at position 5 are likely to cause disturbances. The requirement of only on prism in the search cone can only be fulfilled by instruments supporting an automatic aiming with 9’.

Figure 13

Enhanced flowchart regarding the 1st and 2nd order design for geodetic network planning: Standard procedure (black) and enhanced modules (green).

5.4 Results of the feasibility study

For all placed items in the scene, the required mounting points on the tunnel wall are calculated automatically. A list of ASCII coordinates with the drilling locations for the wall plugs can be exported and delivered to the contractor. Also the instrument height and the prism orientation can be extracted to support an optimal installation of the system. This should guarantee the operability of the real permanent installation as it was simulated in the VR application. For future applications also the sharing of the simulated 3D objects would be possible if the contractor prefers to work in a BIM related environment. Therefore, the shared datasets in an object orientated approach deliver more information, because the result is not only a list of coordinates with an accuracy estimation, moreover, each coordinate can be linked to a 3D object and its metadata (e. g. costs or installation procedure).

The automatization of the quality checks of the network design resulted in a reduced time effort. For comparison and evaluation purposes several automatic aiming scenarios have been checked also manually inside the 3DReshaper software by drawing lines in a CAD like environment and checking their intersection with objects visually. Double the time was needed for a single configuration compared to the VR application. The improvement can be seen in the adjustment and the recalculation of setups. In the CAD environment the manual operation time scales linearly with the number of tasks. In an object orientated approach most of the tasks are carried out automatically without human input. In the given task, where four different types of instruments have been simulated at 30 targets and the configuration had to be readjusted twice, the time reduction can be assumed in a total ratio of 1:5 and the result can be reproduced any time by different experts.