Development of GPS time-based reference trajectories for quality assessment of multi-sensor systems

1 Introduction

The continuous development and improvement of multi-sensor systems (MSSs) is still a major research and business area. The beneficial combination of different sensor technologies as well as the optimization of intelligent algorithms for data fusion and interpretation, enables a growing diversity of new possibilities (e.g. save autonomous driving in complex scenarios or efficient mapping in rough environments). What all these ongoing developments have in common is that completely new fields of application are opened up, a more efficient way of operating is made possible or, more particularly, improved accuracy and precision are to be achieved [1–6].

Within the framework of well established quality assurance requirements, it is necessary to carefully validate such complex MSS in terms of their accuracy, precision, reliability, robustness, completeness and integrity using a suitable measurement and evaluation process. Only in this way, reliable statements can be made about the possibilities and limitations of an MSS with regard to the respective area of application. This also highlights the problem that there are no standardized procedures with which the specifications of an MSS can be comprehensively described in terms of their quality, nor can they be easily verified by the user.

The quality of an MSS is basically composed of a multitude of influencing factors [7, 8]. In this paper, the aspects of quality to be assessed will be limited to the pose of the MSS with its six degrees of freedom (6-DoF). In addition to this georeferencing component, other aspects such as the quality of the acquired information from object space may also be relevant, in case the corresponding MSS is operated with acquisition sensors (e.g. laser scanners or cameras). For this aspect, we refer to Refs. [9, 10], for example.

To assess the georeferencing quality of an MSS, appropriate reference information must be available. It therefore requires at least a 3D position or even a 6D pose, which can guarantee superior accuracy (at least 10× more accurate as a rule of thumb), serving as a ground truth. This becomes particularly challenging for kinematic MSS, which are prevalent nowadays. In this case, a time-dependent trajectory has to be realized, which continuously serves as a reference. Since modern sensor systems can meanwhile achieve remarkably accurate pose solutions in the sub-centimeter range [11–14], this imposes high demands on the reference system used.

Basically, there are two approaches for assessing the georeferencing quality. This can be done either directly via independently determined trajectory information or indirectly on the basis of derived quantities. An example of derived quantities is the comparison between the acquired 3D point cloud of the MSS and a suitable reference point cloud [6, 8, 9, 15]. This approach is quite suitable and justified, but it excludes all systems without object space sensing for the quality analysis. In addition, suitable reference point clouds must be provided and the inner quality of the object space sensors significantly limits the validity. This can lead to a mixture of influencing factors and no reliable statement can be made about the quality.

To obtain an unambiguous statement, it is, therefore, advisable to use approaches that directly determine the 3D position or 6D pose independently. Global navigation satellite service (GNSS) measurements correspond to such a direct determination and are widely used to validate kinematic MSSs (e.g. [16, 17]). Even though there are significant advantages in terms of worldwide availability, the achievable accuracies are often no longer sufficient for high-precision kinematic MSSs. This is aggravated by the fact that GNSS can only be used in outdoor spaces and that multipath and shadowing, for example, have an additional negative impact on uncertainty, especially in challenging areas [18–22].

A reference trajectory can also be acquired with other sensor technologies. Motion tracking by means of camera systems, which are mainly used for small measurement ares of less than 20 m × 20 m, should be mentioned in this context. Measurement rates of typically 100–360 frames per second can be achieved [23, 24]. In addition, there is, for example, the real-time 6 DoF tracking system iGPS from 7dkmetrology. Several target objects can be tracked simultaneously by special sensors using infrared laser transmitters via multi-lateration. The measurement environment that can be covered is virtually unlimited by using a multitude of transmitters. Measurement frequency is between 40 and 50 Hz. However, the achievable position accuracy depends not only on the number, position and intersection angles of sensors and transmitters but also on multipath effects [25].

As a flexible solution, applicable both indoors and outdoors, external tracking via robotic total station (RTS) or laser tracker (LT) is a suitable option [20, 22, 26], [27], [28]. Both sensor technologies provide polar sensing elements and can reliably track a kinematic MSS, provided that a continuous line of sight to an attached reflector is maintained. Modern instruments can also overcome very short interruptions in the target beam. A drawback in comparison to GNSS is the limited target range, which is about 100 m for an LT and up to 2000 m for an RTS in radius. However, this should be sufficient for the majority of applications, especially since e.g. motion tracking by cameras cannot compete with this measurement volume anyway. The biggest challenge in the field is to maintain a continuous line-of-sight connection. The quality of such a reference trajectory is further influenced by the measurement configuration, the reflector used (corner cube reflector [CCR], circular prism or 360° prism) and, in particular, the speed and shape of movement of the kinematic MSS. One particular challenge, however, and the main aspect in this paper, is the temporal synchronization of both mentioned tracking sensors to the sensors of the MSS to be assessed or, more generally, to an absolute time base. Both are currently not supported by the manufacturer for RTS and LT.^[1]

As a limitation, both the RTS and the LT can only measure three-dimensional reference trajectories with respect to a reflector. But even here, further independent information may be required, e.g. if the reflector to be tracked does not coincide with the reference point of the MSS and a lever arm has to be considered instead. For the additional determination of orientations, either special tools (e.g. T-Mac or T-Probe for a Leica Geosystems LT) [6, 8] or the simultaneous use of several instruments [20], each pointing at different reflectors on the MSS, are required. In the latter case, reliable and precise synchronization is essential. Thus, to utilize an RTS or LT to generate accurate and reliable reference information for a kinematic MSS, the following questions must be addressed:

1. How does the measurement configuration influence the quality?

2. How does the reflector affect quality in kinematic systems?

3. How to synchronize external instruments for tracking with an MSS?

4. How to establish a link with absolute time?

Multiple studies have already been conducted on the first two aspects (e.g. [27, 29], [30], [31], [32]), while research is still required on the last two questions. With respect to the LT, no absolute synchronization applications are known so far, except for the investigations carried out by the authors themselves [8, 26, 33]. There are a few approaches and procedures for synchronizing an RTS with additional RTSs or other sensors relative to each other. These are based on synchronization of two RTSs by cross-correlation on the software side [34] or on a sophisticated calibration of intrinsic synchronization parameters in the laboratory using a kinematic motion model [35, 36]. In Ref. [34] as well as in Ref. [36] GPS time is also used to some extent. However, this only refers to a reference solution for investigating the internal time of an RTS (in Ref. [34]) and for examining the synchronization quality of communication between an NTP time server and client over different LAN and WLAN connections (in Ref. [36]). In both investigations, it would be possible in principle to realize an absolute temporal reference of the RTS observations to the GPS time in the respective setups. However, this was not the focus of these investigations. In Vaidis et al. [20], three individual RTSs are synchronized to determine the 6 DoF of a mobile platform in real-time. Using long-range radio communication, each RTS connected to an embedded computer is synchronized with a central master on the mobile platform. Consideration of sensor-specific internal latencies of an RTS are not mentioned. Stempfhuber [29] proposed a real-time acquisition system for the combination of geodetic measurement systems (specifically RTS and GNSS) in kinematic applications based on the Coordinated Universal Time (UTC). Such a realization is presented in Hesse et al. [37]. They rely on a self-developed real-time computer, which performs the synchronization of an RTS from Leica Geosystems regarding GPS time. However, details about this approach as well as investigations of the quality are not presented.

Previous research on quality has therefore been limited to the relative synchronization of RTSs. How reliably the original measurement elements of an RTS can be related to GPS time has not been investigated and validated independently and with superior accuracy in the articles mentioned above. Therefore, a more detailed approach for synchronization with absolute GPS time was already presented in Vogel et al. [33] and compared with a reference solution in a first test setup. The continuation of this work, in particular the detailed long-term investigation of the achievable quality in kinematic tracking, is presented in the following. The methods for absolute synchronization of the polar measurement elements refer to those of both an LT and an RTS from Leica Geosystems. While for the LT a precise trigger signal is used for the temporal relationship, for an RTS the Measure & Stream application distributed by the instrument manufacturer is to be investigated for its performance. This software is intended to provide a simplified option for both internal RTS and external data synchronization, as opposed to the complex calibration routines in Refs. [34, 36]. The disadvantage is that it is a black box solution, especially for RTS internal synchronization. This makes it difficult to overcome latency issues. The investigations are based on a rotating disk as well as a rotating robot arm, each with a mounted reflector and constant speed of movement. The solution based on the LT can also be regarded as an independent reference for the approach based on the RTS.

In the following, the respective concept for linking the raw measuring elements with precise absolute time information is presented in section two for both measuring instruments. In the context of two experiments, the concepts are validated and current limitations are discussed within section three. Finally, future improvements and further investigations as well as application scenarios are addressed in section four.

2 Concepts for GPS time-based reference trajectories

The implementation of an absolute time reference for a local instrument clock is highly dependent on the instrument design. Basically, one can distinguish between sensors that have a trigger input/output and those that do not. The first allows significantly simplified procedures and is only affected by the quality of the trigger signal and its processing. However, if this interface does not exist, there is a direct dependence on the control capabilities of the instrument as well as internal sensor clocks and signal processing. Both cases are shown in the following for the triggerable LT and the non-triggerable RTS, along with individual procedures for absolute synchronization. It should be mentioned that the concepts were developed with reference to Leica Geosystems devices. Specifically, this refers to the laser tracker AT 960 and the robotic total station TS60. When using a trigger signal, a transfer to other hardware with the same functionality is possible in principle. This does not apply to the RTS, which uses manufacturer-specific software and may have a different internal concept. Transferability is only possible to an RTS with Leica Captivate, e.g. MS60 or TS16.

2.1 Laser tracker tracking

An LT is a high-precision instrument for 3D positioning in the micrometer range and thus serves as a reliable and independent referencing device. The available trigger input enables simple and precise synchronization possibilities. The proposed synchronization method for the LT has already been tested and successfully applied in various projects (e.g. [8, 26, 33]). An external GNSS receiver (here a Javad Delta) generates a trigger signal with an adjustable frequency which is fed into the LT’s trigger input. This causes single-point measurements of the LT to be acquired at the predefined sampling rate. Simultaneously, the trigger signal is fed back into the trigger input of the GNSS receiver via a splitter. Thus, absolute GPS time stamps can be logged with identical sampling rate. As a result, there is a unique link between the raw single point measurements of the LT with the absolute time information, based on GPS time. The general structure of this synchronization approach is shown schematically in Figure 1.

Figure 1:

Schematic setup for synchronization of LT observations with GPS time [33].

The quality of this synchronization depends significantly on the trigger signal generated in the receiver and the respective processing via the trigger input of the two devices. Theoretically, there is also a dependency on the length of the antenna and trigger cable, which is negligible. According to Balch [38], the delay time in the cable depends on the conductive material and is typically 5 ns per meter. The width and frequency of the trigger signal are individually adjustable at the receiver and can thus be adapted to the requirements of the trigger input of the LT. This includes in particular whether the rising or the falling edge should be used as triggering moment. We have deliberately chosen a short signal width of 50 ns to keep any negative impact low. According to the manufacturer, the time uncertainty here for the LT is in the microsecond range [39].

After synchronization to absolute GPS time, the time-stamped polar measurement elements can be transformed into Cartesian coordinates and describe a time-dependent trajectory. The individual measurements are usually related to a CCR and provide spatial 3D information. Of particular note is the use of a Leica T-Mac or T-Probe, which can be tracked over time instead of a CCR. With these devices, the complete pose, i.e. also the 3D orientation, can be determined. Several diodes on the tool are the basis for this, which are detected by the camera of the LT and provide rotational information. The permissible measuring distances are reduced here compared to the CCR, but the devices allow the advantage of realizing a complete pose, provided they can be reasonably mounted on the corresponding MSS [8, 40].

Although the accuracy potential and the achievable measurement frequency (up to 1000 Hz) of an LT is significantly superior to that of an RTS due to its interferometric measurement principle, it can generally only be reliably used for corresponding tracking tasks under constant atmospheric conditions indoors. At least when it comes to highest accuracy requirements. Otherwise, uncertainties at least on the level of a high-grade RTS can be assumed for outdoor applications. Because of the measurement principle, reliable measurements on a 360° reflector are not possible, which limits the mobility of the MSS to be tracked. However, this can be remedied by a active self-aligning reflector in an MSS, as shown in Refs. [1, 5, 41], [42], [43]. Still, the target ranges of LTs are significantly shorter and the instrument is not yet as widely used as RTSs due to its current purchase price. However, recent trends to combine RTS and LT at a lower price may create new opportunities in the future.

2.2 Robotic total station tracking

Synchronization of an RTS with an absolute time information is far more challenging compared to the LT, since no trigger signal can be utilized. The actual difficulty comes from the fact that the required polar measuring elements (distance, horizontal and vertical angle) of a single measurement do not take place simultaneously. Rather, it is a combination of different subsystems which each have an individual temporal reference [29, 44]. This circumstance is irrelevant if the observed reflector is static – which was the main use case in the past. But in kinematic applications, depending on the speed, shape, and length of the movement, this understanding is essential for getting accurate measurement results. Furthermore, latencies may additionally occur between the acquisition of the measured value and its availability, especially during the signal transmission to an external controller [45].

The approach presented in the following is based on the extended features of the so-called Measure & Stream application distributed by the instruments manufacturer. The options of this additional software are mainly related to two aspects. Firstly, the sensor-internal synchronization of the individual sensor subsystems such as the electronic distance measuring unit (EDM), automatic target recognition and tracking (ATR) and the angle measuring unit. Secondly, there is the additional possibility to stream the raw measuring elements to an external controller with a significantly more precise relative time information, taking into account latencies during communication [44]. With the help of these advanced functionalities, an absolute synchronization with respect to GPS time can be realized subsequently, as described below. External communication with and programming of a Leica Geosystems RTS via so-called GeoCOM commands has basically been possible for a long time. Thus, the Measure & Stream application offers extended GeoCOM commands, which allow an improved consideration of timing. So far, however, there have been no long-term studies of the achievable quality and reliability of the Measure & Stream application with respect to the use for absolute time synchronization.

To outline the approach, the central internal time systems of an RTS must be introduced first. Apart from the clock of the operating system used (Windows CE), this is mainly the relative time scale of the sensor board. This time reference essentially relates to the instrument’s start time and tracks the milliseconds that have passed since start (SSS). The individual subsystems relate their measured values to this relative time scale, which is referred to as RTS clock in the following. The actual concept for synchronization with absolute time reference is based on three steps and has already been introduced in Ref. [33].

2.2.1 RTS-internal synchronization

For a set of observation parameters to refer to a consistent point in time, the individual RTS-internal sensor time systems must be synchronized with each other as a basic prerequisite. This involves the identification of latencies between the individual subsystems as well as the consideration of their possible temporal variation. This task is performed by the Measure & Stream application, being a black box. As a result, all subsystems should be synchronized with each other. For subsequent streaming of the individual measured values, the time reference to the relative RTS clock is available to millisecond accuracy [46].

2.2.2 Synchronization of the controller with UTC

The superordinate absolute time information is realized on the basis of the highly precise primary civil time source UTC. This internationally standardized time can be used as a master time for an external controller. UTC is based on the International Atomic Time (TAI) and can be received worldwide by means of GNSS messages through an antenna-receiver combination. The actual time information is available as a combination of absolute (but rather inaccurate) National Marine Electronics Association (NMEA) sentences and precise relative time information via the additional pulse-per-seconds (PPS) for the start of each UTC second [47]. Due to the constant availability, UTC can be obtained via a receiver (here a uBlox with LEA-M8T chip was used) and the operating system time of a Raspberry Pi microcontroller can be synchronized continuously. This continuous correction is done in real-time by defining the uBlox receiver (connected via the General Purpose Input/Output [GPIO] pins) as the Raspberry Pi’s reference clock. Since there is a known constant temporal relationship between UTC and GPS time, a direct transition is possible.

2.2.3 Offset determination between the RTS clock and the system clock of the controller

Third, an offset between the RTS clock and the system clock of the microcontroller must be determined during communication via the Network Time Protocol (NTP) [46]. This again requires the capabilities of Measure & Stream, which should take into account all existing network and cable latencies. This NTP offset can be variable in time and should therefore be determined repeatedly over the measurement period on a regular basis. For the single determination, two time points each are required from the RTS in SSS (T₂, T₃) and from the controller in UTC (T₁, T₄), which characterize the start and the end of a data transmission, respectively. Here, T₂ is the time of the last measurement when the NTP command was received and T₃ is the time of the next valid measurement before the response is sent. Accordingly, T₁ is the OS timestamp of the controller when the request is sent, and T₄ is the OS timestamp when the response is received. This allows the NTP offset τ between the RTS clock and the controller clock to be calculated according to [46, 48]

(1) τ = T 2 − T 1 + T 3 − T 4 / 2 .

Thus, in theory and according to Ref. [46] all required synchronizations are performed and the associated offsets are available. The corresponding validation follows in Section 3 as part of the empirical investigations.

The controller receives the observation data for each measurement (distance, horizontal and vertical angles, and associated SSS) over TCP and stores it locally on a memory card. By applying the NTP offset τ to each set of observation parameters with respect to the specified sensor board milliseconds, the observation data should be synchronized to UTC in post-processing. This makes it possible to link the exact local sensor board timing of the raw measurement data from the RTS to an external absolute time information (UTC in this case). Thus, the time-stamped polar measurements are finally available with reference to UTC and can be used as trajectory information after transformation into Cartesian coordinates. An overview of the relevant time systems and clocks of an RTS in interaction with an external controller is shown in Figure 2. Schematically, it also presents the synchronization strategy by means of Measure & Stream.

Figure 2:

Main clocks and subsystems including their synchronization via Measure & Stream application (gray areas) between RTS (left box) and controller (right box) [33].

The synchronization procedure for the RTS is similar in its basic structure to the routine in Ref. [36], as they also define synchronization parameters for RTS-internal processing and for communicating with an external controller. Using the Measure & Stream application is the main difference in how we approach this. Where Thalman & Neuner [36] has to perform a complex calibration process in the lab using an industrial robot to determine the internal RTS latencies, we rely entirely on the manufacturer’s software solution. Using these advanced features, we can also determine the NTP offset between the RTS and the controller in a simplified manner. In addition, our external controller is continuously adjusted to the GPS time in real time, allowing for absolute synchronization.

With both synchronisation concepts presented for an LT and an RTS, reliable reference trajectories with absolute time reference can be realized in spatially limited environments, provided there is a continuous line of sight between the tracking instrument and the reflector to be tracked. In contrast to trajectories based on GNSS observations, this is also possible in indoor environments. Only a connection to a static GNSS antenna is required to obtain the absolute time signal. This is usually not an organizational challenge and is often already implemented by permanently installed antennas using cables routed indoors.

3 Empirical investigations

The realization of reference trajectories with absolute time reference to GPS time is highly different for the two described sensor types with proven tracking capabilities. The LT relies solely on the use of the hardware and software processed trigger signal. The time information is obtained by using a GNSS receiver with identical trigger capabilities. If the trigger signal is processed differently, synchronization deviations are theoretically possible and must be investigated.

The concept of synchronizing an RTS with absolute GPS time is based on an external controller. To enable a user-friendly application, the necessary components (microcontroller, GNSS receiver and external memory card) were built into a 3D printed box. Using an integrated resistive touch display and an intuitive GUI application, the RTS can be synchronized with visual feedback. Figure 3 shows the corresponding realization of the synchronization box.

Figure 3:

Front view with touchable user interface (a) and inside view (b) of the 3D printed synchronization box with touchscreen (in the cover on the left), Raspberry Pi 4 (board in the middle), ublox GNSS module (chip on the right) and associated wiring [33].

The extent to which the two concepts presented can serve to reliably and accurately synchronize the LT and the RTS, and thus be used to validate the trajectory of a kinematic MSS, will be investigated in the following. Therefore, the processing quality of the trigger signal when used on the LT is investigated in Section 3.1. As part of a long-term study, the temporal behavior of the NTP offset for an RTS is analyzed and its influence on the synchronization quality is shown in Section 3.2 using a rotating disk. For higher speeds, a robotic arm is used in Section 3.3 to demonstrate position deviations from the reference.

3.1 Analysis of the trigger signal

Independent of the measurement sensors used and the reference trajectory observed, the required processing time of an external trigger signal for the Raspberry Pi and the Javad Delta GNSS receiver can be examined. From this, it can be deduced to what extent internal processing times of both central hardware components have an influence on the subsequent synchronization accuracies. For this purpose, a shared trigger signal is transmitted via both the GPIOs of the Raspberry Pi and the trigger input of the Javad Delta and recorded each with a direct time reference to UTC. Possible deviations in the logging of the signal could indicate insufficient synchronization potentials. Furthermore, it will be investigated to what extent the observed differences behave over a longer period of time and possible offsets and trends can be identified.

The results for the slightly different processing times required for an identical external 10 Hz trigger signal are shown in Figure 4. Over a period of almost 3 h, the differences between the registered UTC reception times on the Raspberry Pi and the Javad Delta are plotted. It can be seen, that for the examined period, apart from very few (approx. 0.07 ‰ of all values) individual outliers of a maximum of 2.4 ms, there is a median deviation of 43 μs with a standard deviation of 11 μs. No significant trend can be identified over the 3-h period. Repeated investigations have confirmed the results. We assume that the outliers in particular, but also the low bias, are caused by the strictly non-real-time capability of the Raspberry Pi. Processes can be delayed internally when computing power is needed elsewhere. Theoretically, interference during signal transmission, e.g. due to insufficiently shielded cables, is also a potential cause.

Figure 4:

Differences of the UTC reception times of a split 10 Hz trigger signal between Raspberry Pi (RPI) and Javad Delta GNSS receiver in milliseconds. Note the logarithmic scaling of the Y-axis.

As different levels of frequency are apparent in this plot, the differences are also shown as a histogram in Figure 5. Considering the logarithmic scaling of the Y-axis, the median is prominent. In addition, further clusters can be identified at 43 μs and at 87 μs, although the absolute frequency of the latter is significantly lower. These individual accumulations cannot be interpreted and are attributed to the internal processes of the microcontroller. In conclusion, the measured deviations are well below a tenth of a millisecond and therefore have no relevant influence on the synchronization accuracy.

Figure 5:

Histogram showing the differences of the UTC reception times of a split 10 Hz trigger signal between Raspberry Pi (RPI) and Javad Delta GNSS receiver in milliseconds. Note the logarithmic scaling of the Y-axis.

3.2 Long-term study of the synchronization box

To compare the UTC synchronized tracking solutions, it is useful to observe a moving CCR simultaneously with an RTS and an LT. Here, it is necessary to implement a setup that repeatedly ensures constant conditions. This allows the investigation of temporal effects over a longer period of time. An independent verification of the synchronization quality of the LT has not yet been carried out. Even if the information about the quality of the trigger signal processing in the LT is provided by the manufacturer, an independent validation is advisable. However, this is not straightforward due to the high measurement frequency and temporal precision of the trigger signal. Nevertheless, due to its superior measurement uncertainty and the use of a hardware trigger, the LT solution can be used as a reference solution for the synchronization quality of an RTS. A measurement frequency of 100 Hz was selected for the LT. The maximum possible measurement frequency of 20 Hz was set for the RTS, although it is known from previous investigations that this is practically not achievable [34, 49, 50].

3.2.1 Experimental setup

A CCR is mounted on a disk that rotates at a constant speed to provide repeatable and consistent motion. By moving, the CCR describes a recurring circular trajectory with a diameter of 15.55 cm over time. The average duration of about 8.5 s for one revolution corresponds to a speed of 0.07 m/s. LT and RTS are located approximately 3.8 m away and continuously track the CCR while it is being moved. This experimental setup (cf. Figure 6) is installed in a laboratory with constant atmospheric conditions over a period of eight weeks without any other influencing factors. During this period, a total of 12 data sets were recorded, in each of which the CCR was in motion for approximately 30 min. Recurring measurements over a longer period of time allow statements to be made about the repeatability of the synchronization quality and deviations from the reference that occur. To establish a coordinate reference between LT and RTS, the sensors are transformed into the laboratory coordinate system using identical control points prior to each data acquisition. The standard deviations of the estimated transformation parameters are less than 1 mm for translations and 0.01° for rotations.

Figure 6:

Experimental setup with the LT (background on the left) and the RTS (background on the right) aiming at the rotating disk (foreground). Close-up of the front view of the turntable with CCR (bottom) and counterweight (top) and direction of rotation (green arrow).

3.2.2 Change of the NTP offset over time

As mentioned in Section 2.2.3, the offset τ between the RTS clock and the system clock of the external controller should be determined regularly by NTP commands. In the context of our investigations with the CCR mounted on the rotating disk, this NTP offset is determined once every 30 s. With a total measurement time of 30 min, this results in 60 individual determinations. The temporal behavior of τ is shown in Figure 7 for the first recorded dataset. A distinct noise of the individual realizations is noticeable. From experience [34], a linear drift is expected, which is only present here with a coefficient of determination R² of 56.4 % due to the noisy data. However, interpolation is required to use the latest NTP offset for individual points in time. A linear time drift of 39.5 ppm is obtained. The high standard deviation σ _τ of the NTP offset has thereby a direct influence on the slope of the robustly estimated regression line.

Figure 7:

Changes in the individual realizations of the NTP offset τ every 30 s and robustly estimated regression line for the first data set in the sixth calendar week.

The other subsequent independent measurements also yield such noisy determinations of τ. For clarity, Figure 8 hides the individual realizations and focuses on the robustly estimated regression lines for all 12 measurements within the eight-week study period. One notices the basically similar slope of the individual runs. Since it is the same device in each case and internal device-specific drift effects are already known [34], this is to be expected. The time drift varies over the individual runs from a minimum of 30.8 ppm to a maximum of 48.3 ppm. The median amounts to 39.3 ppm and the standard deviation is 4.4 ppm. The mean coefficient of determination R² is 60.2 %. Thus, noisy realizations for τ are present throughout, leading to uncertainty in the time drift. In general, this averaged value corresponds well with the finding in Ref. [33] (39.3 ppm) for the identical instrument. In Ref. [36], similar results were also obtained (39.5 ppm and 37.8 ppm), but for two Leica Geosystems TS16.

Figure 8:

Changes in NTP offset τ according to linear regression lines for all 12 data sets. Color reference of each measurement in relation to Figure 13.

It is also noticeable in Figure 8 that the y-axis intercept varies between the individual data sets. Thus, in addition to a slightly varying drift, there is also a relevant absolute variation of the NTP offset for the communication between the RTS and the external controller. The constant part depends on the distribution and the mean value of the individual realizations of the NTP offset. No temporal correlation could be detected over the 8 weeks. This could be caused by sensor-specific factors (temperature, CPU load, etc.) and especially by the NTP conditions at the time of execution.

The time interval of 30 s to determine τ is apparently too high. To make statistically reliable statements, multiple additional measurements with significantly longer observation times were performed purely for the determination of τ. Figure 9 shows exemplary realizations for a 3.5 h measurement with a τ determination frequency of 10 s. There is still considerable noise in the individual realizations, but compared to Figure 7, there is now a clear linear trend (R² = 98.8 %). Furthermore, it turns out that the standard deviation σ _τ of NTP offsets determined immediately one after the other is about 40 ms. Multiple measurements determined a mean time drift of 39.1 ± 0.1 ppm, which is close to the median of the 12 shorter 30-min measurements.

Figure 9:

Changes in the individual realizations of the NTP offset τ every 10 s (gray dots) and robustly estimated regression line (red line) for a 3.5 h measurement.

3.2.3 Synchronization quality of the RTS

On this basis, each measurement epoch of the RTS is corrected with the linearly interpolated NTP offset τ as a supplement to the sensor-internal SSS. The temporal reference to UTC is thus established for the observations of the RTS. As a result, the measurements can be plotted over time for the RTS and the LT in a joint chart. Figure 10a shows this for the first 10 revolutions as an example for the Y-axis, after transformation from polar to Cartesian coordinates. In total, there are up to 238 full revolutions in each of the 12 measurements, each showing a sinusoidal curve. The significant difference in the achieved measurement frequency between RTS (7.9 ± 0.9 Hz) and LT (100 ± 0 Hz) is striking. In particular, the deviation of the targeted 20 Hz for the RTS is remarkable, but is already known [49]. In addition, the measurement rate of the RTS is inconsistent and contains gaps of different lengths at irregular intervals. Also, a slight temporal lead of the LT observations over the RTS can be observed.

Figure 10:

Temporal course of the Y-component of the CCR for the first 10 of in total up to 238 revolutions for RTS (red circles) and LT (blue dots) in (a). Identified extreme points (circles) of the RTS (red) and LT (blue) time series for an exemplary full revolution of the CCR at the rotating disk using individually fitted Fourier curves in (b).

The time series can be used to determine the time deviation of the RTS solution from the LT reference. For this purpose, all measured 3D points are first transformed by singular value decomposition so that the circle is located in the XY coordinate plane with the origin in the center. The two extreme points of each full revolution are used as target variables. These can be determined independently by curve fitting and the difference in time can be evaluated from them. Therefore, a Fourier curve

(2) f x = a 0 + a 1 ⋅ cos x ⋅ w + b 1 ⋅ sin x ⋅ w

is estimated independently for each revolution for both time series. Where a₀, a₁, b₁ and w represent the coefficients and x the input parameters (time values). This allows the extreme points to be detected precisely in time. A single revolution with the determined extreme points as distinctive features of the curves is shown as an example in Figure 10b. The estimation of a curve function is necessary to avoid the influence of the different measurement frequencies on the extreme point estimation. The coefficient of determination R² for the fitting is at least 99.7 % over all revolutions in all 12 measurements. This enables reliable estimation of the extreme points. The time offset is therefore estimated twice per epoch and is averaged.

After synchronizing the original measurement data to UTC based on NTP, the absolute quality of the RTS time stamps can be evaluated. Figure 11 shows the variation of the time offset over the individual revolutions for all 12 measurements. The offsets vary overall between approximately 46 ms and 75 ms. There are different linear increases and decreases as well as different degrees of noise in the comparison between the 12 measurements. The linear drift is caused by the inaccurate regression line used to interpolate the NTP offsets (cf. Section 3.2.2). If these time drifts are set to 39.1 ppm, based on the more reliable long-term measurements, this will have a direct effect on the determined offset.^[2] Applied to all 12 data sets, the more realistic time offsets are obtained as shown in Figure 12. The previous trends are corrected accordingly and the averaged offsets remain constant over all revolutions. Statistical measures can be derived from the associated boxplot in Figure 13. Averaged over all 12 measurements, the time offset is 63.9 ± 5.2 ms. This value is very similar to the latencies found by Ref. [36]. It is apparent, that individual measurements (e.g. calendar weeks 7 and 12c) show a higher dispersion of 6.8 ms and 4.5 ms, respectively. Calendar week 14b has the lowest dispersion at 1.4 ms. Similar behavior applies to the indicated outliers. For each measurement, we expect the time offset to be relatively constant and not subject to significant short-term variations. The observed varying noise in the individual runs is rather due to the procedure for determining the two extreme points of each individual revolution. Especially for the RTS data, the determination of the extreme points is partly challenging and uncertain (e.g. regarding curve fitting and interpolation) due to the low measurement frequency. Averaged for each of the measurement series, the offset varies from a minimum of 53.0 ms to a maximum of 72.1 ms. The reason for the variations between weeks cannot be finally clarified. However, since we have set the time drift for the NTP offset (cf. Section 3.2.2) to 39.1 ppm for all measurements performed, the variation has to be caused by the RTS-internal processes and their determination using the Measure & Stream application as a black box solution (cf. Section 2.2.1). Depending on the speed of an object to be tracked, this significant time offset must be taken into account. Even though the offset can be assumed to be almost constant over the 30 min, it varies by about 19.1 ms over several weeks in the individual measurements. However, its specific determination can only be done with additional sensors (here the LT) and a suitable reference trajectory (here the recurring circular motion). Using the average value of 63.9 ms over all measurements performed, deviations of up to 11 ms can be expected for our data set.

Figure 11:

Time offsets between reference solution from LT and RTS over the individual revolutions of the CCR on the rotating disk for the 12 measurements. Color reference of each measurement in relation to Figure 13.

Figure 12:

Time offsets between reference solution from LT and RTS over the individual revolutions of the CCR on the rotating disk for the 12 measurements, with fixed time drift of 39.1 ppm. Color reference of each measurement in relation to Figure 13.

Figure 13:

Boxplot of the time offsets between reference solution from LT and RTS over the individual revolutions of the CCR on the rotating disk for the 12 measurements, with fixed time drift of 39.1 ppm.

3.3 Impact of the synchronization deviations on the kinematic position estimation

To what extent the inaccurate UTC synchronization for the RTS affects the position estimation can also be investigated with the described experimental setup. Since the tracking speeds achieved with the rotating disk are limited, additional measurements were performed with the aid of a robotic arm. The experimental setup is basically identical to the experiment from Section 3.2.1. To perform a consistent and reproducible movement, the CCR is mounted in the handle of a 6-axis Yuanda robot (cf. Figure 14). According to the manufacturer, the robot can reach speeds of up to 2 m/s with a repeatability of ±0.05 mm. Thus, there are more possibilities compared to the rotating disk. The robot was programmed to make the CCR perform a circular motion with a radius of 19.0 cm continuously. Different speeds can be specified, which were selected in percentages of 50 %, 75 % and 100 %. This results in 0.22 m/s, 0.33 m/s and 0.44 m/s for the specified radius. The LT and the RTS are located at a distance of about 3.1 m perpendicular to the circular plane created by the robot. The experimental setup is shown in Figure 15.

Figure 14:

CCR attached to the Yuanda robot.

Figure 15:

Experimental setup with the LT (left) and the RTS (right) aiming the CCR on the robot (center background).

The acquired time series are significantly shorter, ranging from five to 10 minutes. The NTP offset was determined every 10 seconds but was again set to 39.1 ppm for the drift because of the short duration. Subsequent determination of the temporal offset of the RTS compared to the LT is performed analogously to the first experiment. The measurements were all taken on one day within 2 h. Nevertheless, the determined offset also varies between 61.8 ms and 66.6 ms and thus basically in a similar range as in the long-term investigation.

Depending on the speed, the influence of the prevailing offset on the position determination can be investigated. For this purpose, the temporally associated reference coordinate is interpolated for each 3D coordinate determined by the RTS on the basis of the LT measurements. The spatial deviations between the coordinates of the LT and the RTS can thus be determined at the interpolated time points. The tangential component is of particular interest in terms of its impact on synchronization. This metric measure can be determined based on different position solutions as an indicator of the deviation from the actual position. Apart from the raw unchanged measurement data of the RTS, this includes the 3D coordinates corrected temporally by Measure & Stream. Figure 16 shows the corresponding median values with standard deviations over all revolutions. The detailed results are also listed in Table 1. Depicted are the results for four data sets, which differ with respect to the rotation speed of the CCR. In addition to the three robotic arm data sets, a single lower speed data set from the long-term study is also included.

Figure 16:

Median tangential deviations to the reference solution by the LT. Shown for the raw measurement data of the RTS (magenta) and the 3D coordinates corrected temporally by Measure & Stream (green). The lowest speed corresponds to the data set from calendar week 15 using the rotating disk. The other three data sets relate to the robot arm.

Table 1:

Median tangential deviations to the reference solution by the LT. Shown for the raw measurement data of the RTS and the 3D coordinates corrected temporally by Measure & Stream. The lowest speed corresponds to the data set from calendar week 15 using the rotating disk. The other three data sets relate to the robot arm.

Speed	Raw [mm]	Measure & Stream [mm]
0.06 m/s	2.04 ± 0.59	1.18 ± 0.40
0.22 m/s	4.72 ± 0.39	2.66 ± 0.14
0.33 m/s	10.82 ± 1.10	5.90 ± 0.23
0.44 m/s	19.04 ± 2.24	11.52 ± 0.51

Using Measure & Stream, synchronization yields an improvement over the raw measurement data. Nevertheless, the improvement is limited and a significant deviation from the actual position remains. These deviations become more significant as the speed increases (from 1.2 mm at 0.06 m/s up to 11.5 mm at 0.44 m/s). For the four data sets shown, this represents an improvement of about 43 % over the raw readings. Improvements beyond this cannot be expected without considering the offset of about 63.9 ms (cf. Section 3.2.3). In principle, similar to Ref. [36], it would be possible to account for this temporal offset in the measurement data. However, this would always require an industrial robot (cf. [36]) or, as here, an LT to determine this parameter separately. This contradicts the actual goal of using simple synchronization via Measure & Stream without the use of additional sensors or measurement and evaluation processes. In addition, the question arises whether a constant offset is justified at all, since it is subject to variations, as shown in Section 3.2.3. Therefore, the use of Measure & Stream allows for improved position determination, although significantly better determination would be possible with more accurate synchronization.

Independent of the synchronization and tangential deviations, the kinematic measurements show another systematic deviation. There is an almost constant offset of about 1 mm in the Z-component between LT and RTS coordinates (cf. Figure 17). This offset can be related to the distance component of the polar measurement elements. While no offset can be detected in the successive stationing of LT and RTS over the CCR, it occurs in the kinematic and simultaneous measurements on the same CCR. The physical reason for this could not be identified with certainty. Therefore, further investigations on the behavior of the offset with respect to the CCR used and possible interferences during simultaneous tracking are required. It should be noted, however, that this effect has no relevant influence on the results presented above, since the investigations are almost independent of the Z-component. In conclusion, it can be seen that, in addition to synchronization, several other influencing factors also affect the quality of trajectory determination.

Figure 17:

Plot of the measured coordinates of the RTS (red) and the LT (blue) for all CCR rotations of the data set from calendar week 15 in three-dimensional space.

4 Conclusion and future work

Reliable and accurate synchronization with an absolute time reference is a necessity for quality assessment of kinematic multi-sensor systems. Today’s RTSs can only be used for this purpose to a limited extent. Linking UTC timestamps to raw observations from an RTS is generally possible using Measure & Stream for supported Leica Geosystems instruments. For an LT, however, this is much easier, more reliable, and more accurate using a trigger signal. Whether Measure & Stream and its synchronization can provide a satisfactory kinematic position determination depends strongly on the motion speed of the object being tracked. Temporal offsets of about 65 ms can lead to a position deviation in the centimeter range even at low speeds of 0.22 m/s. Furthermore, it must be taken into account that besides synchronization, the measurement configuration, prism type and stationing, among others, also affect the trajectory quality. Despite these limitations, RTSs, along with LTs, offer an efficient solution for tracking tasks due to their flexible indoor and outdoor use as well as their long target ranges.

For the consideration of the NTP offsets τ it has to be investigated how often its determination is required to obtain a reliable and accurate estimation of the time drift. This depends on the instrument used, the expected tracking speed, and the required accuracy level. Especially for short measurement periods of a few minutes, a reliable internal determination of τ within the Measure & Stream application will not be possible. Even if the drift factor is determined accurately, there is still a significant time difference to the actual time, which leads to inaccurate trajectories.

In the course of further research, the method for determining the temporal offset should be improved to allow for more precise statements. This mainly refers to the accurate estimation of the extreme points and also includes, for example, the use of cross-correlation methods. In addition, it is necessary to closely examine the effect observed in the distance measurement of the RTS, which appeared in all simultaneous kinematic measurements with RTS and LT.

For further investigation, the identified drift factors and time offsets need to be analyzed for other instruments as well. This applies in particular to the MS60 from Leica Geosystems, which according to the manufacturer is better suited for tracking tasks. In this context, the factors influencing the trend (e.g. temperature) should also be examined as well as the extent to which the locking of this value is acceptable and can be improved. In addition, the synchronization box will be used for specific applications where an absolute temporal reference to other sensors is required, but only low motion speeds are present. Application scenarios with low tracking speeds can include direct georeferencing of a terrestrial laser scanner [51] or navigation of a printing robot in the construction area [5]. In general, transferability to applications outside the laboratory under practical conditions is aspired This includes the use of larger target ranges and the analysis of the resulting effects.