COMPASS: localization in laparoscopic visceral surgery

Related work

There are standard sensors applied in the operating room for laparoscopic surgeries in the last decade. Widely used are infrared tracking systems based on reflective spheres and infrared light detectors. Another common type is electromagnetic tracking systems. The drawback of this approach is the common distortion of the metallic objects used during surgery. Furthermore, robotic arms can determine the position using a forward kinematic and measuring the joint’s angles. Other approaches focus on matching 3D CT scans to real-time images obtained either by ultrasound sensors [2], 2D cameras [3], [4] or 3D cameras.

Others [5], [6], [7] use SLAM methods for movement detection of laparoscope and organ to detect whether organs or the camera shifts and distortions by deformations are the main hazards for this approach. Similar to our approach is [8], which uses the remote center of motion (RCM) constraints for SLAM methods and show that it outperforms conventional SLAM approaches. Another method [9] uses an IMU and 2D camera information to implement a SLAM algorithm.

Often the use of IMU sensors in laparoscopic navigation is restricted to orientation information. The position trajectories are often computed by double integration of the accelerometer, which is erroneous as soon as the IMU is tilted [10].

Other researchers also investigated on integrating IMUs in sensor fusion methods to compensate the drawbacks of other sensors.

In contrast to previous approaches, we combine the RCM constraint with IMU sensors and produce a position estimate with that information. We use infrared tracking system for initialization and bypass timespans when line-of-sight is lost.

This method is not in conflict with other methods as we plan to merge several approaches to increase the accuracy of localization information up to the point that the variance is small enough for our use case.

Coordinate system definition and representation

The definition of coordinate systems should efficiently cover the laparoscopic setting and is essential for the later formulation of transformation matrices and their computation. Figure 1a depicts the essential coordinate systems. WIMU and WIR denote native sensor coordinate systems of IMU and IR sensors, respectively. We further define C to be the Camera tip with x-axis aligned to the laparoscope axis and L the coordinate system of the attached sensor. The trocar entry point is equal to the origin of T. The coordinate system W2 mounted on the patient is only needed if we expect either the infrared camera or the patient’s bed to move. We denote the homogenous 4×4 transformation from coordinate system A to coordinate system B by T, consisting of a translation in coordinate system A and a rotation, i.e.,

Figure 1:

(a) System Setup, (b) Position Estimates, Errors: (c) Total Position Error (d) Deviation due to erroneous angle (e) Deviation in Trocar entry point (f) Deviation due to erroneous penetration length.

Trocar position

When the laparoscope is inside the abdomen, we collect axis of the camera and compute the trocar position in a least-squares sense.

The tuple (x(j),t(j)) of vector and point defines the jth laparoscope axis. We use an interactive algorithm to collect new lines of

only when the optical tracking system has free line-of-sight to the target. Furthermore, to add a new line (x(j),t(j)) to the set of lines {(x(i),t(i))}i=1,…,I we request a minimum deviation

to all lines i=1,…,I in order to make the problem well posed.

In order to find the trocar entry point p with minimum square distances d(i) to all lines (x(i),t(i)) for i=1,…,I we minimize the cost function

which yields

with

We can then store the transformation

Laparoscope position estimation

Based on the trocar entrypoint and the orientation we get a position estimate as follows:

with

Here, Rot is the transformed rotation obtained by the IMU sensor. TransLT is the transition from trocar to laparoscope in coordinate system L, using the last known translation measured by the IR tracking system. The position estimator is, hence, unable to detect movements along the laparoscope axis. Therefore, in the COMPASS navigation setup we plan to include the stereoscopic view to determine movements along that axis. The camera view is thus received by

with previously calibrated rigid transformation TCL dependent on the laparoscope.

Sensors

Our prototype comprises an NDI Polaris Vega IR camera (Waterloo, Ontario, Canada) and unique geometry tools as well as MbientLab Metawear IMU sensor boards (San Francisco, California, USA). Communication with the infrared camera is established via ethernet and we read from the IMU via Bluetooth. The receiver publishes all data via unique topics via MQTT (IoT framework). Also the sensor-fusion and calibration procedures publish their data to the centralized broker. For the IoT setup there has to be a network setup with a running MQTT broker in the operating room (OR).

A 3D printed mounting is attached to the Storz 3D endoscope (Tuttlingen, Baden-Württemberg, Germany), statically connecting the reflecting spheres with the IMU sensor in a way that both coordinate systems are equal in orientation.

Evaluation and discussion

We tested six moving patterns of the laparoscope. We calibrate the trocar position in the beginning and use afterward no information of the IR Tracking System to compute the position. We observe that the position trajectory shows clear correspondence to the ground truth infrared tracking for all movements (see Figure 1b). The error of the position estimate is shown in Figure 1c. To analyze its source we investigate possible error contributions. The inaccurate part of the orientation measured by the IMU is propagating by

with the angle between the measured orientations Δθ=2 arccos(〈q1,q2〉) and the length outside the abdomen l.

The error based on an incorrect trocar entry point

is linearly contributing as well as the error based on incorrect length outside the abdomen

To find ε2 we let the trocar entry point be updated during the whole testing period using the IR Tracking by Equation (5). We test the movement of the updated trocar entry point and computed its error, assuming the mean to be the correct value (see Figure 1e). The measurement shows that the trocar entry point can be found in a comparatively short period of time. The movement of the entry point is due to imprecise calibration of TCL. Since we calibrated the trocar entry point at the beginning the error term is approximately 22 mm.

The error term ε1 is shown in Figure 1d. There exists a strong correlation with the total error in Figure 1c, showing that it is the main error source.

The last error is the length outside the abdomen, changes in that value can not be detected. For this moving pattern (pivoting) the laparoscope does not change its penetration length and the error is therefore comparatively small (see Figure 1f).

Conclusion

It was shown that the approach of estimating position by IMU and trocar entry point is feasible for moving patterns without a change in penetration length. We defined the contributing errors and found that the error is determined by the accuracy of the IMU’s orientation. We further showed that the trocar entry point could be determined fast.

In addition, we suggested an IoT framework for extending the localization framework and announced that studies incorporating other information are planned. For that purpose, we gathered research papers, which determine the localization information in a different way. That can serve as a starting point for extensions.