US/MRI fusion with new optical tracking and marker approach for interventional procedures inside the MRI suite

Yeshaswini Nagaraj; Bjoern Menze; Michael Friebe

doi:10.1515/cdbme-2016-0101

Article Open Access

US/MRI fusion with new optical tracking and marker approach for interventional procedures inside the MRI suite

Yeshaswini Nagaraj , Bjoern Menze and Michael Friebe

Published/Copyright: September 30, 2016

Published by

Become an author with De Gruyter Brill

Author Information Explore this Subject

From the journal Current Directions in Biomedical Engineering Volume 2 Issue 1

Abstract

Interventional MRI in closed bore high-field systems is a challenge due to limited space and the need of dedicated MRI compatible equipment and tools. A possible solution could be to perform an ultrasound procedure for guidance of the therapy tools outside the bore, but still on the MRI patient bed. That could track and subsequently combine the superior images of MRI with the real-time features of ultrasound. Conventional optical tracking systems suffer from line of sight issues and electromagnetic tracking does not perform well in the presence of magnetic fields. Hence, to overcome these issues a new optical tracking system called inside-out tracking is used. In this approach, the camera is directly attached to the US probe and the markers are placed onto the patient to achieve the location information of the US slice. The evaluation of our novel system of framed fusion markers can easily be adapted to various imaging modalities without losing image registration. To confirm this evaluation, phantom studies with MRI and US imaging were carried out using a point-registration algorithm along with a similarity measure for fusion. In the inside-out system approach, image registration was found to yield an accuracy of upto 4 mm, depending on the imaging modality and the employed marker arrangement and with that provides an accuracy that cannot be easily achieved by combining pre-operative MRI with live ultrasound.

Keywords: fusion; inside-out; MRI; tracking; US

1 Introduction

The principal motivation for image fusion between two modalities is to improve the quality of the information contained in the output image called synergy [1]. For accurate diagnosis and surgical procedure of tumours, multi-modal medical imaging is very often required [2]. Due to high level contrast during US-guided interventional procedures, either computed tomography (CT) or magnetic resonance (MR) images are used as reference data for the image registration [3]. Multimodality image registration (3-D alignment) or fusion (data merging) is well established in certain areas where complementary information is available. But this method faces the problem of magnetic field distortions in MR imaging caused by conventional stereotactic frames and line of sight issue of the optical tracking. To prevent these problems, a setup with a combined MRI and US fusion inside the MRI suite is required.

The new optical tracking system called inside-out tracking is a technique, where a camera is placed on an US-probe that observes feature markers of the surrounding environment to provide location information [4]. We have developed framed fusion of US and MRI using combinational markers for transient fixation around the area of interest with this tracking approach. This helps in efficient patient management, where our marker system can be used for different cross-sectional imaging modalities on the same patient without losing image registration [5]. This also allows a high precision in image registration and subsequent image fusion can be achieved with optimal patient comfort, as the patient is not required to change his position. This paper includes the 2D registration of MRI and US imaging in the MRI suite directly employing the inside-out tracking and newly developed combination markers for position estimation of the US probe and subsequent slicing of the corresponding 3D MRI data for 2D fusion in real-time.

2 Material and methods

The 3D MRI sequences were obtained on a 3T high-field closed-bore unit (Siemens Skyra, Germany). For the phantom imaging a surface coil was used and typical clinical MRI sequences used 2-D time-of-flight spoiled gradient echo sequence with flow compensation, 2-mm slice thickness, 256 × 128 matrix, 21 cm field-of-view (FOV) producing 0.391 mm × 0.782 mm pixel size in axial and oblique-axial planes). Circular structures (rubber balls with 2.4 cm radius placed in the phantom) were segmented from MRI volumes in some cases using a simple threshold method and in other cases using a 3-D region growing segmentation algorithm [2]. The threshold level for a set of images was determined by visual inspection, and the region growing settings were optimised for segmentation of expected tissue, which generated a negative mask for image registration. US data was acquired (Venue 70 tablet with Extend Research Package, GE Medical Systems, USA). collecting and saving individual frames or CINE loops of b-scan and color flow data. The tracking camera (Logitech) was attached to the US probe for position estimation with respect to the world co-ordinates. After acquisition the data files were transferred from the US hard disks to a workstation for off-line image processing and analysis.

The overall system workflow is shown in Figure 1. As a first step the camera is calibrated for intrinsic and extrinsic parameters. Then the markers are used to detect the position of the camera in the world co-ordinate system. These two steps together form the inside-out tracking approach which is indicated in red. Then the pose estimation is transferred along with corresponding 2D US-MRI data to a third party software (Imfusion, Munich, Germany) to achieve the image fusion or registration procedure.

Figure 1

The step by step system workflow from image capture to image registration using our new approach.

2.1 Position estimation and data transfer

The image captured by calibrated camera using conventional Matlab calibration function and the camera attached onto the US probe detects the marker to localize its position in real world. Figures 2 and 3 explain the detailed procedure. Suppose if the point has co-ordinates (x,y,z)^T in the co-ordinate frame of the camera, its projection onto the image plane is (x/z,y/z,1)^T. The camera captures the marker attached to the phantom and then the marker points are detected using the detect code library.

Figure 2

Detailed steps of pose estimation using the camera.

Figure 3

The optical marker placed onto the patient next to the area of interest which can easily be tracked by the camera for the pose estimation.

(1)bj=∑j=1n∑k=jnai⁢j⁢k⁢xi⁢xj.

where the right hand side of equation 1 is homogeneous in x_i and homogeneous matrix H using the direct linear transform (DLT). Then the linear equation is solved to achieve the rotational matrix at each step by initially assuming the solution to be single point in Rⁿ.

(2)M⁢x¯=0.

The detailed explanation of the procedure is as shown in Figure 2. Finally by reimposing the quadratic equation behavior of the original equation 1, we attempt to isolate the solution. Since x¯ ∈ Ker(M), there exist real numbers λ_i such that

(3)x¯=∑i=1nλi⁢vi

(4)(x′y′1)=K[R|t](XYZ1)

The position estimated in equation 4 is transferred into the already existing software imfusion and then the transformation is visualized on the MRI. Also, the MRI marker is clearly visible in the 3D MRI data, which provides the direct transformation of MRI data with respect to the marker which is the double check data for the corresponding US slice (Figure 4).

Figure 4

MRI compatible Vitamin E skin marker which of the size 2.4 cm in radius produce no artifacts in MRI data and is clearly visible enabling direct transformation.

2.2 Image fusion/registration

Once the spatial relationship between the just acquired MRI and tracked ultrasound is established, both modalities have to be presented in such a way that their fusion provides meaningful information for a given clinical application. Image registration of 2D single slice MRI to corresponding US slice is done manually. For the scenarios described above, MRI and ultrasound have to be precisely aligned within a common coordinate system, such that the anatomy can be correctly superimposed from both modalities.

The registration is achieved using small fiducial markers attached to the patient’s skin/phantom 4. Those radio-opaque markers are visible in the MRI scan, and can be located with a calibrated, tracked pointer. By computing the 3D rigid motion of the corresponding points from MRI to the tracked points, the MRI data is registered to the tracking coordinate system, which is also used by the tracked ultrasound probe. The MRI data set is fixed in the absolute coordinate system of the Explorer environment and never moved in space. The US data set was selected to be the moveable volume. Using the computer mouse, a user first manually moves the US volume into an overlapping position with the MRI volume, completing a coarse fusion alignment. By manipulating the 2D US single slice and checking its position with respect to the markers and position estimation in preoperative MRI volume, the user can obtain an approximation to the landmarks.

The single slice MRI and US was registered manually in a step process using a point registration algorithm. The use of a manual technique in this process greatly enhances the efficiency and accuracy of the fusion when compared to the automatic one using the 3D software.

2.2.1 Similarity measure

The comparison between MRI and US images, or image patches, is a fundamental operation in many image fusion algorithms. The similarity measure assesses the quality of anatomical alignment between the two modalities, and is a function of the image gray values [6]. A similarity measure should have its global maximum when the two images are correctly aligned.

In our approach, the similarity measure of 2D single slice of MRI and US on a purpose defined phantom at different position estimation using inside- out approach was measured. The evaluation of the US and MRI slices was done using LC2 similarity metric presented by [6] with varying patch size and down-sampling ratio to determine appropriate parameters for both metrics.

Table 1

The table illustrates the similarity measure for 2D single slice of MRI and US on purpose defined phantom at different position estimation using Inside out approach.

DATA	Patch size =10	Patch size =5
SLICE 1	0.6589	0.7066
SLICE 2	0.7890	0.6543
SLICE 3	0.6290	0.7467

3 Results

The inside-out approach was validated using conventional optical tracking and achieved a mean error of transformation of around 2.3 mm in all directions. The accuracy for the image registration or fusion was evaluated by phantom studies using different marker combination in two steps 1) local averaging and 2) using image location. Similarity measures can be extended by incorporating the location vectors. First of all, the location can be used to derive a weight for every intensity pair. The weight might be defined as proximity to the position of a clinical target, where the maximum accuracy is desired. We used the information of the position estimated using our inside out approach to define weights at locations where the intensity measurement is more accurate. In our case we had circular structure where the intensity is well defined as shown in Figure 5.

Figure 5

The co-registered images of US and MRI of a phantom using point registration algorithm around the circular structure.

The image location can be directly integrated into information theoretic measures using both markers and pose estimation. When designing and tweaking a similarity measure, one usually computes a large number of plots that show the measure value with respect to a transformation parameter changed within some range from the correct alignment. The excellent visibility and good delineation of the displayed markers in the MR images (Figure 3), makes the use of automated marker detection algorithms possible, allowing the determination of marker positions with sub pixel accuracy. The 2D image registration with downsampling and resizing of each modality was carried out using point registration algorithm in Imfusion software. The comparable mean error was achieved around the predefined circular structure of phantom.

4 Discussion

We have designed and evaluated a new system of combinational markers, that can be used for multi-imaging modalities. The design of the system allows an easy and fast change between different image modalities without loosing accuracy and patient comfort. The small dimensions of the presented markers minimize field distortions due to the stereotactic system and will thus help in accurate interventions based on MR-US image registration and fusion.

More work needs to be invested towards optimizing the algorithmic components and to conduct an overall evaluation. The similarity measure is often the most crucial component in a new registration algorithm, it therefore deserves special attention with respect to the evaluation. Investigating the measure usually happens at an earlier stage in the development. It should be designed based on all available knowledge about the involved imaging modalities. The results obtained with the presented phantom studies will provide the ethical justification for future application. The only problematic area found is the one between camera and markers that only provide a rather limited field of view since the camera is mounted directly onto the US probe. The inside-Out with combinational markers approach allows the clinician to carry out hybrid MRI-US image guided Interventions inside the MRI suite. For future work completely automated fusion and registration systems could correct any patient motion and non rigid deformations for improved outcome of an image guided therapy procedure.

Acknowledgement

We thank the STIMULATE campus for the use of MRI system and the staff members of the Chair of Catheter Technologies, namely Axel Boese, Dr. Johannes Krug, Alexander van Oepen, Dr. Alfredo Illanes for their valuable support.

Author’s Statement

Research funding: This research was funded by the German BMBF (grant 03IPT7100X). Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent is not applicable. Ethical approval: The conducted research is not related to either human or animal use.

References

[1] I. Fusion. Image Fusion – Theories, Techniques and Applications.Search in Google Scholar

[2] Ewertsen C, Săftoiu A, Gruionu LG, Karstrup S, Nielsen MB. Real-time image fusion involving diagnostic ultrasound. AJR Am J Roentgenol. 2013;200:W249–55.10.2214/AJR.12.8904Search in Google Scholar

[3] Kremser C, Plangger C, Secke RB, Aichner F, Felber SR. Image registration of MR and CT images using a frameless fiducial marker system. Magn Reson Imaging. 1997;15:579–85.10.1016/S0730-725X(97)00023-4Search in Google Scholar

[4] Matthies P, Okur A, Wendler T, Navab N, Friebe M. Combination of intra-operative freehand SPECT imaging with MR images for guidance and navigation. Conf Proc IEEE Eng Med Biol Soc. 2013;2013:3383–6.10.1109/EMBC.2013.6610267Search in Google Scholar PubMed

[5] Wein W, Ladikos A, Fuerst B, Shah A, Sharma K, Navab N. Global registration of ultrasound to MRI using the LC2 metric for enabling neurosurgical guidance. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinfor- matics). 2013;8149 LNCS:34–41.10.1007/978-3-642-40811-3_5Search in Google Scholar PubMed

[6] Pardasani U, Baxter JSH, Peters TM, Khan AR. Single slice US-MRI registration for neurosurgical MRI- guided US. 2016;9786:97862D.10.1117/12.2214265Search in Google Scholar

Published Online: 2016-9-30

Published in Print: 2016-9-1

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Articles in the same Issue

https://doi.org/10.1515/cdbme-2016-0101

Keywords for this article

fusion; inside-out; MRI; tracking; US

Creative Commons

BY-NC-ND 4.0