Home Using smart glasses for ultrasound diagnostics
Article Open Access

Using smart glasses for ultrasound diagnostics

  • Stefan Maas EMAIL logo , Marvin Ingler and Heinrich Martin Overhoff
Published/Copyright: September 12, 2015

Abstract

Ultrasound has been established as a diagnostic tool in a wide range of applications. Especially for beginners, the alignment of sectional images to patient’s spatial anatomy can be cumbersome. A direct view onto the patient’s anatomy while regarding ultrasound images may help to overcome unergonomic examination.

To solve these issues an affordable augmented reality system using smart glasses was created, that displays a (virtual) ultrasound image beneath a (real) ultrasound transducer.

1 Introduction

Ultrasound has been established as a diagnostic tool in a wide range of applications. But especially for beginners, the alignment of sectional images to patient’s spatial anatomy can be cumbersome. The examing physician has to switch his/her view between the screen and patient frequently, what impedes an undisturbed diagnostic. A direct view onto the patient’s anatomy while regarding ultrasound images may help to overcome unergonomic examination.

Augmented reality in medicine has been presented since the 1990s [1, 2]. Still solutions suffer from high cost, regarding to immense technical requirements, and unergonomic designs [3, 4]. To solve these issues an affordable augmented reality system using smart glasses was created, that displays a (virtual) ultrasound image beneath a (real) ultrasound transducer.

2 Methods

The used hardware consists of an ultrasound device (Esaote MyLab70 XVG) including a linear 2-D ultrasound transducer, a PC (Intel Core2Duo, 2.6 GHz CPU, Geforce GTX 560) and smart glasses (Epson Moverio BT-200) including a smartphone (TI OMAP 4460, 1.2 GHz, 1 GB RAM) and an built-in camera.

As illustrated in Figure 1 the PC collects raw data via Ethernet from the ultrasound device, recalculates the image data and holds these data available as byte arrays. Subsequent the smart phone collects the byte arrays via WLAN and reconstructs the ultrasound image. An integrated camera enables the smart glasses to collect images of the transducer and the optical marker on it. These images are sent to the smartphone, where the position of the optical marker can be tracked. The ultrasound images are scaled and, if necessary, rotated to the correct viewing angle. Finally the smart glasses display the (virtual) image at the marker position under the (real) ultrasound transducer (s. Figure 2).

Figure 1 Augmented reality ultrasound data flow.
Figure 1

Augmented reality ultrasound data flow.

Figure 2 View through the smart glasses. On top the ultrasound transducer with the attached optical marker. Below the corresponding ultrasound image.
Figure 2

View through the smart glasses. On top the ultrasound transducer with the attached optical marker. Below the corresponding ultrasound image.

The system needs no initial calibration on startup because the client knows the fixed marker position on the transducer. The image position and size is calculated from the position and size of the marker.

The server, running on the PC, was implemented using C#. The client on the smartphone consists of C#-scripts and an app, created with Unity3D and the Qualcomm Augmented Reality SDK “Vuforia”. While the scripts manage the communication between the server and the client, the app creates the ultrasound images, detects the optical marker and places the images with the calculated rotation and scale at the correct position.

3 Results

The ultrasound images can be visualized inside the smart glasses as seen in Figure 2 at about 25 fps and a resolution of 128x256 pixels. Detecting the optical marker is possible within a viewing angle between about 70 and 110 degrees. When moving the transducer the image follows without noticeable delay.

4 Conclusion and outlook

Smart glasses can be used for ultrasound examinations in principle. Those systems are ergonomic and low cost due to minor technical requirements. The framerate and resolution is adequate for most applications. However the detection of the optical marker needs to be improved to realize higher viewing angles.

Current works determine to the implementation of faster data transfer algorithms to realize higher resolutions and framerates.

Funding

This work was funded by the Landesregierung NordrheinWestfalen in the Med in.NRW-program, grant no. GW01-078.

Author's Statement

  1. Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.

References

[1] Y. Sato, M. Nakamoto, Y. Tamaki, T. Sasama, I. Sakita, Y. Nakajima, M. Monden and S. Tamura. Image Guidance of Breast Cancer Surgery Using 3-D Ultrasound Images and Augmented Reality Visualization. IEEE Transactions on Medical Imaging 1998; 17: 681-693.10.1109/42.736019Search in Google Scholar PubMed

[2] M. Bajura, H. Fuchs and R. Ohbuchi. Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient. Computer Graphics 1992; 26: 203-210.10.1145/133994.134061Search in Google Scholar

[3] D. Magee, Y. Zhu, R. Ratnalingam, P. Gardner and D. Kessel. An augmented reality simulator for ultrasound guided needle placement training. Medical & Biological Engineering & Computing 2007; 45: 957-967.10.1007/s11517-007-0231-9Search in Google Scholar PubMed

[4] M. Nakamoto, Y. Sato, M. Miyamoto, Y. Nakamjima, K. Konishi, M. Shimada, M. Hashizume and S. Tamura, “3D Ultrasound System Using a Magneto-optic Hybrid Tracker for Augmented Reality Visualization in Laparoscopic Liver Surgery,” Medical Image Computing and Computer-Assisted Intervention MICCAI 2002: 148-15510.1007/3-540-45787-9_19Search in Google Scholar

Published Online: 2015-9-12
Published in Print: 2015-9-1

© 2015 by Walter de Gruyter GmbH, Berlin/Boston

This article is distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Articles in the same Issue

  1. Research Article
  2. Development and characterization of superparamagnetic coatings
  3. Research Article
  4. The development of an experimental setup to measure acousto-electric interaction signal
  5. Research Article
  6. Stability analysis of ferrofluids
  7. Research Article
  8. Investigation of endothelial growth using a sensors-integrated microfluidic system to simulate physiological barriers
  9. Research Article
  10. Energy harvesting for active implants: powering a ruminal pH-monitoring system
  11. Research Article
  12. New type of fluxgate magnetometer for the heart’s magnetic fields detection
  13. Research Article
  14. Field mapping of ballistic pressure pulse sources
  15. Research Article
  16. Development of a new homecare sleep monitor using body sounds and motion tracking
  17. Research Article
  18. Noise properties of textile, capacitive EEG electrodes
  19. Research Article
  20. Detecting phase singularities and rotor center trajectories based on the Hilbert transform of intraatrial electrograms in an atrial voxel model
  21. Research Article
  22. Spike sorting: the overlapping spikes challenge
  23. Research Article
  24. Separating the effect of respiration from the heart rate variability for cases of constant harmonic breathing
  25. Research Article
  26. Locating regions of arrhythmogenic substrate by analyzing the duration of triggered atrial activities
  27. Research Article
  28. Combining different ECG derived respiration tracking methods to create an optimal reconstruction of the breathing pattern
  29. Research Article
  30. Atrial and ventricular signal averaging electrocardiography in pacemaker and cardiac resynchronization therapy
  31. Research Article
  32. Estimation of a respiratory signal from a single-lead ECG using the 4th order central moments
  33. Research Article
  34. Compressed sensing of multi-lead ECG signals by compressive multiplexing
  35. Research Article
  36. Heart rate monitoring in ultra-high-field MRI using frequency information obtained from video signals of the human skin compared to electrocardiography and pulse oximetry
  37. Research Article
  38. Synchronization in wireless biomedical-sensor networks with Bluetooth Low Energy
  39. Research Article
  40. Automated classification of stages of anaesthesia by populations of evolutionary optimized fuzzy rules
  41. Research Article
  42. Effects of sampling rate on automated fatigue recognition in surface EMG signals
  43. Research Article
  44. Closed-loop transcranial alternating current stimulation of slow oscillations
  45. Research Article
  46. Cardiac index in atrio- and interventricular delay optimized cardiac resynchronization therapy and cardiac contractility modulation
  47. Research Article
  48. The role of expert evaluation for microsleep detection
  49. Research Article
  50. The impact of baseline wander removal techniques on the ST segment in simulated ischemic 12-lead ECGs
  51. Research Article
  52. Metal artifact reduction by projection replacements and non-local prior image integration
  53. Research Article
  54. A novel coaxial nozzle for in-process adjustment of electrospun scaffolds’ fiber diameter
  55. Research Article
  56. Processing of membranes for oxygenation using the Bellhouse-effect
  57. Research Article
  58. Inkjet printing of viable human dental follicle stem cells
  59. Research Article
  60. The use of an icebindingprotein out of the snowflea Hypogastrura harveyi as a cryoprotectant in the cryopreservation of mesenchymal stem cells
  61. Research Article
  62. New NIR spectroscopy based method to determine ischemia in vivo in liver – a first study on rats
  63. Research Article
  64. QRS and QT ventricular conduction times and permanent pacemaker therapy after transcatheter aortic valve implantation
  65. Research Article
  66. Adopting oculopressure tonometry as a transient in vivo rabbit glaucoma model
  67. Research Article
  68. Next-generation vision testing: the quick CSF
  69. Research Article
  70. Improving tactile sensation in laparoscopic surgery by overcoming size restrictions
  71. Research Article
  72. Design and control of a 3-DOF hydraulic driven surgical instrument
  73. Research Article
  74. Evaluation of endourological tools to improve the diagnosis and therapy of ureteral tumors – from model development to clinical application
  75. Research Article
  76. Frequency based assessment of surgical activities
  77. Research Article
  78. “Hands free for intervention”, a new approach for transoral endoscopic surgery
  79. Research Article
  80. Pseudo-haptic feedback in medical teleoperation
  81. Research Article
  82. Feasibility of interactive gesture control of a robotic microscope
  83. Research Article
  84. Towards structuring contextual information for workflow-driven surgical assistance functionalities
  85. Research Article
  86. Towards a framework for standardized semantic workflow modeling and management in the surgical domain
  87. Research Article
  88. Closed-loop approach for situation awareness of medical devices and operating room infrastructure
  89. Research Article
  90. Kinect based physiotherapy system for home use
  91. Research Article
  92. Evaluating the microsoft kinect skeleton joint tracking as a tool for home-based physiotherapy
  93. Research Article
  94. Integrating multimodal information for intraoperative assistance in neurosurgery
  95. Research Article
  96. Respiratory motion tracking using Microsoft’s Kinect v2 camera
  97. Research Article
  98. Using smart glasses for ultrasound diagnostics
  99. Research Article
  100. Measurement of needle susceptibility artifacts in magnetic resonance images
  101. Research Article
  102. Dimensionality reduction of medical image descriptors for multimodal image registration
  103. Research Article
  104. Experimental evaluation of different weighting schemes in magnetic particle imaging reconstruction
  105. Research Article
  106. Evaluation of CT capability for the detection of thin bone structures
  107. Research Article
  108. Towards contactless optical coherence elastography with acoustic tissue excitation
  109. Research Article
  110. Development and implementation of algorithms for automatic and robust measurement of the 2D:4D digit ratio using image data
  111. Research Article
  112. Automated high-throughput analysis of B cell spreading on immobilized antibodies with whole slide imaging
  113. Research Article
  114. Tissue segmentation from head MRI: a ground truth validation for feature-enhanced tracking
  115. Research Article
  116. Video tracking of swimming rodents on a reflective water surface
  117. Research Article
  118. MR imaging of model drug distribution in simulated vitreous
  119. Research Article
  120. Studying the extracellular contribution to the double wave vector diffusion-weighted signal
  121. Research Article
  122. Artifacts in field free line magnetic particle imaging in the presence of inhomogeneous and nonlinear magnetic fields
  123. Research Article
  124. Introducing a frequency-tunable magnetic particle spectrometer
  125. Research Article
  126. Imaging of aortic valve dynamics in 4D OCT
  127. Research Article
  128. Intravascular optical coherence tomography (OCT) as an additional tool for the assessment of stent structures
  129. Research Article
  130. Simple concept for a wide-field lensless digital holographic microscope using a laser diode
  131. Research Article
  132. Intraoperative identification of somato-sensory brain areas using optical imaging and standard RGB camera equipment – a feasibility study
  133. Research Article
  134. Respiratory surface motion measurement by Microsoft Kinect
  135. Research Article
  136. Improving image quality in EIT imaging by measurement of thorax excursion
  137. Research Article
  138. A clustering based dual model framework for EIT imaging: first experimental results
  139. Research Article
  140. Three-dimensional anisotropic regularization for limited angle tomography
  141. Research Article
  142. GPU-based real-time generation of large ultrasound volumes from freehand 3D sweeps
  143. Research Article
  144. Experimental computer tomograph
  145. Research Article
  146. US-tracked steered FUS in a respiratory ex vivo ovine liver phantom
  147. Research Article
  148. Contribution of brownian rotation and particle assembly polarisation to the particle response in magnetic particle spectrometry
  149. Research Article
  150. Preliminary investigations of magnetic modulated nanoparticles for microwave breast cancer detection
  151. Research Article
  152. Construction of a device for magnetic separation of superparamagnetic iron oxide nanoparticles
  153. Research Article
  154. An IHE-conform telecooperation platform supporting the treatment of dementia patients
  155. Research Article
  156. Automated respiratory therapy system based on the ARDSNet protocol with systemic perfusion control
  157. Research Article
  158. Identification of surgical instruments using UHF-RFID technology
  159. Research Article
  160. A generic concept for the development of model-guided clinical decision support systems
  161. Research Article
  162. Evaluation of local alterations in femoral bone mineral density measured via quantitative CT
  163. Research Article
  164. Creating 3D gelatin phantoms for experimental evaluation in biomedicine
  165. Research Article
  166. Influence of short-term fixation with mixed formalin or ethanol solution on the mechanical properties of human cortical bone
  167. Research Article
  168. Analysis of the release kinetics of surface-bound proteins via laser-induced fluorescence
  169. Research Article
  170. Tomographic particle image velocimetry of a water-jet for low volume harvesting of fat tissue for regenerative medicine
  171. Research Article
  172. Wireless medical sensors – context, robustness and safety
  173. Research Article
  174. Sequences for real-time magnetic particle imaging
  175. Research Article
  176. Speckle-based off-axis holographic detection for non-contact photoacoustic tomography
  177. Research Article
  178. A machine learning approach for planning valve-sparing aortic root reconstruction
  179. Research Article
  180. An in-ear pulse wave velocity measurement system using heart sounds as time reference
  181. Research Article
  182. Measuring different oxygenation levels in a blood perfusion model simulating the human head using NIRS
  183. Research Article
  184. Multisegmental fusion of the lumbar spine a curse or a blessing?
  185. Research Article
  186. Numerical analysis of the biomechanical complications accompanying the total hip replacement with NANOS-Prosthetic: bone remodelling and prosthesis migration
  187. Research Article
  188. A muscle model for hybrid muscle activation
  189. Research Article
  190. Mathematical, numerical and in-vitro investigation of cooling performance of an intra-carotid catheter for selective brain hypothermia
  191. Research Article
  192. An ideally parameterized unscented Kalman filter for the inverse problem of electrocardiography
  193. Research Article
  194. Interactive visualization of cardiac anatomy and atrial excitation for medical diagnosis and research
  195. Research Article
  196. Virtualizing clinical cases of atrial flutter in a fast marching simulation including conduction velocity and ablation scars
  197. Research Article
  198. Mesh structure-independent modeling of patient-specific atrial fiber orientation
  199. Research Article
  200. Accelerating mono-domain cardiac electrophysiology simulations using OpenCL
  201. Research Article
  202. Understanding the cellular mode of action of vernakalant using a computational model: answers and new questions
  203. Research Article
  204. A java based simulator with user interface to simulate ventilated patients
  205. Research Article
  206. Evaluation of an algorithm to choose between competing models of respiratory mechanics
  207. Research Article
  208. Numerical simulation of low-pulsation gerotor pumps for use in the pharmaceutical industry and in biomedicine
  209. Research Article
  210. Numerical and experimental flow analysis in centifluidic systems for rapid allergy screening tests
  211. Research Article
  212. Biomechanical parameter determination of scaffold-free cartilage constructs (SFCCs) with the hyperelastic material models Yeoh, Ogden and Demiray
  213. Research Article
  214. FPGA controlled artificial vascular system
  215. Research Article
  216. Simulation based investigation of source-detector configurations for non-invasive fetal pulse oximetry
  217. Research Article
  218. Test setup for characterizing the efficacy of embolic protection devices
  219. Research Article
  220. Impact of electrode geometry on force generation during functional electrical stimulation
  221. Research Article
  222. 3D-based visual physical activity assessment of children
  223. Research Article
  224. Realtime assessment of foot orientation by Accelerometers and Gyroscopes
  225. Research Article
  226. Image based reconstruction for cystoscopy
  227. Research Article
  228. Image guided surgery innovation with graduate students - a new lecture format
  229. Research Article
  230. Multichannel FES parameterization for controlling foot motion in paretic gait
  231. Research Article
  232. Smartphone supported upper limb prosthesis
  233. Research Article
  234. Use of quantitative tremor evaluation to enhance target selection during deep brain stimulation surgery for essential tremor
  235. Research Article
  236. Evaluation of adhesion promoters for Parylene C on gold metallization
  237. Research Article
  238. The influence of metallic ions from CoCr28Mo6 on the osteogenic differentiation and cytokine release of human osteoblasts
  239. Research Article
  240. Increasing the visibility of thin NITINOL vascular implants
  241. Research Article
  242. Possible reasons for early artificial bone failure in biomechanical tests of ankle arthrodesis systems
  243. Research Article
  244. Development of a bending test procedure for the characterization of flexible ECoG electrode arrays
  245. Research Article
  246. Tubular manipulators: a new concept for intracochlear positioning of an auditory prosthesis
  247. Research Article
  248. Investigation of the dynamic diameter deformation of vascular stents during fatigue testing with radial loading
  249. Research Article
  250. Electrospun vascular grafts with anti-kinking properties
  251. Research Article
  252. Integration of temperature sensors in polyimide-based thin-film electrode arrays
  253. Research Article
  254. Use cases and usability challenges for head-mounted displays in healthcare
  255. Research Article
  256. Device- and service profiles for integrated or systems based on open standards
  257. Research Article
  258. Risk management for medical devices in research projects
  259. Research Article
  260. Simulation of varying femoral attachment sites of medial patellofemoral ligament using a musculoskeletal multi-body model
  261. Research Article
  262. Does enhancing consciousness for strategic planning processes support the effectiveness of problem-based learning concepts in biomedical education?
  263. Research Article
  264. SPIO processing in macrophages for MPI: The breast cancer MPI-SNLB-concept
  265. Research Article
  266. Numerical simulations of airflow in the human pharynx of OSAHS patients
Downloaded on 7.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/cdbme-2015-0049/html
Scroll to top button