Abstract
The establishment of modern workflow management technologies requires the integration of dated devices. The extraction of the essential device data and usage time spans is a central requirement for an integrated OR environment. Therefore, methods are required that extract such information from the output provided by older generation devices, namely video signals. We developed a four-level approach for video-based device information extraction. Usually, video streams contain all relevant patient data and device usage information. We propose an approach consisting of defining regions of interest, grabbing video signals, analyzing the signals and storing the data in a centralized and structured location. The analysis considers textual information and graphical visualization. A prototype of the analysis approach was implemented and applied to a neurosurgical case. An evaluation study was conducted to measure the performance of the approach on video recordings of real interventions. Three medical devices were considered: intraoperative ultrasound, neuro-navigation and microscope. Overall, recognition rates for device usage higher than 95% were obtained. The approach is not limited to a single surgical discipline and does not require modification of medical devices. Furthermore, the analysis of microscopic video streams expands the detectable aspects of the surgical workflow beyond the recognition of device usage.
Acknowledgments
The authors would like to thank the following people for their helpful hints and their support: The OR staff of Department of Neurosurgery of the University Hospital in Leipzig. From the Innovation Center Computer Assisted Surgery at the University of Leipzig (Germany): Stefan Bohn, Bernhard Glaser, Philipp Liebmann, Jens Meier. ICCAS is funded by the German Federal Ministry of Education and Research (BMBF), the Saxon Ministry of Science and Fine Arts (SMWK) in the Unternehmen Region with grant number 03Z1LN12, the European Regional Development Fund (ERDF) and the state of Saxony within the framework of measures to support the technology sector.
References
[1] Agarwal S, Joshi A, Finin T, Yesha Y, Ganous T. A pervasive computing system for the operating room of the future. Mob Netw Appl 2007; 12: 215–228.10.1007/s11036-007-0010-8Search in Google Scholar
[2] Aggarwal R, Grantcharov T, Moorthy K, et al., An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 2007; 245: 992–999.10.1097/01.sla.0000262780.17950.e5Search in Google Scholar PubMed PubMed Central
[3] AXIS Communications, AXIS Commun, http://www.axis.com/global/en/products/axis-video-motion-detection/overview (2015). Accessed on 13 November, 2015.Search in Google Scholar
[4] Bhatia B, Oates T, Xiao Y, Hu P. Real-time identification of operating room state from video. In: Proc 19th Natl Conf Innov Appl Artif Intell – Vol. 2, AAAI Press, 2007; 1761–1766.Search in Google Scholar
[5] Blum T, Feußner H, Navab N. Modeling and segmentation of surgical workflow from laparoscopic video. In: Jiang, T, Navab N, Pluim J, Viergever M, editors. Med Image Comput Comput-Assist Interv – MICCAI 2010, Springer Berlin/Heidelberg, 2010: 400–407.10.1007/978-3-642-15711-0_50Search in Google Scholar PubMed
[6] Bohn S, Franke S, Burgert O, Meixensberger J, Lindner D. First clinical application of an open standards based OR integration system. Biomed. Eng.-Biomed. Tech. 2011; 56 Supplement 1.Search in Google Scholar
[7] Bohn S, Franke S, Burgert O, Neumuth T. An integration architecture with centralized control for medical devices in the digital operating room. In: Proc Fifth Int Workshop Syst Archit Comput Assist Interv SACAI 15th Int Conf Med Image Comput Comput Assist Interv, 2012.Search in Google Scholar
[8] Bouarfa L, Jonker PP, Dankelman J. Discovery of high-level tasks in the operating room. J Biomed Inform 2011; 44: 455–462.10.1016/j.jbi.2010.01.004Search in Google Scholar PubMed
[9] bytedeco/javacv, GitHub, https://github.com/bytedeco/javacv. (2015). Accessed on 13 November, 2015.Search in Google Scholar
[10] Cleary K, Chung HY, Mun SK. OR2020 workshop overview: operating room of the future. Int Congr Ser 2004; 1268: 847–852.10.1016/j.ics.2004.03.287Search in Google Scholar
[11] Dosis A, Aggarwal R, Bello F, et al., Synchronized video and motion analysis for the assessment of procedures in the operating theater. Arch Surg Chic Ill 1960 2005; 140: 293–299.10.1001/archsurg.140.3.293Search in Google Scholar PubMed
[12] Dosis A, BelloF, RockallT, et al., ROVIMAS: a software package for assessing surgical skills using the da Vinci telemanipulator system. In: Inf Technol Appl Biomed 2003 4th Int IEEE EMBS Spec Top Conf On, 2003; 326–329.Search in Google Scholar
[13] DresslerC, Fischer M, Burgert O, Strauß G. Evaluation of a context sensitive system for intra-operative usage of the electronic patient record. Laryngorhinootologie 2011.Search in Google Scholar
[14] Endress A, Brucker S, Wallwiener D, Aydeniz B, Kurek R, Zubke W. Systems integration in the operating room: the challenge of the decade. Gynecol Surg 2006; 3: 6–11.10.1007/s10397-005-0148-ySearch in Google Scholar
[15] Franke S, Liebmann P, Neumuth T. Connecting workflow management to the OR network: Design and evaluation of a bridge to enable dynamic systems behaviour. Biomed. Eng.-Biomed. Tech. 2012; 57 (Suppl 1).10.1515/bmt-2012-4192Search in Google Scholar PubMed
[16] Gessat M, Bohn S, Voruganti A, Franke S, Burgert O. TiCoLi: an open software infrastructure for device integration in the digital OR. Int J CARS 2011; 6 (Suppl 1): 226–295.Search in Google Scholar
[17] Ibach B, Benzko J, Schlichting S, Zimolong A, Radermacher K. Integrating medical devices in the operating room using service-oriented architectures. Biomed. Eng.-Biomed. Tech. 2012; 57: 221–228.10.1515/bmt-2011-0101Search in Google Scholar PubMed
[18] Lacassagne L, Manzanera A, Dupret A. Motion detection: fast and robust algorithms for embedded systems. 2009, 3265–3268.10.1109/ICIP.2009.5413946Search in Google Scholar
[19] Lalys F, Riffaud L, Bouget D, Jannin P. A framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Trans Biomed Eng 2012; 59: 966–976.10.1109/TBME.2011.2181168Search in Google Scholar PubMed PubMed Central
[20] Lalys F, Riffaud L, Morandi X, Jannin P. Surgical phases detection from microscope videos by combining SVM and HMM. In: Menze B, Langs G, Tu Z, Criminisi A, editors. Med Comput Vis Recognit Tech Appl Med Imaging, Springer Berlin Heidelberg, 2011: 54–62.10.1007/978-3-642-18421-5_6Search in Google Scholar
[21] Lemke HU, Vannier MW. The operating room and the need for an IT infrastructure and standards. Int J Comput Assist Radiol Surg 2006; 1: 117–121.10.1007/s11548-006-0051-7Search in Google Scholar
[22] Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O. Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 2009; 16: 72–80.10.1197/jamia.M2748Search in Google Scholar PubMed PubMed Central
[23] Neumuth T, Liebmann P, Wiedemann P, Meixensberger J. Surgical workflow management schemata for cataract procedures. Process model-based design and validation of workflow schemata. Methods Inf Med 2012; 51: 371–382.10.3414/ME11-01-0093Search in Google Scholar PubMed
[24] OpenCV, OpenCV, http://opencv.org/. (2015). Accessed on 13 November, 2015.Search in Google Scholar
[25] Panasonic, Panasonic, http://security.panasonic.com/pss/security/products/hd/i-VMD/ (2015). Accessed on 13 November, 2015.Search in Google Scholar
[26] Piccardi M. Background subtraction techniques: a review. In: 2004 IEEE Int Conf Syst Man Cybern 2004; 4: 3099–3104.10.1109/ICSMC.2004.1400815Search in Google Scholar
[27] Ray S. An overview of the tesseract OCR engine. In: Int Conf Conf Doc Anal Recognit 2007; 629–633.Search in Google Scholar
[28] Rockstroh M, Franke S, Neumuth T. A workflow-driven information source management. Int J Comput Assist Radiol Surg 2013; 8: 189–191.Search in Google Scholar
[29] Rockstroh M, Franke S, Neumuth T. Requirements for the structured recording of surgical device data in the digital operating room. Int J Comput Assist Radiol Surg 2014; 9: 49–57.10.1007/s11548-013-0909-4Search in Google Scholar PubMed
[30] Speidel S, Zentek T, Sudra G, et al., Recognition of surgical skills using hidden Markov models. In: 2009: 726125– 726125–8.10.1117/12.811140Search in Google Scholar
[31] Wu J, Lai C, Gupta R, Abosch A, Nelson D. Video-motion detection for objectively quantifying movements in in patients with Parkinson’s disease. Mov Disord 2013; 28: 322.Search in Google Scholar
©2016 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Research articles
- Development of antibiotic-loaded silk fibroin/hyaluronic acid polyelectrolyte film coated CoCrMo alloy
- Adhesion and proliferation of cells and bacteria on microchip with different surfaces microstructures
- Hydrophobic forces as a key factor in crystalline biofilm formation on ureteral stents
- Topography and nanostructural evaluation of chemically and thermally modified titanium substrates
- Thermodynamic effects after Diode and Er:YAG laser irradiation of grade IV and V titanium implants placed in bone – an ex vivo study. Preliminary report
- Influence of a bonding agent on the bond strength between a dental Co-Cr alloy and nine different veneering porcelains
- Digital templating for THA: a simple computer-assisted application for complex hip arthritis cases
- Finite element study of the acetabulum in cemented hip arthroplasty investigating retention or removal of the subchondral bone plate
- Failure analysis of ParaPost drills that fractured in service: a retrieval analysis study
- Development and verification of a mathematical model to quantify the joint spaces of the elbow
- Development of a smart IUD launcher for prevention of uterine perforation
- Modeling and performance evaluation of a robotic treatment couch for tumor tracking
- Video-based detection of device interaction in the operating room
Articles in the same Issue
- Frontmatter
- Research articles
- Development of antibiotic-loaded silk fibroin/hyaluronic acid polyelectrolyte film coated CoCrMo alloy
- Adhesion and proliferation of cells and bacteria on microchip with different surfaces microstructures
- Hydrophobic forces as a key factor in crystalline biofilm formation on ureteral stents
- Topography and nanostructural evaluation of chemically and thermally modified titanium substrates
- Thermodynamic effects after Diode and Er:YAG laser irradiation of grade IV and V titanium implants placed in bone – an ex vivo study. Preliminary report
- Influence of a bonding agent on the bond strength between a dental Co-Cr alloy and nine different veneering porcelains
- Digital templating for THA: a simple computer-assisted application for complex hip arthritis cases
- Finite element study of the acetabulum in cemented hip arthroplasty investigating retention or removal of the subchondral bone plate
- Failure analysis of ParaPost drills that fractured in service: a retrieval analysis study
- Development and verification of a mathematical model to quantify the joint spaces of the elbow
- Development of a smart IUD launcher for prevention of uterine perforation
- Modeling and performance evaluation of a robotic treatment couch for tumor tracking
- Video-based detection of device interaction in the operating room