Home Kinect based physiotherapy system for home use
Article Open Access

Kinect based physiotherapy system for home use

  • Dominik Haas , Somphong Phommahavong , Jing Yu , Sabine Krüger-Ziolek , Knut Möller and Jörn Kretschmer EMAIL logo
Published/Copyright: September 12, 2015

Abstract

In physiotherapy, rehabilitation outcome is majorly dependent on the patient continuing exercises at home. To support a continuous and correct execution of exercises composed by the physiotherapist it is important that the patient stays motivated. With the emergence of game consoles such as Nintendo Wii, Sony PlayStation or Microsoft Xbox360 that employ special controllers or camera based motion recognition as means of user input those technologies have also been found to be interesting for other real-life applications. We present a concept to employ the Microsoft Kinect system as means to support patients during physiotherapy exercises at home. The system is intended to allow a physiotherapist to compose an individual set of exercises and to control the correct execution of those exercises through tracking the patient’s motions.

1 Introduction

Physiotherapy is a therapy, in which mechanical force and movements are applied by a specialized therapist to prevent and cure injuries. The manual therapy consisting of specific motions and exercises are applied by the therapist and executed by the patient. Therapeutic exercise programs are designed by the therapist to fit the unique needs of an individual patient. Besides manual therapy and exercises applied during appointments with the therapist, the patient is usually instructed to perform additional exercises at home to support the rehabilitation process. To achieve satisfying results, the patient has to perform the exercises on a daily basis, which, exercising at home alone, the patient often is not motivated enough to perform the exercises on a regular basis [1]. Additionally, the patient needs to pay close attention to both the body posture and the range of motions, repeating the exercises the exact same way as shown by the therapist to prevent unnecessary strain to the joints and muscles, possibly leading to further injuries [2].

Game consoles employing various ways of user interaction have nowadays been established as a form of home-based exercise. Systems such as Nintendo Wii (Nintendo AG, Kyołto, Japan), PlayStation Eye (Sony Computer Entertainment Inc., Tokyo, Japan) or Microsoft Kinect (Microsoft Corporation, Redmond, USA) allow the user to interact with a computer game by either using a special controller with acceleration and tilt sensors (Wii) or through motion recognition (Eye, Kinect).

Additionally, tablet computers have replaced desktop computers and notebooks in a wide area of applications. The large amount of Apps available for such systems helps users with limited computer knowledge to easily use tablets in daily life.

Combining a tablet computer with a camera based controller such as the Microsoft Kinect system would allow to create a novel physiotherapy system for home use. It would enable the therapist to select or record an individual set of exercises, which the patient should execute at home. The physiotherapist would have the chance to control and improve the patient’s movements, repetitions and most importantly the body posture based on the recordings of the exercises whereas the patient would be able to stay motivated. We thus propose a system that employs a Kinect camera system to track the movements of human extremities and that allows a patient to execute various exercises. Figure 1 depicts a schematic representation of the intended setup. Infrared depth sensors are used to track the patient’s movements while the RGB camera allows the user to reflect the motions on the tablet screen. The tablet software tells the user which exercises to execute and – if necessary – to correct the current posture.

Figure 1 Schematic overview of the intended setup. Infrared sensors are used to track the user’s motions while the RGB camera is used to show a video of the user on the tablet screen.
Figure 1

Schematic overview of the intended setup. Infrared sensors are used to track the user’s motions while the RGB camera is used to show a video of the user on the tablet screen.

2 Methods

2.1 Kinect system

The Kinect camera is a compact system, consisting of a VGA camera and a pair of infrared sensors, which serve as depth sensors. The system may be connected to a computer via USB and is usable with various programming languages, e.g.

C# (Microsoft Corporation) or C++. The Kinect system is able to track up to six people under normal conditions with two people being active, i.e. moving, at the same time. Based on the depth information it tracks movements of the human extremities by computing the position of various skeleton joints. The origin of the skeleton structure is the hip centre, from which the skeletal tracking algorithm builds the whole skeleton. The position of all joints can be accessed by calling the respective joint functions provided in the Kinect code library. In total, the position of 20 joints or body parts can be accessed. Figure 2 shows all available joints.

Figure 2 Skeletal joints available through the joint functions provided in the Kinect code library.
Figure 2

Skeletal joints available through the joint functions provided in the Kinect code library.

2.2 Calculation of extremity lengths

To individualize physiotherapy treatment it might be necessary to calculate certain body measurements as different body types may lead to individual workouts. In the proposed system, the length between two joints A and B is calculated as:

(1)AB=(xAxB)2+(yAyB)2+(zAzB)2

with x, y and z denoting the Cartesian coordinates of joints A and B in three dimensional space.

2.3 Calculation of angles

Calculating the angle in which the extremities are abducted from the body is an important measure to evaluatev correctness of the exercised workouts. All angles can be calculated through triangle relations, using three given sides. Figure 3 gives an example on how to calculate the angle in which the elbow is adducted.

Figure 3 Example on how to calculate the angle in which the elbow is adducted.
Figure 3

Example on how to calculate the angle in which the elbow is adducted.

Solving the triangular equation, the elbow angle can be calculated as:

(2)β=acos(a2+b2c22ab)

with a denoting the length of the humerus (AB), b denoting the length of the ulna (BC) and c being the distance between wrist and shoulder (CA).

2.4 Calculating distance between floor and body joints

Some physiotherapy exercises (e.g. push-ups or squats) might need to use the distance between the floor and certain body joints to determine whether the exercise is done correctly. The Kinect systems allows to read the floor coordinates, thus distance between a joint J and the floor (F) can be calculated as [3]:

(3)JF¯=AxJ+ByJ+CzJ+DA2+B2+C2

with A, B, C and D denoting the floor plane coordinates and xJ, yJ and zJ describing the coordinates of the joint.

3 Results

Figure 4 shows an exemplary user interface to control and record physiotherapy exercises. On the left it provides a live image of the patient showing the current position of all skeleton joints. Upper and lower spine joint as well as the hip center are marked in yellow, shoulder and hip joints are shown in red, wrists and ankles are marked in pink, while the feet and the head are shown in blue and light blue, respectively. The presented example instructs the user to do five squats while keeping lower legs in parallel to reduce strain to the knee ligaments. A pictographic image shows a schematic overview of the exercise. The user is able to start the exercise by waving the left hand. During exercise, the user is instructed to bend and extend the knees, indicated by an arrow. A repetition is counted as valid when the knee angle is decreased below 120°. During exercise, the distance between knee joints is controlled to be equal to the distance between ankles to ensure parallelism of the lower legs. If the lower legs are not kept in parallel, the current repetition is not counted and the user has to restart that specific repetition. After five repetitions, the user is informed that the exercise is finished. Additionally, the user is enabled to save the exercise data. The recorded data can later be analyzed by a physiotherapist. The recorded data includes the position of all joints along with the length of all extremities during the exercises.

Figure 4 Screenshot of the exemplary application in which the user is instructed to perform squats while keeping lower legs parallel. Exercise is started by waiving the left hand.
Figure 4

Screenshot of the exemplary application in which the user is instructed to perform squats while keeping lower legs parallel. Exercise is started by waiving the left hand.

4 Discussion

The proposed system allows tracking body movements during exercise and calculating the position of body joints, the length of extremities and the distance between a body joint and the floor. That information may be used to instruct and control home-based physiotherapy exercise.

The use of the Microsoft Kinect system as a tool for rehabilitation or physiotherapy has previously been reported in several publications. Luna-Oliva et al. reported on the use of such a system on children with cerebral palsy to train various motor and process skills [4]. Chang et al. showed that using a Kinect-based system for physical rehabilitation in young adults helps in increasing motivation during exercises and thus improving exercise performance [5]. Kinect-based physiotherapy systems currently available on the market (e.g. [6]) use patient-computer interaction to provide individualized exercises and an encouraging rehabilitation routine. However, aside from counting the number of exercise executions they do not control the exact posture during the workout. The presented system intends to include a tight control of body posture and to give feedback to the patient during exercise.

The employed Kinect camera is a low-cost platform allowing implementing the proposed system into current physiotherapy routine without stressing health insurance costs. However, due to its focus on being used in a computer game environment, where the exact tracking of all joints might not be the primary goal of the Kinect system, the calculated joint coordinates might be not exact enough to be used in a physiotherapy setting. Thus, a general analysis of the exactness of joint positions computed by the Kinect might be necessary. Preliminary tests employing a high-resolution 3D tracking system with eight infrared cameras and reflective markers showed an acceptable accuracy of the Kinect calculations.

The presented user interface currently includes only a limited number of exercises. The implementation of additional exercises and the possibility to compose individual workout plans through physiotherapists is planned for future work. Additionally, an evaluation including a greater number of participants to assess both the effectiveness of the posture control and the improvement of the outcome of the physical therapy is planned for future work.

Acknowledgement

This work was supported by the Bundesministerium für Bildung und Forschung (TREFFER, Grant 01PL11008).

Author’s Statement

  1. Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.

References

[1] D. Lloyd-Jones, R. J. Adams, T. M. Brown, M. Carnethon, S. Dai, G. De Simone, T. B. Ferguson, E. Ford, K. Furie, et al., ”Heart disease and stroke statistics–2010 update: a report from the American Heart Association,” Circulation, vol. 121 (7), pp. e46-e215, 2010.10.1161/CIRCULATIONAHA.109.192667Search in Google Scholar PubMed

[2] C. Kisner and L. A. Colby, Therapeutic Exercise - Foundations and Techniques, 6 ed. Philadelphia: F.A. Davis Company, 2012.Search in Google Scholar

[3] Z. A. Mundher and J. Zhong, ”A Real-Time Fall Detection System in Elderly Care Using Mobile Robot and Kinect Sensor,” Int J Mater Mech Man, vol. 2 (2), pp. 133-138, 2014.10.7763/IJMMM.2014.V2.115Search in Google Scholar

[4] L. Luna-Oliva, R. M. Ortiz-Gutierrez, R. Cano-de la Cuerda, R. M. Piedrola, I. M. Alguacil-Diego, C. Sanchez-Camarero, and C. Martinez Culebras Mdel, ”Kinect Xbox 360 as a therapeutic modality for children with cerebral palsy in a school environment: a preliminary study,” NeuroRehabilitation, vol. 33 (4), pp. 513-21, 2013.10.3233/NRE-131001Search in Google Scholar PubMed

[5] Y. J. Chang, S. F. Chen, and J. D. Huang, ”A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities,” Res Dev Disabil, vol. 32 (6), pp. 2566-70, 2011.10.1016/j.ridd.2011.07.002Search in Google Scholar PubMed

[6] RespondWell Rehab Platform, Available under: www.respondwell.com.Search in Google Scholar

Published Online: 2015-9-12
Published in Print: 2015-9-1

© 2015 by Walter de Gruyter GmbH, Berlin/Boston

This article is distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Articles in the same Issue

  1. Research Article
  2. Development and characterization of superparamagnetic coatings
  3. Research Article
  4. The development of an experimental setup to measure acousto-electric interaction signal
  5. Research Article
  6. Stability analysis of ferrofluids
  7. Research Article
  8. Investigation of endothelial growth using a sensors-integrated microfluidic system to simulate physiological barriers
  9. Research Article
  10. Energy harvesting for active implants: powering a ruminal pH-monitoring system
  11. Research Article
  12. New type of fluxgate magnetometer for the heart’s magnetic fields detection
  13. Research Article
  14. Field mapping of ballistic pressure pulse sources
  15. Research Article
  16. Development of a new homecare sleep monitor using body sounds and motion tracking
  17. Research Article
  18. Noise properties of textile, capacitive EEG electrodes
  19. Research Article
  20. Detecting phase singularities and rotor center trajectories based on the Hilbert transform of intraatrial electrograms in an atrial voxel model
  21. Research Article
  22. Spike sorting: the overlapping spikes challenge
  23. Research Article
  24. Separating the effect of respiration from the heart rate variability for cases of constant harmonic breathing
  25. Research Article
  26. Locating regions of arrhythmogenic substrate by analyzing the duration of triggered atrial activities
  27. Research Article
  28. Combining different ECG derived respiration tracking methods to create an optimal reconstruction of the breathing pattern
  29. Research Article
  30. Atrial and ventricular signal averaging electrocardiography in pacemaker and cardiac resynchronization therapy
  31. Research Article
  32. Estimation of a respiratory signal from a single-lead ECG using the 4th order central moments
  33. Research Article
  34. Compressed sensing of multi-lead ECG signals by compressive multiplexing
  35. Research Article
  36. Heart rate monitoring in ultra-high-field MRI using frequency information obtained from video signals of the human skin compared to electrocardiography and pulse oximetry
  37. Research Article
  38. Synchronization in wireless biomedical-sensor networks with Bluetooth Low Energy
  39. Research Article
  40. Automated classification of stages of anaesthesia by populations of evolutionary optimized fuzzy rules
  41. Research Article
  42. Effects of sampling rate on automated fatigue recognition in surface EMG signals
  43. Research Article
  44. Closed-loop transcranial alternating current stimulation of slow oscillations
  45. Research Article
  46. Cardiac index in atrio- and interventricular delay optimized cardiac resynchronization therapy and cardiac contractility modulation
  47. Research Article
  48. The role of expert evaluation for microsleep detection
  49. Research Article
  50. The impact of baseline wander removal techniques on the ST segment in simulated ischemic 12-lead ECGs
  51. Research Article
  52. Metal artifact reduction by projection replacements and non-local prior image integration
  53. Research Article
  54. A novel coaxial nozzle for in-process adjustment of electrospun scaffolds’ fiber diameter
  55. Research Article
  56. Processing of membranes for oxygenation using the Bellhouse-effect
  57. Research Article
  58. Inkjet printing of viable human dental follicle stem cells
  59. Research Article
  60. The use of an icebindingprotein out of the snowflea Hypogastrura harveyi as a cryoprotectant in the cryopreservation of mesenchymal stem cells
  61. Research Article
  62. New NIR spectroscopy based method to determine ischemia in vivo in liver – a first study on rats
  63. Research Article
  64. QRS and QT ventricular conduction times and permanent pacemaker therapy after transcatheter aortic valve implantation
  65. Research Article
  66. Adopting oculopressure tonometry as a transient in vivo rabbit glaucoma model
  67. Research Article
  68. Next-generation vision testing: the quick CSF
  69. Research Article
  70. Improving tactile sensation in laparoscopic surgery by overcoming size restrictions
  71. Research Article
  72. Design and control of a 3-DOF hydraulic driven surgical instrument
  73. Research Article
  74. Evaluation of endourological tools to improve the diagnosis and therapy of ureteral tumors – from model development to clinical application
  75. Research Article
  76. Frequency based assessment of surgical activities
  77. Research Article
  78. “Hands free for intervention”, a new approach for transoral endoscopic surgery
  79. Research Article
  80. Pseudo-haptic feedback in medical teleoperation
  81. Research Article
  82. Feasibility of interactive gesture control of a robotic microscope
  83. Research Article
  84. Towards structuring contextual information for workflow-driven surgical assistance functionalities
  85. Research Article
  86. Towards a framework for standardized semantic workflow modeling and management in the surgical domain
  87. Research Article
  88. Closed-loop approach for situation awareness of medical devices and operating room infrastructure
  89. Research Article
  90. Kinect based physiotherapy system for home use
  91. Research Article
  92. Evaluating the microsoft kinect skeleton joint tracking as a tool for home-based physiotherapy
  93. Research Article
  94. Integrating multimodal information for intraoperative assistance in neurosurgery
  95. Research Article
  96. Respiratory motion tracking using Microsoft’s Kinect v2 camera
  97. Research Article
  98. Using smart glasses for ultrasound diagnostics
  99. Research Article
  100. Measurement of needle susceptibility artifacts in magnetic resonance images
  101. Research Article
  102. Dimensionality reduction of medical image descriptors for multimodal image registration
  103. Research Article
  104. Experimental evaluation of different weighting schemes in magnetic particle imaging reconstruction
  105. Research Article
  106. Evaluation of CT capability for the detection of thin bone structures
  107. Research Article
  108. Towards contactless optical coherence elastography with acoustic tissue excitation
  109. Research Article
  110. Development and implementation of algorithms for automatic and robust measurement of the 2D:4D digit ratio using image data
  111. Research Article
  112. Automated high-throughput analysis of B cell spreading on immobilized antibodies with whole slide imaging
  113. Research Article
  114. Tissue segmentation from head MRI: a ground truth validation for feature-enhanced tracking
  115. Research Article
  116. Video tracking of swimming rodents on a reflective water surface
  117. Research Article
  118. MR imaging of model drug distribution in simulated vitreous
  119. Research Article
  120. Studying the extracellular contribution to the double wave vector diffusion-weighted signal
  121. Research Article
  122. Artifacts in field free line magnetic particle imaging in the presence of inhomogeneous and nonlinear magnetic fields
  123. Research Article
  124. Introducing a frequency-tunable magnetic particle spectrometer
  125. Research Article
  126. Imaging of aortic valve dynamics in 4D OCT
  127. Research Article
  128. Intravascular optical coherence tomography (OCT) as an additional tool for the assessment of stent structures
  129. Research Article
  130. Simple concept for a wide-field lensless digital holographic microscope using a laser diode
  131. Research Article
  132. Intraoperative identification of somato-sensory brain areas using optical imaging and standard RGB camera equipment – a feasibility study
  133. Research Article
  134. Respiratory surface motion measurement by Microsoft Kinect
  135. Research Article
  136. Improving image quality in EIT imaging by measurement of thorax excursion
  137. Research Article
  138. A clustering based dual model framework for EIT imaging: first experimental results
  139. Research Article
  140. Three-dimensional anisotropic regularization for limited angle tomography
  141. Research Article
  142. GPU-based real-time generation of large ultrasound volumes from freehand 3D sweeps
  143. Research Article
  144. Experimental computer tomograph
  145. Research Article
  146. US-tracked steered FUS in a respiratory ex vivo ovine liver phantom
  147. Research Article
  148. Contribution of brownian rotation and particle assembly polarisation to the particle response in magnetic particle spectrometry
  149. Research Article
  150. Preliminary investigations of magnetic modulated nanoparticles for microwave breast cancer detection
  151. Research Article
  152. Construction of a device for magnetic separation of superparamagnetic iron oxide nanoparticles
  153. Research Article
  154. An IHE-conform telecooperation platform supporting the treatment of dementia patients
  155. Research Article
  156. Automated respiratory therapy system based on the ARDSNet protocol with systemic perfusion control
  157. Research Article
  158. Identification of surgical instruments using UHF-RFID technology
  159. Research Article
  160. A generic concept for the development of model-guided clinical decision support systems
  161. Research Article
  162. Evaluation of local alterations in femoral bone mineral density measured via quantitative CT
  163. Research Article
  164. Creating 3D gelatin phantoms for experimental evaluation in biomedicine
  165. Research Article
  166. Influence of short-term fixation with mixed formalin or ethanol solution on the mechanical properties of human cortical bone
  167. Research Article
  168. Analysis of the release kinetics of surface-bound proteins via laser-induced fluorescence
  169. Research Article
  170. Tomographic particle image velocimetry of a water-jet for low volume harvesting of fat tissue for regenerative medicine
  171. Research Article
  172. Wireless medical sensors – context, robustness and safety
  173. Research Article
  174. Sequences for real-time magnetic particle imaging
  175. Research Article
  176. Speckle-based off-axis holographic detection for non-contact photoacoustic tomography
  177. Research Article
  178. A machine learning approach for planning valve-sparing aortic root reconstruction
  179. Research Article
  180. An in-ear pulse wave velocity measurement system using heart sounds as time reference
  181. Research Article
  182. Measuring different oxygenation levels in a blood perfusion model simulating the human head using NIRS
  183. Research Article
  184. Multisegmental fusion of the lumbar spine a curse or a blessing?
  185. Research Article
  186. Numerical analysis of the biomechanical complications accompanying the total hip replacement with NANOS-Prosthetic: bone remodelling and prosthesis migration
  187. Research Article
  188. A muscle model for hybrid muscle activation
  189. Research Article
  190. Mathematical, numerical and in-vitro investigation of cooling performance of an intra-carotid catheter for selective brain hypothermia
  191. Research Article
  192. An ideally parameterized unscented Kalman filter for the inverse problem of electrocardiography
  193. Research Article
  194. Interactive visualization of cardiac anatomy and atrial excitation for medical diagnosis and research
  195. Research Article
  196. Virtualizing clinical cases of atrial flutter in a fast marching simulation including conduction velocity and ablation scars
  197. Research Article
  198. Mesh structure-independent modeling of patient-specific atrial fiber orientation
  199. Research Article
  200. Accelerating mono-domain cardiac electrophysiology simulations using OpenCL
  201. Research Article
  202. Understanding the cellular mode of action of vernakalant using a computational model: answers and new questions
  203. Research Article
  204. A java based simulator with user interface to simulate ventilated patients
  205. Research Article
  206. Evaluation of an algorithm to choose between competing models of respiratory mechanics
  207. Research Article
  208. Numerical simulation of low-pulsation gerotor pumps for use in the pharmaceutical industry and in biomedicine
  209. Research Article
  210. Numerical and experimental flow analysis in centifluidic systems for rapid allergy screening tests
  211. Research Article
  212. Biomechanical parameter determination of scaffold-free cartilage constructs (SFCCs) with the hyperelastic material models Yeoh, Ogden and Demiray
  213. Research Article
  214. FPGA controlled artificial vascular system
  215. Research Article
  216. Simulation based investigation of source-detector configurations for non-invasive fetal pulse oximetry
  217. Research Article
  218. Test setup for characterizing the efficacy of embolic protection devices
  219. Research Article
  220. Impact of electrode geometry on force generation during functional electrical stimulation
  221. Research Article
  222. 3D-based visual physical activity assessment of children
  223. Research Article
  224. Realtime assessment of foot orientation by Accelerometers and Gyroscopes
  225. Research Article
  226. Image based reconstruction for cystoscopy
  227. Research Article
  228. Image guided surgery innovation with graduate students - a new lecture format
  229. Research Article
  230. Multichannel FES parameterization for controlling foot motion in paretic gait
  231. Research Article
  232. Smartphone supported upper limb prosthesis
  233. Research Article
  234. Use of quantitative tremor evaluation to enhance target selection during deep brain stimulation surgery for essential tremor
  235. Research Article
  236. Evaluation of adhesion promoters for Parylene C on gold metallization
  237. Research Article
  238. The influence of metallic ions from CoCr28Mo6 on the osteogenic differentiation and cytokine release of human osteoblasts
  239. Research Article
  240. Increasing the visibility of thin NITINOL vascular implants
  241. Research Article
  242. Possible reasons for early artificial bone failure in biomechanical tests of ankle arthrodesis systems
  243. Research Article
  244. Development of a bending test procedure for the characterization of flexible ECoG electrode arrays
  245. Research Article
  246. Tubular manipulators: a new concept for intracochlear positioning of an auditory prosthesis
  247. Research Article
  248. Investigation of the dynamic diameter deformation of vascular stents during fatigue testing with radial loading
  249. Research Article
  250. Electrospun vascular grafts with anti-kinking properties
  251. Research Article
  252. Integration of temperature sensors in polyimide-based thin-film electrode arrays
  253. Research Article
  254. Use cases and usability challenges for head-mounted displays in healthcare
  255. Research Article
  256. Device- and service profiles for integrated or systems based on open standards
  257. Research Article
  258. Risk management for medical devices in research projects
  259. Research Article
  260. Simulation of varying femoral attachment sites of medial patellofemoral ligament using a musculoskeletal multi-body model
  261. Research Article
  262. Does enhancing consciousness for strategic planning processes support the effectiveness of problem-based learning concepts in biomedical education?
  263. Research Article
  264. SPIO processing in macrophages for MPI: The breast cancer MPI-SNLB-concept
  265. Research Article
  266. Numerical simulations of airflow in the human pharynx of OSAHS patients
Downloaded on 12.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/cdbme-2015-0045/html
Scroll to top button