Home Usability evaluation of a locomotor therapy device considering different strategies
Article Open Access

Usability evaluation of a locomotor therapy device considering different strategies

  • Sonja Langthaler EMAIL logo , Alexander Lassnig , Christian Baumgartner and Jörg Schröttner
Published/Copyright: September 30, 2016

Abstract

Usability of medical devices is one of the main determining factors in preventing use errors in treatment and strongly correlates to patient safety and quality of treatment. This thesis demonstrates the usability testing and evaluation of a prototype for locomotor therapy of infants. Therefore, based on the normative requirements of the EN 62366, a concept combined of evaluation procedures and assessing methods was created to enable extensive testing and analysis of the different aspects of usability. On the basis of gathered information weak points were identified and appropriate measures were presented to increase the usability and operating safety of the locomotor prototype. The overall outcome showed an usability value of 77.4% and an evaluation score of 6.99, which can be interpreted as “satisfactory”.

1 Introduction

Usability has become more and more relevance in the development of medical products. Deficits through insufficient usability can lower the operating safety as well as the quality of treatment and therefore increase costs in health care. To increase safety an intuitive and easy handling should be provided [1], [2].

To lower use associated risks and ensure safety, the standard EN 62366 provides, in the context of risk management, methods to implement an usability-orientated-process and verify the usability accordingly [3].

Beside device safety an optimized usability can improve the whole treatment process and thereby raise the quality of treatment. Potential negative effects on the compliance of the patient may be avoided which furthermore can enhance the process as well as the therapeutic success. Hence, the motivation of this work was to deal with different usability methods and assess the usability of a locomotor therapy device.

2 Material and methods

2.1 Proceeding

The usability testing was split into two phases: The first phase covers usability tests with volunteers (n of 13). The test was restricted to software operations and offered a first feedback about usability and general weak points in the software design. Based on this information, improvements of usability could be achieved.

The second phase comprises usability tests with the intended users, involving the whole user interface and application process. This allows the evaluation of usability and definition of selective measures to optimize usability and increase operation safety of the device. The number of test subjects was defined by 5–10 participants, according to recommendations from the literature (cp. [1], [4]). The test was finally performed with four physical therapists for infants.

The usability test in both phases was realised as “think aloud”. The tasks were selected based on the main operation functions and thus involves all frequently-used and safety-related functions. The effectiveness was determined by the task performance and the number and severity of use errors, respectively. Efficiency was evaluated by the handling time. To receive a comprehensive insight of the subjective user satisfaction the inquiry comprised a face to face interview and a questionnaire. The applied questionnaire accords to the “System Usability Scale” (SUS) from Brooke [5], adapted to the medical context as well as the specific problem. Furthermore, a heuristic evaluation based on 10 defined heuristics according to Backhaus [1] was performed.

2.2 Evaluation methods

The analysis of data and evaluation of usability was carried out by three methods – “PROMEDIKS” [6], “UseProb” [2] and defined usability aims, as suggested by the standard EN 62366 [3].

PROMEDIKS from Backhaus [6] represents an evaluation method based on the influence factors “operability”, “functionality” and their deficits. The identification of deficits occurs on a given set of criteria, which were modified for the evaluation of the prototype (see Table 1) and not identified by the median as suggested. Similarly the relevance of processes and operations must be assessed and considered in the evaluation. Therefore, for processes and operations, which can affect the compliance of infants or the safety, a weighting factor from 4 to 5 for the relevance of tasks (TRi) and processes (PRi) was used. A weighting factor from 1 to 3 was considered depending on the frequency of use of tasks and processes. Quantitative operability (QO) and process support (QPS) can be calculated as defined in Equations 1 and 2. The geometric mean of these parameters leads to the usability value (UV), see Equation 3 [6].

(1)QO=(TRi.(AUAi))(TRi.UAmax).100%
Table 1:

Modified criteria for identification of operation deficits.

Operation deficitValueCriteriaMeasure
No deficit1<30%, Slight failureNo
Potential deficit2≥30%, Slight failureSensible
<15%, Severe failure
Severe deficit3≥15%, Severe failureNecessary

QO: quantified operability; TRi: relevance of task, UAi: assessment of deficit, UAmax: maximum of usability assessment, A: n + 1, n: number of estimation levels

(2)QPS=(PRi.PAai)(PRi.PAmax).100%

QPS: quantified process support, PRi: relevance of process, PAai: actual functional process support, PAmax: maximum of process support

(3)UV=QO.QPS

UV: Usability Value, QO: quantified operability, QPS: quantified process support

UseProb from Büchel [2] presents a standardized evaluation method for medical devices based on a risk analysis. Usability problems, which were identified through a heuristic evaluation, the usability test and the user inquiry were estimated according to severity, incidence rate as well as the potential harm. The evaluation is based on the most serious problem of each task. The evaluation result can be calculated and the usability can be assessed by the given scale [2].

The defined Usability Aims comprise objective aims and imply dimensions for effectiveness and efficiency. They were derived from main operation functions and consist of several aims for each task [3]. In this stage the aims simply represent handling attributes of the system. To get a result of the usability, the achievement of objectives was used and examined.

3 Results

The use of PROMEDIKS in phase 1 shows an impressive usability value of 85.3% (see Figure 1, UVTP_1). The analysis of the questionnaire leads to a SUS Score of 89.1 which represents an “excellent” user satisfaction [7]. The examination of usability aims in contrast to the results of PROMEDIKS showed a target achievement of 45.5% which is not acceptable. The main goal of phase 1 was to detect general weak points of the software operations and to derive suitable measures. Major measures after testing comprise an increase of type size for better legibility, highlighting and labelling of relevant functions as well as the variation of terms to prevent inconsistencies.

Figure 1: Graphic presentation of the usability value, based on [6].
Figure 1:

Graphic presentation of the usability value, based on [6].

The outcome of phase 2, as shown in Table 2, represents a lower usability value of 77.4%. This lower value results from the process of patient fixation, whereby seven severe operation deficits occurred. Figure 1 depicts the quantified results of the handling process separated from tests including software operations. The operability of the software could be improved up to 96% (see Figure 1), mainly based on the implemented measures from phase 1. The usability aims could also be increased to 68% and lead to an acceptable result. The user satisfaction was slightly reduced from 89.1 to 82.9, which still represents a “good” to “excellent” user assessment [7]. Major points of criticism from the inquiry related to the handling of patient positioning and fixation.

Table 2:

Results of test phase 1 and 2.

MethodPhase 1Phase 2
Usability value (PROMEDIKS)85.3%77.4%
Questionnaire89.182.9
Usability aims45.5%68.0%
Evaluation result (UseProb)a6.99

aThe UseProb method was not used in phase 1.

The application of UseProb leads to at least 51 usability problems and 13 items which describe good or excellent solutions. The evaluation result is 6.99. In summary, this leads to an evaluation level of seven which corresponds to a satisfying usability where personal or material injuries are highly improbable [2].

4 Conclusion

On the basis of the results from phase 2 the usability of the locomotor therapy device was assessed as “satisfactory”. In spite of different evaluation strategies the used methods approximately lead to equal results.

Most of the detected weak points and deficiencies slightly affect the usability and can be assessed as “trivial” in regard to the good learnability. The general high user satisfaction proves this conclusion. Furthermore it could be shown that the implemented measures from phase 1 increased the usability of the software operation.

Usability deficiencies, which possibly affect patient safety, mainly concern the procedure of patient positioning and patient fixation. Special training courses as wells as a setting-in period should be considered to decrease the risk for this handling procedure. In addition to that checklists for attachment and fixation could be implemented.

These measures should lead to optimized usability and higher operation safety of the locomotor therapy prototype. The effectiveness of these measures needs to be evaluated in further testing. Therefore a field testing is planned, whereby under the consideration of the intended use of the device a final and valid evaluation should be carried out.

Author’s Statement

Research funding: The author state no funding involved. Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent is not applicable. Ethical approval: The conducted research is not related to either human or animal use.

References

[1] Backhaus C. Usability-Engineering in der Medizintechnik. Heidelberg-Dordrecht-London-New York: Springer; 2010.10.1007/978-3-642-00511-4Search in Google Scholar

[2] Büchel D. Entwicklung einer allgemeingültigen Standardprozedur zur Überprüfung der Gebrauchstauglichkeit medizintechnischer Produkte. Eberhard Karls Universität zu Tübingen, Tübingen, Dissertation; 2010.Search in Google Scholar

[3] EN ISO 62366. Medizinprodukte, Anwendung der Gebrauchstauglichkeit auf Medizinprodukte. 2008.Search in Google Scholar

[4] Faulkner N. Beyond the five-user assumtion: Benefits of increased sample sizes in usability testing. Behav Res Meth Ins C. 2003;35:379–83.10.3758/BF03195514Search in Google Scholar

[5] Brooke J. A quick and dirty usability scale. Usability Evaluation in Industry: Taylor &amp; Francis Ltd; 1996, ch. 21. p. 189–94.Search in Google Scholar

[6] Backhaus C. Entwicklung einer Methodik zur Analyse und Bewertung der Gebrauchstauglichkeit von Medizintechnik. Technische Universität Berlin, Berlin, Dissertation; 2004.Search in Google Scholar

[7] Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. Journal of Usability Studies. 2009;4:114–23.Search in Google Scholar

Published Online: 2016-9-30
Published in Print: 2016-9-1

©2016 Sonja Langthaler et al., licensee De Gruyter.

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Articles in the same Issue

  1. Synthesis and characterization of PIL/pNIPAAm hybrid hydrogels
  2. Novel blood protein based scaffolds for cardiovascular tissue engineering
  3. Cell adhesion and viability of human endothelial cells on electrospun polymer scaffolds
  4. Effects of heat treatment and welding process on superelastic behaviour and microstructure of micro electron beam welded NiTi
  5. Long-term stable modifications of silicone elastomer for improved hemocompatibility
  6. The effect of thermal treatment on the mechanical properties of PLLA tubular specimens
  7. Biocompatible wear-resistant thick ceramic coating
  8. Protection of active implant electronics with organosilicon open air plasma coating for plastic overmolding
  9. Examination of dielectric strength of thin Parylene C films under various conditions
  10. Open air plasma deposited antimicrobial SiOx/TiOx composite films for biomedical applications
  11. Systemic analysis about residual chloroform in PLLA films
  12. A macrophage model of osseointegration
  13. Towards in silico prognosis using big data
  14. Technical concept and evaluation of a novel shoulder simulator with adaptive muscle force generation and free motion
  15. Usability evaluation of a locomotor therapy device considering different strategies
  16. Hypoxia-on-a-chip
  17. Integration of a semi-automatic in-vitro RFA procedure into an experimental setup
  18. Fabrication of MEMS-based 3D-μECoG-MEAs
  19. High speed digital interfacing for a neural data acquisition system
  20. Bionic forceps for the handling of sensitive tissue
  21. Experimental studies on 3D printing of barium titanate ceramics for medical applications
  22. Patient specific root-analogue dental implants – additive manufacturing and finite element analysis
  23. 3D printing – a key technology for tailored biomedical cell culture lab ware
  24. 3D printing of hydrogels in a temperature controlled environment with high spatial resolution
  25. Biocompatibility of photopolymers for additive manufacturing
  26. Biochemical piezoresistive sensors based on pH- and glucose-sensitive hydrogels for medical applications
  27. Novel wireless measurement system of pressure dedicated to in vivo studies
  28. Portable auricular device for real-time swallow and chew detection
  29. Detection of miRNA using a surface plasmon resonance biosensor and antibody amplification
  30. Simulation and evaluation of stimulation scenarios for targeted vestibular nerve excitation
  31. Deep brain stimulation: increasing efficiency by alternative waveforms
  32. Prediction of immediately occurring microsleep events from brain electric signals
  33. Determining cardiac vagal threshold from short term heart rate complexity
  34. Classification of cardiac excitation patterns during atrial fibrillation
  35. An algorithm to automatically determine the cycle length coverage to identify rotational activity during atrial fibrillation – a simulation study
  36. Deriving respiration from high resolution 12-channel-ECG during cycling exercise
  37. Reducing of gradient induced artifacts on the ECG signal during MRI examinations using Wilcoxon filter
  38. Automatic detection and mapping of double potentials in intracardiac electrograms
  39. Modeling the pelvic region for non-invasive pelvic intraoperative neuromonitoring
  40. Postprocessing algorithm for automated analysis of pelvic intraoperative neuromonitoring signals
  41. Best practice: surgeon driven application in pelvic operations
  42. Vasomotor assessment by camera-based photoplethysmography
  43. Classification of morphologic changes in photoplethysmographic waveforms
  44. Novel computation of pulse transit time from multi-channel PPG signals by wavelet transform
  45. Efficient design of FIR filter based low-pass differentiators for biomedical signal processing
  46. Nonlinear causal influences assessed by mutual compression entropy
  47. Comparative study of methods for solving the correspondence problem in EMD applications
  48. fNIRS for future use in auditory diagnostics
  49. Semi-automated detection of fractional shortening in zebrafish embryo heart videos
  50. Blood pressure measurement on the cheek
  51. Derivation of the respiratory rate from directly and indirectly measured respiratory signals using autocorrelation
  52. Left cardiac atrioventricular delay and inter-ventricular delay in cardiac resynchronization therapy responder and non-responder
  53. An automatic systolic peak detector of blood pressure waveforms using 4th order cumulants
  54. Real-time QRS detection using integrated variance for ECG gated cardiac MRI
  55. Preprocessing of unipolar signals acquired by a novel intracardiac mapping system
  56. In-vitro experiments to characterize ventricular electromechanics
  57. Continuous non-invasive monitoring of blood pressure in the operating room: a cuffless optical technology at the fingertip
  58. Application of microwave sensor technology in cardiovascular disease for plaque detection
  59. Artificial blood circulatory and special Ultrasound Doppler probes for detecting and sizing gaseous embolism
  60. Detection of microsleep events in a car driving simulation study using electrocardiographic features
  61. A method to determine the kink resistance of stents and stent delivery systems according to international standards
  62. Comparison of stented bifurcation and straight vessel 3D-simulation with a prior simulated velocity profile inlet
  63. Transient Euler-Lagrange/DEM simulation of stent thrombosis
  64. Automated control of the laser welding process of heart valve scaffolds
  65. Automation of a test bench for accessing the bendability of electrospun vascular grafts
  66. Influence of storage conditions on the release of growth factors in platelet-rich blood derivatives
  67. Cryopreservation of cells using defined serum-free cryoprotective agents
  68. New bioreactor vessel for tissue engineering of human nasal septal chondrocytes
  69. Determination of the membrane hydraulic permeability of MSCs
  70. Climate retainment in carbon dioxide incubators
  71. Multiple factors influencing OR ventilation system effectiveness
  72. Evaluation of an app-based stress protocol
  73. Medication process in Styrian hospitals
  74. Control tower to surgical theater
  75. Development of a skull phantom for the assessment of implant X-ray visibility
  76. Surgical navigation with QR codes
  77. Investigation of the pressure gradient of embolic protection devices
  78. Computer assistance in femoral derotation osteotomy: a bottom-up approach
  79. Automatic depth scanning system for 3D infrared thermography
  80. A service for monitoring the quality of intraoperative cone beam CT images
  81. Resectoscope with an easy to use twist mechanism for improved handling
  82. In vitro simulation of distribution processes following intramuscular injection
  83. Adjusting inkjet printhead parameters to deposit drugs into micro-sized reservoirs
  84. A flexible standalone system with integrated sensor feedback for multi-pad electrode FES of the hand
  85. Smart control for functional electrical stimulation with optimal pulse intensity
  86. Tactile display on the remaining hand for unilateral hand amputees
  87. Effects of sustained electrical stimulation on spasticity assessed by the pendulum test
  88. An improved tracking framework for ultrasound probe localization in image-guided radiosurgery
  89. Improvement of a subviral particle tracker by the use of a LAP-Kalman-algorithm
  90. Learning discriminative classification models for grading anal intraepithelial neoplasia
  91. Regularization of EIT reconstruction based on multi-scales wavelet transforms
  92. Assessing MRI susceptibility artefact through an indicator of image distortion
  93. EyeGuidance – a computer controlled system to guide eye movements
  94. A framework for feedback-based segmentation of 3D image stacks
  95. Doppler optical coherence tomography as a promising tool for detecting fluid in the human middle ear
  96. 3D Local in vivo Environment (LivE) imaging for single cell protein analysis of bone tissue
  97. Inside-Out access strategy using new trans-vascular catheter approach
  98. US/MRI fusion with new optical tracking and marker approach for interventional procedures inside the MRI suite
  99. Impact of different registration methods in MEG source analysis
  100. 3D segmentation of thyroid ultrasound images using active contours
  101. Designing a compact MRI motion phantom
  102. Cerebral cortex classification by conditional random fields applied to intraoperative thermal imaging
  103. Classification of indirect immunofluorescence images using thresholded local binary count features
  104. Analysis of muscle fatigue conditions using time-frequency images and GLCM features
  105. Numerical evaluation of image parameters of ETR-1
  106. Fabrication of a compliant phantom of the human aortic arch for use in Particle Image Velocimetry (PIV) experimentation
  107. Effect of the number of electrodes on the reconstructed lung shape in electrical impedance tomography
  108. Hardware dependencies of GPU-accelerated beamformer performances for microwave breast cancer detection
  109. Computer assisted assessment of progressing osteoradionecrosis of the jaw for clinical diagnosis and treatment
  110. Evaluation of reconstruction parameters of electrical impedance tomography on aorta detection during saline bolus injection
  111. Evaluation of open-source software for the lung segmentation
  112. Automatic determination of lung features of CF patients in CT scans
  113. Image analysis of self-organized multicellular patterns
  114. Effect of key parameters on synthesis of superparamagnetic nanoparticles (SPIONs)
  115. Radiopacity assessment of neurovascular implants
  116. Development of a desiccant based dielectric for monitoring humidity conditions in miniaturized hermetic implantable packages
  117. Development of an artifact-free aneurysm clip
  118. Enhancing the regeneration of bone defects by alkalizing the peri-implant zone – an in vitro approach
  119. Rapid prototyping of replica knee implants for in vitro testing
  120. Protecting ultra- and hyperhydrophilic implant surfaces in dry state from loss of wettability
  121. Advanced wettability analysis of implant surfaces
  122. Patient-specific hip prostheses designed by surgeons
  123. Plasma treatment on novel carbon fiber reinforced PEEK cages to enhance bioactivity
  124. Wear of a total intervertebral disc prosthesis
  125. Digital health and digital biomarkers – enabling value chains on health data
  126. Usability in the lifecycle of medical software development
  127. Influence of different test gases in a non-destructive 100% quality control system for medical devices
  128. Device development guided by user satisfaction survey on auricular vagus nerve stimulation
  129. Empirical assessment of the time course of innovation in biomedical engineering: first results of a comparative approach
  130. Effect of left atrial hypertrophy on P-wave morphology in a computational model
  131. Simulation of intracardiac electrograms around acute ablation lesions
  132. Parametrization of activation based cardiac electrophysiology models using bidomain model simulations
  133. Assessment of nasal resistance using computational fluid dynamics
  134. Resistance in a non-linear autoregressive model of pulmonary mechanics
  135. Inspiratory and expiratory elastance in a non-linear autoregressive model of pulmonary mechanics
  136. Determination of regional lung function in cystic fibrosis using electrical impedance tomography
  137. Development of parietal bone surrogates for parietal graft lift training
  138. Numerical simulation of mechanically stimulated bone remodelling
  139. Conversion of engineering stresses to Cauchy stresses in tensile and compression tests of thermoplastic polymers
  140. Numerical examinations of simplified spondylodesis models concerning energy absorption in magnetic resonance imaging
  141. Principle study on the signal connection at transabdominal fetal pulse oximetry
  142. Influence of Siluron® insertion on model drug distribution in the simulated vitreous body
  143. Evaluating different approaches to identify a three parameter gas exchange model
  144. Effects of fibrosis on the extracellular potential based on 3D reconstructions from histological sections of heart tissue
  145. From imaging to hemodynamics – how reconstruction kernels influence the blood flow predictions in intracranial aneurysms
  146. Flow optimised design of a novel point-of-care diagnostic device for the detection of disease specific biomarkers
  147. Improved FPGA controlled artificial vascular system for plethysmographic measurements
  148. Minimally spaced electrode positions for multi-functional chest sensors: ECG and respiratory signal estimation
  149. Automated detection of alveolar arches for nasoalveolar molding in cleft lip and palate treatment
  150. Control scheme selection in human-machine- interfaces by analysis of activity signals
  151. Event-based sampling for reducing communication load in realtime human motion analysis by wireless inertial sensor networks
  152. Automatic pairing of inertial sensors to lower limb segments – a plug-and-play approach
  153. Contactless respiratory monitoring system for magnetic resonance imaging applications using a laser range sensor
  154. Interactive monitoring system for visual respiratory biofeedback
  155. Development of a low-cost senor based aid for visually impaired people
  156. Patient assistive system for the shoulder joint
  157. A passive beating heart setup for interventional cardiology training
Downloaded on 12.9.2025 from https://www.degruyterbrill.com/document/doi/10.1515/cdbme-2016-0018/html
Scroll to top button