Volume 13, Issue 3 - Special issue: Virtual reality based rehabilitation applications for motor, cognitive and sensorial disorders / Editors: Evelyne Klinger, Jaime Sánchez, Paul M. Sharkey and Joav Merrick

During the past decade, there has been a continuous exploration of how virtual environments can be used to facilitate motor recovery and relearning after neurological impairment. There are two goals for using virtual environments: to improve patients’ rehabilitation outcomes beyond our current capabilities or to supplement labor-intensive and time consuming therapies with technology-based interventions. After over a decade of investigation, it seems appropriate to determine whether we are succeeding in meeting such goals.

Virtual reality (VR)-based technologies are one of the emerging tools that appear to have great potential for use in cognitive rehabilitation. However, it still is unclear how brain capacities are involved and what is the best approach to such training. Quantitative aspects are encouraging because some improvements have been shown after few training sessions. By contrast, qualitative design of VR tools is more questionable. Choice of errorless or error-full designs may depend on the severity of disturbances. Most VR tools emphasize the explicit component of tasks; even procedural aspects comprise the main strength of VR retraining programs. VR and augmented reality tools give various stimuli and indicators, but their best modalities stay unclear, given that most data are coming from learning studies in normal subjects more than rehabilitation studies in brain-injured patients. Specific studies to explore the impact of sensorial transmodal effects and emotional involvement in VR tasks are required. Rehabilitation protocols utilizing virtual environments are moving from single applications to cognitive impairment (i.e., alertness, memory, neglect, language, executive functions) to comprehensive rehabilitation programs, with the aim of achieving efficient improvement in autonomy and transfer of benefits in real life conditions. A core issue that presents challenges to rehabilitation is decreased ability of persons with brain injury to transfer learning from one situation or context to another. The multi-context approach to cognitive rehabilitation proposes treatment methods for teaching the use of strategies across a wide range of meaningful activities, which can promote generalization and enhance functional performance.

Balance disorders are a common impairment after an acquired brain injury (ABI). Neurorehabilitation programs focus on the rehabilitation of balance skills to enhance patients’ self-dependency. The Wii Balance Board has been adopted with rehabilitative purposes due to its low cost and widespread battery of exercises. However, this entertainment system is oriented to healthy people and cannot easily adapt to the patient’s motor (and possible cognitive) deficits. The objective of this study was two-fold: a) to study whether custom-made rehabilitative exercises on a force platform could improve the balance condition of ABI patients compared with conventional physical therapy programs; and b) to study if their clinical effects persisted in absence of the virtual training. To prove the first hypothesis, a randomized controlled trial (RCT) was carried out involving 17 ABI participants (control group: 8 participants; experimental group: 9 participants). To prove the second hypothesis a follow-up study (FUS) was carried out involving 7 ABI participants. The participants of both studies underwent 20 1-h sessions, from 3 to 5 sessions per week. The participants were assessed at the beginning and at the end of the treatment (RCT, FUS) and 1 month after the therapy (FUS). Significant improvements were detected in some scales for those participants who underwent the virtual therapy (RCT). The effects persisted over time (FUS). Balance training through low-cost force platforms and custom-made exercises can provide lasting clinical benefits to ABI chronic patients when compared to conventional treatments.

The authors produced a dual task (DT) that provided a dynamic balance task and a cognitive task in a game system using motion sensors and virtual images. There had been no DT where a cognitive task needs a dynamic balance task that requires full-body motions. We developed and evaluated a game system to assess the performance of the DT. The DT was to solve a Sudoku puzzle using full-body motions like Tái Chi. An ability to perform a DT is intimately related to risk of falls. To evaluate the developed system, we compared the performance of elderly people and young people. Generally, elderly people are at a higher risk of falls. Twenty elderly community-dwelling adults (mean age, 73.0±6.2 years) and 16 young adults (mean age, 21.8±1.0 years) participated in this study. To compare the two groups, we applied an independent-samples t -test. The time taken for the elderly people was 60.6±43.2 s, whereas the time taken for the young people was 16.0±4.8 s. The difference is statistically significant ( p <0.05). This result suggests that the developed game system is useful for the evaluation of the DT performance.

The cognitive rehabilitation of stroke is often related to improvement of executive functioning through repeated and systematic training of memory and attention exercises, in which virtual reality may be a valid approach. Several devices have been used as visual outputs. Head mounted displays (HMD) and desktop screens displays are among them. HMDs are usually perceived as being more immersive than screens. However, the HMD presents several shortcomings if a widespread use is the objective. In this way, this study aims to assess the prospect of opting for screen displays as an alternative to HMD within virtual reality (VR)-based applications to rehabilitate memory and attention impairments in stroke patients. A sample of 17 patients with memory and attention deficits resulting from stroke were recruited from the hospital Centro de Medicina da Reabilitação do Alcoitão. The patients were randomly assigned to two different groups: a) HMD-based VR; and b) desktop screen-based VR. The patients in the experimental groups underwent a VR training program with 12 sessions of memory and attention exercises. These patients were assessed before and after the VR training sessions with the Wechsler Memory Scale for memory and the Toulouse-Piéron for attention functioning. The results showed increased working memory and sustained attention from initial to final assessment regardless of the VR device used. These data may suggest better functional independence following VR-based intervention and support the use of non-expensive displays as an alternative to high-end setups commonly used in VR applications devised for rehabilitation purposes.

Upper body motion impairment is a common after-effect of a stroke. We are developing a novel multisensory therapy system that makes use of augmented feedback and is expected to enhance rehabilitation outcomes. The first phase of the project involved developing algorithms to automatically differentiate between normal and impaired upper body motions. Seven healthy subjects performed two types scripted actions: (a) elbow flexion and extension and (b) reaching over via elbow flexion and shoulder flexion and adduction. Each action was repeated 10 times as each participant felt most comfortable and also 10 times simulating a common post-stroke impaired motion according to the literature. Principal component analysis was applied to the upper body trajectories during each action, as observed with a Kinect sensor, to extract the dominant modes of motion. Three classification algorithms and up to three motion modes were examined in order to distinguish between normal and impaired motions. A statistical analysis of the Kinect skeletal tracking data vs. manual annotation confirmed a significant bias in the tracking of an elevated shoulder joint. Despite this bias, leave-one-subject-out cross validation experiments confirmed the effectiveness of the proposed methods in detecting impaired motions (accuracy >95%), even when the impairment involved elevating a shoulder. A single, most dominant motion mode provided sufficient information for the binary classification task. The high accuracy rates validate the use of vision-based pose tracking technologies in identifying motion deficiencies.

This paper presents the ARM Trainer, a new augmented reality-based system that can be used to train amputees in the use of myoelectric prostheses. The ARM Trainer provides users with a natural and intuitive method to develop the muscles used to control a myoelectric prosthetic. In addition to improving the training process, the new interface has the potential to mitigate psychological issues arising from amputation that are not addressed by existing approaches (e.g., self-image, phantom limb pain). We conducted an empirical study comparing our system to an existing commercial solution (Myoboy) and found the ARM Trainer to be superior along a number of subjective dimensions (enjoyment, perceived effort, competency, and pressure). We also found no significant difference in terms of muscle control development between the two systems. This study shows the potential of augmented reality-based training systems for myoelectric prostheses.

This paper evaluates the patient engagement and clinical feasibility of an Augmented Reflection Technology (ART) system for use in physical rehabilitation of the upper limb following stroke. In particular, TheraMem, an extension of the ART system, was assessed on its potential to engage patients in meaningful therapeutic exercise. Six patients participated in a total of 24 sessions of upper limb training using the system. Tailored support for patients performing the exercises was provided based on the severity and level of their impairment. Various configurations of the system were evaluated and adjusted to best match the therapeutic requirements and the patient’s preferences. All patients were able to successfully participate and complete the TheraMem intervention at a high level of engagement and motivation over the course of the therapy sessions.

Tele-rehabilitation refers to the use of information and communication technologies to provide rehabilitation services to people in their homes or other environments. The objective of this paper was to present the development, validation, and usability testing of a low-cost, markerless, full-body tracking virtual reality system designed to provide remote rehabilitation of the upper extremity in patients with stroke. The Methods and Results sections present the progress of our work on system development, system validations, and a feasibility/usability study. We conclude with a brief summary of the initial stages of an intervention study and a discussion of our findings in the context of the next steps. The validation study demonstrated considerable accuracy for some outcomes (i.e., shoulder “pitch” angle, elbow flexion, trunk forward, and side-to-side deviation). In addition, positive responses were received from the clients who participated in the feasibility study. We are currently in the process of improving the accuracy of the system and analyzing results from a randomized clinical trial, which assessed the effectiveness of the system to improve upper extremity function post-stroke.

This paper presents a novel application of virtual environments to assist in encouraging behavior change in individuals who misuse drugs or alcohol. We describe the user-centered design of a series of scenes to engage users in the identification of triggers and to encourage discussions about relevant coping skills. Results from the initial testing of this application with six service users showed variation in user responses. Results also suggested that the system should encourage group discussion and that it was linked to a small improvement in users’ confidence in understanding and identifying triggers.

During recent years, researchers showed the feasibility and the interest of using virtual reality (VR)-based activities of daily living (vADL) among patients with cognitive impairments for the recovery of capacities. While interacting, the VR system provides various kinds of information for different purposes, including display of the virtual environment and understanding of the task, and also highlighting the functionalities or delivery of instructions. Generally, in order to improve the patient’s performance, additional cues are provided to enhance information saliency, such as arrows, change of colors, and so on. We give the definition of “contextual Additional Software Stimulus” (cASS) to any additional information related to the interaction and whose absence from the virtual environment does not have any effect on the unfolding of the task. This work was designed to study the effects of cASS on the performance in a daily living simulated task, including purchasing items in the Virtual Action Planning Supermarket (VAP-S). For this purpose, after implementation of cASS in the VAP-S, we carried out experiments, in which 23 healthy subjects (12 M and 11 F) and 12 patients with brain injury (12 M) took part. Results showed that the deliverance of cASS during the virtual task significantly improved some variables that described the performance of healthy subjects and patients with brain injury.

In the context of the admissions process for Chilean state universities, there is a paper and pencil test knowledge-measuring instrument called the University Selection Test (PSU, using its Spanish acronym). This instrument of evaluation is designed to measure the level of knowledge in various learning sub-sectors such as language and communication, mathematics, history and social sciences, and finally science. For each learning sub-sector, students use a paper facsimile with questions that each has five possible answer choices, which are recorded on a separate answer sheet. Based on a contextual analysis of the problems that people who are blind have with participating in the regular admissions process for Chilean universities, the purpose of this study was to design, implement, and evaluate a digital pilot system that adapts the Chilean university entrance system, PSU, in the area of Language and Communication for people with disabilities based on audio and haptics. This pilot system allowed for the inclusive, equitable, and autonomous participation of people with visual disabilities in the university admissions processes. The results demonstrate the creation of a system called AudioPSU, which provides the necessary autonomy and respects the working time that each user needs to respond to the questions in the PSU. In addition, the system is shown to help users to map the structure of the PSU facsimile for Language and Communication. Finally, initial results show that AudioPSU allows for the integration of people with visual disabilities in the admissions process for Chilean universities.

We introduce a novel rehabilitation technology for upper limb rehabilitation after stroke that combines a virtual reality (VR) training paradigm with a myoelectric robotic limb orthosis. Our rehabilitation system is based on clinical guidelines and is designed to recruit specific motor networks to promote neuronal reorganization. The main hypothesis is that the restoration of active movement facilitates the full engagement of motor control networks during motor training. By using a robotic limb orthosis, we are able to restore active arm movement in severely affected stroke patients. In a pilot evaluation, we have successfully deployed and assessed our system with three chronic stroke patients by means of behavioral data and self-report questionnaires. The results show that our system is able to restore up to 60% of the active movement capability of patients. Further, we show that we can assess the specific contribution of the biceps/triceps movement of the paretic arm in a VR bilateral training task. Questionnaire data show enjoyment and acceptance of the developed rehabilitation system and its VR training task.

A majority of studies examining repetitive task practice facilitated by robots for the treatment of upper extremity paresis utilize standardized protocols applied to large groups. This study will describe a virtually simulated, robot-based intervention customized to match the goals and clinical presentation of a gentleman with upper extremity hemiparesis secondary to stroke. MP, the subject of this case, is an 85-year-old man with left hemiparesis secondary to an intracerebral hemorrhage 5 years prior to examination. Outcomes were measured before and after a 1-month period of home therapy and after a 1-month virtually simulated, robotic intervention. The intervention was designed to address specific impairments identified during his PT examination. When necessary, activities were modified based on MP’s response to his first week of treatment. MP’s home training program produced a 3-s decline in Wolf Motor Function Test (WMFT) time and a 5-s improvement in Jebsen Test of Hand Function (JTHF) time. He demonstrated an additional 35-s improvement in JTHF and an additional 44-s improvement in WMFT subsequent to the robotic training intervention. A 24-h activity measurement and the Hand and Activities of Daily Living scales of the Stroke Impact Scale improved following the robotic intervention. Based on his responses to training we feel that we have established that a customized program of virtually simulated, robotically facilitated rehabilitation was feasible and resulted in larger improvements than an intensive home training program in several measurements of upper extremity function in our patient with chronic hemiparesis.