Volume 10, Issue 1 - Special Issue on Disability, Virtual Reality, ArtAbilitation and Music Editors: Anthony L. Brooks, Paul Sharkey and Joav Merrick

No abstract available

The field of rehabilitation has increasingly adopted commercially available games using perceptual interfaces as a means for physically training patients. The adaptability of such systems to match each person’s need and rehabilitation goal remains problematic. This paper presents a rapid prototyping approach for customising gaming technology using various affordable commercial devices and open source software. We first demonstrate how a freely available game is adapted for training disabled people through different sensors and control modes. We then show how an open online virtual world such as Second Life ® offers sufficient conditions for quickly building custom content for testing with interactive devices. When presented with these prototyping possibilities, people from the target groups (healthcare professionals, patients, people with disabilities, older people, families) related such systems to their needs and further elaborated on the use of such systems. Our research indicates how availability of simple prototyping platforms expands upon the possibilities for developers and practitioners.

Aphasia is an acquired communication disorder caused by brain damage that can affect the ability to speak and understand, as well as to read and write. Aphasia is most commonly caused by a stroke, but it can also result from a traumatic brain injury or a tumour. Having lost normal communication skills, an aphasic victim will often hide and isolate him or herself. This can also occur as a result of a reduced level of activity following rehabilitation. To help cope with this condition and to help victims regain their self-esteem, the Aphasic Theatre was created in 1992. The objective was to involve interested victims in drama and theatre as a way of rehabilitating their communication skills and self-esteem. The Aphasic Theatre is currently a recognized theatre company which has put on plays in Quebec, as well as elsewhere in Canada and Europe. There is now an accumulation of recorded evidence, from specialists, aphasic participants and their relatives, and the audience attending Aphasic Theatre performances, as well as a study completed by the ESPACE group of the University of Montreal, to confirm the validity of this innovative social rehabilitation method.

The Warriors’ Journey is an interactive narrative experience that is part of the online, virtual world healing space of the Coming Home Project (www.cominghomecenter.org) at the University of Southern California’s Institute for Creative Technologies. This project is exploring new ways to reach veterans who are returning from recent conflicts and who often have mental health issues or difficulty fitting back into their home communities. We are looking at the affordances of online 3D virtual worlds to address some of the issues that may prevent or deter veterans from getting the help they need. We are exploring the use of specific therapies within the virtual world space, as well as social and enrichment activities designed around the needs of this group. The Warriors’ Journey is one such activity that engages participants in stories to emphasize and reinforce common ideals of honorable warriors throughout history. After experiencing these stories, participants are then encouraged to construct their own warrior’s story within the Coming Home space. It is hoped that such activities may help contribute to a more positive sense of self-esteem, a restructured life narrative, and a feeling of control over the stresses incurred during their service to their country.

This new multi-user interactive sound installation (for up to eight persons simultaneously) implements proprietary glob-recognition and tracking software to allow visitors to a large empty space (ca. 5×7 m) to move an avatar – projected on a screen at the end of the space – simply by moving about the space, with the objective of taking it to specific, recognisable locations. Success in this endeavour causes musical sounds to be triggered.

This article discusses a range of interactive body-worn systems and devices for performance, play, rehabilitation and disability or altered-ability support. The systems combine experimental and off-the-shelf technologies to arrive at outcomes that require and inspire extended physical and expressive engagement, and afford a range of different learning opportunities. Notions of extended bodies, shared augmented environments, magical thinking and play are examined for their poetic valence, as well as for therapeutic potential. Much of the work is in its early stages. Several scenarios of use are outlined for each of the devices, and relevance to ArtAbilitation proposed. The aim is to generate collaborative interest and inform development.

This paper describes the technical and aesthetic approach ­utilised for the development of an interactive audiovisual performance system designed specifically for use by children with multiple learning difficulties, including deafness and autism. Sound is transformed in real-time through the implementation of a Fast Fourier Transform and translated into a moving image. This image is adapted so that relevant information can be understood and manipulated visually in real-time. Finally, the image is turned back into sound with only minimal delay. The translation process is based on research in computer music, neuroscience, perception and abstract film studies. The system is achieving a significant user base amongst the deaf and disabled community through projects involving the British Broadcasting Corporation and the London Philharmonic Orchestra. Alongside continuing collaboration with the Sonic Arts Network, the system has now been made freely available to the public, and is being developed as a speech therapy tool.

The design and rationale behind a novel music and sound controller is described. Using an array of eight low-cost infrared distance measuring sensors, the system enables users to trigger and manipulate sounds using MIDI messages. Although the controller can facilitate complex musical interaction, providing eight note polyphony and expressive control, the central theme of the project has been one of accessibility. The controller is designed to be used in a variety of settings by users with special needs and has been designed to be both intuitive to play and easy to set up. An ongoing programme of user-testing is described and discussed alongside design modifications and future directions within the project.

This paper aims to describe the different stages of the artistic project SOUND=SPACE Opera , held at Casa da Música, Porto, Portugal and presented during the festival Ao Alcance de Todos (Accessible to All) during 15–19 April, 2008. A series of creative workshops using an interactive musical environment, created by Rolf Gehlhaar in 1984, were run for several weeks with a group of six adolescents with special needs, mainly suffering from Down Syndrome. The translation of their day-by-day experiences into sounds and movements resulted in two final performances which were the public surface of a very empowering and rewarding process that intended to improve the quality of life of those who participated.

Background: The use of video games has shown benefits for the ageing population. This paper studies the cognitive effects of video games on the elderly, as well as these effects on self-concept and quality of life. The mediator effect of self-concept is also evaluated. The instruments used are the Cognitive Subscale of Alzheimer’s Disease Assessment Scale (ADAS-Cog), the Clinical Inventory of Self-Concept (ICAC) and the World Health Organisation Quality of Life Questionnaire (WHOQOL-bref). Methods: The study involved the participation of 43 elderly people distributed between three experimental conditions (15 used video games, 17 relaxation and 11 had no intervention). There were two periods of assessment, before the intervention and after 8 weeks. Results: It was found that the participants who used video games showed a decline in cognitive deterioration, unlike the control groups. The self-concept measure deteriorated significantly under relaxation condition and on the passive control group. The quality of life test did not show any differences. Nor were any correlations found between the time of use of video games and larger effects. The mediator effect of self-concept on differences obtained in the ADAS-Cog (r=0.57, p=0.014) and in the ICA (r=−0.039) was confirmed. Conclusions: In summary, the results show that the use of video games leads to the improvement of cognitive functioning in the elderly and to the maintenance of self-concept and quality of life. The results also suggest that the higher the self-concept, the better the cognitive effects achieved.

A novel input device is presented for use with a personal computer by persons with physical disabilities who would otherwise not be able to enjoy computer gaming. This device is simple to manufacture and low in cost. A gaming application was especially designed for this input device, requiring four participants to collaborate to accomplish the set task. We describe the constituent parts of this device and how colour and simple pictures were incorporated into the physical input device.

The authors have developed a vocalization training system for the auditory impaired using a talking robot. The training system mainly consists of a talking robot that has mechanical organs like a human. With an adaptive learning strategy using an auditory feedback control, the robot autonomously learns the vocalization, and then reproduces the speech articulation from inputted sounds. In the previous system, the speech-learning algorithm of the robot was constructed by employing a self-organizing neural network (SONN), which consists of the combination of a self-organizing map (SOM) and a neural network (NN). However, improper maps were occasionally generated in the results of the speech articulation learning. To solve this problem, a new algorithm introducing two three-dimensional SOMs, called a dual-SOM, was employed for the autonomous learning of the robotic articulations. By applying the robot and its properties, we have constructed an interactive training system. The training is divided into two approaches; one is to use the talking robot to show the shape and the motion of the vocal organs, and the other is to use a topological map to present the difference of phonetic features of a trainee’s voices. In this study, first, the construction of the training system is described together with the autonomous learning of robotic vocalization using the dual-SOM algorithm, and then the analysis of the speech training progress is presented based on the phonetic features and the mechanical vocal articulations.

Describing real and imaginary three-dimensional scenes from the observer’s viewpoint is an intuitive activity for visually non-impaired people, but it is difficult for congenitally blind people, since it involves abstract concepts, such as perspective, depth planes, occlusion, etc. This paper discusses these problems and presents physical environments and procedures supported by an augmented reality tool in order to help blind people to understand, describe and convert three-dimensional scenes into two-dimensional embossed representations. To verify how blind people can acquire these concepts, we developed an augmented reality application that worked as an audio spatial tutor to make the perspective-learning process easy. The application was tested with 10 congenitally blind people, who learned to understand the perspective concepts and who reported on the experience. Finally, we discuss the learning method and technical aspects, suggesting ways to improve the augmented reality application and how it can be released.

A person who is blind can be capable of locating objects and also other people, such as a sighted person, by using audio cues alone. In this research we present the design, development and evaluation of ARTAB (Augmented Reality Tags for Assisting the Blind), a technological assistant for people who are blind, which uses Augmented Reality to identify a set of objects in an indoor environment. As a result, we generated audio-based representations that allow a user to determine the position of an object relative to the angle of vision of the video capture device for navigation purposes. The usability testing performed allowed us to detect that it is not trivial to assign sound effects so that the variation of such effects would imply changes in the position of an object. The continual variation of the sound pitch does not generate the contrast necessary for the user who is blind to be able to obtain a certain kind of information. However, users generally perceive ARTAB as a useful tool for assisting orientation and mobility tasks.

This paper presents data from two independent case studies: a 15-year-old female with cerebral palsy and related profound and multiple learning difficulties, and a 7-year-old male with extreme behaviour associated with autistic spectrum disorder. An audiovisual immersive interactive environment was developed to encourage creative interaction and expression from the participants. In both case studies, there were support workers present and it was the interventions of these support staff which are the main focus of this paper. Results indicated that profuse but unintentional interventions from the staff could have distorted interaction with, dissuaded or diverted participants from meaningful engagements with the reactive feedback provided by the system.

Students who are blind are usually integrated at public schools with sighted students. Since most of science education curriculum resources are based on visual representations such as diagrams, charts, models (physical and computational), and experimentation in science laboratories, students who are blind lack opportunities for participating and collecting first-hand information. The current research project is based on the assumption that the supply of appropriate information through compensatory sensory channels may contribute to science education performance. In the research system, Listening to Complexity, the user interacts with dynamic objects in a real-time agent-based sonified computer model.