1 An Adaptive Virtual Biofeedback System for Neuromuscular Rehabilitation O. Barzilay 1 , A. Wolf 1 1 Technion Israel Institute of Technology / Dept. of Mechanical Engineering, Biorobotics and Biomechanics Lab, Haifa, Israel Abstract— The purpose of this research is to enhance the methods used in biomechanical rehabilitation by combining virtual reality, artificial intelligence, motion capture and bio- feedback instrumentation. During training, the subject wears 3D glasses, in which virtual tasks (i.e. trajectories) are dis- played to him. The subject is then asked to follow the virtual task displayed to him and match his own movement to it. Dur- ing task performance, the subject's kinematics and electromy- ograms (E.M.G.) signals are tracked and recorded by Vicon motion tracking system. We use an intelligent learning system to model on-line the performance of the subject. Once trained, the system changes and adapts the task displayed to the sub- ject, producing a patient-specific task for better neuromuscu- lar rehabilitation. Moreover, the system creates a more enter- taining environment which increases the efficiency of physiotherapy, in adults and especially in pediatric. Besides physiotherapy, this system can be used in other ap- plications, such as performance enhancement in sports train- ing and as an educational tool in any application requiring precise and controlled movement and coordination. Keywords— Rehabilitation, virtual reality, artificial intelli- gence, biofeedback, electromyogram. I. INTRODUCTION Rehabilitation helps a significant number of people eve- ryday to recover from surgical operations, strokes, or inju- ries. Physiotherapy strives to develop, maintain, and restore maximal movement and functionality to the injured organ. For good results, the physiotherapist has to tailor specific physical exercises that best fit the subjects' pathology and needs. Accurate repetitions of these biomechanical exercises are key points for successful physiotherapy treatment. Moreover, the physiotherapist must adapt the exercises as a response to the subject's performances. This adaptive physi- otherapy is very difficult to perform online and is subjected to the trainer interpretation of the patient's performance. In the last decade, the use of virtual reality in neuromus- cular therapy has exponentially increased. Virtual environ- ment technology has been used to assist rehabilitation in neurological diseases [1, 2], for patients with balance disor- ders [3], or even sports and musical performance enhance- ment. The results obtained in the last few years with this technique have been very encouraging and researches even claim that motor learning in virtual environments can sur- pass training in the real world (e.g. [4]). Virtual motor learning generates repetitive training with enhanced feed- back (and even real-time feedback). Furthermore, studies [5, 6] have shown that most patients are greatly motivated by the virtual reality therapy. In our laboratory, children have been strongly stimulated by the visual feedback and imme- diately turned to be more cooperative during testing and training. In this paper, we introduce a virtual adaptive biofeedback rehabilitation approach that optimizes neuromuscular train- ing using an artificial intelligence learning system that learns from real-time biofeedback and produces online new patient-specific virtual physiotherapy missions. With the help of a motion capture system and electromyograms (EMG), the system tracks at any time the kinematics of a subject and his muscles activation. The subject is exposed, via a head mounted display unit, to virtual tasks, which he then is asked to perform. A neural network is trained to respond to the subject's biofeedback information having the desired muscles activation and motions as references. Once training is complete, the network calculates a new trajectory as biomechanical exercise, subjected to the previous per- formance of the subject. This adaptive loop is repeated continuously, resulting in an online biofeedback-based adaptive rehabilitation virtual training system. II. SYSTEM DESCRIPTION A. Experimental Setup Using the Vicon motion capture system capabilities [7], we track the subjects' motions in real time and gather mo- mentarily kinematic data. Markers are placed on the sub- ject's body or on part of it (we first focused our study on the arms). During training, the subject is immersed in a virtual environment in which he is shown floating targets (see Fig. 1). The subject is then asked to follow the motions of the targets with his pointing finger. His motions are continu- ously recorded by the system while following the virtual missions presented to him. Moreover in order to record the subject muscular activation, we place electromyograms sensors on key muscles associated related to the motion.