Virtual Human Physiotherapist Framework for Personalized Training and Rehabilitation Sabarish Babu Catherine Zanbaka Jonathan Jackson TaeOh Chung Benjamin Lok 1 Min C. Shin Larry F. Hodges Department of Computer Science University of North Carolina at Charlotte 1 Computer and Information Science and Engineering Department, University of Florida 1 Motivation and Related Works For patients recovering from injuries, or individuals that require a system to provide feedback on aspects of ex- ercise performance, physical therapy sessions with a professional trainer can be expensive and inconvenient. In addition, privacy might be an issue for some patients who are not comfortable closely interacting with strangers. Nonetheless, personalized feedback and monitoring of the individual’s performance on a spe- cific exercise routine is crucial for faster rehabilitation and effective training. Moreover, improper training may even lead to injury. We present a framework that employs a Virtual Human Physiotherapist (VHP) towards personalized training and rehabilitation. The VHP is an inexpensive alternative to training with a real person. Using a virtual human for training allows the user to train on their own schedule and in the privacy of their own homes. In ad- dition, the gender, race and age of the virtual human can be altered to match the needs of the user, making the user more comfortable in interacting with the VHP. Finally, the VHP provides a uniform and accurate train- ing experience. Unlike real people, virtual humans are not affected by ‘having a bad day’ or being tired. The VHP is always happy and ready to assist the user in training for a particular exercise. There is also no limit on the amount times the user can repeat an exercise. Researchers have shown that a virtual human inter- face can provide feedback to individuals using a variety of verbal and non-verbal behaviors. Anecdotal evi- dence suggests that human communication consists of a high bandwidth of several modalities such as gestures, facial expressions, speech, and body language [1]. In addition, researchers have found that demonstrating a task can be far more effective than trying to perform it, especially when that task involves spatial motor skills [2]. Using a virtual human interface minimizes the need for training users, since they already know how to in- teract with other people [1]. Finally, Zanbaka, et al., found that people respond to virtual humans similarly to the way they respond to real humans. The authors were able to elicit social inhibition from female partici- pants, using a virtual human [3]. Virtual humans have already been used in training. Slater, et al., examined the use of virtual humans by actors and a director to rehearse for a live performance and found that a performance level was reached in the virtual rehearsal which led to a successful live perform- ance, one that could not have been achieved by only learning of lines or video conferencing [4]. Virtual humans are also being used to allow medical students to experience the interaction between a patient and a medical doctor using natural methods of interaction with a high level of immersion [5]. At UNCC, a VR training tool for nursing students is being developed which will allow a nurse to verbally interview, observe, and examine a digital patient similar to the way she would interact with a real patient [6]. Researchers at Carnegie Mellon University have developed a virtual reality Tai Chi training application where the user is able to observe and mimic the presented virtual charac- ter of a Tai Chi master. However, the system does not give automatic feedback. The accuracy of the user’s body position is determined only by the user’s own perception of the virtual character [7]. 2 System Framework The VHP framework uses the Straps system which en- ables the tracking and recovery of 3D position of color markers in the scene [8]. The color markers are used in order to obtain skeletal joint position of the user. The color markers were placed in the positions on the exer- cising arm and shoulders of the user. The proof-of- concept exercise used in VHP is the bicep-curl. Figure 1 shows the position of the color markers on the user’s body which were tracked by the Straps system. The Straps system sends skeletal joint position information of the user to the rendering and animation component of the VHP framework.