An RGB-D Camera based Walking Pattern Detection Method for Smart Rollators He Zhang and Cang Ye Dept. of Systems Engineering University of Arkansas at Little Rock 2801 S. University Ave, Little Rock, AR 72204 Email: {hxzhang1, cxye}@ualr.edu Abstract. This paper presents a walking pattern detection method for a smart rollator. The method detects the rollator user’s lower extremities from the depth data of an RGB-D camera. It then segments the 3D point data of the lower ex- tremities into the leg and foot data points, from which a skeletal system with 6 skeletal points and 4 rods is extracted and used to represent a walking gait. A gait feature, comprising the parameters of the gait shape and gait motion, is then constructed to describe a walking state. K-means clustering is employed to cluster all gait features obtained from a number of walking videos into 6 key gait features. Using these key gait features, a walking video sequence is mod- eled as a Markov chain. The stationary distribution of the Markov chain repre- sents the walking pattern. Five SVMs are trained for walking pattern detection. Each SVM detects one of the five walking patterns. Experimental results demonstrate that the proposed method has a better performance in detecting walking patterns than three existing methods. 1 Introduction Walking therapy is a particular physical therapy (or physiotherapy) that assists a mo- tor-impaired patient to recover their walking ability. This treatment requires interac- tion and cooperation between a therapist and a patient. The patient is offered instruc- tions to perform the physiotherapy exercises in a monitored manner that provides feedbacks to the therapists for evaluating the effectiveness of the exercises and adjust- ing the therapy parameters. However, due to the lengthy recovery process and the need of travel, one-to-one in-clinic treatment is prohibitively expensive. As a result, the patient is taught in clinic about the therapy exercises and performs the exercises at home. While it is cost-effective and save the patient time in travel, at-home physio- therapy does not provide the therapist with feedback in a timely fashion for evaluation and adjustment of the exercises. Often, a patient uses a rolling walker (aka rollator) [2], [6], [7], [8] as a walking aid and to support the therapy exercises during the re- covery process. Our work is therefore to develop a computer vision method for auto- matic detection of walking patterns and devise a smart rollator system that is able to provide persistent monitor on the user’s walking patterns for at-home walking thera- py. The system can be used to score a physiotherapy exercise by monitoring the change in the user’s walking patterns during the course of recovery.