Skill Learning using Temporal and Spatial Entropies for Accurate Skill Acquisition Sang Hyoung Lee, Gyung Nam Han, Il Hong Suh † , and Bum-Jae You Abstract— In manipulation tasks, skills are usually modeled using the continuous motion trajectories acquired in the task space. The motion trajectories obtained from a human’s multi- ple demonstrations can be broadly divided into four portions, according to the spatial variations between the demonstrations and the time spent in the demonstrations: the portions in which a long/short time is spent, and those in which the spatial variations are large/small. In these four portions, the portions in which a long time is spent and the spatial variation is small (e.g., passing a thread through the eye of a needle) are usually modeled using a small number of parameters, even if such portions represent the movement that is essential for achieving the task. The reason for this is that these portions are slightly changed in the task space as compared with the other portions. In fact, such portions should be densely modeled using more parameters (i.e., overfitting) to improve the performance of the skill because the movements of those portions must be accurately executed to achieve the task. In this paper, we propose a method for adaptively fitting these skills based on the temporal and the spatial entropies calculated by a Gaussian mixture model. We found that it is possible to retrieve accurate motion trajectories as compared with those of well-fitted models, whereas the estimation performance is generally higher than that of an overfitted model. To validate our proposed method, we present the experimental results and evaluations when using a robot arm that performed two tasks. I. I NTRODUCTION In manipulation tasks, skills are usually learned using continuous motion trajectories. An intelligent robot should therefore be able to learn such skills, using a set of the continuous motion trajectories obtained from a human’s mul- tiple demonstrations. In such a set of motion trajectories, the trajectories can be categorized into four portions, according to the spatial variations between the demonstrations and the duration (i.e., time spent) of the demonstrations. Let us consider an example to intuitively understand the portions that constitute a manipulation task. A robot learns a skill for painting an assembly part based on a human’s multiple demonstrations. The procedure is as follows: the robot first lifts up a brush to the part. Next, it uses the brush to paint *This work was supported by the Global Frontier R&D Program on <Human-centered Interaction for Coexistence> funded by the National Re- search Foundation of Korea grant funded by the Korean Government(MEST) (NRF-M1AXA003-2011-0028353) S. H. LEE is with the Department of Electronics and Computer Engi- neering, Hanyang University, Seoul, Korea. zelog@hanyang.ac.kr G. N. Han is with the Department of Electronics and Computer Engineer- ing, Hanyang University, Seoul, Korea. mudian@hanyang.ac.kr † I. H. Suh is with the Department of Computer Science and Engineer- ing, Hanyang University, Seoul, Korea. ihsuh@hanyang.ac.kr, All correspondence should be addressed to I. H. Suh. B. J. You is with Korea Institute of Science and Technology, Seoul, Korea. ybj@kist.re.kr the part along a zigzag path. Finally, the brush is withdrawn from the part. In this painting example, Fig. 1 shows the set of the motion trajectories obtained from a human’s multiple demonstrations. In the figure, portion (1) involves the painting of the assembly part, and the other portions involve the lifting of the brush and withdrawal of the brush from the part. Portion (1) takes up the longest time, although the movement is slightly changed in the demonstrations when the part is fixed in a specific location, as shown in Fig. 1. Next, let us consider modeling a Gaussian Mixture Model (GMM) using the motion trajectories in Fig. 1. The GMM is estimated using Bayesian Information Criterion (BIC) and Expectation-Maximization (EM) algorithms, as shown in Fig. 2-(a). In the GMM, portion (1) in Fig. 1 is sparsely modeled, even though the GMM is well fitted by the BIC and EM algorithms. The problem can be formulated in terms of the differences between Fig. 1 and Fig. 2-(b). There is no zigzag path in the motion trajectories retrieved by a Gaussian Mixture Regression (GMR) process. The GMM should be modeled to use more parameters to retrieve the zigzag path because this path is essential in achieving the task. When the number of parameters (i.e., the number of Gaussians) is forcefully increased, in this context, the rest of the portions are more densely overfitted than portion (1), as shown in Fig. 3. The reason for this is that the changes of the movements in the rest of the portions are larger than they are in portion (1). Although in portion (1), slightly accurate motion trajectories can be retrieved by the GMM in Fig. 3, the estimation performance is lower than that of the GMM in Fig. 2. As shown in Fig. 3-(b), the motion trajectories are overfitted in all portions. The model should be able to remodeled using more parameters only in portion (1), while maintaining the rest of the portions. In this context, we propose a method for learning skills that consider the spatial variations between multiple demon- strations and the time spent in these demonstrations, based on the entropies involved. Fig. 4 shows the entire process of the proposed method. The set of the continuous mo- tion trajectories obtained from multiple demonstrations is temporally aligned using a Dynamic Time Warping (DTW) algorithm, as shown in Fig. 4-(a). The motion trajectories are projected onto the reduced dimensional subspace by Principal Component Analysis (PCA), as shown in Fig. 4- (b). A GMM is estimated to contain the temporal and the spatial information based on the BIC and EM algorithms for the calculation of the entropies of each portion, as shown in Fig. 4-(c). The temporal and the spatial entropies are calculated as per the Gaussians from the estimated GMM, 2013 IEEE International Conference on Robotics and Automation (ICRA) Karlsruhe, Germany, May 6-10, 2013 978-1-4673-5642-8/13/$31.00 ©2013 IEEE 1315