A Hierarchical Approach to Landmarks Detection in Taekwondo Poomsae Videos Munther Abualkibash 1 , James Gedney 1 , Yongbom Kim 2 , Jeongkyu Lee 1 1 Department of Computer Science and Engineering, 2 Department of Martial Arts University of Bridgeport, CT USA {mabualki, jgedney, ybkim, jelee}@bridgeport.edu Abstract—Taekwondo Poomsae performance is a series of basic movements for offense and defense techniques. Despite the high popularity and long history of Taekwondo, there has been less effort to systemize Taekwondo Poomsae competition. In this paper, we propose a hierarchical approach to landmarks detection in Taekwondo Poomsae videos, which is a significant change in a series of movements. First, we propose a kinematic model for basic Poomsae movements based on the anatomic analysis of player’s body parts. Second, we measure a player’s movement from a Poomsae video using changed pixels. Third, we segment a Poomsae video into a number of movements, each of which contains the same semantic, i.e., basic Poomsae movement. Since the initial segments are usually over- segmented, we classify the segmented movements into higher level that represent significant movements of Poomsae performance. Finally, we identify landmarks from the created movement hierarchy. The experimental results show that the 70% of landmarks are detected correctly. Keywords-Landmark detection;Taekwondo video;Poomsae I. INTRODUCTION Over 177 countries, more than five million people world- wide practice Taekwondo as their martial art style. Specifically, the Poomsae (called as form, kata, or hyung) 1 is a series of basic movements in Taekwondo for offense and defense techniques. Despite the high popularity and long history of Taekwondo, there has been less effort to systemize Taekwondo Poomsae competition, which may cause several issues, such as fair judging and accurate scoring, which are caused by the subjective decision in Poomsae competition. In order to address the judging issues, various techniques are adapted into many sports games. The most popular technique is instant replay [1,2] using video technology to correct referee’s mistake in NFL (National Football League), MLB (Major League Baseball), and tennis games. Another approach is to utilize electric devices. World Taekwondo Federation has been developing a new electric guard that can automatically detect punch and kick in two players’ sparring match [3]. 1 The term ‘Poomsae’ is Korean that is an official language in Taekwondo as well as English. We use Taekwondo terms in Korean with comments in English for more accurate information. Taekwondo Poomsae cannot be free from the issue of referee’s judgments. Therefore, it is highly desired to model and systemize a Poomsae performance using a captured video. In this paper, we propose a hierarchical approach to detection of landmarks in Taekwondo Poomsae videos. A landmark in Taekwondo Poomsae videos can be defined as follows: Definition 1. A landmark of Taekwondo Poomsae video is continuous frames that indicate a boundary of two different movements with significant change. The examples of landmarks in Taekwondo Poomsae videos are “ready stance”, “turning (changing direction)”, and “combination of several movements (kick and punch)”. Since a landmark is a significant change along a series of movements in a Poomsae performance, it is highly desired to detect and identify such landmarks from a video, which can be used for further analysis of Poomsae performance. In order to detect landmarks from Taekwondo Poomsae videos, we employ a hierarchical method, i.e., bottom-up approach. The proposed approach consists of two main components, i.e., characterization and analysis, and four sub- components as follows: (1) Movement modeling: Based on the anatomic analysis of player’s body parts, a kinematic model is proposed for basic movements of Poomsae performance; (2) Movement measurement: Using a frame-based pixel difference technique, we measure a player’s movement from a Poomsae video that is captured by a camera; (3) Movement Hierarchy: Using the measurement, we segment a Poomsae video into a number of movements, each of which contains the same semantic. Since the initial segments are usually over-segmented, we classify more significant movements into higher level; and (4) Landmarks Detection: We identify landmarks from the upper level of created movement hierarchy. Our contributions in this paper are as follows: (1) We propose a new kinematic model of basic movements in Taekwondo Poomsae based on the anatomic analysis of player’s body; (2) We characterize a player’s movement of Taekwondo Poomsae video based on the kinematic model; and, (3) We propose a hierarchical approach to detect landmarks from a Poomsae video, which can be used for basic unit of Poomsae video analysis.