1 3 Recovering 3D human pose based on biomechanical constraints, 4 postures comfort and image shading 5 6 7 Leandro Dihl Q1 ⇑ , Soraia Raupp Musse 8 Pontifícia Universidade Católica do Rio Grande do Sul – PUCRS, Av. Ipiranga, 6681, Porto Alegre, RS, Brazil Q2 9 10 12 article info 13 14 Keywords: 15 Image processing 16 Human pose estimation 17 18 abstract 19 This paper presents a new model to identify 3D human poses in pictures, given a single input image. The 20 proposed approach is based on a well known model found in the literature, including improvements in 21 terms of biomechanical restrictions aiming to reduce the number of 3D possible postures that correctly 22 represent the pose in the 2D image. Since the generated set of poses can have more than one possible 23 posture, we propose a ranking system in order to suggest the best generated postures according to a 24 ‘‘comfort’’ criterion and shading characteristics in the image as well. The comfort criterion adopts 25 assumptions in terms of pose equilibrium, while the shading criterion eliminates the ambiguities of pos- 26 tures taken into account the image illumination. We must emphasize that the removal of ambiguous 3D 27 poses related to a single image is the main focus of this work. The achieved results were analyzed w.r.t. 28 visual inspection of users as well as a state of the art technique and indicate that our model contributed in 29 some way to the solution of that challenge problem. 30 Ó 2014 Published by Elsevier Ltd. 31 32 33 34 1. Introduction 35 The way a person poses in front of camera can indicate his/her 36 emotions, attitudes and intentions. Recovering 3D human pose 37 from video streams or images can be useful in areas such as sport 38 performance analysis, automatic search in image databases, avatar 39 reconstruction, and person identification systems, among other 40 applications. Indeed, many applications can benefit from this tech- 41 nology that can deal with data coming from single images or vid- 42 eos. However, besides the large range of applications, this is an 43 open research field due to the variability of possible human move- 44 ments, partial occlusions and the restrictions imposed by loss of 45 information when the 3D world is mapped into a single 2D image. 46 Considerable effort has been done in order to solve such problems. 47 Several papers surveying the state of the art in the area are avail- 48 able in the literature, providing good summaries of the models cur- 49 rently being developed (Agarwal & Triggs, 2006; Moeslund, Hilton, 50 & Krüger, 2006). Q4 The recovery of human poses can deal with infor- 51 mation from a single image, where there is no depth information, 52 or stream video, which presents motion and time information in- 53 volved in the process. The output of pose recovery algorithms also 54 can vary in nature, being a pose in two or three dimensions, 55 depending on the needs of the solution. 56 An important problem in 3D human pose recovery is the ambi- 57 guity. Such ambiguity is generated when estimating three-dimen- 58 sional positions from a 2D image, since many 3D postures can 59 present same 2D projections, and it is inherent to the loss of depth 60 information present in 2D images (Hua, Yang, & Wu, 2005). Many 61 authors (Jiang, 2010; Pishchulin, Jain, Andriluka, Thormahlen, & 62 Schiele, 2012; Wei & Jinxiang, 2009) mention this as one of the 63 main problems to obtain the 3D human posture. In the study 64 developed by Agarwal and Triggs (2004), the authors define this 65 as an intrinsic challenge to estimate 3D poses. The treatment of 66 ambiguity is dealt in several ways. For instance, Wei and Jinxiang 67 (2009) propose a method that uses a set of biomechanical restric- 68 tions on the angles of the body joints to eliminate ambiguity. In the 69 study by Lee and Cohen (2006), the ambiguity is handled through 70 an approach that employs Markov Chains. Moeslund et al. (2006) 71 uses techniques based on kinematic constraints and movements 72 to treat ambiguity based on motion capture. Moreover, in the study 73 of state of the art we find methods that are focused on a particular 74 controlled situation, usually requiring databases containing pos- 75 ture samples. It is the case of the model proposed by Mori, Ren, 76 Efros, and Malik (2004), who developed an approach for obtaining 77 poses of baseball players. 78 In this paper, we propose a new model to handle the ambiguity 79 problem, by initially using a set of biomechanical restrictions to http://dx.doi.org/10.1016/j.eswa.2014.03.049 0957-4174/Ó 2014 Published by Elsevier Ltd. ⇑ Corresponding author Q3 . E-mail addresses: leandro.dihl@acad.pucrs.br (L. Dihl Q1 ), soraia.musse@pucrs.br (S.R. Musse). URL: http://www.inf.pucrs.br/~vhlab (L. Dihl Q1 ). Expert Systems with Applications xxx (2014) xxx–xxx Contents lists available at ScienceDirect Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa ESWA 9262 No. of Pages 10, Model 5G 16 April 2014 Please cite this article in press as: Dihl Q1 , L., & Musse, S. R. Recovering 3D human pose based on biomechanical constraints, postures comfort and image shad- ing. Expert Systems with Applications (2014), http://dx.doi.org/10.1016/j.eswa.2014.03.049