Multi-Constrained Inverse Kinematics for the Human Hand Ali-Akbar Samadani 1 , Dana Kuli´ c 1 and Rob Gorbet 2 Abstract— Measuring the spatial and temporal characteris- tics of hand movement is a challenging task due to the large number of degrees of freedom (DOF) in the hand. This paper presents a multi-constrained inverse kinematics (IK) approach for hand motion estimation from motion capture data. The IK approach satisfies a set of prioritized motion and postural constraints for each hand joint and link. The high-priority constraint is fully satisfied, while the fulfilment of the low- priority constraints is achieved as long as no conflict with the high-priority constraint exists. The proposed approach can aid marker-based motion capture technologies in accurately reconstructing discontinuities or erroneous marker trajectory segments resulting from occluded, missing, or flipped markers. The performance of the multi-constrained IK approach for the hand is tested for a full range of continuous hand motion. I. INTRODUCTION Fine hand finger tracking finds application in a wide range of fields including robotics, computer animation, ergonomics and rehabilitation (e.g.,[1]). A popular technology for mea- suring human movement is optical motion capture, in which reflective markers are secured to landmarks on the body and tracked using high speed cameras [2]. Optical motion capture systems have been used in clinical research as a tool to measure abnormalities in finger motions due to pathologi- cal conditions or neurological impairments (e.g., [3], [4]). However, accurately measuring fine hand and finger motions using optical motion capture is challenging due to issues of occlusion and marker flipping (misidentification of markers when two markers come into close proximity) caused by the large number of markers attached to a small area. Post- processing of the motion capture data to correct flipped markers and apply spline fits to fill-in occluded markers is a tedious task as it requires retracing the marker trajectories frame by frame for discontinuities and unrealistic behaviours (due to marker flipping). Therefore, there is a need for a tool that can automatically reconstruct the defective motion segments. Commercial motion capture systems (e.g., [2]) provide tools such as a reference kinematic model (skeleton) and virtual markers to help acquire clean and continuous motion data. Missing marker trajectories are recovered from neigh- bouring marker trajectories located on the same rigid body segment using inter-marker distance constraints in [5]. Inter- polation techniques are used for reconstructing short missing marker trajectories (e.g.,[6]). The interpolation techniques 1 A.A Samadani and 1 D. Kuli´ c are with the Electrical and Computer Engineering Department at the University of Waterloo, Waterloo, ON N2L 3G1 CANADA (e-mail: asamadani@uwaterloo.ca, dkulic@uwaterloo.ca). 2 R. Gorbet is with the Centre for Knowledge Integration at the Uni- versity of Waterloo, Waterloo, ON N2L 3G1 CANADA (e-mail: rbgor- bet@uwaterloo.ca) require data samples before and after the occlusion; hence, they are only applicable at the post-processing stage. In [7], an extrapolation approach for reconstructing short missing marker trajectory segments from the previously observed marker positions is presented with the assumption that the underlying human motions are linear or circular. Human mo- tions are modeled using a conditional restricted Boltzmann machine in [8], and the resulting models are used to recover the missing marker positions during motion capture. In [9], a dynamic Bayesian network is used to reconstruct the missing marker trajectories assuming that the trajectories are smooth and that there is a correlation between different marker trajectories. Motion sequences are modeled using principal component analysis (PCA) as a hierarchy of linear models, and then a motion sequence with discontinuous marker trajectories is recovered through least squares optimization using available marker positions and the closest linear model representing the motion [10]. These approaches are data- driven; hence, their performance depends on the richness of the human motion data used for the model training. Another approach is the use of filtering methods, such as Unscented Kalman filtering [11]. This approach uses the velocity and acceleration of the tracked marker as the observation states along with those of the neighbouring markers to enhance the reconstruction accuracy. An inverse kinematic solver is then applied to ensure constant link length. The velocity and acceleration information can be very noisy if derived from the captured marker positions. Additional equipment can be used to directly measure velocity and acceleration, however this results in difficulties such as synchronization between separate measurement systems. Furthermore, this approach requires the placement of three markers with constant inter- marker distances on each rigid body segment (in our case, the small area of the finger phalanges), which may result in increased marker occlusion and/or flipping. Another class of approaches to determine the joint con- figuration given a measured posture or end-point position is Inverse Kinematics (IK). There are two main approaches in IK: analytical and differential. Analytical IK is obtained by finding a closed form solution for the inverse of the forward kinematic function, and is specific to a particular structure being studied. However, a closed-form solution is not guaranteed to exist for complex kinematic structures. In contrast with the analytical approach, the differential ap- proach is applicable to any kinematic structure. Differential IK linearly maps the Cartesian velocity of a point along a kinematic chain to the joint velocities along the chain using the structure Jacobian. Comparisons between different IK approaches for articulated body motions can be found in [12] 34th Annual International Conference of the IEEE EMBS San Diego, California USA, 28 August - 1 September, 2012 6780 978-1-4577-1787-1/12/$26.00 ©2012 Crown