Finger Kinematic Modeling and Real-Time Hand Motion Estimation P. CERVERI, 1 E. DE MOMI, 1 N. LOPOMO, 1 G. BAUD-BOVY, 2 R. M. L. BARROS, 3 and G. FERRIGNO 1 1 Bioengineering Department, Politecnico di Milano University, Piazza Leonardo da Vinci, 32, I-20133 Milan, Italy; 2 Faculty of Psychology, San Raffaele Vita-Salute University, and IIT Network Research Unit of Molecular Neuroscience, San Raffaele Foundation, via Olgettina, 58, I-20132 Milan, Italy; and 3 Laborato´rio de Instrumentac¸a˜o para Biomecaˆnica, Faculdade de Educac¸a˜o Fı´sica – Universidade Estadual de Campinas, Campinas, Brazil (Received 28 September 2005; accepted 2 August 2007; published online 15 August 2007) Abstract—This paper describes methods and experimental studies concerned with quantitative reconstruction of finger movements in real-time, by means of multi-camera system and 24 surface markers. The approach utilizes a kinematic model of the articulated hand which consists in a hierarchical chain of rigid body segments characterized by 22 functional degrees of freedom and the global roto-translation. This work is focused on the experimental evaluation of a kinematical hand model for biomechanical analysis purposes. From a static posture, a completely automatic calibration procedure, based on anthropometric measures and geometric constraints, computes axes, and centers of rotations which are then utilized as the base of an interactive real-time animation of the hand model. The motion tracking, based on automatic marker labeling and predictive filter, is empowered by introducing constraints from functional finger postures. The validation is performed on four normal subjects through different right-handed motor tasks involving voluntary flexion-extension of the thumb, voluntary abduction–adduc- tion of the thumb, grasping, and finger pointing. Perfor- mances are tested in terms of repeatability of angular profiles, model-based ability to predict marker trajectories and tracking success during real-time motion estimation. Results show intra-subject repeatability of the model cali- bration both to different postures and to re-marking in the range of 0.5 and 2 mm, respectively. Kinematic estimation proves satisfactory in terms of prediction capability (index finger: maximum RMSE 2.02 mm; thumb: maximum RMSE 3.25 mm) and motion reproducibility (R 2 coefficients—index finger: 0.96, thumb: 0.94). During fast grasping sequence (60 Hz), the percentage of residual marker occlusions is less than 1% and processing and visualization frequency of 50 Hz confirms the real-time capability of the motion estimation system. Keywords—Hand kinematic model, Real-time tracking, Thumb motion, Reaching, grasping, Finger movements. INTRODUCTION Since last years the analysis of in-vivo hand motion, specifically focused on finger joints, has gained grow- ing attention in several scientific domains like biome- chanics, 3,5,9,14,18,27,34,37 neurophysiology, 17,20,25,28,32 and hand surgery. 6,8,33,36 Quantitative evaluation of traumatic and pathologic effects on hand functions, planning of surgical intervention, effectiveness of rehabilitative therapy and physiological understanding represent biomedical applications that can benefit from accurate kinematic and dynamic models of the hand structure. In particular, the ability to replicate in real- time articulated hand movements is advocated by re- search domains like psychological investigation of multi-sensorial integration, tele-manipulation, device remote control and human-computer interac- tion. 22–24,31 However, because of the great number of involved degrees of freedom (DoF) and of the com- plexity of the mapping between external measurements and internal functional variables, the accurate assess- ment of in-vivo hand kinematics and dynamics repre- sents a complex task. Electromyography (EMG) and kinetic data are ra- ther common measures for investigating muscle coor- dination or synergy, but they do not supply accurate information about segmental motion evolution. 11,28,32 Electromagnetic and video-based systems have been largely proposed in the field of motion analysis and they proved effectiveness for gait analysis, but their use for hand motion analysis has been poorly investigated. While electromagnetic motion analysis systems were applied to measure gross hand movements, 14 the suc- cess of video-based systems coupled with surface markers to estimate fine finger motion was demon- strated through fluoroscopy validation, where wave- forms obtained from surface markers compared favorably with ‘‘gold standard’’ fluoroscopy wave- forms. 26 However, only planar movements of the forefinger were analyzed. Address correspondence to P. Cerveri, Bioengineering Depart- ment, Politecnico di Milano University, Piazza Leonardo da Vinci, 32, I-20133 Milan, Italy. Electronic mail: pietro.cerveri@polimi.it Annals of Biomedical Engineering, Vol. 35, No. 11, November 2007 (Ó 2007) pp. 1989–2002 DOI: 10.1007/s10439-007-9364-0 0090-6964/07/1100-1989/0 Ó 2007 Biomedical Engineering Society 1989