IEEE TRANSACTIONS ON VISUALIZATION AND GRAPHICS, VOL. 17, NO. 3, MARCH 2011 to appear 1 Scan-Based Volume Animation Driven by Locally Adaptive Articulated Registrations Taehyun Rhee, Member, IEEE, J.P. Lewis, Member, IEEE, Ulrich Neumann, Member, IEEE, and Krishna Nayak, Member, IEEE Abstract—This paper describes a complete system to create anatomically accurate example-based volume deformation and animation of articulated body regions, starting from multiple in vivo volume scans of a specific individual. In order to solve the correspondence problem across volume scans, a template volume is registered to each sample. The wide range of pose variations is first approximated by volume blend deformation (VBD), providing proper initialization of the articulated subject in different poses. A novel registration method is presented to efficiently reduce the computation cost while avoiding strong local minima inherent in complex articulated body volume registration. The algorithm highly constrains the degrees of freedom and search space involved in the non-linear optimization, using hierarchical volume structures and locally constrained deformation based on the biharmonic clamped spline. Our registration step establishes a correspondence across scans, allowing a data-driven deformation approach in the volume domain. The results provide an occlusion free person-specific 3D human body model, asymptotically accurate inner tissue deformations, and realistic volume animation of articulated movements driven by standard joint control estimated from the actual skeleton. Our approach also addresses the practical issues arising in using scans from living subjects. The robustness of our algorithms is tested by their applications on the hand, probably the most complex articulated region in the body, and the knee, a frequent subject area for medical imaging due to injuries. Index Terms—Registration, Deformation, Volume Animation. ✦ 1 I NTRODUCTION 1.1 Motivation A N anatomically accurate 3D human body model including the bones, muscles, tendons, and other anatomical layers is important in various fields. Artic- ulated body regions such as the human knee or hand are capable of a wide range of skeletal movements, resulting in complex deformation of the surrounding soft tissues. Although a physically-based model of important anatomical layers can approximate such deformations, manually creating an accurate physical model including many anatomical layers is difficult and may not exactly mimic the complex interaction and compression among different tissue layers for a specific person. The difficulty in manually or algorithmically defining complex articulated body structures of an individual subject can be avoided by adopting a data-driven ap- proach. Since scans of a living subject at multiple poses can be used as the training samples, accurate deformable models can be built from actual data. Also, a model con- structed from living human scans reflects characteristics of the subject and provides personalized information, which is often essential to create a virtual clone for medical and other applications. Volume data obtained from 3D medical image scans (e.g. MRI or CT) represents 3D interior anatomy with- • Taehyun Rhee is with the Samsung Advanced Institute of Technology and U. of Southern California. E-mail: thrhee@samsung.com • J.P. Lewis is with Weta Digital LTD. and Victoria University. • Ulrich Neumann and Krishna Nakay are with U. of Southern California. Manuscript received 17 Dec. 2008; revised 29 May 2009, accepted 3 Oct. 2009. out any occlusion. Translucent volume rendering can successively visualize all these anatomical layers with- out losing the overall context of the subject. Previous scan based approaches have focused on surface scans and deformation. We develop a data-driven approach in the volume domain using appropriate deformation algorithms, resulting in accurate volume deformation informed by multiple scans of articulated body regions from a living person. Based on our literature survey, we believe this is the first achievement of this type. One of the challenging issues in scan-based deforma- tion is to obtain geometric correspondences across the volume samples [1]. In case of medical image volumes, the geometrical information is represented by voxel properties without explicit geometric parameterizations, and creating iso-surfaces of each layer from in vivo MRI volumes is very difficult due to poor delineation of different tissue layers. Our approach uses the only available voxel intensity information without any fiducial markers. To accomplish this, the non-rigid volume registration requires complex non-linear optimization, and the total degrees of freedom (DOF) in the deformation must be carefully controlled. In addition, the optimization must start from proper initialization to avoid the strong local minima that arise in matching articulated subjects. There are also issues involving the use of in vivo MRI that do not arise with cadavers or non-articulated subjects; these concerns are described in section 2.