1592 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 58, NO. 6, JUNE 2011 Multiple-Object 2-D–3-D Registration for Noninvasive Pose Identification of Fracture Fragments Ren Hui Gong, James Stewart, and Purang Abolmaesumi ∗ Abstract—This paper presents a multiple-object 2-D–3-D regis- tration technique for noninvasively identifying the poses of fracture fragments in the space of a preoperative treatment plan. The plan is made by manipulating and aligning computer models of indi- vidual fracture fragments that are segmented from a diagnostic computed tomography. The registration technique iteratively up- dates the treatment plan and matches its digitally reconstructed radiographs to a small number of intraoperative fluoroscopic im- ages. The proposed approach combines an image similarity metric that integrates edge information with mutual information, and a global–local optimization scheme, to deal with challenges associ- ated with the registration of multiple small fragments and limited imaging orientations in the operating room. The method is easy to use as minimum user interaction is required. Experiments on sim- ulated fractures and two distal radius fracture phantoms demon- strate clinically acceptable target registration errors with capture range as large as 10 mm. Index Terms—Computer-assisted fracture reduction, covariance matrix adaptation evolution strategy (CMA-ES), digitally recon- structed radiograph (DRR), edge enhancement, fluoroscopic im- age, global–local alternating optimization, multiple-object 2-D– 3-D registration, mutual information (MI), noninvasive pose iden- tification, treatment plan. I. INTRODUCTION I N computer-assisted surgery (CAS), a fundamental task is to accurately establish a spatial correspondence between a preoperative treatment plan and the patient in operating room (OR). In the context of fracture treatment in orthopedics, the task becomes to identify the poses of the fracture fragments in the space of the plan in order to highlight deviations between the actual and planned positions of the fragments. Conventionally, optical tracking is used to identify the po- sition of each fracture fragment. This pose identification tech- nology is accurate, fast, and reliable. However, it requires line of sight between the camera and the reference bodies that are Manuscript received July 15, 2010; revised November 26, 2010; accepted December 21, 2010. Date of publication January 13, 2011; date of current ver- sion May 18, 2011. This work was supported in part by the Natural Sciences and Engineering Research Council (NSERC) and the Canadian Institutes of Health Research (CIHR). Asterisk indicates corresponding author. R. H. Gong and J. Stewart are with the School of Computing, Queen’s University, Kingston, ON K7L 3N6, Canada (e-mail: rhgong@cs.queensu.ca; jstewart@cs.queensu.ca). ∗ P. Abolmaesumi is with the Department of Electrical and Computer En- gineering, University of British Columbia, Vancouver, BC V6T 1Z4, Canada (e-mail: purang@ece.ubc.ca). Digital Object Identifier 10.1109/TBME.2011.2105487 mounted on each fragment. Such approach has several draw- backs: first, the size and weight of the reference bodies may limit their application in certain fractures that contain multiple small fragments; second, the line-of-sight constraint limits the surgeon’s flexibility in the OR; and third, the mounting process is invasive, which may lead to longer recovery time. 2-D–3-D registration is an alternative pose identification tech- nique that overcomes the limitations of optical tracking. To iden- tify the pose, a treatment plan generated from preoperative data, such as computed tomography (CT) images of the trauma re- gion, is mapped to intraoperative fluoroscopic images through an image registration process. This process is generally slow and less accurate than optical tracking; however, it provides a noninvasive solution for the cases where optical tracking is impossible or costly. The registration process is usually per- formed by maximizing a similarity metric between simulated fluoroscopic images generated from the CT data, called digi- tally reconstructed radiographs (DRRs), and the actual fluoro- scopic images. A number of DRR-based 2-D–3-D registration techniques are suggested (see, e.g., [1]–[6]). However, none of these techniques have been reported for the pose identification problem in fracture treatment, primarily due to the following challenges: 1) Involvement of multiple moving objects: Most current 2-D–3-D registration techniques handle only one large or long bone such as pelvis or femur. In fracture treatment, multiple and maybe small bones are involved, which not only increase the computation complexity, but also in- crease the likelihood of occlusion in fluoroscopic images. 2) Limitations from the constrained OR environment: The imaging orientations are very limited due to the collision of the fluoroscopy imaging device with the OR table. So, the best fluoroscopic imaging views that are optimum for registration may not be available. A few studies explore multiple-object 2-D–3-D registration. Dalvi et al. [7] used phase-based mutual information (MI) as the similarity metric for registering femur and tibia to fluoro- scopic images of the knee joint. Each bone is separately reg- istered to the joint images as a conventional 2-D–3-D registra- tion, and the phase information is used to reduce the impact of the outliers, i.e., the other bone shown in the joint images. The algorithm required very good initialization (±3 ◦ and ±3 pixels). Ma et al. [8] registered multiple bones to two fluoroscopic im- ages: one anteroposterior (AP) view and other lateral view. They use correlation of edge information to deal with overlaps be- tween bones, and an optimization algorithm that takes advantage 0018-9294/$26.00 © 2011 IEEE