Expert Systems With Applications 50 (2016) 75–88 Contents lists available at ScienceDirect Expert Systems With Applications journal homepage: www.elsevier.com/locate/eswa Expert system for automated bone age determination Jinwoo Seok a,∗ , Joséphine Kasa-Vubu b , Michael DiPietro c , Anouck Girard a a Department of Aerospace Engineering, University of Michigan, Ann Arbor, Michigan 48109, United States b Department of Pediatrics-Division of Pediatric Endocrinology, University of Michigan Health System, Ann Arbor, Michigan 48109, United States c Department of Radiology, Section of Pediatric Radiology University of Michigan Health System, Ann Arbor, Michigan 48109, United States article info Keywords: Bone age Expert system Automated system Fusion rule Classification abstract A novel automated bone age determination algorithm using left hand X-ray images, which provides con- sistent overall bone age as well as five part bone ages, is presented in this paper. Based on the descriptive narrative from the Greulich and Pyle atlas as well as those from other more recent studies, 17 region of interests are selected and based on anatomical similarity, five clinically relevant groupings (or “parts”) are defined on left hand X-ray images. When disharmonious maturations for different regions of interest are large, providing part bone ages with overall bone age is helpful to pediatricians. Based on interviews with two experts to get input on their bone age determination strategy, overall bone age determination can be viewed as the weighted sum of “part bone ages” of the five parts. Using the method of least squares and inputs from five (human) readers, we extract weights for bone age determination using all five parts (as well as reduced algorithms using only four, three or two parts). The weights indicate that part 1 (distal joints) has the highest priority. Overall bone age is then estimated based on the weights and bone ages of available parts. In our work, a computer vision algorithm provides bone ages of individual regions of interest. To combine the region of interest computer classifiers and generate each of the five part bone ages, we develop and analyze fusion rules of multiple classifiers with more than three classes each. The fusion rules take into account performance of each region of interest classifier. Once the part bone ages are obtained based on the fusion rule and region of interest classifiers, the overall bone age is determined in a fully automated way. Finally, we give a use case for the whole automated bone age determination system and validation of the algorithm based on given performance of each region of interest computer classifier. Results indicate that the algorithm works well with reasonably good classifiers. © 2015 Elsevier Ltd. All rights reserved. 1. Introduction 1.1. Motivation Early detection of possible growth disorders or abnormal pu- bertal maturation is an important aspect for wellbeing. The assess- ment of growth and pubertal maturation is central to the practice of pediatric endocrinology. Bone age (BA) is a measure of the de- gree of maturation of a child’s skeleton. Its reliable assessment is a key reference in growth and maturation evaluation. The Greulich and Pyle (GP) (Greulich & Pyle, 1959) atlas is the predominant clinical reference to determine BA in pediatric en- docrinology in the United States. To determine BA, a radiologist compares a patient’s X-ray to those contained in the reference ∗ Corresponding author. Tel.: +17183168589. E-mail addresses: sjinu@umich.edu (J. Seok), jzkv@med.umich.edu (J. Kasa- Vubu), dipietro@med.umich.edu (M. DiPietro), anouck@umich.edu (A. Girard). atlas. The patient’s X-ray is then assigned the BA corresponding to the closest matching atlas image. When comparing the X-ray and the atlas, clinicians focus on a number of areas of special interest, subsequently called Re- gions of Interest (ROI). Disharmonious maturations may be found in BA assessment for different ROIs. However, there is no stan- dard rule to determine the overall BA when the maturation of each ROI is different. In this situation, the clinicians rely on their experience and personal opinions. From an engineering perspec- tive, the method is human-centric and subject to bias in inter- pretation. In other words, different radiologists may use differ- ent criteria when matching the patient’s X-ray to the atlas, which may cause different overall BA readings. Furthermore, if dishar- monious maturations are significant, overall BA may not be rep- resentative, and clinicians may benefit from additional informa- tion on maturation of different parts in the hand. However, no formal method to provide this information exists. Generating and providing that additional information would be helpful for the clinicians. http://dx.doi.org/10.1016/j.eswa.2015.12.011 0957-4174/© 2015 Elsevier Ltd. All rights reserved.