ORIGINAL PAPER Identification and analysis of photometric points on 2D facial images: a machine learning approach in orthodontics Gururajaprasad Kaggal Lakshmana Rao 1 & Arvind Channarayapatna Srinivasa 2 & Yulita Hanum P. Iskandar 3 & Norehan Mokhtar 1 Received: 7 November 2018 /Accepted: 15 March 2019 # IUPESM and Springer-Verlag GmbH Germany, part of Springer Nature 2019 Abstract The lack of an effective and automated facial landmark identification tool has prompted us to design and develop a smart machine learning approach. The study aims to address two objectives. The primary objective is to assess the effectiveness and accuracy of algorithmic methodology in identifying and analysing facial landmarks on two dimensional (2D) facial images and the secondary objective is to understand the clinical application of automation in facial landmark identification. The study has utilised 418 facial landmark points and 220 landmark measures from 22 2D facial images of volunteers. The study has used a deep learning algorithm ‘You Only Look Once (YOLO)’ to determine the accuracy of the developed system and its clinical applications. The system identified 418 landmarks in total with facial recognition being 100%. Of the total 220 landmark measures, the system provided 48 (21.81%) measures in the error range of 0 to 1 mm, 75 (34.09%) measures in the error range of 2 to 3 mm, 92 (41.81%) measures in the error range of 4 to 5 mm followed by 5 (2.2%) measures in the range of 6 mm. The smart and innovative approach provides valuable training and a helpful tool for the students performing the clinical facial analysis. The automated system with its effective and efficient algorithm delivers fast and reliable landmark identification and analysis. Keywords Orthodontic photometric points . Orthodontic facial measures . Frontal facial photography . YOLO . Machine learning algorithm . Deep learning . Orthodontics . Smart learning 1 Introduction Photography is an ideal method for analysing the pre- operative dental conditions for recording a baseline of oral health. The extraoral photographs provide an excellent re- source for the student and the clinician in arriving at a firm diagnosis and treatment option to restore health, function and aesthetics. The clinical photographic documentation of a pa- tient’ s face contributes immensely towards examination, diag- nosis and treatment planning in the orthodontic management of a patient. The clinical photographs which aid facial mea- surements are a valuable source of information in evaluating orthodontic needs of the patient [1, 2]. The facial measure- ments help relate the facial soft tissues to their corresponding hard tissues. The facial structures exhibit variations creating disproportions. The orthodontic application of these facial measures involves the assessment, identification and treat- ment to correct the disproportions into functional and aesthet- ically pleasing forms [3]. The facial measures which are usu- ally recorded manually are an inefficient method as the ana- tomical points can be erroneously located leading to improper measures. The changing facial expressions and movement of the head create difficulties for manual measurements. The process can be tedious for both the student/clinician and the patient. These drawbacks necessitate the development of an effective tool for recording facial landmarks using a new ma- chine learning algorithm. The innovative algorithm can per- form automated facial landmark identification with facial landmark measurements on 2D images of patients. The devel- oped system uses YOLO deep learning algorithm for facial localisation and the landmarks are predicted on the localised * Gururajaprasad Kaggal Lakshmana Rao drgururajaprasad@student.usm.my 1 Craniofacial and Biomaterial Science Cluster, Advanced Medical and Dental Institute, Universiti Sains Malaysia, 13200 Penang, Malaysia 2 Cognitive Computing and Data Science Research Lab, Global Technology Office, Cognizant Technology Solutions, Bengaluru, India 3 Graduate School of Business, Universiti Sains Malaysia, 11800 Penang, Malaysia Health and Technology https://doi.org/10.1007/s12553-019-00313-8