http://www.iaeme.com/IJMET/index.asp 465 editor@iaeme.com International Journal of Mechanical Engineering and Technology (IJMET) Volume 10, Issue 05, May 2019, pp. 465-475, Article ID: IJMET_10_05_047 Available online at http://www.iaeme.com/ijmet/issues.asp?JType=IJMET&VType=10&IType=5 ISSN Print: 0976-6340 and ISSN Online: 0976-6359 © IAEME Publication MULTIPLE HUMAN TRACKING USING RETINANET FEATURES, SIAMESE NEURAL NETWORK, AND HUNGARIAN ALGORITHM Dina Chahyati, Aniati Murni Arymurthy Machine Learning and Computer Vision Laboratory Faculty of Computer Science, Universitas Indonesia, Depok, Indonesia ABSTRACT Multiple human tracking based on object detection has been a challenge due to its complexity. Errors in object detection would be propagated to tracking errors. In this paper, we propose a tracking method that minimizes the error produced by object detector. We use RetinaNet as object detector and Hungarian algorithm for tracking. The cost matrix for Hungarian algorithm is calculated using the RetinaNet features, bounding box center distances, and intersection of unions of bounding boxes. We interpolate the missing detections in the last step. The proposed method yield 43.2 MOTA for MOT16 benchmark. Key words: RetinaNet, tracking by detection, Hungarian algorithm, Siamese neural network, interpolation Cite this Article: Dina Chahyati, Aniati Murni Arymurthy, Multiple Human Tracking Using Retinanet Features, Siamese Neural Network, and Hungarian Algorithm, International Journal of Mechanical Engineering and Technology 10(5), 2019, pp. 465-475. http://www.iaeme.com/IJMET/issues.asp?JType=IJMET&VType=10&IType=5 1. INTRODUCTION Multiple object tracking has been a challenge for researchers over a decade. Vehicle and human tracking have been dominating this field since they are very important for surveillance system. Human tracking has its own challenges because of a wide variety of human appearance and severe occlusions in most of the scenes. In the beginning, researchers focused on tracking by trying to predict the path. Methods such as Kalman filter and optical flow [1][2][3] were commonly used in this approach. Nowadays many researchers change the approach to tracking by detection [4]–[6]. Detections by using HOG or its variation such as DPM are quite popular and has been used in multiple object tracking (MOT) benchmark [7]. However, DPM has limitation for recognizing more complex objects and its properties such as variation of clothes, bags, activities, gender, etc. Our future research topic is to track movements of people with a specific gender, therefore we cannot rely on DPM for the detection step. A more reasonable option would be to use deep learning approach for the detection step because they can be trained to detect more specific categories by using transfer learning.