Journal of Signal and Information Processing, 2019, 10, 167-199 https://www.scirp.org/journal/jsip ISSN Online: 2159-4481 ISSN Print: 2159-4465 DOI: 10.4236/jsip.2019.104010 Nov. 29, 2019 167 Journal of Signal and Information Processing Deep Learning Based Target Tracking and Classification for Infrared Videos Using Compressive Measurements Chiman Kwan 1* , Bryan Chou 1 , Jonathan Yang 2 , Trac Tran 3 1 Applied Research LLC, Rockville, MD, USA 2 Google, Inc., Mountain View, CA, USA 3 Department of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD, USA Abstract Although compressive measurements save data storage and bandwidth usage, they are difficult to be used directly for target tracking and classification without pixel reconstruction. This is because the Gaussian random matrix destroys the target location information in the original video frames. This paper summarizes our research effort on target tracking and classification di- rectly in the compressive measurement domain. We focus on one particular type of compressive measurement using pixel subsampling. That is, original pixels in video frames are randomly subsampled. Even in such a special com- pressive sensing setting, conventional trackers do not work in a satisfactory manner. We propose a deep learning approach that integrates YOLO (You Only Look Once) and ResNet (residual network) for multiple target tracking and classification. YOLO is used for multiple target tracking and ResNet is for target classification. Extensive experiments using short wave infrared (SWIR), mid-wave infrared (MWIR), and long-wave infrared (LWIR) videos demon- strated the efficacy of the proposed approach even though the training data are very scarce. Keywords Target Tracking, Classification, Compressive Sensing, SWIR, MWIR, LWIR, YOLO, ResNet, Infrared Videos 1. Introduction There are many applications such as traffic monitoring, surveillance, and secu- rity monitoring that use optical and infrared videos [1]-[6]. Object features in How to cite this paper: Kwan, C., Chou, B., Yang, J. and Tran, T. (2019) Deep Learning Based Target Tracking and Clas- sification for Infrared Videos Using Com- pressive Measurements. Journal of Signal and Information Processing, 10, 167-199. https://doi.org/10.4236/jsip.2019.104010 Received: October 10, 2019 Accepted: November 26, 2019 Published: November 29, 2019 Copyright © 2019 by author(s) and Scientific Research Publishing Inc. This work is licensed under the Creative Commons Attribution International License (CC BY 4.0). http://creativecommons.org/licenses/by/4.0/ Open Access