1 Edge-Based Street Object Detection Sushma Nagaraj, Bhushan Muthiyan, Swetha Ravi, Virginia Menezes, Kalki Kapoor, Hyeran Jeon Computer Engineering Department College of Engineering, San Jose State University San Jose, CA, US Abstract—Nowadays everything is becoming smart and intelligent with the help of Internet of Things (IoT) and artificial intelligence (AI). One of the promising applications of the integration of IoT and AI is smart city. The typical design pattern of smart cities is to install cameras and various sensors as many spots as possible and connect them to data center servers that can make smart decisions based on the inputs from the cameras and sensors. In such structure, network bandwidth may hinder the real-time processing because sensory data should be sent to the servers in the remote location. To solve this problem, recent a few studies demonstrated edge computing. In edge computing, IoT devices can handle basic recognition. Thus, only sophisticated inputs that are not handled by IoT devices are sent to remote servers. This paper further demonstrates that the edge computing can provide stand-alone processing power for handling one of the fundamental applications of smart city, which is street object detection. Correct detection of various street objects is the core function of traffic systems and public safety applications of smart cities. A convolutional neural network model is developed by training with traffic data captured at California and Nebraska. Our model detects 14 objects with 25% average accuracy on NVIDIA Jetson TX2. Keywords—deep learning, autonomous cars, object detection, Detect-Net, YOLO. I. INTRODUCTION Machine learning has given the ability to process various tasks without human intervention such as playing chess, diagnosing cancer, recognizing various objects. One of the major branches of machine learning is deep learning. With multiple layers of feature extraction, deep learning has enabled recognizing complicated patterns of target objects thereby solving various problems like human brain. One of the most trending applications that is accelerated by deep learning is traffic objects detection. Traffic objects detection is the core component of smart traffic control of smart cities and auto- pilot system of self-driving cars. These applications not only make the urban life convenient but also safe by reducing car accidents. However, correct traffic object detection is not easy. The first hurdle is diverse shapes of different objects. On street, there are pedestrians, vehicles, bicycles, and traffic signals. Vehicles again have different shapes depending on their types such as sedan, truck, van, and SUV. Traffic signals impose different meaning based on the lighted color such as red, yellow, and green. Thus, traditional machine learning that uses shallow network cannot recognize these objects altogether. With multiple layers of feature extraction, deep learning can be the solution of traffic object detection. In this paper, we demonstrate a deep learning model that recognizes 14 objects that are captured by traffic cameras. Different approaches were being used to achieve this task. Approach one was working with Digits and DetectNet model, we used pre-trained model which was best suited for detecting one class, but we found that the results were biased towards the classes which had more number of images. So, our second approach was to use Yolo model [2] with Darknet framework. Darknet is written in C and Cuda [8] and the state of the art says that it only looks once over the images to train the model. So, we choose Darknet framework for our second approach. Our model has small footprint to be deployed on a mobile GPU platform. Thanks to the huge advance in processor technology, mobile platforms are recently providing over one Tera-flops of performance [7]. These powerful compute engines enable edge computing. In edge computing, mobile devices serve a few core functions that used to be handled only by data center and servers. Edge computing is beneficial for many real-time applications such as traffic object detection because it can timely solution without needing to communicating with remote servers through slow network. We deployed the trained model on NVIDIA Jetson TX2 without any detection accuracy loss. This paper organizes as follows. Chapter II explains related work. Chapter III describes the design of our model. Chapter IV explains various approaches used for solving the problem of Traffic management. Chapter V is dedicated to results and conclusion. Chapter VI gives information about various references used for this task to accomplish. II. RELATED WORK The ideology of deep learning and neural networks has been in existence since 1965 [4]. Since then researchers have been working on how to make it better and improve computing in machines. Deep learning is a technique where the machine is fed with an algorithm, typically called a neural network, and tons of data. Machine is expected to learn certain patterns in the data. It is then tested to identify the pattern in a completely new test data. User can choose between two types of learning: Supervised and Unsupervised. When the data is labelled and provided for deep learning, it is called supervised learning else unsupervised learning. Deep learning has been very successful in solving different image classification, object detection and many other problems. We are trying to solve an object detection in traffic as a problem. Object detection is one of the most challenging problems in computer vision and is the first step in several computer vision applications. Two well-known models for object detection are