Automatic Visual Reading of Meters Using Deep Learning Karlo Koˇ sˇ cevi´ c Faculty of Electrical Engineering and Computing University of Zagreb Zagreb, Croatia Email: karlo.koscevic@gmail.com Marko Subaˇ si´ c Faculty of Electrical Engineering and Computing University of Zagreb Zagreb, Croatia Email: marko.subasic@fer.hr Abstract—In this paper, we present a novel approach to the problem of reading residential meters using deep learning algorithms. As a starting point we use Faster R-CNN method and, to acquire more precise readings, we modify its functionality. As there were no databases for this kind of task, one had to be collected and properly annotated. This paper also provides a brief introduction to methods for image augmentation and a technique to augment annotated image dataset. For each part of the presented method as well as the whole method as one unit experiments were conducted to show the overall successfulness. I. I NTRODUCTION Electricity, gas and water meters are common devices in all residential areas. Every day, many experts inspect these devices and many owners have a need to track their consump- tion either for personal statistics or because of their obligations to service providers. There are already some remote reading solutions for the problem of residential meter reading, but they are often very expensive and require some additional hardware on top of the meter itself or a new meter. Therefore, our goal was to develop a solution that is easy to use, accessible to everyone and inexpensive. Smartphones, deep neural networks and new frameworks that can connect these two concepts together are the main motivation for our solution. A piece of software that is accessible to all smartphone users drastically reduces costs of meter reading as it does not require custom hardware and simplifies interactions with systems for gas, water or electricity consumption. Our goal was to develop a deep learning method that would fulfill such requirements. The starting point of our solution is a deep neural network for object detection Faster R-CNN [10]. To achieve better readings we modified that network. As there were no databases for this sort of problem we manually annotated our own database and, additionally, implemented augmentation methods to bulk up our database. II. RELATED WORK The technology of telemetry has successfully been ap- plied for the problem of reading of residential meters. Some PLC [1], RF [2], GSM [3], and etc. technologies are still widely used for automatic meter reading. In [4] a method for residential meter reading that uses low- level image features is proposed. The process of determining meter type is done by searching nearest neighbors in a k-d tree which is made of features extracted from all sample images of devices and image of interest. To extract features SURF [5] method is used. Once the type of meter is known input image is transformed to match the perspective of the corresponding sample. For each region of interest, digits are extracted using some image processing methods and recognized using the HOG [6] descriptor. For more in-depth insight into this method please refer to [4]. Examples of existing smartphone solutions include [7], [8], [9] and etc. III. OBJECT DETECTION USING DEEP NETWORKS Faster R-CNN [10] is an object detection method split up into two parts: Region Proposal Network (RPN) and Fast R- CNN detector [11]. Faster R-CNN is a deep neural network with attention [12] mechanism as RPN module tells Fast R- CNN detector where to look for objects [13]. Figure 1 shows Faster R-CNN architecture. To make this paper more self- contained we briefly describe this method in following para- graphs but for a more in-depth description of Faster R-CNN method please take a look at [10]. The goal of this method was to share RPN and Fast R-CNN detector computations and because of that, both networks share a common set of convolutional layers [10]. Fig. 1: Faster R-CNN architecture. Region proposal network (RPN) [10] takes an image as input and outputs objects proposals. Region proposals are generated using the small convolutional neural network [14] on top of the last shared convolutional layer. This network works Proceedings of the Croatian Computer Vision Workshop, Year 6 October 24, 2018, Zagreb, Croatia https://doi.org/10.20532/ccvw.2018.0002 CCVW 2018, Oral Session 1