Development of a Vision Aided Automated Ball Retrieving Robot for Tennis Training Sessions Dulanjana M. Perera * , G.M.D. Menaka † , W.V.K.M. Surasinghe ‡ , D.G. Kanishka Madusanka § Thilina Dulantha Lalitharathne ¶ Robotics and Control Systems Laboratory, Department of Mechanical Engineering University of Moratuwa Moratuwa, Sri Lanka. 10400 * dulmanope@gmail.com, † menaka.gmd@gmail.com, ‡ kolimigara@gmail.com, § kanishkam@uom.lk, ¶ thilinad@uom.lk Abstract—A single training session of tennis requires 30-40 balls and these scattered balls have to be collected at the end of each session which costs time and induces unnecessary physical stress on players. This paper proposes a method to retrieve the balls which are scattered in the Tennis court back to the user. The proposed method comprises image processing techniques for object localization and the adaptation of Dijkstra’s algorithm for optimum path planning for efficient navigation. Both localization and the path planning algorithms are developed on a Python platform and robot’s differential drive system is controlled by a microcontroller. Communication between robot and the computer is established through Wi-Fi. Novel development of this research is the implementation of the global view method for a vast range of image acquisition and verifying its suitability in Tennis field. In order to collect balls efficiently, an adaption of an existing mechanism is proposed. Results show that the mechanism is 80% efficient in collecting balls and 100% efficient in object identification for a given environmental condition. Index Terms—Tennis, mobile robot, Image Processing, Self- navigation I. I NTRODUCTION Tennis is one of the most popular racket games in the world among every age category as a professional sport or cardio sport. In general, a tennis training session will take place around 3 hours. During that period of time, several practice sessions are carried out. In a single session, 30 to 40 balls are used to practise different ball serving or shooting techniques. However, at the end of each training session, all the scattered balls have to be collected for the next session. This task costs more time and energy of the player. Many research studies have been done to address this problem and various kind of solutions have been proposed. In 1968 Stap Jacob received a patent for developing a device to address this issue regarding collecting Tennis balls [1]. His design can squeeze the balls into a container between steel rods at the bottom of a receptacle. Even though it is an acceptable solution for the problem, the user feels discomfort while carrying the cage and has to stoop every time to retrieve the balls. After 6 years, G. Seewagen and J. Markisz have proposed a method to overcome the drawbacks of Stap’s development, that is implementing retractable legs which facilitate for the user to hold the container at a convenient height [2]. In 1982 by Delbert Lee developed a ball retriever which is similar to a baby carrying cart. When user moves the device towards a ball, they get stuck between the disks and transport the balls into the container [3]. It was very popular at This work is supported by Vice-chancellor fund, University of Moratuwa, Sri Lanka. We thank all the staff members at the Robotics and Control Systems Laboratory, the University of Moratuwa who provided that great support during the research. that time due to easy usage and the cheap cost however, it has to be operated manually. In order to overcome the stooping to retrieve balls, a device with an adjustable ball container was proposed by H. Chen et al. in 1992 [4]. A modernized but still manually operated retrieval device was proposed by Robert Valdes-Rodriguez in 1998 as a solution to stooping. This device scoops up the balls into the container at a higher position through a hopper [5]. In 2012, Charles J. Mailman got the patent for an electrical ball retrieval device designed for tennis [6]. His motivation was to implement a vacuum suction system in a conventional receptacle, so that the balls can be conveyed quicker than previous methods. However, the device has to be navigated manually and require high power to transport heavy objects such as Tennis balls (60g). Moreover, it is suitable for clay grounds only. A similar implementation can be seen in Nan- thawam Am-Eam’s research in 2010 which is a combination of solar cell energy and hydraulic power [7]. Balls were conveyed to a container by a Direct Current (DC) motor. Height of the container can be adjusted by a hydraulic motor. As previous developments, it required considerably high power and produce noise during the operation. After several years from N. Am-Eam’s research, H. K. Hen and J. M. Dai proposed an intelligent tennis ball collecting robot in 2016 [8]. However, the robot is navigated on a pre- defined path automatically while avoiding obstacles to some extent. Moreover, the user can manually navigate the robot through an Android application. Ball collecting mechanism is similar to the Delbert Lee’s mechanism. However, the ball capacity and the collecting efficiency is very low. A year before Hen and Dai’s research, Jeffery Law and team presented a research about a ball collecting robot which can navigate automatically on a predefined path as Hen’s research [9]. However, the collecting mechanism is quite primitive. As per the results, the capacity and the rate of the mechanism is low. Significant improvements in adapting technologies for this particular field can be observed in a research done by J. Zhao and his team [10]. Development of ball collecting mechanism was inspired by conventional spin blade drum mechanism and also the computer vision technology was used to distinguish the balls from other objects in similar size and color. More importantly, fuzzy logic facilitates the robot to self-navigate towards the balls with the help of 6 sonar sensors while allowing cloud-based manual navigation. This development is also comprised of similar drawbacks, as mentioned in previous developments. 978-1-7281-3706-3/19/$31.00 ©2019 IEEE