Next Generation Image Guided Citrus Fruit Picker Christopher Aloisio, Ranjan Kumar Mishra, Chu-Yin Chang and James English Energid Technologies Corporation, One Mifflin Place, Cambridge, MA 02138, USA ranjan@energid.com Abstract— Supply and availability of labor for fruit harvesting is a problem worldwide, and attempts to mechanize the fruit harvesting process have been carried out for decades. During the last century, fruit harvesters have tried mechanical methods such as trunk shaking, canopy shaking, raking, and mass mechanical penetration. Though they are economical alternative options for harvesting nuts, olives, cherries, and prunes, these brute-force mechanical methods have had limited results for citrus fruit. Robotics has also been tried. Technical challenges in robotics include recognizing and locating the fruit and detaching it according to prescribed criteria without damaging either the fruit or the tree. The key practical problem, though, is economic. A robotic system needs to be economically sound to warrant its use as an alternative to hand picking. Cambridge, Massachusetts, based Energid Technologies Corporation is developing a practical robotic fruit picking system whose concept is driven by the economics of mass harvesting while leveraging machine vision and robotic guidance and control. The system uses flexible tubes with removal tools at one end that can be individually fired pneumatically and steered robotically, with sensor input coming from a grid of machine vision cameras. Keywords: robotic fruit picking, citrus harvesting, machine vision BACKGROUND A typical citrus grove has about 130 trees to the acre, and each tree on average produces roughly 500 individual fruits, though this number varies considerably (some trees produce over 2,000). Figure 1 below shows oranges on a tree—they are solidly attached, hanging both singly and in clusters, both on the periphery and internal to the tree. Figure 1: Citrus fruit hangs singly and in bunches near the surface and inside the tree, and may be obscured or hindered by limbs and leaves. Oranges may have inconsistent coloring. Florida Hamlin juice oranges are shown. OUR APPROACH Energid Technologies has developed an approach that combines the economy of mass removal with the intelligence of robotics. It uses a large number of low-cost mechanisms controlled through visual sensors and high-throughput computer processing. The fast-moving parts of the system are inexpensive and disposable, reducing maintenance costs. The system will harvest an estimated 32 pieces of fruit per second at peak performance. Energid’s approach applies 16 inexpensive, disposable mechanisms organized into a pneumatically actuated grid. Multiple grids are used simultaneously, allowing 64 or more picking mechanisms to be applied to a tree by one operator. The picking mechanisms have multiple picking modalities, including sharp end effectors that strike the stems of the fruit and remove the fruit from the tree. Each grid of picking mechanisms integrates an array of color vision cameras that are used for visual guidance. The top- level concept is illustrated in Figure 2 below, where two picking mechanisms have been mounted to a boom arm on a truck as a test system. The concept is inspired by the way a frog tracks prey with its eyes and captures it by extending the tongue. Each “Frog Tongue” mechanism in Energid’s harvesting system has an inexpensive flexible tube that rapidly extends/retracts from its base. Multiple cameras provide input to tracking algorithms that direct the flexible tubes and guide them while extending. This way the expensive cameras are isolated and protected, while only the inexpensive flexible tubes are exposed. The flexible tube is powered by commonly available compressed air while a two degree-of-freedom planar mechanism at the base of the tube aims and guides the tube. SYSTEM DESCRIPTION The overall system as show in Figure 2 will be mounted on a goat truck with a modified boom arm and will house a central computing cluster with a high-throughput data capture interface, air compressor with accumulator tank, and a collection of multiple Frog Tongue modules. The Frog Tongue module has undergone design iteration primarily driven by cost and simplicity. Our latest design (Figure 3) is described in detail in this article. 978-1-4673-0856-4/12/$31.00 ©2012 IEEE 37