Interactive Learning of Visually Symmetric Objects Wai Ho Li and Lindsay Kleeman Intelligent Robotics Research Centre Department of Electrical and Computer Systems Engineering Monash University, Clayton, Victoria 3800, Australia waiholi@gmail.com, Lindsay.Kleeman@eng.monash.edu.au Abstract— This paper describes a robotic system that learns visual models of symmetric objects autonomously. Our robot learns by physically interacting with an object using its end effector. This departs from eye-in-hand systems that move the camera while keeping the scene static. Our robot leverages a simple nudge action to obtain the motion segmentation of an object in stereo. The robot uses the segmentation results to pick up the object. The robot collects training images by rotating the grasped object in front of a camera. Robotic experiments show that this interactive object learning approach can deal with top- heavy and fragile objects. Trials confirm that the robot-learned object models allow robust object recognition. I. I NTRODUCTION Autonomous object learning is an inherently interesting concept as humans use it regularly to adapt to new en- vironments. A robot with the ability to learn new objects on its own can adapt to different operating environments while shifting the burden of training data collection and model construction away from human users. By doing so, the robot may now be able to operate in environments such as the household where the large number of unique objects make exhaustive modelling and training intractable. Given the increasing ratio of workers versus retirees in developed nations [1] and positive public opinion towards domestic robots [2], the case for autonomous object learning has never been stronger. There are many bilaterally symmetric objects in the house- hold, including container objects such a cups and bottles. As such, the ability to autonomously learn symmetric objects is a useful addition to any domestic robot performing tasks such as cleaning or setting the table. In a previous paper [3], the authors demonstrated a robotic system that segments objects autonomously. Segmentation is performed by observing the object motion induced using a controlled pushing action called the robotic nudge. The robotic nudge removed the need for object models, allowing the robot to segment new objects autonomously, including near-symmetric objects such as a mug with a handle. We suggested that it maybe possi- ble to use robot-obtained segmentations to perform further interactions and object learning. This paper confirms these suggestions by demonstrating a robotic system that learns visual models of new symmetric objects via robotic interaction. The learning process is au- tonomous and model-free, which frees our robot from having to rely on training data and prior object models. Experiments on beverage bottles show that models learned by the robot allow reliable and robust object recognition. II. CONTRIBUTIONS Contributions are made in the areas of interactive object learning and object recognition. The autonomous nature of the entire robotic system, from object segmentation to grasping to modelling, also contributes to current research. A. From Simple to Advanced Interactions Fitzpatrick suspected that it maybe possible to leverage simple object interactions to perform advanced interactions such as object grasping [4]. This paper confirms Fitzpatrick’s suspicion experimentally. Our robotic system investigates objects by moving them a very short distance across the table using a robotic nudge. The information gained from this simple interaction is then used by the robot to pick up the object. The robot’s ability to move autonomously from a nudge to a grasp is novel and useful in situations where the robot has to deal with new objects. B. Object Learning using Robot-collected Training Images In our previous paper [3], we suggested that object seg- mentations obtained autonomously by our robot can be used as training data for an object recognition system. While these segmentations are accurate, nudging an object on a table only provides a single view of the moved object. The robot presented here grasps the nudged object and rotates it to collect training images over the entire 360 degrees of the grasped object. Object models are constructed using these robot-collected training images. The proposed approach differs from the traditional ap- proach of offline image collection and feature detection using a turntable-camera rig as surveyed in [5]. Our approach also differs from semi-autonomous systems, such as [6], that require a human user to provide the robot with different views of test objects. Instead, our robot autonomously learns new objects by modelling them online. Object recognition experiments suggests that the robot is able to learn useful visual models of new objects. C. Robust Object Recognition by Pruning SIFT Descriptors The robot’s gripper has two wide foam fingers, which can be seen in the photos of Figure 1. The foam-padded gripper ensures a stable grasp but does not allow an accurate pose