Distinguishing Sliding from Slipping during Object Pushing Martin Meier and Guillaume Walck and Robert Haschke and Helge J. Ritter {mmeier,gwalck,rhaschke,helge}@techfak.uni-bielefeld.de Neuroinformatics Group, Center of Excellence Cognitive Interaction Technology (CITEC), Bielefeld University, Germany Abstract— The advent of advanced tactile sensing technology triggered the development of methods to employ them for grasp evaluation, online slip detection, and tactile servoing. In contrast to recent approaches to slip detection, distinguishing slip from non-slip conditions, we consider the more difficult task of distinguishing different types of slippage. Particularly we consider an object pushing task, where forces can only be applied from the top. In that case, the robot needs to notice when the object successfully moves vs. when the object gets stuck while the finger slips over its surface. As an example, consider the task of pushing around a piece of paper. We propose and evaluate three different convolutional net- work architectures and proof the applicability of the method for online classification in a robot pushing task. I. I NTRODUCTION In robotics, object manipulation is a key capability – especially in service robotics when interaction with com- plex environments is necessary. Applying a motion to an object doesn’t need to involve grasping. In this case the term non-prehensile manipulation or more concretely push- manipulation is used. Push-manipulation has been well ana- lyzed to model the mechanics of such motion [1], [2]. In most of the work it is supposed that the friction properties between the pusher and the slider are known or at least controlled. This allows for feed-forward execution of planned motion trajectories. For example, [3] studied pre-grasp manipulation involving object pushing. Particularly, they considered get- ting in contact with the object from top and using friction to move the object. However, the contact point was assumed to have enough friction for proper power transmission. As a consequence, there is little work considering reactive controllers monitoring and adjusting the contact state during motion. In [4], a visual tracking system was employed to maintain a suitable angle of contact in order to ensure proper friction. However, online slippage monitoring and active force control call for tactile sensors within the feedback loop. The increasing availability of tactile sensors triggered a lot of research to employ them for grasp evaluation, see [5] for a review of current technologies. Some methods were also proposed for slip detection – typically for object grasping. Corresponding methods can be distinguished into two major approaches: 6D contact wrench estimation and detection of micro-vibrations at the contact. This work was supported by the DFG Center of Excellence EXC 277: Cognitive Interaction Technology (CITEC) and has received funding from the EU projects WEARHAP (grant 601165) and SaraFun (grant 644938). Bicchi [6] has shown, that it is possible to determine the local contact position and the local contact wrench given a wrench measurement at the fingertip base – provided that the fingertip shape is a known quadratic surface, e.g. spherical, ellipsoid, or planar. Knowing the friction coefficient, one can then estimate slippage from the ratio of observed tangential and normal forces at the contact point [7]. In [4], the friction coefficient to be considered during pushing is estimated in advance by sliding over the fixated object and estimating the contact force from measured torques at the hand joints. In contrast to these approaches that suppose an exact friction model to be available, vibration-based approaches attempt to predict incipient slippage from high-frequency oscillations (≈ 200-400 Hz) in the tactile sensor signals, much like Pacinian corpuscles do in human skin [8], [9]. To this end, machine learning techniques are employed to directly predict slippage from data (typically only the normal force or the overall pressure value is available). In [10], [11], slippage-induced micro-vibrations are mea- sured with tactile sensors that allow for high-speed acquisi- tion rates and thus enabled the detection of slippage before visual sensors or inertial measurement units (IMU) can notice. Several machine learning approaches were studied to detect slippage, (i) including SVM and random forests [12] achieving an F score of 0.75, and (ii) multilayer perceptrons (MLP) achieving an accuracy of 80% [13]. The authors of the latter work also pointed out, that it is particularly important to distinguish between translational and rotational slippage, because counteracting rotational slip typically requires higher forces to be exerted onto the object for stabilization. How- ever, to the best of our knowledge, this task wasn’t solved yet. Having successfully applied convolutional neural networks to this task of distinguishing translational, rotational and non- slip conditions recently [14], in the present paper we transfer that work from the previously used 16 × 16 flat sensor matrix to a 12-taxel, 3D-shaped fingertip. As a consequence of the curved fingertip shape, the contact area is typically much smaller, only activating 4-6 taxels at a time, rendering the classification task even harder. Furthermore, we consider the important task of distinguishing object sliding from object slippage in non-prehensile manipulation. When pushing thin objects, e.g. a piece of paper, one can only apply forces from the top, not from the side. In some cases, the object will then not move on the ground, but the finger(s) will slip on the object. For successful execution of such tasks, it is