630 IEEE TRANSACTIONS ON ROBOTICS, VOL. 27, NO. 3, JUNE 2011 Short Papers A Probabilistic Approach to Tactile Shape Reconstruction Martin Meier, Matthias Sch ¨ opfer, Robert Haschke, and Helge Ritter Abstract—In this paper, we present a probabilistic spatial approach to build compact 3-D representations of unknown objects probed by tactile sensors. Our approach exploits the high frame rates provided by modern tactile sensors and utilizes Kalman filters to build a prob- abilistic model of the contact point cloud that is efficiently stored in a kd-tree. The quality of generated shape representations is compared with a naive averaging approach, and we show that our method provides superior accuracy. We also evaluate the feasibility of object classification combin- ing the generated object representations, together with the iterative closest point algorithm. Index Terms—Grasping, recognition, tactile sensing, 3-D reconstruction. I. INTRODUCTION Tactile data provides valuable object shape information and greatly contributes to enable sophisticated robot–environment interaction tasks such as allowing autonomous robots to perform stable grasps. If a priori unknown objects are to be handled, an appropriate 3-D model needs to be acquired as input to available planning algorithms [1], [2] or for object identification. While modern vision-based depth sensors can provide a good shape representation of the frontal surface of an object, they cannot reconstruct its whole shape. Photometric effects like reflections or transparency further reduce the feasibility of this approach. With the advent of multifingered hands equipped with tactile sensors of good temporal resolution, another modality to reconstruct 3-D shapes comes into view, which can be used to augment visual data. Reconstruction of the object shape from contact points acquired from palpating sequences gives precise and robust results but is time consuming. In contrast to vision-based methods, which instantaneously acquire data, tactile sensing is sequential and local. That is, during a single hand-object contact, e.g., due to a grasp, only a very small amount of the object’s surface is in contact with tactile sensors cells. (We will call a single sensor cell “tactel,” i.e., tactile element, hereafter.) This requires the collection and sequential alignment of acquired contact points in order to build a consistent tactile point cloud. Two other properties of tactile sensors, namely their spatial and temporal resolution, have to be considered as well. The coarse spatial resolution of available sensors—typically in the range of 3–5 mm— make the detection of fine contours a difficult task. However, for object manipulation, a holistic shape representation, which includes all faces, Manuscript received December 15, 2010; revised February 11, 2011; ac- cepted February 21, 2011. Date of publication April 5, 2011; date of current version June 9, 2011. This paper was recommended for publication by Guest Editor G. Cannata and Editor W. K. Chung upon evaluation of the reviewers’ comments. The authors are with the Bielefeld University, 33615 Bielefeld, Germany (e-mail: mmeier@techfak.uni-bielefeld.de; mschoepf@techfak.uni-bielefeld. de; rhaschke@techfak.uni-bielefeld.de; helge@techfak.uni-bielefeld.de). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TRO.2011.2120830 is more important than high precision as targeted in 3-D scanning. High frame rates offered by recently developed tactile sensor arrays [3] can be utilized to compensate for these shortcomings, e.g., by sliding over the object’s surface. However, methods need to cope with the overwhelming amount of data. Existing methods to reconstruct the object shape are intended as offline methods and collect all available, typically redundant contact information, which is subsequently fitted to an existing object model, e.g., using superquadrics [4]. To speed up the exploration process, some attention driven active exploration approaches that determine tactual regions of interest have been proposed [5], [6]. More general techniques create probabilistic haptic maps of the object to recognize previously examined regions of interest [7]. An often-studied application is the identification and localization of objects based on haptic information. Some of these approaches incor- porate additional information, such as joint angle data [8] or vision data [9]. These methods do not need to explicitly build a shape recon- struction, but often directly derive the classification result from data sequences [10]. Again, methods exist to reduce the amount of required palpation actions, e.g., extracting sets of higher level features [11] or estimating surface curvature from 1-D palpation curves only [12]. This paper aims for the reconstruction of an object shape with a dense and compact tactile point cloud, which can be subsequently utilized for applications like grasp planning and monitoring or real- time object classification. Although we rely on tactile data only, the proposed method can also be applied to sequentially acquired visual data, e.g., range scans. We propose to use a combination of three well-known methods to achieve this goal: 1) An incrementally growing space partitioning tree [13] is used to efficiently build a point cloud of surface points. 2) Each point is represented by a Kalman filter [14] that realizes an efficient spatiotemporal integration of tactile sensor data and accounts for inherent measurement uncertainty. 3) The iterative closest point (ICP) algorithm [15] is used to align two acquired point clouds. The remaining paper is organized as follows. In the next section, we introduce the employed methods and their adaption to the task of object shape reconstruction. Section III describes the autonomous data acqui- sition process employing a three-fingered robot hand. In Section IV, we compare the quality of the shape representations built 1) with the proposed Kalman filter approach; 2) a simple averaging method; and 3) utilizing the raw data directly. A technique for object classification is presented in Section V as an application of the generated shape representations. Finally, Section VI concludes the paper. II. METHODOLOGY Collecting all tactile sensor data in order to represent the shape of an explored object results in a tactile point cloud, which is rapidly growing in size due to the high available frame rates of nowadays sensors. To yield a more compact representation and to reduce the redundancy of data points, we propose to employ a nearest neighbor matching to assign a newly acquired contact point m n to an already existing point μ k in the reduced point cloud set X = {μ k }, which is subsequently adapted to account for the new data point. For this update process, we compare a simple approach, which com- putes the sliding average of the 3-D surface point with a Kalman filter approach, which explicitly models the measuring uncertainty. 1552-3098/$26.00 © 2011 IEEE