IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, 2015 1 Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery Claudio Pacchierotti, Domenico Prattichizzo, Senior Member, IEEE, and Katherine J. Kuchenbecker, Member, IEEE Abstract—Despite its expected clinical benefits, current tele- operated surgical robots do not provide the surgeon with haptic feedback largely because grounded forces can destabilize the sys- tem’s closed-loop controller. This article presents an alternative approach that enables the surgeon to feel fingertip contact de- formations and vibrations while guaranteeing the teleoperator’s stability. We implemented our cutaneous feedback solution on an Intuitive Surgical da Vinci Standard robot by mounting a SynTouch BioTac tactile sensor to the distal end of a surgical instrument and a custom cutaneous display to the corresponding master controller. As the user probes the remote environment, the contact deformations, DC pressure, and AC pressure (vibrations) sensed by the BioTac are directly mapped to input commands for the cutaneous device’s motors using a model-free algorithm based on look-up tables. The cutaneous display continually moves, tilts, and vibrates a flat plate at the operator’s fingertip to optimally reproduce the tactile sensations experienced by the BioTac. We tested the proposed approach by having eighteen subjects use the augmented da Vinci robot to palpate a heart model with no haptic feedback, only deformation feedback, and deformation plus vibration feedback. Fingertip deformation feedback significantly improved palpation performance by reducing the task completion time, the pressure exerted on the heart model, and the subject’s absolute error in detecting the orientation of the embedded plastic stick. Vibration feedback significantly improved palpation performance only for the seven subjects who dragged the BioTac across the model, rather than pressing straight into it. I. I NTRODUCTION T ELEOROBOTIC surgical systems involve a slave robot, which interacts with the patient, and a master console, operated by the human surgeon. The slave robot reproduces the hand movements of the surgeon, who in turn needs to observe the operative environment with which the robot is interacting. C. Pacchierotti is with the Dept. of Information Engineering and Math- ematics, University of Siena, Siena, Italy, with the Dept. of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy, and with the Depts. of Mechanical Engineering & Applied Mechanics and Computer & Information Science, GRASP Laboratory, University of Pennsylvania, Philadelphia, PA, USA. E-mail: pacchierotti@dii.unisi.it. D. Prattichizzo is with the Dept. of Information Engineering and Mathemat- ics, University of Siena, Siena, Italy and with the Dept. of Advanced Robotics, Istituto Italiano di Tecnologia, Genova, Italy. E-mail: prattichizzo@dii.unisi.it. K. J. Kuchenbecker is with the Depts. of Mechanical Engineering & Ap- plied Mechanics and Computer & Information Science, GRASP Laboratory, University of Pennsylvania, Philadelphia, PA, USA. E-mail: kuchenbe@seas.upenn.edu. The research leading to these results has received funding from the European Union Seventh Framework Programme FP7/2007-2013 under grant agreement n ◦ 601165 of the project “WEARHAP - WEARable HAPtics for humans and robots.” Copyright (c) 2014 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending an email to pubs-permissions@ieee.org If the surgeon receives sufficient information about the slave system and the operative environment, he or she will feel present at the operative site, a condition commonly referred to as telepresence [1], [2]. The strength of the telepresence illusion depends on the type and quality of information that flows from the operating table to the surgeon. Visual feedback is already available in commercial robotic surgery systems (e.g., the Intuitive Surgical da Vinci Si), but current surgical robots have very limited haptic feedback. This omission is mainly due to the negative effect that haptic force feedback has on the stability of the teleoperation loop; outputting grounded forces with the master console can lead to undesired oscillations of the system, which interfere with the surgery and may be dangerous for the patient [3], [4]. However, haptic feedback is still widely believed to be valuable for teleoperated surgical procedures [5], [6], [7], [8], [9]. It has been shown to enhance surgeon performance in a wide range of applications including microneedle positioning [10], telerobotic catheter insertion [11], suturing simulation [12], cardiothoracic pro- cedures [13], and cell injection [14]. Its benefits typically include increased manipulation accuracy, increased perception accuracy, decreased completion time, and decreased peak and mean force applied to the remote environment [15], [16], [17], [18], [4], [19], [20]. Given the expected benefits of haptic feedback and the challenges of stable implementation, many researchers have turned to sensory substitution techniques, wherein force in- formation is presented via an alternative feedback channel, such as vibrotactile [21], auditory [22], or visual cues [23]. Because no haptic forces are displayed to the operating surgeon, sensory substitution techniques make teleoperation systems intrinsically stable [4], [18], [20]. However, although the stability of the system is guaranteed, the provided stimuli differ substantially from the ones being substituted (e.g., a beep sound instead of force feedback). Therefore, sensory substitution often shows performance inferior to that achieved with unaltered force feedback [4], [20]. Cutaneous feedback has recently received great attention from researchers looking for an alternative to sensory substi- tution of force feedback. Cutaneous stimuli are detected by mechanoreceptors in the skin, enabling humans to recognize the local properties of objects such as shape, edges, and texture. Cutaneous perception for exploration and manipula- tion principally relies on measures of the location, intensity, direction, and timing of contact forces on the fingertips [24], [25]. Delivering this type of haptic cues to the surgeon’s skin has been proved to convey rich information and does not affect