Performance metrics for robotic grasping systems Rajkumar Muthusamy and Ville Kyrki Being able to measure grasp performance of complex robotic hands is crucial for building robust manipulation systems. Various grasp planners for determining grasps and grasp quality metrics to evaluate the established grasps exist overwhelmingly in the robotic grasping literature [1]. However, replicating planned contact locations and grasp configuration with real hardware is difficult. In the case of under-actuated hands, achieving planned grasps is even more challenging due to the hand’s intrinsic complexity. In this work, we propose experimental performance metrics for grasps that allow quantitative evaluation of real performance and robustness against disturbances. We also analyse how the proposed metrics relate to existing wrench space grasp quality measures. Measures of performance are an essential component in evaluating capabilities of robot hands. We propose a process and five metrics that allow to analyse the performance of a grasp made by a robot hand. (I) After grasping an object, the object is lifted to a predetermined height. Metric 1 (M1) is the tilt of the object measured using an inclinometer, which describes the overall success of the grasp. Lower tilt corresponds to better performance. (II-III) After the lift, a weight (0.25 kg) is placed in the center (II) and edge (III) of the object and the tilt measurement is repeated. Metric 2 (M2) is the tilt with weight at the center while Metric 3 (M3) is the tilt with the weight at the edge. The metrics quantify the ability to hold the grasped object subject to external disturbances. Again, smaller tilt corresponds to better performance. (IV) A force disturbance of 5N is applied three times in different locations of the object by hooking down the fish scale. While the disturbance is applied, the robot and object motions are observed visually and Metric 4 (M4) is scored as 0 for a grasp or lifting failure, 0.5 for object slip/pose deviation in robot hand and 1 for no slip/success in withstanding disturbances. The metric evaluates the ability of the grasp to reject disturbance forces. (V) After all of the above steps, the object is lowered back on the table and the deviation in the object location is measured in centimetres, resulting in Metric 5 (M5). The metric provides insight on the total effects of disturbances. Physical experiments with a two robot system are used to evaluate the metrics in a real-world setting. We use Jaco and Barrett hands for cooperative handling of a light and a heavy object illustrated in Fig. 1 (a) and (c). To study the performance, three (CZ-1 to 3) collaborative grasps were executed for the two objects and the real time quality *This work was supported by the EU FP7 RECONFIG project 600825, and Academy of Finland grant 286580. R. Muthusamy and V. Kyrki are with School of Electrical Engineering, Aalto University, Finland {rajkumar.muthusamy , ville.kyrki} @aalto.fi (a) Coordinated lift: Light object (b) Table: Top Grasps and Metrics (c) Coordinated lift: Heavy object (d) Chair: Top Grasps and Metrics Fig. 1: Performance study carried out in a Multi-robot setting for effective co-manipulation using underactuated hands. metrics were recorded (Fig. 1(b) and (d)). The cooperative grasps selected for the physical experiment are based on the W L1 wrench space grasp quality measure (ε) [2]. The performance metrics are consistent with ε metric for the lighter object, even in the presence of significant modeling uncertainties. For the heavier object, the correlation between the success of the physical system and the ε metric is weak. The wrench space metric did not predict the success of the physical system well. Proposed metrics give insight on different strengths and weakness of the executed grasp. Analysing the proposed experimental metrics, the inclination metrics (M1-M3) correlate well with the overall success under disturbances (M4) while the alignment metric (M5) does not exhibit such a clear correlation. The reason for the smaller correlation of the alignment metric seems to be that the alignment error is caused by a series of uncertainties stemming from the entire system. Task specific wrench space metrics [3] will be useful for heavy objects to predict better grasps for physical system since forces and torques need to be applied to counteract the gravity. REFERENCES [1] J. Bohg, A. Morales, T. Asfour, and D. Kragic, “Data-driven grasp synthesis—a survey,” IEEE Trans. Robot., vol. 2, pp. 289–309, 2013. [2] R. Muthusamy and V. Kyrki, “Decentralized approaches for cooperative grasp planning,” in Int. Conf. Contr. Autom. Robot. Vis., 2014. [3] R. Muthusamy, C. P. Bechlioulis, K. J. Kyriakopoulos, and V. Kyrki, “Task specific cooperative grasp planning for decentralized multi-robot systems,” in Proc. IEEE Int. Conf. Robot. Autom., pp. 6066–6073, 2015.