Contact Mode Guided Motion Planning for Dexterous Manipulation Xianyi Cheng, Eric Huang, Yifan Hou, and Matthew T. Mason Carnegie Mellon University {xianyic,erich1,yifanh}@andrew.cmu.edu, mattmason@cmu.edu Abstract—Within the field of robotic manipulation, a central goal is to replicate the human ability to manipulate any object in any situation using a sequence of manipulation primitives such as grasping, pushing, inserting, sliding, etc. Conceptually, each manipulation primitive restricts the object and robot to move on a lower-dimensional manifold defined by the primitive’s dynamic equations of motion. Likewise, a manipulation sequence repre- sents a dynamically feasible trajectory that traverses multiple manifolds. To manipulate any object in any situation, robotic systems must include the ability to automatically synthesize ma- nipulation primitives (manifolds) and sequence those primitives into a coherent plan (find a path across the manifolds). This paper investigates a principled approach for solving dexterous manipulation planning. This approach is based on rapidly-exploring random trees which use contact modes to guide tree expansion along primitive manifolds [10]. This paper extends this algorithm from 2D domains to 3D domains. We validated our algorithm on a large collection of simulated 3D manipulation tasks 1 . These tasks required our algorithm to sequence between 6-42 manipulation primitives (i.e. distinct contact modes). We believe this work represents an important step towards robotic manipulation capabilities which generalize across objects and environments. I. I NTRODUCTION Manipulation skills used by humans and animals are numer- ous. For foraging mountain gorillas, Byrne et al. [7] document 72 functionally distinct manipulation primitives. With regard to human manipulation, Nakamura et al. [35] find that existing taxonomies cannot categorize many human grasping behav- iors in a grocery store. Researchers have explored a variety of manipulation skills in depth, including grasping, pushing [29, 26], tumbling [27], pivoting [2, 15, 16], whole-body manipulation [38], extrinsic dexterity [9], prehensile pushing [8], regrapsing [42], shared grasping [17], etc. However, most of these work still requires significant human engineering to design manipulation motions or modeling assumptions to simplify the tasks for the robots, such as pre-defining a sequence of contact modes. Automatically generating such dexterous manipulation skills and motions in various scenarios becomes desirable but still challenging. The dexterity in manipulation comes from two resources: the dexterity of robot hands (intrinsic dexterity) and the properties of the environments (extrinsic dexterity). A good general dexterous manipulation motion planner needs to generate strategies that use both intrinsic and extrinsic dexterity cleverly. Moreover, manipulation is contact-rich by 1 The video is available at https://youtu.be/o63dLjIsZmw. 1: Push Slide up Grasp Push Object trajectory Fingertips 2: Slide up 3: Grasp Figure 1: One example of the problems solved in this paper. The goal is to get the object out of the box while the gaps are too small for both fingers to get in. Our method plans first to push the object on the top to create a wider gap that lets the fingers slide up the object from the side. nature. Making and breaking contacts bring more complexity into the system by changing its kinematics and dynamics. This hybrid nature makes planning through contacts difficult. This paper aims to move one step closer to general dexterous manipulation planning by considering the dexterity from plan- ning through both the robot hand contacts and environment contacts. The key idea of this paper is to use contact modes as the guidance for dexterous manipulation motion generation. The planner we propose is an RRT [22] based planner with instantly enumerated contact modes as guidance to extend the tree. We can think of enumerated contact modes as au- tomatically generated “motion primitives” as they both define submanifolds of configuration space. Compared with manually designed motion primitives, contact modes can help generate more varied motions, because they can easily capture the discrete changes of the system introduced by contacts, as the set of all feasible contact modes enumerates all possible transitions of the states of contacts in the system. Combining contact mode guidance with sampling based planning, our method can perform fast exploration over large continuous and discrete spaces to plan contact-rich motions. One example for peg-out-of-hole/unpacking planning is shown in Figure 1. The planner generates the strategy of first pushing the object on the top to create a wider gap that lets the fingers slide up the object from the side, and then form a grasp. arXiv:2105.14431v1 [cs.RO] 30 May 2021