SMT-Based Synthesis of Integrated Task and Motion Plans for Mobile Manipulation Srinivas Nedunuri *2 , Sailesh Prabhu *2 , Mark Moll *1 , Swarat Chaudhuri *2 and Lydia Kavraki *1 Abstract— Satisfiability Modulo Theories (SMT) solvers have recently emerged as a core technology in automated reasoning about systems. In this paper, we demonstrate the utility of these solvers in integrated task and motion planning (ITMP) for robots performing mobile manipulation. Specifically, we present a system—called ROBOSYNTH—for integrated task and motion planning that uses discrete search based on SMT-solvers as a complement to motion planning algorithms. As far as we know, this is the first application of SMT-solving in ITMP. The inputs to our version of ITMP are: (1) a scene description that specifies the physical space that the robot manipulates; (2) a plan outline that syntactically defines a space of plausible integrated plans; and (3) a set of logical requirements that we want the generated plan to satisfy. Given these inputs, our method uses a motion planning algorithm to construct a discrete placement graph whose paths represent feasible, low- level motion plans. An SMT-solver is now used to symbolically explore the space of all integrated plans that correspond to paths in the placement graph, and also satisfy the constraints demanded by the plan outline and the requirements. We have evaluated our approach on a generalization of an ITMP problem investigated in prior work. The experiments demonstrate that our method is capable of generating inte- grated plans that are interesting in a qualitative sense. We also find the method to scale well with an increase in the number of objects and locations manipulated, as well as the size of the space of plausible integrated plans. I. I NTRODUCTION Over the last decade, Satisfiability Modulo Theory (SMT) solvers [1]–[3] have emerged as a core technology in many areas that demand analysis of large, discrete state spaces [4]. In a nutshell, SMT-solvers are fully automatic, highly engineered satisfiability checkers for logical formulas in quantifier-free first-order theories. Their utility in state space analysis lies in the fact that large, even infinite, sets of system states can be compactly represented as (quantifier-free) first-order formulas. This allows us to frame problems of state space search (for example, the search for a length-k trajectory satisfying a temporal constraint) using a small number of calls to an SMT-solver. As state-of-the-art SMT-solvers can handle formulas with hundreds of thousands of clauses and variables, this formulation often leads to dramatic improvements in the scalability of state space analysis. In this paper, we show that SMT-solvers can be a key component of solving the difficult problem of integrated task and motion planning (ITMP) for robots. ITMP (see, * Dept. of Computer Science, Rice University, Houston, TX 77251. {nedunuri | snp3 | mmoll | swarat | kavraki}@rice.edu 1 Work supported in part by NSF grants 1317849, 1139011, and ARL/ARO grant W911NF-09-1-0383 2 Work supported in part by NSF Award 1162076 and NSF CAREER award 1156059 e.g., [5]) is a challenging class of planning problems involving a complex combination of high-level task planning and low- level motion planning. As far as we know, we are the first to explore the application of SMT-solvers in ITMP. In the version of ITMP considered here, the task planning level is discrete and requires combinatorial exploration of the space of possible integrated plans, while the motion planning level is responsible for finding paths in continuous spaces. The task level planner has to search a space that is exponential in the number actions required to achieve a goal, while the continuous planning problem is PSPACE-complete in the degrees of freedom of the robot [6]. It is thus not surprising that the seamless integration of these two levels is difficult and eludes simple recipes. A strictly hierarchical approach where the task planner operates on an abstraction and passes the solution to a continuous motion planner does not work: it either sacrifices completeness or forces the continuous planner to solve an infeasible problem. While backtracking in the physical space [5] can mitigate the second problem, this can be very time consuming. Our approach to solving the above problem—embodied in a system called ROBOSYNTH—uses an SMT-solver to efficiently explore the combinatorial space of integrated plans. The inputs to ROBOSYNTH include: (1) a scene description that specifies the robot and the physical workspace in which it operates; (2) a plan outline that describes high-level partial knowledge that the programmer has about plausible plans, and syntactically defines a space of integrated plans that we search; and (3) a set of logical, semantic requirements that the generated plan must satisfy. For example, in the benchmark on which we have evaluated ROBOSYNTH, the workspace is a kitchen scene, and requirements include properties like “The path taken by the robot from the dishwasher to the storage area must avoid the food preparation area and be less than 10m long.” The plan outline captures the fact that in many realistic situations, the programmer might know that cups should be loaded before plates from high-level insights. What he or she would not know are the locations of dirty dishes or where they should go in the dishwasher or the paths the robot should take, and these are the unknowns) that the planning system should synthesize. The workspace specification, plan outline, and requirements are written as a C-like “program” where the objects that the robot moves around (for example, cups and dishes) are declared as symbolic variables, and actions that the robot performs (for example, moving and picking up an object) appear as function calls. The scene description can consist of CAD models, sensor data-derived models, or some combination of both.