Robust Dexterous Manipulation under Object Dynamics Uncertainties Yongxiang Fan 1 , Liting Sun 1 , Minghui Zheng 1 , Wei Gao 2 , Masayoshi Tomizuka 1 Abstract— Dexterous manipulation has broad applications in assembly lines, warehouses and agriculture. To perform broad- scale manipulation tasks, it is desired that a multi-fingered robotic hand can robustly manipulate objects without knowing the exact objects dynamics (i.e. mass and inertia) in advance. However, realizing robust manipulation is challenging due to the complex contact dynamics, the nonlinearities of the system, and the potential sliding during manipulation. In this paper, a dual- stage grasp controller is proposed to handle these challenges. In the first stage, feedback linearization is utilized to linearize the nonlinear uncertain system. Considering the structures of uncertainties, a robust controller is designed for such a linearized system to obtain the desired Cartesian force on the object. In the second stage, a manipulation controller regulates the contact force based on the Cartesian force from the first stage. The dual-stage grasp controller is able to realize robust manipulation without contact modeling, prevent the slippage, and withstand 40% mass and 50% inertia uncertainties. More- over, it does not require velocity measurement or 3D/6D tactile sensor. Simulation results on Mujoco verify the efficacy of the proposed method. The simulation video is available at [1]. I. INTRODUCTION Dexterous manipulation is essential for manipulators to execute complicated tasks, such as circuit assembly, com- modity organizing and fruit harvesting. To perform broad- scale manipulations, a robotic hand usually has to manipulate objects with various shapes and dynamics properties such as mass and inertia. In many applications, accurate models of the object dynamics are unknown in advance. They are estimated from 3D sensing, as well as prior knowledge such as density and statistical models. Consequently, uncertainties are introduced into the system. It is difficult to deal with such uncertainties in dexterous manipulation. First, the object is not directly controlled by actuators. Alternatively, energy is transferred from the fingertips to the object through unknown contact dynamics. Second, the robotic hand for dexterous manipulation can be a high degree-of-freedom (DOF) nonlinear system and can not be directly written into linear time-invariant (LTI) or linear parametric-varying (LPV) form. Moreover, the potential sliding between the fingertips and the object would degrade the object motion tracking performance. As a result, robust dexterous manipulation for nonlinear systems has received significant attention. A robust controller for contact uncertainties was proposed in [2]. The controller 1 Yongxiang Fan, Liting Sun, Minghui Zheng, and Masayoshi Tomizuka are with Department of Mechanical Engineering, University of California, Berkeley, Berkeley, CA 94720, USA yongxiang fan, litingsun, minghuizheng, tomizuka@berkeley.edu 2 Wei Gao is with School of Aerospace, Tsinghua University, Beijing, 100084, P. R. China gaow13@mails.tsinghua.edu.cn is designed for a LTI system linearized around an equilibrium point. A force-position controller using 6D tactile sensors was implemented to realize adaptive grasping [3]. Nonlinear- ities were ignored due to its constant-pose grasping property. In order to consider parameter variations caused by nonlin- earities, a LPV control with smooth scheduling was applied in [4], assuming that the nonlinearities can be approximated through linear varying parameters. To deal with dynamics uncertainties, a disturbance observer (DOB) was proposed in [5] for tracking control. The nonlinearities and parameter uncertainties are lumped into a disturbance term. It assumes full state feedback, while in dexterous hand, the velocity feedback is difficult due to the size constraints and cost issue. Feedback linearization was applied to control an unmanned aerial vehicle [6]. A linear state observer and a DOB are combined to observe the state and the lumped disturbance. Similar to [5], the structures of the parameter uncertainties are ignored, and the linear state observer assumes a perfect model for state estimation. This paper proposes a dual-stage grasp controller for dexterous manipulation under object dynamics uncertainties and external disturbances. Distinctive features of this pa- per include: 1) The nonlinearities are reduced by feedback linearization on a nominal model. Compared with LPV that assumes linear variations of parameters, the proposed method is more computationally efficient for broad-scale manipulations. 2) The robust controller is formulated as a μ-synthesis problem, and the structures of the uncertain- ties are considered by descriptor form, instead of treating uncertainties as a lumped disturbance, which results in information loss and a larger disturbance to resist. 3) By the dual-stage formulation, the complicated contact modeling is bypassed, and the contact force is regulated and the slippage is prevented. 4) Moreover, the dual-stage grasp controller does not require expensive 3D/6D tactile sensors or velocity measurements of objects/joints. The remaining of this paper is organized as follows. Sec- tion II introduces the dual-stage grasp controller framework. Section III describes the system dynamics and its combina- tion with the feedback linearization. The robust controller and the manipulation controller are presented in Section IV and Section V, respectively. Section VI shows the simulation results. Section VII concludes the paper. II. DUAL-STAGE GRASP CONTROLLER FRAMEWORK Figure 1 shows the proposed framework of the dual- stage grasp controller. In this figure, r, y, n and e denote the reference pose, the actual pose, the measurement noise