Development of Closed Loop Coordinated Control of a Robot Guided SMA Actuated Flexible Active Needle with Multimodal Sensory Feedbacks Felix Orlando Maria Joseph Electrical Engineering Department Indian Institute of Technology Roorkee Roorkee, India felixfee@iitr.ac.in Abstract— An experimental evaluation of a two loop coordinated control strategy of a robot guided shape memory alloy (SMA) actuated flexible needle system is presented in this paper. The control of the flexible needle is deployed as the inner loop and the robot driving the needle in correspondence with the needle’s movement is deployed as the outer loop. These two control loops are coordinated in such a way that the desired curvilinear trajectory for the needle tip is achieved with an acceptable accuracy. The control performance in trajectory tracking with three feedback modalities, i.e. Electromagnetic (EM) sensor, Vision and Ultrasound (US) imaging is presented. Results from the experiments reveal that the control performance with EM sensory feedback has smaller error (RMSE =0.04mm) compared to the other two feedbacks. However, tracking performance with US imaging feedback is comparable to EM sensory feedback, thereby, may be a good option for imaging based robot-assisted needle intervention for clinical applications. Keywords— Flexible Needle, Robot, Sensory Feedback, Coordinated control, Multimodal Feedback. I. INTRODUCTION (HEADING 1) Many medical interventional procedures such as, biopsy, brachytherapy, Radio Frequency based thermal ablation, anesthetic drug delivery, neural stimulation and so on, require accurate needle placement. Most of these interventional procedures are accomplished in a percutaneous manner. However, it is challenging to place the needle accurately at the specific target. For instance, in prostate brachytherapy, because the radioactive seeds could expose considerable amount of radiation dose to neighboring sensitive organs (urethra, bladder and rectum), the placement of these seeds into the specific location of the prostate accurately is a challenge. One of the major reasons for this challenge is due to the lack of proper needle actuation and also the needle geometry. Moreover, the sensory feedbacks of the needles in real-time are seldom available or inadequate, and most of the needles deployed in surgical procedures are in general straight and rigid. It is beneficial in several medical procedures by controlling a robot driven needle to reach a target, precisely through a desired trajectory along with the guidance of potential sensory feedbacks. Once the target is reached accurately, the subsequent steps involved could be taking samples (biopsy), or permanent implantation of radioactive seeds (brachytherapy) which are relatively simpler compared to the primary task of accurate target reaching. In order to follow a curvilinear path, a flexible needle is required. However, it is a challenging task to control the flexible needle to reach the target location in percutaneous intervention. Several researchers have performed the control of robot driven flexible needles [1-12]. The closed-loop control of a shape memory alloy (SMA) actuated flexible needle in air and in water experimental conditions with electromagnetic (EM) sensory feedbacks at different sampling rates is performed by Ruiz et al. [1]. Through pulse width modulation (PWM) control strategy, Ayvali et al. [2] have evaluated the performance of a multi-degrees of freedom (DOF) SMA-actuated cannula in a switched mode with visual and temperature feedbacks for position control. A Proportional plus Integral (PI) controller based visual feedback control of actuators surrounding the breast for positioning the tumor within the line of insertion of the needle is proposed by Mallapragada et al. [3]. Bernardes et al. [4] have presented a duty cycling based control of a 2D semi- automatic robot assisted needle steering system with manual imagery feedback and adaptive path planning approach for insertion in adaptive arcs. Through a visual servoing control strategy, Chatelain et al. [5] have presented a real-time needle detection by having the ultrasound (US) probe within the field of view and maintaining probe axis alignment with the US beam. A state-observer control strategy for flexible needle tip steering with needle placement accuracy improvement and reduction in needle planar deflection due to tissue uncertainty is presented through simulation by Zhao et al. [6]. In order to improve the operator force perception, a robotic co-axial needle insertion assistant which is admittance controlled thereby giving a quantitative scaled measure of force applied on the tissue by the needle tip is developed by Lorenzo et al. [7]. Clinical analysis and practicality via testing of a robot assisted needle for prostate brachytherapy is performed by Song et al. [8]. Kallem and Cowan [9] have performed vision based control of a flexible bevel-tipped needle by taking into account the  l -))) 