1090 IEEE TRANSACTIONS ON ROBOTICS, VOL. 28, NO. 5, OCTOBER 2012 Automated Multiprobe Microassembly Using Vision Feedback John D. Wason, Member, IEEE, John T. Wen, Fellow, IEEE, Jason J. Gorman, and Nicholas G. Dagalakis, Life Senior Member, IEEE Abstract—This paper describes the algorithm development and experimental results of a vision-guided multiprobe microassembly system. The key focus is to develop the capabilities required for the construction of 3-D structures using only planar microfabricated parts. Instead of using grippers, multiple sharp-tipped probes are coordinated to manipulate parts by using vision feedback. This novel probe-based approach offers both stable part grasping and dexterous part manipulation. The light weight of the part and relatively slow motion means that only kinematics-based control is required. However, probe motions need to be carefully coordinated to ensure reliable and repeatable part grasping and manipulation. Machine vision with multiple cameras is used to guide the mo- tion. No contact force sensor is used; instead, vision sensing of the probe bending is used for the grasp force control. By combining preplanned manipulation sequences and vision-based manipula- tion, repeatable spatial (in contrast with planar) manipulation and insertion of a submillimeter part have been demonstrated with an experimental testbed consisting of two actuated probes, a passive probe, an actuated die stage, and two cameras for vision feedback. Index Terms—Computer vision, dexterous manipulation, force and tactile sensing, grasping, micro/nano robots. I. INTRODUCTION M ICROASSEMBLY research arose from optoelectronics packaging needs (placement, alignment, and bonding of heterogeneous parts) in the communication industry [1]. The field has since broadened to address the construction of com- plex microscale structures from heterogeneous basic blocks. Microassembly has the potential to overcome some inherent limits of monolithic bulk micromachined microelectromechan- ical system (MEMS) devices [2]–[4]. Standard micromachin- ing processes work well on planar devices, but it is difficult to fabricate complex spatial mechanisms. The ability to layer Manuscript received November 10, 2011; revised April 9, 2012; accepted May 8, 2012. Date of publication June 13, 2012; date of current version Septem- ber 28, 2012. This paper was recommended for publication by Associate Editor Y. Sun and Editor B. J. Nelson upon evaluation of the reviewers’ comments. This work was supported in part by the Center for Automation Technologies and Systems under a Block Grant from the New York State Foundation for Science, Technology, and Innovation; in part by the Engineering Laboratory of the National Institute of Standards and Technology; and in part by the National Science Foundation Smart Lighting Engineering Research Center under Grant EEC-0812056. J. D. Wason and J. T. Wen are with Rensselaer Polytechnic Institute, Troy, NY 12180 USA (e-mail: wasonj@rpi.edu; wenj@rpi.edu). J. J. Gorman and N. G. Dagalakis are with the National Institute of Standards and Technology, Gaithersburg, MD 20899-8230 USA (e-mail: gorman@nist.gov; nicholas.dagalakis@nist.gov). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TRO.2012.2200991 mechanical components is limited by the underlying pho- tolithography approach. It is also hampered by the difficulty in fabrication using different types of materials. The wafer material is homogeneous at the start of the MEMS fabrication process, and different materials can only be deposited in layers on the same wafer substrate. Microassembly can be used to overcome these limitations by assembling components fabricated with different micromachining processes that would otherwise be incompatible. Microassembly technology may be divided into two general categories: the assembly of discrete components on a larger wafer, such as attaching photonics components to a microchip [5]–[8], or the assembly of a microstruture, such as a microrobot or multipart optical device from discrete components [9]–[13]. The design requirements are quite different—the discrete place- ment task often favors speed over accuracy and does not require 3-D dexterity over the orientation of the part, while a multipart robot or structure will require very tight tolerances and high spatial dexterity with speed being a secondary requirement. Previous work on microassembly has primarily focused on the use of microgrippers. The grippers may be based on bimorph metals, piezoelectric ceramics [14], or are silicon MEMS de- vices themselves [15]–[19]. The use of microgrippers is attrac- tive, as conventional macroscale robot task and path planning techniques may be applied. However, adhesion forces, consist- ing of electrostatic attraction, van der Waals force, and capillary forces [20], tend to dominate in the microscale. As a result, parts may stick to the gripper even after opening, which then requires additional mechanism or manipulation to overcome (such as the three-prong microgripper in [21]). These grippers can also be brittle and easily damaged. To avoid these prob- lems, parts may have specialized grip points [22], [23] or have built-in “snap” connectors [1], [17]. Multi-degree-of-freedom (DOF) macroscale robotic mechanisms are typically needed to provide dexterous control of the microgripper [16]. This adds to the complexity, size, and is a potential source of disturbance to the microassembly system. Our research is motivated by the following inquiry: Can two probes be used more effectively than a conventional two-tine microgripper? This paper provides a partial answer—the mul- tiprobe micromanipulation approach is at least a viable alter- native, particularly for parts with a wide range of scales and geometries. The problem addressed in this paper is to simply reorient a small (submillimeter) planar part and insert it verti- cally into a slot. The part needs to be lifted, rotated out of plane, aligned, and inserted. Fig. 1 shows a photo of the part before it is picked up and after it has been inserted. This simple task 1552-3098/$31.00 © 2012 IEEE