Received May 8, 2022, accepted May 31, 2022, date of publication June 10, 2022, date of current version June 23, 2022. Digital Object Identifier 10.1109/ACCESS.2022.3182114 Generic Development of Bin Pick-and-Place System Based on Robot Operating System CHING-CHANG WONG 1,2 , CHI-YI TSAI 1,2 , (Senior Member, IEEE), REN-JIE CHEN 2 , SHAO-YU CHIEN 2 , YI-HE YANG 2 , SHANG-WEN WONG 2 , AND CHUN-AN YEH 2 1 Department of Electrical and Computer Engineering, Tamkang University, New Taipei City 25137, Taiwan 2 Doctoral Program in Robotics, College of Engineering, Tamkang University, New Taipei City 25137, Taiwan Corresponding author: Ching-Chang Wong (wong@ee.tku.edu.tw) This work was supported in part by the Ministry of Science and Technology (MOST), Taiwan, under Grant MOST 109-2221-E-032-038 and Grant MOST 110-2221-E-032-046. ABSTRACT Bin pick-and-place is an important topic in factory automation and warehouse automation. In this paper, a bin pick-and-place system based on robot operating system (ROS) is implemented to make a six-degree-of-freedom (6-DOF) robot manipulator to complete multiple pick-and-place tasks. The proposed system uses ROS to integrate an object perception module and a pick-and-place module, where the former uses an RGB-D camera to capture images inside the bin, and the latter controls a 6-DOF robot manipulator and two self-made vacuum tools. To estimate the pose of the target object, a YOLOv4 object detector is implemented, and an object sorting method is proposed to find the target object in the image. Then, a pose estimation method based on computer aided design (CAD) is proposed to estimate the pose of target object. To perform the object pick-and-place operations, a coordinate transformation node is designed to transfer the pose of the target object into the workspace. Then, a link distance-based bin collision avoidance method is proposed to avoid collisions. Finally, the angle of the 1-DOF vacuum tool and the picking and placement poses of the robot manipulator are obtained from the result of the bin collision avoidance and the pose of the target object. In this study, a total of ten ROS nodes are designed, and the solutions that make each function easier to implement and reproduce are proposed. In the experiments, we set up four experiments with two task types and two object types to verify the effectiveness of the implemented bin pick-and-place system. INDEX TERMS Bin pick-and-place, robot operating system, object detection, pose estimation, collision avoidance. I. INTRODUCTION The integrated applications of robot manipulator have been developed in automation technology for many years, and bin pick-and-place is still one of the challenging research issues. The ability of a robot manipulator to autonomously pick and place objects from a cluttered bin can be applied to many scenarios, such as pick-and-place objects in warehouse automation, or pick-and-place parts in factory automation [1]. One of the goals of research related to automation technology is to establish a system with the ability to work independently, and different types of objects also bring different challenges. The design of the bin pick-and-place system mainly includes two parts: object perception and object pick-and-place. The The associate editor coordinating the review of this manuscript and approving it for publication was Shun-Feng Su . purpose of the object perception part is to detect the object or estimate its pose so that the robot manipulator can correctly complete the pick-and-place task. In general, the types of common objects can be divided into textured objects and texture-less objects according to whether they have surface textures. For example, most groceries for warehouse automation are textured objects with rich surface textures, while parts for factory automation are mostly texture-less objects. Therefore, different object detection methods are usually designed under different object types and scenes. On the other hand, the object pick-and-place part converts the result of the object detection into the motion command of the robot manipulator when it performs a pick-and-place task. The types of pick-and-place tasks can be divided into orientation non-required and orientation required depending on whether the objects need to be placed in a particular pose. VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 65257