Markerless Feature-Registration-Based Image Processing Framework for Oil Palm Fresh Fruit Bunch Robotic Harvester Tiong Yew Tang* a , Chian C. Ho b , Geetha Nadarajan c a Research Centre For Human-Machine Collaboration (HUMAC), School of Engineering and Technology, Sunway University, No. 5, Jalan Universiti, Bandar Sunway, 47500 Selangor Darul Ehsan, Malaysia; b Dept. of Electrical Engineering, National Yunlin University of Science and Technology, Taiwan; c Department of Marketing Strategy & Innovation, Sunway Business School, Sunway University, No. 5, Jalan Universiti, Bandar Sunway, 47500 Selangor Darul Ehsan, Malaysia. ABSTRACT Malaysia's oil palm industry faces many challenges in sustaining manual oil palm harvesting operations. This work investigates an effective oil palm Fresh Fruit Bunch (FFB) image processing method for robot harvesting automation. This research explores the proposed image processing method that first detects the Fresh Fruit Bunch (FFB) category which involves 6 different categories of FFB growth stages and then detects its 6D pose estimation for harvesting. Next, this research proposes a novel image processing framework that utilises the convolutional neural network deep learning classification and is followed by markerless feature-registration-based oil palm FFB for 6D pose estimation with the public FFB dataset. Furthermore, this work introduced view obstruction to the public FFB dataset as noise for practical robot harvester applications in plantation field operation. Moreover, the experiment results show the proposed model can maintain a high F1 score performance up until 70% of view obstruction before the F1 score performance is reduced. Keywords: FFB, Image Processing, Markerless, Oil Palm, 6D pose estimation, Classification, Deep Learning 1. INTRODUCTION Malaysia is the world’s second-largest palm oil agriculture industry producer [1]. However, Malaysia's agriculture industry faces many difficult challenges such as Work-Related Musculoskeletal Disorders (WRMSDs) issues among oil palm plantation workers [1] and socioeconomic matters [2] which are not sustainable in the long run. Therefore, robotic automation solutions (see Figure 1) to resolve Malaysia’s agricultural challenges are crucial for Malaysia's agriculture industry [2]. To ensure the sustainability of the Malaysian agriculture industry to operate efficiently, we proposed the novel oil palm Fresh Fruit Bunches (FFB) image processing for the robot harvester in the oil palm plantation field. This research objective focuses on the image processing contribution of the research where we aim to detect the oil palm FFB categories and then detect FFB 6D pose estimation for mechanisation of oil palm robot harvesting operations. Based on our knowledge, our proposed method is novel in the oil palm robot harvesting research area. The 6D pose estimation is an important feature for robot harvesting because it enables the robot to manipulate the FFB for harvesting. The research questions are: 1) How does the proposed image processing detect the oil palm FFB? 2) What is the F1 score performance comparison between different view obstruction percentages with our proposed method? The oil palm FFB robotic harvesting operation involves removing objects (e.g., leaves, branches, dirt) and other view obstacles before oil palm FFB can be harvested. Hence, the robotic harvester’s image processing must be equipped with the capability of image object detection with view obstruction tolerance. Therefore, the robotic harvester can effectively locate the oil palm FFB from the robot’s view during operation in the plantation. Next, the harvester robot needs the 6D object pose estimation (See Figure 2) to locate oil palm FFB on the ground. Once the oil palm FFB 6D pose is estimated on the plantation ground, the robot harvester utilises the technique of visual servoing [3] to move itself nearer to the FFB for the grabbing or harvest process (See Figure 3). Furthermore, the markerless visual servoing approach is utilised instead because it is not feasible to implement a marker approach in a large oil palm plantation environment. *tiongyewt@sunway.edu.my; phone +60194195541; https://sunwayuniversity.edu.my/school-of-engineering- technology/staff-profiles/dr-tang-tiong-yew; https://github.com/robotictang/IWAIT2025