Citation: Ji, S.; Pan, J.; Li, L.; Hasegawa, K.; Yamaguchi, H.; Thufail, F.I.; Brahmantara; Sarjiati, U.; Tanaka, S. Semantic Segmentation for Digital Archives of Borobudur Reliefs Based on Soft-Edge Enhanced Deep Learning. Remote Sens. 2023, 15, 956. https://doi.org/10.3390/rs15040956 Academic Editors: Anastasios Doulamis, Nikos Grammalidis and Kosmas Dimitropoulos Received: 11 December 2022 Revised: 4 February 2023 Accepted: 7 February 2023 Published: 9 February 2023 Copyright: © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). remote sensing Article Semantic Segmentation for Digital Archives of Borobudur Reliefs Based on Soft-Edge Enhanced Deep Learning Shenyu Ji 1, *, Jiao Pan 1 , Liang Li 2 , Kyoko Hasegawa 1 , Hiroshi Yamaguchi 3 , Fadjar I. Thufail 4 , Brahmantara 5 , Upik Sarjiati 4 and Satoshi Tanaka 2 1 Research Organization of Science and Technology, Ritsumeikan University, Shiga 525-8577, Japan 2 College of Information Science and Engineering, Ritsumeikan University, Shiga 525-8577, Japan 3 Nara National Research Institute for Cultural Properties, Nara 630-8577, Japan 4 Research Center for Area Studies, National Research and Innovation Agency, Jakarta 12710, Indonesia 5 Borobudur Conservation Office, Magelang 56553, Indonesia * Correspondence: jishenyu@gst.ritsumei.ac.jp Abstract: Segmentation and visualization of three-dimensional digital cultural heritage are important analytical tools for the intuitive understanding of content. In this paper, we propose a semantic seg- mentation and visualization framework that automatically classifies carved items (people, buildings, plants, etc.) in cultural heritage reliefs. We also apply our method to the bas-reliefs of Borobudur Temple, a UNESCO World Heritage Site in Indonesia. The difficulty in relief segmentation lies in the fact that the boundaries of each carved item are formed by indistinct soft edges, i.e., edges with low curvature. This unfavorable relief feature leads the conventional methods to fail to extract soft edges, whether they are three-dimensional methods classifying a three-dimensional scanned point cloud or two-dimensional methods classifying pixels in a drawn image. To solve this problem, we propose a deep-learning-based soft edge enhanced network to extract the semantic labels of each carved item from multichannel images that are projected from the three-dimensional point clouds of the reliefs. The soft edges in the reliefs can be clearly extracted using our novel opacity-based edge highlighting method. By mapping the extracted semantic labels into three-dimensional points of the relief data, the proposed method provides comprehensive three-dimensional semantic segmentation results of the Borobudur reliefs. Keywords: cultural heritage; digital archive; semantic segmentation; deep learning; edge extraction from relief; Borobudur temple; Borobudur reliefs 1. Introduction Recently, with the development of three-dimensional (3D) digitization of large-scale cultural heritage properties [1–4], efficient analysis of digitized properties has been the focus of an increasing amount of research [5–7]. Segmentation and visualization of three- dimensional scanned cultural heritage are important analytical tools for interpretation and intuitive understanding of the associated content. Semantic segmentation for three- dimensional point clouds has been widely investigated, such as outdoor scene under- standing for autonomous driving [8–11] and robotics [12–14]. Semantic segmentation of three-dimensional digital cultural heritage has broad applications, such as the identification of architectural elements [15] or analysis of the state of conservation of materials [16]. However, automatic semantic segmentation for three-dimensional point clouds of cultural heritage is challenging because most cultural heritage properties consist of complex struc- tures and are distinct from each other. With the development and application of deep learning in recent years, the semantic segmentation of three-dimensional point clouds in cultural heritage has made a significant breakthrough [15–17]. In this study, we focus on semantic segmentation for three-dimensional point cloud- type digital archives of cultural heritage reliefs. Cultural reliefs were often carved to Remote Sens. 2023, 15, 956. https://doi.org/10.3390/rs15040956 https://www.mdpi.com/journal/remotesensing