Robust Shadow Detection by Exploring Efective Shadow Contexts Xianyong Fang Anhui University Hefei, China fangxianyong@ahu.edu.cn Xiaohao He Anhui University Hefei, China hexiaohaoahu@163.com Linbo Wang Anhui University Hefei, China wanglb@ahu.edu.cn Jianbing Shen University of Macau Macau, China shenjianbingcg@gmail.com (a) Input Image (b) GT (c) Ours (d) MTMT-Net [1] (e) DSDNet [44] Figure 1: Example of shadow detections with the SBU dataset [34]. The top and bottom scenes can lead to fake shadow (the dark parts of the pillar) and foreground (the human shadow) respectively. ABSTRACT Efective contexts for separating shadows from non-shadow objects can appear in diferent scales due to diferent object sizes. This paper introduces a new module, Efective-Context Augmentation (ECA), to utilize these contexts for robust shadow detection with deep structures. Taking regular deep features as global references, ECA enhances the discriminative features from the parallelly computed fne-scale features and, therefore, obtains robust features embed- ded with efective object contexts by boosting them. We further propose a novel encoder-decoder style of shadow detection method where ECA acts as the main building block of the encoder to extract strong feature representations and the guidance to the classifca- tion process of the decoder. Moreover, the networks are optimized with only one loss, which is easy to train and does not have the instability caused by extra losses superimposed on the intermediate features among existing popular studies. Experimental results show that the proposed method can efectively eliminate fake detections. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proft or commercial advantage and that copies bear this notice and the full citation on the frst page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specifc permission and/or a fee. Request permissions from permissions@acm.org. MM ’21, October 20ś24, 2021, Virtual Event, China © 2021 Association for Computing Machinery. ACM ISBN 978-1-4503-8651-7/21/10. . . $15.00 https://doi.org/10.1145/3474085.3475199 Especially, our method outperforms state-of-the-arts methods and improves over 13.97% and 34.67% on the challenging SBU and UCF datasets respectively in balance error rate. CCS CONCEPTS · Computing methodologies → Object detection. KEYWORDS Shadow detection, deep learning, encoder-decoder ACM Reference Format: Xianyong Fang, Xiaohao He, Linbo Wang, and Jianbing Shen. 2021. Robust Shadow Detection by Exploring Efective Shadow Contexts. In Proceedings of the 29th ACM International Conference on Multimedia (MM ’21), October 20ś24, 2021, Virtual Event, China. ACM, New York, NY, USA, 9 pages. https: //doi.org/10.1145/3474085.3475199 1 INTRODUCTION Shadow is the light efect due to surface occlusion, which exists almost everywhere in our daily lives. It can be hard or soft, depend- ing on the number of light sources. Accurately detecting shadows is important for computing illumination [20, 26], layout [17], camera calibration [16], object tracking [23], etc. Recently, deep learning based approaches demonstrate better performances [12, 18, 24, 34] than traditional physical [3, 5] or handcrafted methods [33, 41, 46]. They can train optimally deep Session 21: Media Interpretation-I MM ’21, October 20–24, 2021, Virtual Event, China 2927