Using Expert Crowdsourcing to Annotate Extreme Weather Events Dennis Paulino 1(B ) , António Correia 1 , João Barroso 1 , Margarida Liberato 2 , and Hugo Paredes 1 1 INESC TEC and University of Trás-os-Montes e Alto Douro, Vila Real, Portugal {dpaulino,ajcorreia,jbarroso,hparedes}@utad.pt 2 Instituto Dom Luiz and University of Trás-os-Montes e Alto Douro, Vila Real, Portugal mlr@utad.pt Abstract. The harsh impacts of extreme weather events like cyclones or precip- itation extremes are increasingly being felt with hazardous consequences. These extreme events are exceptions to well-known weather patterns and therefore are not forecasted with current automatic computational methods. In this context, the use of human computation to annotate extreme atmospheric phenomena could pro- vide novel insights for computational forecasting algorithms and a step forward in climate change research by enabling the early detection of abnormal weather conditions. However, existing crowd computing solutions have technological lim- itations and show several gaps when involving expert crowds. This paper presents a research approach to fulfill some of the technological and knowledge gaps for expert crowds’ participation. A case study on expert annotation of extreme atmo- spheric phenomena is used as a baseline for an innovative architecture able to support expert crowdsourcing. The full stack service-oriented architecture ensures interoperability and provides an end-to-end approach able to fetch weather data from international databases, generating experts’ visualizations (weather maps), annotating data by expert crowds, and delivering annotated data for processing weather forecasts. An implementation of the architecture suggests that it can deliver an effective mechanism for expert crowd work while solving some of the identified issues with extant platforms. Therefore, we conclude that the proposed architecture has the potential to contribute as an effective annotation solution for extreme weather events. Keywords: Annotation · Atmospheric phenomena · Expert crowdsourcing · Extreme meteorological events · System design · Weather maps 1 Introduction Global warming accentuates the probability of having more unprecedented Extreme Weather Events (EWE) like heat waves or precipitation extremes [1]. As a result, climate change can be noticed in different regions worldwide suffering the effects of EWE [2–4]. Such extreme meteorological events can have significant socio-economic impacts, increasing agriculture production shortfalls, economic losses, and human mortality [5, © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 Á. Rocha et al. (Eds.): WorldCIST 2021, AISC 1366, pp. 522–532, 2021. https://doi.org/10.1007/978-3-030-72651-5_50