Research Challenges in Developing Multimedia Systems for Managing Emergency Situations Mengfan Tang Department of Computer Science University of California, Irvine mengfant@uci.edu Siripen Pongpaichet Department of Computer Science University of California, Irvine spongpai@ics.uci.edu Ramesh Jain Department of Computer Science University of California, Irvine jain@ics.uci.edu ABSTRACT With an increasing amount of diverse heterogeneous data and information, the methodology of multimedia analysis has become increasingly relevant in solving challenging soci- etal problems such as managing emergency situations during disasters. Using cybernetic principles combined with multi- media technology, researchers can develop effective frame- works for using diverse multimedia (including traditional multimedia as well as diverse multimodal) data for situa- tion recognition, and determining and communicating ap- propriate actions to people stranded during disasters. We present known issues in disaster management and then focus on emergency situations. We show that an emergency man- agement problem is fundamentally a multimedia informa- tion assimilation problem for situation recognition and for connecting people’s needs to available resources effectively, efficiently, and promptly. Major research challenges for man- aging emergency situations are identified and discussed. We also present a platform to assimilate heterogeneous data streams for intelligently detecting evolving environmental situations, and discuss the role of multimedia micro-reports as spontaneous participatory sensing data streams in emer- gency responses. Given enormous progress in concept recog- nition using machine learning in the last few years, situation recognition may be the next major challenge for learning approaches in multimedia contextual big data. The data needed for developing such approaches is now easily avail- able on the Web and many challenging research problems in this area are ripe for exploration in order to positively impact our society during its most difficult times. Keywords Disaster; Situation recognition; Situation prediction; EventShop; Micro-reports; 1. INTRODUCTION Disasters happen. And Disasters will always happen. We have to be ready for them, respond to them efficiently, ef- Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita- tion on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re- publish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. MM ’16, October 15-19, 2016, Amsterdam, Netherlands c  2016 ACM. ISBN 978-1-4503-3603-1/16/10. . . $15.00 DOI: http://dx.doi.org/10.1145/2964284.2976761 fectively, and promptly, and then manage their long term consequences. Disasters such as floods, earthquakes, land- slides, forest fires, tsunamis, hurricanes, terrorist bombings, Zika or similar epidemics, global warming, and air quality crisis cause huge damage in both human lives and property across the globe. Furthermore, a disaster sometimes reaches the stage when it becomes an emergency. During an emer- gency, lives are lost and property may be damaged. The right information, at the right time, at the right place, and to the right people minimizes the loss of life, property, and misery. In this paper, we propose the use of multimedia for pre- dicting emerging disastrous situations, identifying people’s needs during disasters, and helping them during disasters. We believe that multimedia computing and communication systems have a pivotal role to play in disaster preparedness, help during disasters, and then in recovery afterward. Mul- timedia computing has evolved from basic audio and video applications in business and entertainment to using infor- mation from different multimodal sources for understanding and helping people and organizations. Researchers started paying attention to situation recognition and appropriate ac- tions to help people in different novel applications [32]. Dis- aster management may benefit significantly from multimedia (including multimodal) information. More importantly, for the emerging broader multimedia community, disaster man- agement in general and emergency response in particular may be a transformative research area. Researchers from environmental science, social science, public health, computer science, and various disciplines put a lot of effort into better predicting the space, time, and severity of disasters for improving response and mitigation. Governments, NGOs, and other organizations concentrate on disaster management, appropriate information sharing and communication, analysis of citizens’ needs, and disaster recovery planning. Large volumes of diverse data are being collected regularly by sensors, IoTs, social media, govern- mental reports, and citizen reports. The data is essentially multimedia big data, because of the volume, variety, and velocity [19]. We believe that people are the best sensors, people are the best responders, and people are the best actuators. Our ap- proach to using multimedia for emergency response is based on using people equipped with mobile phones connected to multimedia disaster management systems, both for collect- ing real time in-situ information and receiving instructions for action at the right place. In this approach, multimedia data resulting in timely actionable information is the key. 938