Challenges in IT Security Preparedness Exercises: A Case Study Maria Bartnes a,b,1,2,* , Nils Brede Moe b a Department of Telematics, Norwegian University of Science and Technology N-7491 Trondheim b SINTEF ICT, N-7465 Trondheim Abstract The electric power industry is currently implementing major technological changes in order to achieve the goal of smart grids. However, these changes are expected to increase the susceptibility of the industry to IT security in- cidents. IT security preparedness exercises are not commonly performed in the electric power industry, even though this industry is considered part of so- ciety’s critical infrastructure. Resolving an IT security incident requires inter- departmental collaborations between various categories of personnel, and to suc- cessfully achieve this, training is required. The process of preparing a response to incidents enhances the nature of collaboration, coordination, and communi- cation within an organization. Our objective is to understand the challenges faced when performing IT security preparedness exercises, as challenges experi- enced during these exercises affect the response process during a real incident. By improving the exercises, the response capabilities would be strengthened ac- cordingly. We have designed a multiple-case study with six teams in three orga- nizations. We collected data by performing semi-structured interviews, partici- pant observations, and from process artifacts. We identified six main challenges involving team composition and external expert involvement, goal definition, documentation, and time management. In summary, there are many ways of conducting preparedness exercises. Therefore, organizations need to both opti- mize current exercise practices and experiment with new ones in order to ensure continuous learning and improvement; hence, they can be adequately prepared to respond to IT security incidents. Keywords: Information security, Incident management, Preparedness * Corresponding author Email addresses: maria.bartnes@sintef.no (Maria Bartnes), nils.b.moe@sintef.no (Nils Brede Moe) 1 Tel.: +47-45218102, Fax: +47-73593350 2 This work is funded primarily by the Norwegian University of Science and Technology through the project Smart Grids as a Critical Infrastructure. Partial funding has been pro- vided by the Norwegian Research Council under Grant no. 217528 (DeVID). The authors would like to thank the three DSOs that participated in this study. Preprint submitted to Computers and Security October 28, 2016