G. Leitner, M. Hitz, and A. Holzinger (Eds.): USAB 2010, LNCS 6389, pp. 1–13, 2010. © Springer-Verlag Berlin Heidelberg 2010 Mapping the Users’ Problem Solving Strategies in the Participatory Design of Visual Analytics Methods Eva Mayr 1 , Michael Smuc 1 , Hanna Risku 1 , Wolfgang Aigner 2 , Alessio Bertone 2 , Tim Lammarsch 2 , and Silvia Miksch 2 1 Research Center KnowComm, Danube University Krems, Dr. Karl Dorrek Str. 30, 3500 Krems, Austria {Eva.Mayr,Michael.Smuc,Hanna.Risku}@donau-uni.ac.at 2 Department of Information and Knowledge Engineering (ike), Danube University Krems, Dr. Karl Dorrek Str. 30, 3500 Krems, Austria {Wolfgang.Aigner,Alessio.Bertone,Tim.Lammarsch, Silvia.Miksch}@donau-uni.ac.at Abstract. Especially in ill-defined problem spaces, more than one exploration way leads to a solution. But often visual analytics methods do not support the variety of problem solving strategies users might apply. Our study illustrates how knowledge on users’ problem solving strategies can be used in the partici- patory design process to make a visual analytics method more flexible for dif- ferent user strategies. In order to provide the users a method which functions as a real scaffold it should allow them to choose their own problem solving strat- egy. Therefore, an important aim for evaluation should be to test the method’s flexibility. Keywords: Problem solving strategies, information visualization, visual analytics, evaluation. 1 Introduction “The goal of visual analytics is to create software systems that will support the ana- lytical reasoning process” [19]. Following this rationale, we are currently engaged in a research project which aims to support the daily work processes of business con- sultants by means of novel visual analytics methods. To ensure that the methods suc- cessfully support data exploration, prototypes are iteratively evaluated in real-world settings with real users and refined based on evaluation results. A successful visual analytics method allows users to generate insights and supports exploratory data analysis. Therefore, evaluation techniques building on task comple- tion time and number of errors were criticized as restricted in the past [2]. In more recent evaluations researchers code and count the insights gained [13][17]. Though insights are an outcome of cognitive processes during exploratory data analysis, they are not directly linked to the task at hand. To understand the users’ cognitive proc- esses while they are completing a task (or failing to do so) we proposed to analyze the problem solving processes [10]. Problems are the users’ subjective representations of