Eurographics Conference on Visualization (EuroVis) 2022 R. Borgo, G. E. Marai, and T. Schreck (Guest Editors) Volume 41 (2022), Number 3 A Programmatic Approach to Applying Visualization Taxonomies to Interaction Logs Sneha Gathani 1 , Shayan Monadjemi 2 , Alvitta Ottley 2 , Leilani Battle 3 1 University of Maryland, College Park, MD 2 Washington University in St. Louis, St. Louis, MO 3 University of Washington, Seattle, WA Abstract Researchers collect large amounts of user interaction data with the goal of mapping user’s workflows and behaviors to their higher-level motivations, intuitions, and goals. Although the visual analytics community has proposed numerous taxonomies to facilitate this mapping process, no formal methods exist for systematically applying these existing theories to user interaction logs. This paper seeks to bridge the gap between visualization task taxonomies and interaction log data by making the taxonomies more actionable for interaction log analysis. To achieve this, we leverage structural parallels between how people express themselves through interactions and language by reformulating existing theories as regular grammars. We represent interactions as terminals within a regular grammar, similar to the role of individual words in a language, and patterns of interactions or non-terminals as regular expressions over these terminals to capture common language patterns. To demonstrate our approach, we generate regular grammars for seven visualization taxonomies and develop code to apply them to three interaction log datasets. In analyzing our results, we find that existing taxonomies at the low-level (i.e., terminals) show mixed results in expressing multiple interaction log datasets, and taxonomies at the high-level (i.e., regular expressions) have limited expressiveness, due to primarily two challenges: inconsistencies in interaction log dataset granularity and structure, and under-expressiveness of certain terminals. Based on our findings, we suggest new research directions for the visualization community for augmenting existing taxonomies, developing new ones, and building better interaction log recording processes to facilitate the data-driven development of user behavior taxonomies. CCS Concepts • Theory of computation → Regular languages; Algebraic language theory; • Human-centered computing → Visualization theory, concepts and paradigms; 1 Introduction A clear understanding of the user’s visual analytic process is critical for designing and evaluating visualization systems. To this end, the visualization community has comprehensively captured their knowledge of users’ visual analytic processes via multiple theoretical frameworks, typologies, and taxonomies [AES05, GZ09, BM13]. We refer to these kinds of structures as taxonomies in this paper. In parallel, researchers are collecting more and more interaction log data to learn how humans analyze information via visualization systems in more data-driven ways [HMSA08, XOW ∗ 20, CGL20, PW18]. This interaction log data can reveal the user’s sensemaking process, analytical strategies, and reasoning behavior empirically much like taxonomies have aimed to capture them theoretically. Furthermore, the analysis of interaction log data could “close the loop” by enabling data-driven approaches to testing, validating, and extending longstanding theoretical taxonomies in the visualization community. However, taxonomies generalize our understanding of user analysis behavior as high-level user goals and strategies, whereas interaction logs aim to capture low-level actions and system events. As a result, inferring high-level goals and analysis strategies from interaction log data often requires an explicit mapping between lower-level interactions captured with the visualization interface and a model of the user’s task. One solution is to manually define user tasks based on the data and visualization system design. For example, Cook et al. [CCI ∗ 15] defined tasks models such as InvestigateCrime and InvestigateSuspectsBehavior to map low-level data interactions to potential high-level goals. This formulation enabled them to create a mixed-initiative system that infers the user’s task as it evolves throughout their analysis and provides suggestions to aid the process. Similarly, Heer et al. [HMSA08] and Battle and Heer [BH19] cat- egorized their observed actions into task types such as analysis-filter, undo, navigate, as well as interface specific actions like shelf-add, show-me and worksheet-add. Customized categorizations can help researchers reveal patterns in users’ analysis strategies with specific systems, but fail to generalize to other visual interfaces [PW18]. Other works have leveraged existing visualization task taxonomies to systematize the analysis of collected interaction logs. For example, Pohl et al. [PWM ∗ 12], Torsney et al. [TWSM17] and Guo et al. [GGZL15] demonstrate the potential of using theoretical taxonomies by mapping interaction logs they gathered to pre-defined task categories. The process first involved selecting the most submitted to Eurographics Conference on Visualization (EuroVis) (2022) arXiv:2201.03740v1 [cs.HC] 11 Jan 2022