Blooms’ Taxonomy: A Framework for Assessing Programmers’ Knowledge of Software Systems Jim Buckley and Chris Exton University of Limerick jim.buckley/chris.exton@ul.ie Abstract Programmers must attain knowledge about a system before they can perform specific software maintenance tasks on it. Traditionally, computer scientists have described the activity of attaining this knowledge as ‘software comprehension’. However, if we look at the educational literature, attainable knowledge has been described with much finer granularity. Blooms’ taxonomy identifies six separate levels of knowledge within the cognitive domain, one of which refers to a (more constricted) definition of comprehension. Several other levels in Bloom’s taxonomy seem to correlate more directly to specific software maintenance tasks. This article reviews Blooms’ taxonomy as a richer descriptive framework for programmers’ knowledge of code and illustrates how various software maintenance tasks map to knowledge levels in this hierarchy. A pilot study is presented showing how participants’ knowledge of software may differ at various levels of this hierarchy. 1. Introduction Singer and Lethbridge [28], found that one of the most prevalent tasks performed by software engineers was searching through source code, and the corresponding documentation, for information. Their analysis shows that programmers attempt to build up knowledge of their systems but it does not attempt to describe the nature of this knowledge. However, several software comprehension studies have probed into the nature of this knowledge. Wiedenbeck [31], for example generated empirical results that showed how experienced programmers’ recall of ‘beacon’ statements was better than their recall of other source code statements. Beacon statements are clichéd statements that strongly indicate a system’s functionality. Still at the code level, Pennington [21] performed several experiments where programmers answered specific types of questions about the systems they had previously viewed. Using their performance on the quiz Pennington was able to illustrate that programmers initially build up knowledge of control-block constructs when studying unfamiliar code (a ‘Program Model’). Only after this control-block knowledge was established did the programmers seem to incorporate data-flow and functional aspects of the system into their representation of the system, thus moving to a model that maps from the states, situations and events of the system to the states, situations and events of the system’s domain (a ‘Situation Model’) [7]. However, other studies have repeated this analysis and found different sequences of knowledge attainment under different experiment conditions [24], [27] At this higher level of abstraction, Boehm Davis et al [5], and Soloway and Ehrlich [29], carried out independent work that tried to identify programmers’ knowledge of the plans contained in code. Plans are clichéd sections of code that implement functional goals of the system [25]. Thus, plans act as a communication mechanism between the original programmer and the maintenance programmers. Boehm-Davis et al. assessed plan knowledge by asking programmers to identify sections of code that should logically be grouped together. Soloway and Ehrlich asked programmers to complete plan-like and unplan-like sections of code to achieve the (implied) goal. At an even higher level of abstraction, Murphy and Notkin [17], tried to attain designers’ knowledge of their system architecture. They asked designers to draw a free- format conceptual graph of the system, showing the macro- structures of the system and their relationship. However, all these studies assess programmers’ knowledge in the context of understanding the system only. Several studies have shown that programmers are very often not interested in understanding the entire system per- se, but are very directed towards obtaining the knowledge required by the task they are performing. Littman et al [15], for example, illustrated how programmers tended to Proceedings of the 11 th IEEE International Workshop on Program Comprehension (IWPC’03) 1092-8138/03 $17.00 © 2003 IEEE