Van Mierlo et al. Reflexivity in and through evaluation – IFSA 2016 NOT TO BE QUOTED page 1 Reflexivity in and through evaluation: shedding light on its meaning for system innovation initiatives Barbara van Mierlo 1 , PJ Beers 2 , Marlèn Arkesteijn 3 1 Wageningen University & Research, 2 DRIFT Erasmus University, 3 Capturing Development PLEASE DO NOT QUOTE OR USE OTHERWISE WITHOUT PERMISSION OF THE FIRST AUTHOR: barbara.vanmierlo@wur.nl Keywords: reflexivity; evaluation; learning; system innovation; transformative change; reflexive monitoring in action; systemic stability Abstract Complex problems need a freeing-up of formal and informal rules and relations that guide problematic standard actions and routine practices. This could take place in an interactive learning process. Several evaluation approaches have emerged to actively support system innovation from a reflexive perspective. The current conceptualisations of reflexivity however, provide insufficient clarity and hence no guidelines for such evaluations. In our paper, we first make a case regarding the need for reflexive evaluation approaches and their key features, based on an earlier paper. We then present a framework to operationalise and investigate reflexivity and its relation with learning empirically, with the purpose of informing reflexive evaluation approaches addressing complex problems. The potential value of this framework is illustrated with a case of a sustainability initiative in the Dutch greenhouse sector, which we supported with Reflexive Monitoring in Action. With an ex-post, secondary analysis of all the data collected, the changes in reflexivity as well as the associated outcomes of learning were traced from a temporal perspective. We conclude that learning among the actors in a system innovation process may indeed contribute to reflexivity. However, the relation between the two is weak; reflexivity is clearly also the outcome of the interactions between initiative and context. This has implications for reflexive evaluation approaches. 1. Introduction Results of interventions acknowledging the uncertain and contested character of complex problems may be short-lived if the dynamics that provide stability to the current status quo remain untouched. Complex problems need a freeing-up of formal and informal rules and relations that guide standard actions and routine practices through an interactive learning process. Several evaluation approaches have emerged to actively support transformative processes (or: system innovation) from a reflexive perspective. The current conceptualisations of reflexivity however, provide insufficient clarity or guidelines for such evaluations. It tends to remain a highly abstract term and its relations with learning and reflection as the presumed condition to learning, so far have hardly been studied empirically. Moreover, the concepts of reflexivity and learning as well as their relationship are loaded with positive, normative connotations, which may prove to be little fruitful for supporting system innovation strategies through evaluation. In our paper, we first make a case regarding the need for reflexive evaluation approaches and their key features, based on an earlier paper. We then present a framework to operationalise and investigate reflexivity and its relation with learning empirically, with the purpose of informing reflexive evaluation approaches addressing complex problems. The potential value of this framework is illustrated with a case of a sustainability initiative in the Dutch greenhouse sector, which we supported with Reflexive Monitoring in Action. With an ex-post, secondary analysis of all the data collected, the changes in reflexivity as well as the associated outcomes of learning were traced from a temporal perspective.