Pullin, A. S., & Knight, T. M. (2009). Data credibility: A perspective from systematic reviews
in environmental management. In M. Birnbaum & P. Mickwitz (Eds.), Environmental
program and policy evaluation: Addressing methodological challenges. New Directions for
Evaluation, 122, 65–74.
Data Credibility: A Perspective From
Systematic Reviews in Environmental
Management
Andrew S. Pullin, Teri M. Knight
Abstract
To use environmental program evaluation to increase effectiveness, predictive
power, and resource allocation efficiency, evaluators need good data. Data
require sufficient credibility in terms of fitness for purpose and quality to
develop the necessary evidence base. The authors examine elements of data
credibility using experience from critical appraisal of studies on environmental
interventions employing systematic review methodology. They argue that criti-
cal appraisal of methodological quality is a key skill to improve both retro-
spective evaluation and prospective planning of monitoring of environmental
programs. Greater transparency and data sharing among evaluators could facil-
itate rapid development in approaches to environmental evaluation that improve
data credibility. © Wiley Periodicals, Inc.
I
f we want to know what works and what doesn’t, we collect data and
analyze them. But the confidence with which we can interpret those data
in the context of our questions depends on their quality and the strength
of the evidence they provide (how credible they are). Is the measured effect
65
6
NEW DIRECTIONS FOR EVALUATION, no. 122, Summer 2009 © Wiley Periodicals, Inc., and the American Evaluation
Association. Published online in Wiley InterScience (www.interscience.wiley.com) • DOI: 10.1002/ev.296