Scale factors and hypothetical referenda: A clarifying note Fredrik Carlsson à , Olof Johansson-Stenman Department of Economics, School of Business, Economics and Law, University of Gothenburg, Box 640, 405 30 Go ¨teborg, Sweden article info Article history: Received 11 October 2008 Available online 7 March 2010 JEL classification: C25 C91 C72 Keywords: Hypothetical referenda Hypothetical bias Scale parameter Contingent valuation abstract In this note we explore in detail the importance of, and problems associated with, correcting for variance differences between data sets obtained from hypothetical and real referenda. We show that a previous discussion in the literature rests on a problematic estimation of the relative scale factor. The implications are illustrated with data from Cummings et al. (1997) [5], as well as with simulated data. Moreover, we propose a concrete methodology for how to analyze cases where it is difficult, or even impossible, to estimate the relative scale factor due to informational limitations, such as when there is no variation of the bid. We conclude that it is valuable to be able to separate behavioral differences into variance differences and parameter differences in the underlying objective function. Yet, we argue that when using the results to interpret the results of other hypothetical referenda, it is sometimes the net effect, i.e., without correction for scale differences, that matters. & 2010 Elsevier Inc. All rights reserved. 1. Introduction Whether hypothetical referenda are valid in the sense of mimicking real referenda is important for policy evaluations. For example, contingent valuation studies of non-market goods are often formulated as a referendum. There is no sign of consensus about whether hypothetical referenda actually mimic real referenda. Some studies find a strong hypothetical bias, while others do not; see for example [5–7], and also see [13] for a meta-analysis. The paper by Cummings et al. [5] (henceforth CEHM) is an important study that compared two treatments: one hypothetical and one real referendum directed towards people living close to a contaminated area. The scenario description in both treatments told the subjects that if everybody paid 10 USD, the amount of money would be sufficient to produce and distribute a ‘‘citizens’ guide’’ that would provide valuable information about safe groundwater. In the hypothetical referendum, 45% voted yes and 55% voted no, whereas in the real referendum, 27% voted yes and 73% voted no. This sizable difference was found to be statistically significant, implying that they rejected the hypothesis that the hypothetical referendum is incentive compatible. However, in a comment, Haab et al. [8] (henceforth HHW) dispute this conclusion and claim that ‘‘the results of the experiments by Cummings et al. do not reject the hypothesis of incentive compatibility of hypothetical referenda’’ (p. 186). They further claim that if one corrects for a difference in variance between the two treatments, then there is no significant difference between them anymore. In this note we explore in detail the importance of and problems with correcting for a possible difference in variance between data sets. The point raised by HHW is indeed potentially very important. However, as we will show, the way they identify and correct for the relative scale factor is inappropriate, and it may indeed be difficult or even impossible to identify a difference in variance when the informational basis for such an estimation is weak, such as in the case with the Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/jeem Journal of Environmental Economics and Management ARTICLE IN PRESS 0095-0696/$ - see front matter & 2010 Elsevier Inc. All rights reserved. doi:10.1016/j.jeem.2010.02.001 à Corresponding author. Fax: + 46 31 7861043. E-mail address: fredrik.carlsson@economics.gu.se (F. Carlsson). Journal of Environmental Economics and Management 59 (2010) 286–292