Advance Access publication July 16, 2011 Political Analysis (2011) 19:287–305 doi:10.1093/pan/mpr023 Choice or Circumstance? Adjusting Measures of Foreign Policy Similarity for Chance Agreement Frank M. Ha ¨ge Department of Politics and Public Administration, University of Limerick, Limerick, Ireland e-mail: frank.haege@ul.ie The similarity of states’ foreign policy positions is a standard variable in the dyadic analysis of international relations. Recent studies routinely rely on Signorino and Ritter’s (1999, Tau-b or not tau-b: Measuring the similarity of foreign policy positions. International Studies Quarterly 43:115–44) S to assess the similarity of foreign policy ties. However, S neglects two fundamental characteristics of the international state system: foreign policy ties are relatively rare and individual states differ in their innate propensity to form such ties. I propose two chance-corrected agreement indices, Scott’s (1955, Reliability of content analysis: The case of nominal scale coding. The Public Opinion Quarterly 19:321–5) p and Cohen’s (1960, A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20:37–46) j, as viable alternatives. Both indices adjust the dyadic similarity score for a large number of common absent ties. Cohen’s j also takes into account differences in individual dyad members’ total number of ties. The resulting similarity scores have stronger face validity than S. A comparison of their empirical distributions and a replication of Gartzke’s (2007, The capitalist peace. American Journal of Political Science 51:166–91) study of the ‘Capitalist Peace’ indicate that the different types of measures are not substitutable. 1 Introduction The similarity of states’ foreign policy positions is a standard variable in the quantitative, dyadic analysis of international relations. The variable is supposed to capture the extent to which pairs of states have shared or opposing interests. Explicitly or implicitly, the degree of similar or opposing state interests forms part of most explanations for international cooperation and conflict. For example, similar state interests are hypothesized to foster bilateral trade (Morrow, Siverson, and Tabares 1998; Kastner 2007), to increase the chances of receiving military and development aid (Neumayer 2003; Derouen and Heo 2004), to im- prove the effective functioning of international institutions (Stone 2004), to reduce the incentives to harbor foreign terrorist groups (Bapat 2007), and, of course, to decrease the risk of conflict and militarized dis- putes (Bearce, Flanagan, and Floros 2006; Long and Leeds 2006; Gartzke 2007; Braumoeller 2008). Yet despite the importance of this variable, the measurement offoreign policy similarity has received little attention. Bueno de Mesquita (1975) originally proposed Kendall’s (1938) rank-order correlation coefficient s b as a measure of similarity. According to this measure, the foreign policy ties of two states are maximally similar if their rankings exhibit perfect covariation. Signorino and Ritter (1999) objected to the use of s b on conceptual grounds. They argue that s b does not indicate the extent to which two states share the same types of foreign policy ties to other states, but only the extent to which the two states rank their foreign policy ties to other states in a similar manner (Signorino and Ritter 1999, 121). Signorino and Ritter (1999) propose S as an alternative measure. According to this measure, the foreign policy tie profiles of two states are maximally similar if they match exactly, regardless of whether or not the strength of foreign policy ties covaries. Signorino and Ritter’s S has since become the prevailing measure of foreign policy positions in the statistical analyses of international relations. 1 Despite its growing popularity, few studies have subsequently examined the properties of S. Although Bennett and Rupert (2003) and Sweeney and Keshk (2005) have pointed to some empirical and conceptual problems of S, they have not suggested feasible alternatives. 1 A search in the Social Science Citation Index for articles citing Signorino and Ritter (1999) returns 126 matches (http://isiwebofknowledge.com [accessed April 26, 2011]). A similar search in Google Scholar returns 273 hits (http://scholar.google.com [accessed April 26, 2011]). Although S might not have completely replaced s b , I am not aware of any recent study that relies exclusively on s b , without reporting results with S as well. Supplementary materials for this article are available on the Political Analysis Web site. Ó The Author 2011. Published by Oxford University Press on behalf of the Society for Political Methodology. All rights reserved. For Permissions, please email: journals.permissions@oup.com 287 at University of Limerick on October 21, 2012 http://pan.oxfordjournals.org/ Downloaded from