Active Citation: A Precondition for Replicable Qualitative Research Andrew Moravcsik, Princeton University ............................................................................................................................................................................................................................................... Q ualitative research dominates political sci- ence. In the field of international relations (IR), for example, about 70% of scholars primarily employ qualitative methods, compared to 21% favoring formal or quantitative analysis (Jor- dan et al. 2009). Since nearly all of the latter make secondary use of textual and historical methods, overall over 90% of IR scholars employ qualitative analysis, whereas 48% use any statistical and only 12% any formal methods. This understates the dominance of qualitative analysis, for many statistical data sets rest ultimately on historical work, and IR scholars, when polled, report that qualitative case studies are more relevant for policy than quantitative or formal work. Hardly any major IR debate—whether that over the end of the cold war, American unipolarity, Chinese foreign policy, the nature of European integration, compliance with international law, democratic peace, the causes of war, or the impact of human rights norms—remains untouched by important qual- itative contributions. Yet qualitative political science finds itself in self-imposed crisis. This crisis stems, above all, from a failure to impose firm standards of replicability. True, as a result of a recent deepening in methodological awareness, qualitative ana- lysts increasingly formalize theories and select cases more carefully. Yet once theories and cases are selected, case study analyses tend to proceed almost entirely without explicit meth- odological rules—particularly with regard the treatment of evidence. The selection, citation, and presentation of sources remain undisciplined and opaque. The drawing of causal infer- ences from evidence lacks transparency, precision, rigor, and, therefore, replicability. The discipline rarely rewards or censures a scholar for collecting or failing to collect primary evidence, for employing or failing to employ rigorous process- tracing techniques, or for revising or failing to revise the con- ventional historiography. This is reflected in disciplinary norms. The methodologi- cal adequacy of case study analysis is only infrequently a pre- condition for publication in journals or books, in striking contrast to rigorous standards for quantitative and formal work. IR and European studies, the two fields I know best, remain rife with case studies resting on citations that convey almost no empirical information. 1 Without standards for cita- tion, there is no consistent means of knowing whether the process-tracing practices are being employed rigorously; hence, for the most part, process-tracing rules are simply ignored. The conventional wisdom—“you can prove anything with a case study”—reflects widespread skepticism about qualitative methods. But the skepticism is often justified. To be sure, exceptional pockets of rigorous qualitative research exist.Yet such scholarship—within the sub-discipline of IR, at least—tends to fall into two narrow categories. One is work within a particular geographical area: Russian, East Asian, or European Union politics, for example. Small communities of scholars exploit local interpretive knowledge, linguistic skills, and a more familiar body of sources, functioning simi- larly to historians. Another exceptional category contains work in which political scientists recapitulate positions and sources from preexisting historical literature. Consider, for example, the debate in security studies over the past two decades about the origins of World War I. This debate tracks the similar discussion—sources, arguments, counterarguments, and all— held a generation earlier among German and European his- torians (e.g., Lieber 2007). Given the absence of documented primary-source content and the derivative nature of such political science debates, it is no surprise academic political science has become increas- ingly isolated from academic history. Outside of a few excep- tional areas—the Cuban missile crisis comes to mind—the relationship between the disciplines of history and political science remains at best one of neglect and at worst one of abuse. Political scientists often lift the work of historians out of its evidentiary context, and historians disregard the efforts of political scientists to advance general theories. Both disci- plines come away poorer. WHY FORMALIZATION, CASE SELECTION AND PROCESS-TRACING ARE INSUFFICIENT The lack of consistent adherence by political scientists to meth- odological standards in qualitative work is puzzling, because there is no shortage of sound guidance. Recent years have wit- nessed a battery of insightful books and articles on qualitative and historical methods (Bates et al. 1998; Elman, Kapiszewski, and Vinuela 2010; George and Bennett 2005; Brady and Collier 2004; King, Keohane, andVerba 1994; Lieberman, Howard, and Lynch 2004; Lustick 1996b; Skocpol 1984; Trachtenberg 2006; Van Evera 1997). Some prominent authors recommend that qualitative scholars should formalize theory. Others suggest they should select cases to create relevant variation on inde- pendent variables. Others set forth multiple alternative process- level implications of alternative theories. Still others argue that they should pay closer attention to evidence and sources. These admirable methodological works have generated new enthusiasm about rigorous qualitative methods. The problem is that political scientists have implemented such advice selec- tively. Most political scientists think of “qualitative methods” as concerned with the formalization of theory, strategic case PSC 43(1) 99078 1/7 12/21/09 9:08 am RE-REVISED PROOF Page: 29 ............................................................................................................................................................................................................................................... SYMPOSIUM ............................................................................................................................................................................................................................................... doi:10.1017/S1049096509990783 PS • January 2010 29