46 Process Tracing with Bayes: Moving Beyond the Criteria of Necessity and Sufficiency Andrew Bennett Georgetown University bennetta@georgetown.edu Qualitative & Multi-Method Research, Spring 2014 applications. As to Boolean logit, my own experience is that its information requirements and the convoluted likelihood func- tions that it produces are its Achilles’ heel: the former can result in inestimable models, while the latter result in frequent violations of the Wald assumptions that underpin estimated standard errors. The remedy in the first case is more data. The remedy in the second is bootstrapped standard errors, which are time-consuming but produce the correct standard error estimates. All in all, the main critiques of these models, unsurprisingly, revolve around their position on the spectrum from high-as- sumption to high-information. In the case of fs/QCA, if the many assumptions fit, you can believe the results; in the case of Boolean logit, if you have enough information you can esti- mate the model. The two models in between, while less ambi- tious in terms of their ability to model causal complexity, are also less ambitious in their requirements. Conclusion: Caveat Emptor Taken as a whole, to reiterate, these models represent points on a spectrum, from the assumption-laden fs/QCA procedure to the data-hungry Boolean logit. The implications of data- intensivity are fairly straightforward: a data-hungry procedure runs the risk of providing null results if too few data are avail- able to estimate all of the necessary quantities. What are the implications of incorrect assumptions? In the case of the three statistical procedures, the main assumption has to do with the form of the interaction. A bad assumption at this stage will, in a nutshell, produce conclu- sions that are inaccurate to an unknowable degree—and that is every bit as bad as it sounds. In fs/QCA, because conclusions depend on a wider range of assumptions, the cumulative implications of violating those assumptions can be even more dire. If fuzzy-set membership is estimated improperly, if the mean rather than the minimum de- fines joint membership in the conjunction of two sets, if the true threshold between possible and impossible cases is really Y=X 2 , and if an independent variable’s contribution to an out- come is partial (or, worse, unconditional), the results can bear shockingly little resemblance to the reality they are meant to capture. In all cases it pays to question assumptions and to do so thoroughly. For statistical models, it is at least possible to use model fit to adjudicate among competing assumptions. We may never arrive at the One True Specification, but we can at least know which is the best given the data we have at hand. fs/QCA offers fewer assurances of this nature, but practitio- ners can at least get a sense of the range of possible conclu- sions by varying the assumptions at each step and exploring the extent to which the results are robust to those changes. References Berry, William D., Jacqueline H. R. DeMeritt, and Justin Esarey. 2010. “Testing for Interaction in Binary Logit and Probit Models: Is a Product Term Essential?” American Journal of Political Sci- ence 54 (1): 248–266. Braumoeller, Bear F. 2003. “Causal Complexity and the Study of Politics.” Political Analysis 11 (3): 209–233. Braumoeller, Bear F. and Austin Carson. 2011. “Political Irrelevance, Democracy, and the Limits of Militarized Conflict.” Journal of Conflict Resolution 55 (2): 292–320. Braumoeller, Bear F. and Gary Goertz. 2000. “The Methodology of Necessary Conditions.” American Journal of Political Science 44 (4): 844–858. Clark, William Roberts, Michael J. Gilligan, and Matt Golder. 2006. “A Simple Multivariate Test for Asymmetric Hypotheses.” Politi- cal Analysis 14 (3): 311–331. Dion, Douglas. 1998. “Evidence and Inference in the Comparative Case Study.” Comparative Politics 30 (2): 127–145. Hildebrand, David K., James D. Liang, and Howard Rosenthal. 1976. “Prediction Analysis in Political Research.” American Political Sci- ence Review 70 (2): 509–535. Hug, Simon. 2013. “Qualitative Comparative Analysis: How Induc- tive Use and Measurement Error Lead to Problematic Inference.” Political Analysis 21 (2): 252–265. Ragin, Charles C. 2013. “QCA versus Statistical Interaction.” De- partment of Sociology, University of California, Irvine. Seawright, Jason. 2013. “Warrantable and Unwarranted Methods: The Case of QCA.” Presented at the Annual Meeting of the Ameri- can Political Science Association, Chicago. Social scientists have long recognized the study of evidence from within individual cases as a fundamental tool for causal inference. This evidence helps guard against the inferential errors that can arise from making causal inferences based only on comparisons among cases. Process tracing, the systematic study of evidence from within a single case to assess alterna- tive explanations of that case, is a key method of within-case analysis. Yet until recently, formal articulation of the underlying logic of process tracing has been incomplete. One line of in- quiry has sought to organize the traditional process tracing tests in terms of whether they provide necessary and/or suffi- cient grounds for inferring that a given piece of evidence con- firms a particular hypothesis (Bennett 2010; Collier 2011). Thus, (1) the results of a straw in the wind test may provide sugges- tive, but far from definitive, support for the hypothesis; (2) the hoop test must be passed for the hypothesis to be seriously Author’s Note: This is an abridged and revised version of “Disci- plining our Conjectures: Systematizing Process Tracing with Baye- sian Analysis,” the technical appendix to Andrew Bennett and Jef- frey Checkel, eds., Process Tracing: From Metaphor to Analytic Tool (Cambridge University Press, forthcoming 2014). I would like to thank Derek Beach, Jeff Checkel, David Collier, Colin Elman, Dimitri Gallow, Macartan Humphreys, Alan Jacobs, James Mahoney, Ingo Rohlfing, and David Waldner for their insightful comments on an earlier draft of this paper. Any remaining errors are my own. https://doi.org/10.5281/zenodo.894552