doi: 10.1111/j.1469-1809.2010.00625.x Detecting Epistasis with Restricted Response Patterns in Pairs of Biallelic Loci Pratyaksha Wirapati 1∗ , Karl Forner 2 , Angelica Delgado-Vega 3 , Marta Alarc ´ on-Riquelme 4,5 , Mauro Delorenzi 1,6 and J ´ er ˆ ome Wojcik 2 1 Bioinformatics Core Facility, Swiss Institute of Bioinformatics, Lausanne, Switzerland 2 Bioinformatics, Merck-Serono S.A., Geneva, Switzerland 3 Department of Genetics and Pathology, Uppsala University, Uppsala, Sweden 4 Arthritis and Immunology Program, Oklahoma Medical Research Foundation, Oklahoma City, USA 5 Human DNA Variability Area, Pfizer-University of Granada-Junta de Andalucia, Center for Genomics and Oncological Research, Granada, Spain 6 Centre Hospitalier Universitaire Vaudois, Lausanne, Switzerland Summary Well-established examples of genetic epistasis between a pair of loci typically show characteristic patterns of phenotypic distributions in joint genotype tables. However, inferring epistasis given such data is difficult due to the lack of power in commonly used approaches, which decompose the epistatic patterns into main plus interaction effects followed by testing the interaction term. Testing additive-only or all terms may have more power, but they are sensitive to nonepistatic patterns. Alternatively, the epistatic patterns of interest can be enumerated and the best matching one is found by searching through the possibilities. Although this approach requires multiple testing correction over possible patterns, each pattern can be fitted with a regression model with just one degree of freedom and thus the overall power can still be high, if the number of possible patterns is limited. Here we compare the power of the linear decomposition and pattern search methods, by applying them to simulated data generated under several patterns of joint genotype effects with simple biological interpretations. Interaction-only tests are the least powerful; while pattern search approach is the most powerful if the range of possibilities is restricted, but still includes the true pattern. Keywords: Epistasis, power study, interaction test, linear models, genome-wide scan Introduction With the recent flood of high-throughput SNP genotyp- ing studies, there is renewed interest in detecting epistasis (Carlborg & Haley, 2004; Phillips, 2008; Cordell, 2009), mo- tivated by the desire to uncover higher-order relationships that might be missed by gene-by-gene scans for associations. Although the concept of ‘epistasis’ is nearly as old as genetics (Bateson, 1907; Fisher, 1918), with many textbook examples in simple, fully penetrant traits (Roth et al., 2009), detecting epistasis in quantitative traits or non-fully penetrant binary ∗ Corresponding author: Pratyaksha Wirapati, Swiss Institute of Bioinformatics, Quartier Sorge, Bˆ atiment G´ enopode, 1015 Lau- sanne, Switzerland. Tel: +41 21 692 4098; Fax: +41 21 692 4055; E-mail: pratyaksha.wirapati@isb-sib.ch traits (such as disease susceptibility) is notoriously difficult due to the lack of power, and still confusing practical formulations of ‘epistasis’ (Cordell, 2002; Phillips, 2008; Cordell, 2009; VanderWeele, 2010), which are often attributed to presum- ably distinct concepts of ‘Fisherian’ versus ‘Batesonian’ epis- tasis (Roth et al., 2009; VanderWeele, 2010). Furthermore, claims of potential ubiquity of epistasis interactions (Carlborg & Haley, 2004) are not undisputed. For example, Hill et al. (2009) pointed out that complex traits could be explained by mainly additive genetic effects. Here, we attempt to clarify these issues by characteriz- ing the power of several detection methods resulting from different formulation of epistasis. Previous power analyses (Ritchie et al., 2003; Marchini et al., 2005; Evans et al., 2006; Chapman & Clayton, 2007; Gay´ an et al., 2008) were focused on one proposed method under various conditions Annals of Human Genetics (2011) 75,133–145 133 C  2010 Merck Serono S.A. Annals of Human Genetics C  2010 Blackwell Publishing Ltd/University College London