¦ 2016 Vol.12 no.3 FormattingandPresentationasConfoundsinOnline Studies JessicaL.Engelbrecht a, andJohnE.Edlund a a RochesterInstituteofTechnology Abstract Participantbiasesareawell-documentedpartofpsychologicalresearch,andaccounting for common biases (e.g., participant acquiescence) is now commonplace. However, the role that online survey construction plays in handling or aggravating these biases has not been as deeply investigated.Thecurrentresearchaskedparticipantstocompleteasimpletraitattributiontaskin theformofanonlinesurveyinwhichthescaletype,orderofstimulipresentation,andpresence ofpagebreaksvaried. Ashypothesized,thesevaryingpresentationfeaturesimpactedparticipant responses.Precautionsforavoidingtheseerrorsarediscussed,aswellasthepossibleimplications forreplicabilityinsocialpsychology. Keywords participantbiases,onlinestudies,surveyconstruction,ordereffects. engej084@newschool.edu JLE: 0000-0003-2397-2892; JEE: 0000-0003-3868-1844 10.20982/tqmp.12.3.p243 Acting Editor De- nis Cousineau (Uni- versit´ ed’Ottawa) Reviewers Oneanonymousre- viewer Introduction Researchers have identified a number of potential con- founds and concerns to consider when designing studies forpsychologicalresearch(e.g.,McDermott&Miller, 2007). Ensuringthereliabilityandvalidityofmeasuresandstim- uliaretheforemostgoalsofresearchers,butthesefactors canbeadverselyaffectedbyunforeseenbiasesduringthe executionofthestudy(Nichols&Edlund, 2015).Biasesthat stemfromtheexpectationsoftheresearchers(Strickland &Suben, 2012)orthedesignofthematerialscanbeantici- patedandcontrolledinmanycases,butoftenparticipants bringwiththemtheirownthinkingbiasesthatcanimpact results on a large scale (e.g., Twedt, Crawford, & Proffitt, 2015). Eliminatingthesebiasesinordertoassuretheva- lidity of the data collected can be challenging, as many heuristics,cognitivebiases,andlogicalfallaciesarenearly ubiquitous,independentofintelligence(Stanovich&West, 2008). Surveys are particularly easy to execute and yet diffi- cult to design in a fashion that controls all possible con- founds. Surveysareahighlyattractiveoptionforpsycho- logical research, as they require minimal interaction be- tween researchers and participants and provide data in formats convenient for analysis. Several guidelines for theirusearetypicallyfollowed;wordingshouldbeunder- standabletoparticipants,questionsshouldnotbiaspartic- ipantstowardsaspecificanswerwiththeirwording,and response items should be standardized and presented in a single index when possible (Krosnick, 1999). Although these basic guidelines have been recognized as effective forsometime,theycannotpreventallpossible flawsthat aparticularsurveydesignmayhave. Next,webrieflyre- viewsomeofcommonbiasesfoundinsurveyresearch. Acquiescence One core flaw inherent in survey design is found in the manner of data collection. As researchers cannot ask follow-upquestionstoparticipantswhenresponsesareun- clear, ensuring that specific answers are received is im- perative. Keeping survey questions as simple as possible is desirable; however, this doesn’talwaysprovidethere- sults researchers hope for. Binary questions (that seek answerssuchasyes/no,agree/disagree,true/false)leadto a phenomenon known as acquiescence, wherein partici- pantsareultimatelymorelikelytoanswerinthepositive thanthenegative(Krosnick, 1999). Thus,researcherswill endupcollectingmore “yes”, “true”,or “agree” answersin responsetosurveyitems,andtheresultsarenotnecessar- ilyinformative.Thisconfirmationbiasisendemicandwell he uantitative ethodsfor sychology 243