.........................................................................................................................................................................................................................................................................................................................
doi:10.1017/S1049096519001161 © American Political Science Association, 2019 PS • 2019 1
Politics
Improving Data Quality in Face-to-Face
Survey Research
Carolyn Logan, Michigan State University
Pablo Parás, Data OPM
Michael Robbins, Princeton University
Elizabeth J. Zechmeister, Vanderbilt University
ABSTRACT
Data quality in survey research remains a paramount concern for those studying
mass political behavior. Because surveys are conducted in increasingly diverse contexts
around the world, ensuring that best practices are followed becomes ever more important
to the field of political science. Bringing together insights from surveys conducted in more
than 80 countries worldwide, this article highlights common challenges faced in survey
research and outlines steps that researchers can take to improve the quality of survey data.
Importantly, the article demonstrates that with the investment of the necessary time and
resources, it is possible to carry out high-quality survey research even in challenging envi-
ronments in which survey research is not well established.
E
nsuring data quality in survey research remains a
paramount—if often under-discussed—issue for
research into mass political behavior. During the
past two decades, this has been especially true as sur-
veys became a more common method of research in
countries around the world. Best practices, often developed in the
United States, have been imported to other contexts despite dif-
ferences in survey modes, techniques, and challenges. Data quality
encompasses a wide range of issues, including implementing sam-
ple plans correctly and ensuring that field teams have the capacity
to carry out the design. Meanwhile, in recent years, the issue of data
fabrication—that is, the intentional departure from specified pro-
tocols by a member of the data-collection team—also has become
a more prominent concern (Bredl, Winker, and Kötschau 2012;
Kuriakose and Robbins 2016; Spagat 2016).
Afrobarometer, AmericasBarometer, and Arab Barometer have
all confronted the challenges of ensuring data quality and prevent-
ing fabrication in the work they have done in more than 80 coun-
tries worldwide. Evidence from these projects makes clear that
it is possible to conduct high-quality, nationally representative,
and reliable public-opinion surveys even in challenging condi-
tions. However, this process is time and resource intensive, and
investments in data quality must be made throughout the life of
the project. Achieving high-quality data is not only a question of
monitoring and policing by principal investigators (PIs) but also of
partner commitment, capacity, and correctly structured incentives.
Following is a summary of best practices developed by these
organizations during each stage of the research process. When
these methods are followed, the likelihood of obtaining valid and
reliable data and reducing fabrication is high. Although our own
focus is on projects in settings that are relatively less developed
with relatively more challenges, many of these practices are rel-
evant to researchers working in other contexts as well, whether
comparative survey projects, single-country surveys, or surveys of
a specific population within a country.
STAGE 1: PRE-FIELDWORK
Investments in data integrity must start before fieldwork
begins, when designing the survey instrument, identifying
suitable partners, selecting and training fieldworkers, and
planning fieldwork logistics. A well-designed questionnaire not
only ensures conceptual equivalence but also helps interviews
to flow quickly and smoothly, reducing the likelihood of unau-
thorized shortcuts. Partner organizations that demonstrate
real commitment and buy-in to the methodology and that have
adequate resources and the necessary capacity to do the work
well will be on the frontlines in producing high-quality data.
Effectively recruited fieldworkers who are adequately trained,
resourced, fairly compensated, and motivated are more likely to
commit to collecting good quality data and foregoing fabrica-
tion. Finally, adequately planning, resourcing, and supervising
Carolyn Logan is associate professor of political science at Michigan State University
and deputy director of the Afrobarometer. She can be reached at clogan@msu.edu.
Pablo Parás is president and founder of Data OPM in Mexico City. He can be reached at
pp@dataopm.net.
Michael Robbins is director of the Arab Barometer based at Princeton University.
He can be reached at mdr7@princeton.edu.
Elizabeth J. Zechmeister is Cornelius Vanderbilt Professor of political science
at Vanderbilt University and director of the Latin America Public Opinion Project
(LAPOP). She can be reached at liz.zechmeister@vanderbilt.edu.
This is an updated version of the original article. For details please see the notice at
https://doi.org/10.1017/S1049096519001689