Promoting Uniform Question Understanding in Today’s and Tomorrow’s Surveys Frederick G. Conrad 1 and Michael F. Schober 2 Survey respondents misunderstand questions more frequently than one might expect but, current methods for collecting data make it hard to detect and correct misunderstanding. The conventional practice has been to leave the interpretation of questions up to respondents; interviewers react to requests for clarification with nondirective probes like “Let me repeat the question.” The current article reviews a research program that has explored alternatives to standardized wording, in which interviewers and web survey systems can define survey concepts as needed as a way to assure uniform comprehension across respondents. One problem is that many respondents fail to recognize that their understanding is not aligned with the survey sponsors’ and so do not ask for clarification – a problem that, we argue, is more serious in the survey response task than other tasks in which information is exchanged. Using today’s survey techniques (telephone and face-to-face interviews, web surveys) it is possible to increase respondents’ sensitivity to their own misunderstanding, increasing their requests for clarification; and, based on respondents’ verbal and visual cues of comprehension difficulty, it is possible to intervene to correct misunderstanding. This approach can be extended in surveys of the future by incorporating mature speech recognition capabilities, modeling respondent uncertainty about question meaning so that when clarification is needed it can be provided automatically, and developing interface agents when appropriate. By evaluating simulated versions of these technologies in the near term researchers will be better able exploit them as they become available. Key words: Clarification; conversational interviewing; conceptual variability; question comprehension; standardization; interviewing techniques; mode effects; interviewer- respondent interaction. 1. Introduction When people misunderstand questions it is not good for their answers. Yet today’s techniques for collecting survey data do not make it easy to correct misunderstanding. Consider, for example, this interchange from a telephone survey, discussed in greater detail in Schober and Conrad 2002. (In this excerpt, overlapping speech is enclosed in asterisks; pauses are indicated by periods surrounded by spaces (.); lengthened sounds are indicated by colons.) q Statistics Sweden 1 University of Michigan, Institute for Social Research, P O Box 1248, Ann Arbor, MI 48106, U.S.A. Email: fconrad@isr.umich.edu 2 New School for Social Research, Dept. of Psychology, New York, NY 10003, U.S.A. Email: schober@newschool.edu. Acknowledgements: We thank the National Science Foundation (Grants SBR-97-0140, IIS-0081550, SES- 0454832), the Institute for Social Research at the University of Michigan, The U.S. Bureau of Labor Statistics, The U.S. Bureau of the Census, and the Vrije Universiteit of Amsterdam. Journal of Official Statistics, Vol. 21, No. 2, 2005, pp. 215–231