When More Is Less: The Paradox of Choice in Search Engine Use Antti Oulasvirta, Janne Hukkinen Helsinki Institute for Information Technology HIIT Helsinki University of Technology TKK Barry Schwartz Psychology Department Swarthmore College ABSTRACT In numerous everyday domains, it has been demonstrated that in- creasing the number of options beyond a handful can lead to pa- ralysis and poor choice and decrease satisfaction with the choice. Were this so-called paradox of choice to hold in search engine use, it would mean that increasing recall can actually work counter to user satisfaction if it implies choice from a more exten- sive set of result items. The existence of this effect was demon- strated in an experiment where users (N=24) were shown a search scenario and a query and were required to choose the best result item within 30 seconds. Having to choose from six results yielded both higher subjective satisfaction with the choice and greater confidence in its correctness than when there were 24 items on the results page. We discuss this finding in the wider context of “choice architecture”—that is, how result presentation affects choice and satisfaction. Categories and Subject Descriptors H.1.2 [User/Machine Systems]: Human information processing. H.3.3 [Information Search and Retrieval]: Information filtering. General Terms Design, Human Factors. Keywords Search engines, relevance judgments, satisfaction, user interfaces. 1. INTRODUCTION If you type in your favorite pop singer’s name to Google, you will be presented with a result set of possibly millions of items. Items within a single page may have perceivable differences, yet the better the engine has done its job, the greater the number of items that will appear relevant. In such a case, can you be content with the link you finally choose, given that you could not consider even an iota of the full number of results available? At the time of writing, Google offered 99,500,000 results for the query “Britney Spears.” The situation is not that different from what Westerners face daily in the offline domain: massive choice. For example, wanting to buy breakfast cereal at a grocery store forces a choice from among some 273 products [28]. Figure 1: Does it matter how many search results are pre- sented? Six-item (left) versus 24-item (right) result listings in Google, materials used in the experiment. Note: In the 24-item list, the final three items are shown on a second page. Recent research in cognitive psychology has revealed an interest- ing effect of choice overload: The paradox of choice: providing more options—particularly if they are highly relevant and success is personally important—will lead to poorer choice and degrade satisfaction [28]. Experimental demonstrations of this paradox are quite compelling and bespeak its generality. For example, passersby are more likely to buy jams on display, and more satisfied as customers when there are six jams to choose from than 24 [15]. University students are more likely to write an extra-credit essay, and write better essays, when they have six topics to choose from than 30 [15]. Employees are more likely to participate in 401(k) retire- ment plans when there are two rather than 59 funds to choose from [16]. But would the same apply to Google with, say, six ver- sus 24 items? Figure 1 illustrates the situation. The existence of this phenomenon could have important implica- tions for how we think about search engine use. One presumption has been that if the user has the persistence to go through the re- sult set, or a sufficient part of it, a larger number of items on the list indicates greater likelihood that (s)he has encountered an item of higher relevance as the end is reached. Ergo, the more results, the higher the effectiveness. If this assumption turns out to be questionable, we can ask whether search engines should be less like slavish “reporters” and more akin to personal assistants who guide customers to the most reasonable options in a store. However, anyone can imagine a number of reasons for the effect not appearing in search engine use. For example, if users jump to Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that cop- ies bear this notice and the full citation on the first page. To copy other- wise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SIGIR’09, July 19–23, 2009, Boston, Massachusetts, USA. Copyright 2009 ACM 978-1-60558-483-6/09/07...$5.00.