Being A Turker David Martin, Benjamin V. Hanrahan, Jacki O’Neill Xerox Research Centre Europe 6 chemin de Maupertuis, Grenoble France {david.martin, ben.hanrahan, jacki.oneill}@xrce.xerox.com Neha Gupta University of Nottingham University Park NG7 2TD Nottingham neha.gupta@xrce.xerox.com ABSTRACT We conducted an ethnomethdological analysis of publicly available content on Turker Nation, a general forum for Amazon Mechanical Turk (AMT) users. Using forum data we provide novel depth and detail on how the Turker Nation members operate as economic actors, working out which Requesters and jobs are worthwhile to them. We show some of the key ways Turker Nation functions as a community and also look further into Turker-Requester relationships from the Turker perspective – considering practical, emotional and moral aspects. Finally, following Star and Strauss [25] we analyse Turking as a form of invisible work. We do this to illustrate practical and ethical issues relating to working with Turkers and AMT, and to promote design directions to support Turkers and their relationships with Requesters. Author Keywords Ethnomethodology; content analysis; crowdsourcing; microtasking; Amazon Mechanical Turk; Turker Nation. ACM Classification Keywords H.5.3 Group and Organizational Interfaces – Computer, Supported Cooperative Work General Terms Human Factors INTRODUCTION The concept of crowdsourcing was originally defined by Jeff Howe of Wired Magazine as “the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call.” [8] This ‘undefined network of people’ is the key topic of this article. We present the findings of an ethnomethodological analysis of posts and threads on a crowdsourcing forum called Turker Nation 1 . We have sought to understand members of the crowd – their reasoning practices, concerns, and relationships with requesters and each other – as they are shown in their posts on the forum. We seek to present them as faithfully as possible, in their own words, in 1 http://turkernation.com/forum.php order to provide more definition to this network of people. We believe that this will be beneficial for researchers and businesses working within the crowdsourcing space. Crowdsourcing encompasses multiple types of activity: invention, project work, creative activities, and microtasking. This latter is our focus here. The most well- known microtask platform is Amazon Mechanical Turk (AMT) 2 , and the Turker Nation forum that we studied is dedicated to users of this platform. The basic philosophy of microtasking and AMT is to delegate tasks that are difficult for computers to do to a human workforce. This has been termed ‘artificial artificial intelligence’. Tasks like image tagging, duplicate recognition, translation, transcription, object classification, and content generation are common. ‘Requesters’(the AMT term for people who have work to be completed) post multiple, similar jobs as Human Intelligence Tasks (HITs), which can then be taken up by registered ‘Turkers’. Turkers (termed ‘Providers’ by AMT) are the users completing the HITs, which typically take seconds or minutes paid at a few cents at a time. For Amazon, the innovative idea was to have an efficient and cost effective way to curate and manage the quality of content on their vast databases (weeding out duplicates, vulgar content, etc.). While Amazon is still a big Requester, AMT has been deployed as a platform and connects a wide variety of Requesters with up to 500,000 Providers. However, Fort et al. [6] have performed an analysis on the available data and suggest that real number of active Turkers is between 15,059 and 42,912; and that 80% of the tasks are carried out by the 20% most active (3,011–8,582) Turkers. While these numbers are useful, the research community still has little deep qualitative knowledge about this workforce. Questions remain unanswered such as: how and what do they look for in jobs; what are their concerns; and how do they relate to requestors? LITERATURE REVIEW To date much of the research on AMT takes the employers’ perspective, e.g. [14, 15, 17, 18], and this has in turn been highlighted [6, 16]. Silberman et al. [23] note that this mainstream research looks at how: “[to] motivate better, cheaper and faster worker performance […] to get good 1 http://turkernation.com/forum.php 2 http://www.mturk.com Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CSCW ’13, February 23–27, 2013, San Antonio, Texas, USA. Copyright 2013 ACM 978-1-4503-1331-5/13/02...$15.00.