Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations Jennifer Hill a,⇑ , W. Randolph Ford b,1 , Ingrid G. Farreras c,2 a School of Engineering and Applied Science, The George Washington University, 800 22nd St. NW, Washington, D.C. 20052, USA b Data Analytics Program, Graduate School, 3501 University Boulevard East, University of Maryland University College, Adelphi, MD 20783, USA c Psychology Dept., Hood College, 401 Rosemont Ave., Frederick, MD 21701, USA article info Article history: Keywords: CMC Instant messaging IM Chatbot Cleverbot abstract This study analyzed how communication changes when people communicate with an intelligent agent as opposed to with another human. We compared 100 instant messaging conversations to 100 exchanges with the popular chatbot Cleverbot along seven dimensions: words per message, words per conversation, messages per conversation, word uniqueness, and use of profanity, shorthand, and emoticons. A MANOVA indicated that people communicated with the chatbot for longer durations (but with shorter messages) than they did with another human. Additionally, human–chatbot communication lacked much of the richness of vocabulary found in conversations among people, and exhibited greater profanity. These results suggest that while human language skills transfer easily to human–chatbot communication, there are notable differences in the content and quality of such conversations. Ó 2015 Elsevier Ltd. All rights reserved. 1. Introduction Artificial intelligence’s (A.I.) efforts in the last half century to model human language use by computers have not been wildly successful. While the idea of using human language to communi- cate with computers holds merit, A.I. scientists have, for decades, underestimated the complexity of human language, in both com- prehension and generation. The obstacle for computers is not just understanding the meanings of words, but understanding the end- less variability of expression in how those words are collocated in language use to communicate meaning. Nonetheless, decades later, we can find an abundance of natural language interaction with intelligent agents on the internet, from airline reservation systems to merchandise catalogs, suggesting that humans have little or no difficulty transferring their language skills to such applications. Because so much of this communication occurs through digital technology rather than in person, computer- mediated communication (or ‘‘CMC’’) has become a prominent area of research in which to explore this simulation of natural human language. One of the most popular forms of CMC today, particularly among adolescents and teenagers, is instant messaging (IM) (Tagliamonte & Denis, 2008). While many specialized applications enable instant messaging, the service is also provided through many other popular media, such as multiplayer online games, email clients, and social networking websites (Varnhagen et al., 2009). Several studies have compared IM and other forms of CMC to other forms of language. Ferrara, Brunner, and Whittemore (1991) determined that CMC possesses uniquely distinguishing linguistic features that display qualities of both written and spoken dialogue. Compared to other standard forms of communication, CMC’s most distinctive trait is its unique, shortened-form language of acronyms and abbreviations, and an informal discursive style that is similar to face-to-face spoken language (Werry, 1996). CMC differs from spoken communication, however, in its lack of cues from features such as body language, communicative pauses, and vocal tones (Hentschel, 1998). Despite this absence of cues, however, CMC has been found to be able to communicate emotion as well as or better than face-to-face communication (Derks, Fischer, & Bos, 2008). Although CMC has been compared to other forms of communi- cation, few studies have compared different forms of CMC to one another. Perhaps the most noteworthy of these studies is Baron’s (Baron, 2007) comparison of the linguistic characteristics of IM and text (or SMS) messages – another form of CMC – among http://dx.doi.org/10.1016/j.chb.2015.02.026 0747-5632/Ó 2015 Elsevier Ltd. All rights reserved. ⇑ Corresponding author. Tel.: +1 301 693 4579. E-mail addresses: jenhill@gwu.edu (J. Hill), rand.ford@umuc.edu (W. Randolph Ford), farreras@hood.edu (I.G. Farreras). 1 Tel.: +1 240 684 5606. 2 Tel.: +1 301 696 3762. Computers in Human Behavior 49 (2015) 245–250 Contents lists available at ScienceDirect Computers in Human Behavior journal homepage: www.elsevier.com/locate/comphumbeh