Poster: Toward Context-aware Proactive Conversation for Smart Speakers Soowon Kang sw.kang@kaist.ac.kr KAIST South Korea Heepyung Kim heepkim@kaist.ac.kr KAIST South Korea Youngtae Noh ytnoh@inha.ac.kr Inha University South Korea Uichin Lee ∗ uclee@kaist.ac.kr KAIST South Korea ABSTRACT Despite the increasing popularity of novel conversational services for smart speakers, the current ones have supported limited proac- tive interactions. In this work, we focus to understand opportune moments for proactive conversational interactions in domestic con- texts. To achieve our goal, we built a speech-based experience sam- pling device and performed a 1-week feld study with 40 students living in university dormitories. From 3,572 in-situ experiences, we identifed eleven categories to fnd contextual features related to opportune moments. We showed that the key determinants for interruptibility are relevant to personal contextual factors, user mobility, and social presence. Through considering the aforemen- tioned factors, we envision that a smart speaker can intelligently manage the timing of conversations. CCS CONCEPTS · Human-centered computing → Ubiquitous and mobile com- puting; User interface management systems. KEYWORDS Smart Speakers, Conversational Interaction, Interruptibility ACM Reference Format: Soowon Kang, Heepyung Kim, Youngtae Noh, and Uichin Lee. 2021. Poster: Toward Context-aware Proactive Conversation for Smart Speakers. In Ad- junct Proceedings of the 2021 ACM International Joint Conference on Per- vasive and Ubiquitous Computing and Proceedings of the 2021 ACM Inter- national Symposium on Wearable Computers (UbiComp-ISWC ’21 Adjunct), September 21–26, 2021, Virtual, USA. ACM, New York, NY, USA, 3 pages. https://doi.org/10.1145/3460418.3479306 1 INTRODUCTION Recently, top shares in the smart speaker market (e.g., Amazon Echo [9], and Google Home [11]) started to pay attention to the ∗ This is the corresponding author. UbiComp-ISWC ’21 Adjunct, September 21–26, 2021, Virtual, USA © 2021 Copyright held by the owner/author(s). This is the author’s version of the work. It is posted here for your personal use. Not for redistribution. The defnitive Version of Record was published in Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers (UbiComp-ISWC ’21 Adjunct), September 21–26, 2021, Virtual, USA, https: //doi.org/10.1145/3460418.3479306. proactive services, while the current smart speakers have generally supported reactive services via voice commands. Although proac- tive services allow the users to obtain useful information, prior studies also informed that timing and relevance of the services are crucial to the user experience [1, 3]. To the best of our knowledge, the interruptibility studies of smart speakers in home contexts lacks depth in spite of the popularity of proactive services in smart speak- ers. In this study, we explained the opportune moments for the proactive services of smart speakers in diverse activities at home. To collect in-situ experiences in home environment, we imple- mented a smart speaker that supports recording user contexts and responses via a voice-based experience sampling method (ESM). Our smart speaker asked users to answer the question, łIS now a good time to talk?ž in yes or no, with detailed contextual informa- tion. The 3,572 ESM responses were collected from 20 pairs (n=40) living in dormitories. We examined the responses by conducting the exit interview to understand the core factors that afects interrupt- ibility related to home activities. Thus, we identifed the following factors relevant to the user interruptibility of conversational inter- actions: (i) personal contextual factors (e.g., engagement, urgency, psychological/physical states, auditory/verbal channel availability), (ii) movement-related factors (e.g., entrance/departure behaviors, ac- tivity switching), and (iii) social presence (e.g., roommate’s current activity). Our fndings emphasize that the proactive conversation management is required to the proactive conversational services for smart speakers. 2 BACKGROUND Traditional smart speakers have mainly provided reactive services. The recent commercial smart speakers, including Amazon and Google, started to look into several proactive services. Amazon Echo supported friendly reminders (e.g., informing to lock the door) and ambient sensing to warn the user of safety issues (e.g., win- dow break) [9]. Google Home can notify the upcoming events that depend on the user’s Google Calendar [11]. These prior research on smart speakers supported limited chances to proactively interact with users. One important factor of intelli- gent services is detecting the appropriate time to open the conver- sation session. Researchers have been studied the interruptibility in wide contexts (e.g., smartphone [8], in-vehicle [7]). However, home environment is more challenging since it is more dynamic compared to other environments due to multi-user, mobility factors, 38