Discourse Processes Understanding indirect reference in visual context --Manuscript Draft-- Manuscript Number: DP-D-16-00054R2 Full Title: Understanding indirect reference in visual context Article Type: Original Article Corresponding Author: Juhani Järvikivi, PhD University of Alberta CANADA Corresponding Author Secondary Information: Corresponding Author's Institution: University of Alberta Corresponding Author's Secondary Institution: First Author: Juhani Järvikivi, PhD First Author Secondary Information: Order of Authors: Juhani Järvikivi, PhD Sarah Schimke, PhD Pirita Pyykkönen-Klauck, PhD Order of Authors Secondary Information: Abstract: We often use pronouns like it or they without explicitly mentioned antecedents. We asked whether the human processing system that resolves such indirect pronouns utilizes the immediate visual-sensory context in multimodal discourse. Our results showed that people had no difficulty understanding conceptually central referents, whether explicitly mentioned or not, whereas referents that were conceptually peripheral were much harder to understand when left implicit than when they had been mentioned before. Importantly, we showed that people could not recover this information from the visual environment. The results suggest that the semantic- conceptual relatedness of the potential referent with respect to the defining events and actors in the current discourse representation is a determining factor of how easy it is to establish the referential link. The visual environment is only integrated to the extent that it is relevant or acts as a fall-back when the referential search within the current discourse representation fails. Powered by Editorial Manager® and ProduXion Manager® from Aries Systems Corporation