Title: Language learning with meanings as stored sensorimotor sequences: a connectionist model Authors: MartiŶ Takáč 1 , ĽuďiĐa Beňuškoǀá 1,2 , Alistair Knott 2 Affiliations: 1 Centre for Cognitive Science, Faculty of Mathematics, Physics and Informatics Comenius University, Bratislava, Slovakia. 2 Dept. of Computer Science, University of Otago, Dunedin, New Zealand E-mail: takac@ii.fmph.uniba.sk, lubica@cs.otago.ac.nz, alik@cs.otago.ac.nz Abstract We will present a connectionist model of language acquisition based on a novel idea that semantic representations are stored as sequences that reflect sensorimotor processing. IŶ aĐĐordaŶĐe ǁith „eŵďodied cognition perspective, we believe that high-level cognitive processes take place in agents situated in a physical world, whose experiences are shaped by their sensory-motor interactions with the world. At a timescale around a third of a second, these interactions often take the form of short sequences of sensorimotor operations with well-defined structure, as first noticed by Ballard et al. (1997) who named them deictic routines. A deictic routine relies on actions of attention mediated by saccadic eye movements that evoke transitory representations in sensorimotor areas of the brain. These evoked representations can be used to plan the next saccade or some other motor action, which will have its own attentional consequences, etc. This idea has been further elaborated by Knott (2012) who argues that experiencing concrete transitive actions such as reaching to grasp an object yields a sensorimotor process structured as a deictic routine, namely a canonical sequence of operations attending to the agent of the action, attending to the target, and monitoring of the action, which itself involves reattending to the agent and the object in a different way. This sequence is the same whether the experiencer experiences performing an action himself, or observes another agent doing so. Knott’s book includes a detailed review of empirical evidence supporting this claim. Drawing on this idea, we created a connectionist model of language acquisition that represents meaning of sentences as canonical temporal sequences of sensorimotor signals. The model assumes a learning mode of a child exposed to sentences of some target language paired with their meanings. The task of the child is to learn to generate correct sentences for given meanings (including meanings never seen during training). Because sentences are sequences of words, the task is to learn mapping from one temporal sequence (meaning) to another (sentence). This task is not straightforward, as the elements are not in perfect synchrony: while the semantic sequence has a cross-linguistically universal structure, the word sequence can feature different word- orders and grammatical rules in different target languages and can contain idioms (fixed expressions in which a single meaning is expressed by multiple words). Moreover, the semantic sequence contain multiple occurrences of the episode constituents, hence the system has to learn on which occasion to express the constituent by an overt word, and when to suppress overt generation. The model has been trained and tested on six artificial languages featuring different canonical word orders of simple transitive sentences (SVO, SOV, VSO, VOS, OSV, OVS) and in all cases it was able to successfully acquire abstract word-order rules, surface idiomatic constructions, and the morphology of the target language. More details of the model architecture and simulation results can be found in Takac et al (in press). References Ballard, D., Hayhoe, M., Pook, P., Rao, R.: Deictic codes for the embodiment of cognition. Behavioral and Brain Sciences 20 (1997) 723-767. Knott, A.: Sensorimotor cognition and natural language syntax. MIT Press, Cambridge, MA, 2012. Takac, M., Benuskova, L., Knott, A.: Mapping sensorimotor sequences to word sequences: A connectionist model of language acquisition and sentence generation. Cognition, in press.