Homogeneity Analysis for Object-Action Relation Reasoning in Kitchen Scenarios * Hanchen Xiong Sandor Szedmak Justus Piater Institute of Computer Science, University of Innsbruck Technikerstr.21a A-6020, Innsbruck, Austria {hanchen.xiong,sandor.szedmak,justus.piater}@uibk.ac.at ABSTRACT Modeling and learning object-action relations has been an active topic of robotic study since it can enable an agent to discover manipulation knowledge from empirical data, based on which, for instance, the effects of different actions on an unseen object can be inferred in a data-driven way. This paper introduces a novel object-action relational model, in which objects are represented in a multi-layer, action- oriented space, and actions are represented in an object- oriented space. Model learning is based on homogeneity analysis, with extra dependency learning and decomposi- tion of unique object scores into different action layers. The model is evaluated on a dataset of objects and actions in a kitchen scenario, and the experimental results illustrate that the proposed model yields semantically reasonable interpre- tation of object-action relations. The learned object-action relation model is also tested in various practical tasks (e.g. action effect prediction, object selection), and it displays high accuracy and robustness to noise and missing data. 1. INTRODUCTION This copy is for personal use, the final version will be publisehd in the 2nd Workshop on Machine Learning for Interactive Systems by ACM International Conference Proceedings Series Manipulations of objects are core and indispensable func- tions in robotic systems to fulfill various practical tasks. However, because of the diversity of real-world objects in shape, material and other properties, manipulation design at the instance level is very effort-consuming and thus pro- hibitive. Learning principles or correlation patterns of dif- ferent actions based on trial experiences is an appealing di- rection of robotics research. In other words, an agent can acquire knowledge of object-action relations in a data-driven manner by making use of a limited number of experiments. * The research leading to these results has received fund- ing from the European Community’s Seventh Framework Programme FP7/2007-2013 (Specific Programme Coopera- tion, Theme 3, Information and Communication Technolo- gies) under grant agreement no. 270273, Xperience. Figure 1: A sample set of kitchen objects In addition, the study of object-action relations has also at- tracted attention within the cognition and psychology com- munities [5, 10], since it is expected to be related to how human beings accumulate knowledge by physically interact- ing with different objects. Humans begin to interact with their environment in their infancy, and in many interactions, two elements are involved: objects and actions. Actions are executed on objects with the humans’ motor capabili- ties, and the effects of these actions are observed with their perception abilities. Based on such repeated interactions, human beings can quickly acquire object-action knowledge, and easily fulfill different actions on various objects by trans- ferring such knowledge to novel objects. Although the exact mechanism of how the human brain organizes and learns object-action relations is still unknown, it has been pointed out that computational modeling of object-action relations can be a plausible perspective for the study of both robotics and human cognition. Nevertheless, modeling and learning object-action relations has been a difficult task. The difficulties mainly stem from two sources. First, the structure of descriptions of both ob- jects and actions can be very complex. The descriptions are derived from several sources, and the corresponding feature spaces are high-dimensional (i.e., objects and actions are characterized by large numbers of parameters). The second difficulty is due to the small number of experiments which can confirm the effects of different actions on objects. Even worse, in some cases, the experiments might provide con- tradicting outcomes. In consequence, the empirical data are rather sparse and noisy. In this paper we put forward a novel model of object-action relations, in which objects are represented in a multi-layer action-oriented space, and actions are represented in an object- oriented space. The object-action relations are encoded in