Robot Navigation Using Qualitative Landmark States from Sketched Route Maps George Chronis Computer Science University of Missouri - Columbia chronisg@missouri.edu Marjorie Skubic Electrical and Computer Engineering University of Missouri – Columbia skubicm@missouri.edu Abstract—The goal of this work is to illustrate and evaluate a novel method for communicating with a mobile robot, namely, by drawing a sketch. The user draws a sketched route map to direct a mobile robot along a specified path. In this paper we focus on the navigation of the sketched path in the real environment. Challenges include sketch inaccuracies such as distortion or abstraction and low sensory resolution of the robot. Our method is based on utilizing spatial relations to extract a sequence of qualitative landmark states from the sketched map, which in turn the robot follows in a real environment to replicate the sketched route. Several examples are included. Keywords- sketch-based navigation, human-robot interaction, spatial relations, histogram of forces I. INTRODUCTION The goal of this work is to illustrate and evaluate a novel method for communicating with a robot, namely, by drawing a sketch. The user sketches a map on a PDA screen and a desired route for the robot to follow. We show in this paper how to make a robot follow the drawn path, without using absolute coordinates from the sketch. Instead, we extract qualitative landmark states and that involve spatial relations between the robot and the surrounding objects. A robot reaching the desired goal by following a specified path is a good indication that the extracted landmark states were adequate and that the robot could successfully follow the desired route. Our navigation approach is shown to handle inaccuracies while sketching, such as skewing, scaling or omission of objects with respect to the real environment. Some limited work has been done in using sketches to direct movement. Igarashi et al. proposed a path drawing technique overlaid on a virtual scene, as a means of specifying a route through the virtual environment [1]. Ferguson et al. developed a sketch interface for military course-of-action diagrams, which are used for strategic planning [2]. Freksa et al. proposed the use of a schematic map for directing robot navigation [3]. A schematic map is described as an abstraction between a sketch map and a topological map; an example is a subway map. The closest work is that of Kawamura et al. in which the user specifies a robot path by selecting via points on a sketch of the environment [4]. Artificial landmarks are placed in the scene and on the sketch to act as landmarks in the navigation process. Our previous work [5][6] discusses the use of a sketched route map for directing mobile robot navigation. A route is extracted from the sketched path in the form of a sequence of landmark states with corresponding actions, where each landmark state is a qualitative condition based on the spatial relationship of landmarks relative to the robot. The previous work was adequate for simple map configurations but was not robust in more complex environments. In this paper we present significant improvements to the navigation methods, which utilize the extracted landmark states. The enhancements presented in this paper support more complex sketches as well as more abstract sketches, i.e. sketches that have scaling differences from the real environment (due to the inaccuracies of the sketch). We have tested the system with sketches drawn by different users (collected as part of a user study). In addition, we have varied the navigation environment to illustrate the ability of the robot to adapt and navigate robustly in spite of these variations. Examples are included in the paper. In the next section we provide background information, and in Sec. III, we explain our approach and the improvements over previous algorithms. Sec. IV presents experimental results using a Nomad 200 robot simulator. Finally, in Sec. V, we summarize and discuss future plans of software enhancements and integration with other frameworks. II. BACKGROUND A sketched route map is drawn to help someone navigate along a path for the purpose of reaching a goal. An example is shown in Fig. 1(a). Although route maps do not generally contain complete map information about a region, they do provide relevant information for the navigation task. People sketch route maps to include landmarks at key points along the path and use spatial relationships to help depict the route, often adding arrows and other notation for clarity [7]. The depiction of the environment structure is not necessarily accurate and may even distort the actual configuration [8]. For example, a 60-degree turn in the physical environment may be sketched as a 90-degree turn. However, as the route follower navigates in the real environment, his motion is constrained by the environment so that the distortion is corrected and the route can be completed. Research by Michon and Denis [9] provides insights into how landmarks are used for human navigation and what are considered to be key route points. In studying route directions, they found that landmarks were used more frequently at four types of critical nodes: (1) at the start, (2) at the end, (3) at a