MARKER AND CHALKBOARD REGIONS Michael N. Wallick, Rachel M. Heck and Michael L. Gleicher {michaelw,heckr,gleicher@cs.wisc.edu} University of Wisconsin-Madison Department of Computer Science 1210 West Dayton Street Madison, WI 53706 USA ABSTRACT When recording a classroom lecture, it is useful to capture the writing on the board for processing by applications. To provide more useful information for applications, we group together strokes based on the meaning of the writ- ing. However, it is currently not possible to automatically obtain this meaning, or general semantic information au- tomatically. Instead we rely on the structure of writing to approximate regions of the board, where each region rep- resents a single thought or idea. These regions provide an abstraction of the board that can be used in several appli- cations, such as a note taker, a lecture indexing program, an automatic video editor, and a program for creating mul- timedia presentations. 1. INTRODUCTION Many lectures center around a lecturer writing on a board. Applications that process this writing can potentially add value to lecture recordings. For example, the writing might be re-arranged to provide a transcript of the lecture (auto- matic note-taking), elements of the writing can be used as indices into the lecture, or information about the writing can be used to aid the creation of novel presentations of the lecture. In order to perform processing on the board, the writing must be represented at an appropriate granularity. The processing that can be done on the entire board at once is extremely limited. Likewise representing writing solely as individual strokes reduces the types of processing that can be done. Moving a single stroke can destroy the mes- sage that the writing conveys. For example, consider a stroke that represents a character in a word. Moving the stroke not only changes the word it was removed from but also the words it was moved close to. To perform nontriv- ial processing operations, strokes can be combined into groups that can be manipulated independently. We call such groups regions. Regions can serve as the atomic ob- jects in board processing applications, ensuring that re- lated writing is kept and processed together. Ideally, a region would represent a complete thought or idea, for example a sentence or a diagram. Unfortunately, the information available for forming regions is quite lim- ited. While spatial and temporal information about the individual strokes is provided by most board capture tech- nologies, acquisition of higher-level information, such as word recognition or semantic interpretation, is oten in- feasible. Therefore, an automatic process for grouping writing into regions must rely on spatio-temporal data and heuristics about how people write on boards. In this paper we present a method for grouping writing into regions. The region model is simple enough that re- gions can be obtained automatically and processed effi- ciently. Our method relies only on readily available in- formation about the strokes and does not require writing interpretation. Unlike similar methods, our region model contains a temporal element, that provides information about the state of the region through time. In the event that no other capture technology is available, we present a method for obtaining strokes from a video of people writ- ing on the board. We also present several sample applica- tions that take advantage of the region concept. The remainder of this paper is organized as follows. In the next section, 2, we discuss other efforts related to our regions concept. In Section 3 we formally define regions, presenting the lifespan of a region and heuristics for form- ing regions. Our algorithm for forming the regions is pre- sented in Section 4. We describe several sample applica- tions in Section 5. 2. RELATED WORK Several researchers have considered the idea of grouping writing together on the board either in service of some larger specific application or as a first class object in itself. The major difference between our regions and the work of others is that regions provide a temporal model which de- scribes the state of a region through time. The ScanScribe system by Saund et. al. [11] groups writing together, in a similar manner to regions. However, their system requires Proceedings of Mirage 2005, INRIA Rocquencourt, France, March, 1-2 2005 223