Four Paradigms for Indexing Video Conferences Rick Kazman and Reem Al-Halimi University of Waterloo Meetings in which participantsare linked by video, audio, and shared computer appli- cations produce several parallel informationstreams. We created a meeting indexer, Jabber, that uses content-based indexing of the audio stream to access these parallel streams. It performs speech recognition on the audio stream, then groups the recognizedwords into semantically linked trees. The user interface is designed to display informa- tion with minimal distraction during meetings. William Hunt and Marilyn Mantei University of Toronto s research and market priorities increasingly emphasize multimedia, videoconferencing, computer-sup- A ported cooperative work, and tele- presence, automatic methods become essential for organizing the enormous amounts of data that such shared work creates. Uncompressedreal-time video alone can produce 150 kilobytes to 1.2 megabytes of data per second. This data is typi- cally stored as a linear stream or, in the case of a multipoint video conference, as a set of parallel streams. Such enormous bodies of information, although valuable, are intractably locked in their inherently linear video format. Occasionally, videotaped meetings are indexed so that they may be queried like a database or browsed like an encyclopedia, but only through large amounts of manual labor. There exists little research into the management and automatic indexing of video repositories of meetings. In par- ticular, we need ways to 1. organize and index meeting records so that people can access the stored information in a semantically meaningful, efficient way, and 2. provide user interfaces that make the annota- tion and retrieval of this information simple, natural, and effective. To address these needs, we built an indexing system called Jabber. We wanted to explore both indexing techniques and the human-computer interaction issues involved in annotating, brows- ing, and searching through large information repositories. Previous work has focused on struc- turing multimedia information in highly con- strained domains (such as television news stories'), but little work has targeted structuring arbitrary multimedia repositories. This article describes several techniques for structuring, and thus providing more convenient random access to, arbitrary multimedia information. We also address how the user will interact with this infor- mation. Although we concentrate on indexing distributed meetings, many types of multiperson interactions would benefit from indexing-lec- tures, contract negotiations, requirements elicita- tion, and joint work sessions, to name a few. The problem Meetings use extensiveorganizational resources and contain an enormous amount of corporate knowledge, culture, and decisions. Large software development projects, for example, regularly spend fifty percent of their budgets on the interactive specification and design process. Because meetings take participants away from their normal work environments, often with significant travel and time costs incurred, organizations are becoming more interested in supporting long-distance meet- ings through technology. Products that address this concern, such as ProShare from Intel, Aspects from Group Technologies, and Vis-a-vis from World- linx, are already commercially available. Typically, however, meetings are either not recorded or are recorded in an abstracted, inter- preted form (through an agenda and a set of min- utes, for example). Given that meetings consume so much personnel time and contain such impor- tant corporate information, more detailed record- ing mechanisms are needed. Our research indicates that post-meeting recall and shared understanding among participants on what tran- spired at a meeting is rather poor. Current distrib- uted meeting support products allow corporations to maintain detailed video and audio recordings of meetings, but not without creating new prob- lems, including providing efficient access to these records later. Keeping control of meetings Meetings are complex human interactions that impose a large cognitive load on the individual 1070-986X/96/$5.00 0 1996 IEEE