Dynamic Composition of Process Federations for Context Aware Perception of Human Activity James L. Crowley and Patrick Reignier Laboratoire GRAVIR-IMAG, INRIA Rhône-Alpes Grenoble, France Abstract—This paper describes a distributed software model for context-aware perception of human activity. The basic building blocks in this model are perceptual modules, composed of a data transformation component and a control component. Modules are assembled into perceptual processes controlled by a reflexive process controller. Process controllers regulate computation, and provide a reflexive description of their internal state and capabilities. Explicit models of context are used to assemble federations of processes for observing and predicting activity. As context changes, the federation is restructured. Restructuring the federation enables the system to adapt to a range of environmental conditions and to provide services that are appropriate over a range of activities. 1. I NTRODUCTION In this paper, we describe a data-flow architecture based on dynamically assembled federations [1], [2]. Our model builds on previous work on process-based architectures for machine perception and computer vision [3], [4], as well as on data flow models for software architecture [5]. We propose a model in which a user’s context is described by a set of roles and relations. A context is translated into a federation of processes for observing the entities that satisfy roles as well as the relations between these entities. This model leads to an architecture in which reflexive elements are dynamically composed to form federations of processes for observing and predicting the situations that make up a context. As context changes, the federation is restructured. Restructuring the federation enables the system to adapt to a range of environmental conditions and to provide services that are appropriate over a range of activities. The result is a software architecture for building systems that act as a silent partner to assist humans in their activities in order to provide appropriate services without explicit commands and configuration. 2. MODULES, PROCESSES AND FEDERATIONS The most basic unit in our system is a module. A module us defined as a transformation applied to a synchronous data stream of to asynchronous events. The transformation may depend on a set of parameters. The data stream may be accompanied by meta-data. In our model, all modules are designed with the capability report on their state. Examples of module state include computation time and quality of result. Module state is discussed below. State Events Data Events Data Parameters Transformation Fig. 1. Modules are defined as transformation over events and data. Modules are assembled into processes, shown in figure 2. State and capabilities Control Transformation Events Data Events Data Control in Fig. 2. A Observational process combines transformation with a control component. A process has two functional facets: A transformation component and a control component. As with modules, the transformation component may be defined to transform data received in a synchronous stream or asynchronous events. The transformation component of a process is generally a composition of transformations provided by modules. The input data to the transformational component is generally composed of some raw numerical values, generally arriving in a synchronous stream, accompanied by meta-data. Meta data includes information such as a time-stamp, a confidence factor, a priority or a description of precision. An input event is a symbolic message that can arrive asynchronously and that may be used as a signal to begin or terminate the transformation of the input data. Output data and the associated meta-data is a synchronous stream produced from the transformation of the input data. We also allow the possibility of generating asynchronous output messages that may serve as events for other processes. This model is similar to that of a contextor [28], which is a conceptual extension of the context widget implemented in the Context Toolkit [29]. The control component of a process enables reflexive control of observational processes and thus provides a number of important functions. The control component receives commands and parameters, supervises the execution of the