FcBD: An Agent-Based Architecture to Support Sketch Recognition Interfaces Danilo Avola, Paolo Bottoni, Alexandru Dafinei, Anna Labella Department of Computer Science, Sapienza University of Rome Via Salaria 113, 00198 Rome, Italy {avola, bottoni, labella}@di.uniroma1.it, a dafinei@yahoo.it Abstract Sketch-based interfaces are increasingly used for in- teraction with desktops or mobile devices, favouring natural interaction through the expression of com- mands and concepts with graphical symbols. How- ever, developing this kind of human-oriented interfaces is still challenging, due to: the limited number and type of graphical symbols definable and recognizable in a device-independent way, the computational cost in- curred for recognizing complex symbols, and the differ- ent users’ styles in tracing symbols. FcBD’s agent- based architecture supports the definition and real-time recognition of sets of 2D graphical symbols, indepen- dently of the input style. FcBD’s layered organisation favours abstraction from the concretely employed de- vice, and proposes an approach to symbol recognition based on associating with each recognition agent dis- joint sets of primary and secondary features. 1 Introduction Multimodal interfaces exploit one or more human- oriented modalities (e.g. gesture, speech) to allow users natural interaction with desktop or mobile devices. Users increasingly access heterogeneous applications and services via different modalities according to envi- ronmental (e.g. indoor, outdoor) or device-dependent (e.g. screen size, computational capacity) characteris- tics. In this context, sketch-based interfaces provide a simple interaction tool that allows users to graphically express concepts and commands. The development of a sketch-based interface meets three major problems: multi-domain definition, real- time recognition and tracing style interpretation. The first problem concerns the construction of en- gines able to define and recognize a wide range of do- mains (i.e. sets of graphical symbols, called libraries). In fact, building a domain-specific sketch-based inter- face is a time-consuming operation that often requires the re-engineering and/or re-designing of the whole in- terface. Moreover, as libraries are extended with new symbols, recognition conflicts grow more relevant. As for the second problem, the computational com- plexity of the recognition algorithms usually depends on both the number of symbol constituents (e.g. a cir- cle inside a square is composed of two elements) and the number of strokes composing each basic element (e.g. a square can be drawn using four strokes/lines). Hence, a generic software architecture should support recognition processes on symbols of arbitrary complex- ity and its implementation should be as independent as possible of specific devices, applications, or services. The last problem concerns the style adopted by users when sketching a symbol: 1) users may draw any given symbol with different numbers of strokes; 2) strokes can be drawn in different directions (e.g. leftwards or rightwards); 3) over-tracing and/or cross-tracing phe- nomena lead to drawing each stroke in solid, bold, or dashed styles. A style-independent recognition proto- col is therefore desirable. This paper describes the Feature calculation Bid Decision (FcBD) agent-based architecture for sketch recognition. FcBD is equipped with two types of agents: feature evaluators and symbol recognizers, which are defined with respect to a specific library of symbols. Every time a user draws a symbol, the active evaluators work concurrently, each computing a specific feature on the drawn sketch. Recognizers read from the pool of features thus obtained only those needed to discriminate the symbol for which they are responsible, and make a bid for classifying the symbol under scrutiny. In case of conflict, a mediator agent progressively applies three solution strategies based on an adaptive choice or relaxation of features. As feature calculation is largely independent of the