From handwriting analysis to pen-computer applications by L. Schomaker In this paper, pen computing, i.e. the use of computers and applications in which the pen is the main input device, will be described from four different viewpoints. Firstly a brief overview of the hardware developments in pen systems is given, leading to the conclusion that the technological developments in this area have not led to the expected user acceptance of pen computing. The reasons underlying this market failure are explored. Problems of pen-user interface design are then described and existing and new applications are summarised. The handwriting process and product are discussed and, finally, automatic recognition methodologies are considered. Four basic factors determining handwriting variation and variability are identified. A handwriting recognition approach using segmentation into velocity-based strokes is considered in somewhat more detail. 1 Introduction Handwriting recognition and pen computing are characterised by an arduous evolution history. Originally identified 30 years ago as a first step towards the more difficult problem of speech recognition, the automatic recognition of unconstrained, natural handwriting is today still a difftcult and scientific challenge. Automatic handwriting-recognition performance profits only indirectly from technological advances such as increased computing power. The inherent variation of styles and the variability in a writer’s behaviour require (a) fundamental insight of the handwriting production process, (b) domain knowledge on the nature of script pattern geometry and (c) powerful algorithms which display both noise tolerance and the ability to integrate multilevel information sources. The handwriting-recognition research groups around the world are very small as compared to the effort spent in speech recognition. Still, the appeal of the idea that written words can be transformed into a neat machine-print font and handled by a computer is so strong that university groups are trying to tackle this problem again and again. Similarly, companies try to put forward pen-based computers, with limited or varying success. Why is it so difficult to translate the relatively simple idea of ‘writing on a computer’ into a reliable, easy and attractive system? It appears that the integration of pattern recognition modules into usable applications is far from trivial. The market failure of pen computing in the early nineties played an important role in motivating a reassessment of pen- computing technology at a number of levels. In this paper, four different aspects of handwriting recognition and pen computing are presented: pen-computing hardware, software and user interfaces, the handwriting process and product and recognition of on-line handwriting. 2 Pen-computing hardware The late sixties and early seventies witnessed the birth of a wide rang of XY-position-sensing devices. These transducers used either resistive, capacitive, electro- magnetic, acoustic or pressure-sensitive technologies for the measurement of pen-tip position as a function of time. The technological developments enabled accurate planar position-sensing, such as was needed for graphical input in computer-aided design (CAD), especially in the automotive industry. Fig. 1 is a schematic diagram of the electromagnetic approach using tetherless pens. Visionary ideas, such as Alan Kay’s’ Dynabook (1968)) gave the impetus to a new hardware development: the integration of position-sensing technology with graphical display technology to provide a form of ‘electronic paper’ (EP). Early experiments involved standard cathode ray tube (CRT) screens, which were embedded in an office desk and equipped with some form of position sensing. In the mid eighties, the first real electronic-paper prototypes appeared: the British National Physical Laboratory (NPL) produced a plasma display with an integrated XI’ tablet, and IBM developed early prototypes of electronic paper. ELECTRONICS & COMMUNICATION ENGINEERING JOURNAL JUNE 1998