Human Supervision and Control in Engineering and Music: Similarities, Dissimilarities, and Their Implications TOSHIYUKI INAGAKI, SENIOR MEMBER, IEEE, AND JOHAN STAHRE Invited Paper The human supervisory control model has been used successfully for decades and in numerous contexts to describe human interac- tion with complex technical systems. Recently, it has been argued that supervisory control modeling can also be applied in a musical context, to describe the interaction between a conductor and an or- chestra. This paper reviews a range of applications of supervisory control modeling. Further, the authors attempt to identify similar- ities and dissimilarities between its use in technical and musical settings. The aim is to identify valuable knowledge and experience from the musical field. Such experience could be used to enrich the design of interaction between humans and complex technical sys- tems. The key to successful knowledge transfer is the general appli- cability of human supervisory control modeling. Keywords—Authority in decision and action, human-centered automation, human supervisory control, intent recognition, levels and strategies of automation. I. INTRODUCTION Many complex industrial processes are semiautonomous, where computers control the processes according to direc- tives given by human operators. The configuration of such human–machine systems is called human supervisory con- trol [1]. Why are these processes semiautonomous, rather than being fully automated? A most obvious reason may be that we cannot foresee in the design phase all possible events that may happen during the expected lifetime of the pro- cesses. Thus, although designers tried to replace human op- erators by machines for higher efficiency or reliability, their attempt was not successful [2]. Actually, human operators Manuscript received January 30, 2003; revised November 7, 2003. T. Inagaki is with the Institute of Information Sciences and Elec- tronics, University of Tsukuba, Tsukuba 305-8573, Japan (e-mail: inagaki@is.tsukuba.ac.jp). J. Stahre is with the Department of Product and Production Development, Chalmers University of Technology, Göteborg SE-412 96, Sweden (e-mail: johan.stahre@hfe.chalmers.se). Digital Object Identifier 10.1109/JPROC.2004.825876 have to be on-site to perform tasks of “completing the system design,” adapting the system for situations that the designers did not anticipate [3]. It then becomes an important design decision to deter- mine what humans do and what machines do. One design strategy is to allocate to machines every function that can be automated and to allocate to operators the leftover functions to which no automation technologies are available. Another strategy may be to find an allocation that ensures econom- ical efficiency. Such design strategies are typical examples of technology-centered automation [4]. This may remind readers of C. Chaplin’s classic film Modern Times, which portrays a comic, yet unpleasant world in which seemingly intelligent machines demand humans to obey or adapt to the machines. It is human-centered automation that we need for real- ization of an environment in which humans and machines can work cooperatively in more sound and comfortable man- ners [4]–[6]. However, in spite of popularity of the term, it is still not clear what human-centered automation really means. Sheridan [7], [8] distinguishes ten different meanings of human-centered automation. He also states the possible and even probable contradictions that may be found in those “definitions” of human-centered automation [7], [8]. Human–machine systems are not yet free from prob- lems, such as: 1) loss of situation awareness, in which operators fail to grasp the process state exactly [9]; 2) automation-induced surprises, in which operators fail to understand what computers are doing, and why [10], [11]; and 3) complacency, in which operators monitor processes less often than is required (or optimal) [12], [13]. These problems tell us that human–computer interaction is not well designed even in modern processes created with highly advanced technologies. Recently, an argument has emerged on the applicability of the supervisory control concept to the musical sciences [14]. 0018-9219/04$20.00 © 2004 IEEE PROCEEDINGS OF THE IEEE, VOL. 92, NO. 4, APRIL 2004 589