52 1541-1672/07/$25.00 © 2007 IEEE IEEE INTELLIGENT SYSTEMS Published by the IEEE Computer Society I n t e r a c t i n g w i t h A u t o n o m y Operator Performance and Intelligent Aiding in Unmanned Aerial Vehicle Scheduling Mary L. Cummings and Amy S. Brzezinski, Massachusetts Institute of Technology John D. Lee, University of Iowa U nmanned vehicles (UVs) are quickly becoming ubiquitous in almost every aspect of hostile-environment operations. For example, with reduced radar signatures, increased endurance, and the removal of humans from immediate threat, unmanned aer- ial vehicles (UAVs) have become indispensable assets to militarized forces around the world. Remotely guided underwater vehicles have many potential military applications; in addition, the oil and gas industry uses them for maintenance. Sur- face water UVs are under development and testing for harbor patrol. The mining industry is increasingly looking toward remotely operated vehicles for solu- tions in hostile climates. Transcending earthly bounds, the Spirit and Opportunity UVs are explor- ing the surface of Mars. However, even though these vehicles have no onboard crew, they still need human operators for supervisory control. These UVs require human guidance to varying degrees and often through several operators. For example, to be fully operational, the Predator UAV requires a crew of two. However, with the current mil- itary focus on streamlining operations and reducing staffing, researchers are increasingly trying to design systems that will invert the current many-to-one ratio of operators to vehicles. While this inversion has received the most interest in the air and ground UV domains, it’s also important in other domains such as mining. To replace the multiple personnel required to operate a single UV with a single operator, the UVs must become more autonomous. Also, instead of the lower-level tasks that today’s UV teams perform, the sole operator will need to focus on high-level super- visory control tasks such as monitoring mission time- lines and reacting to emergent mission events. A key challenge in designing these futuristic one- controlling-many systems will be minimizing peri- ods of excessive operator workload that can arise when critical tasks for several UVs occur simulta- neously. To a certain degree, you can predict and mitigate such periods in advance. However, actions that mitigate a particular period of high workload in the short term might create long-term episodes of high workload that were previously nonexistent. So, we need decision support that helps an opera- tor evaluate alternative actions for managing a mis- sion schedule in real time. To this end, we present an iterative design cycle that tries to leverage intel- ligent, predictive aiding together with human judg- ment and pattern recognition to maximize both sys- tem and human performance in the supervision of four UAVs. The simulation testbed We wanted to determine what types of decision sup- port tools would help an operator schedule multiple UAVs, including what kinds of intelligent aiding would be the most beneficial. So, we developed the Multi-aerial Unmanned Vehicle Experiment interface, a dual-screen simulation testbed (see figure 1). The MAUVE interface lets an operator supervise four inde- pendent UAVs simultaneously and intervene as the situation requires. In this simulation, the UAVs must destroy a set of time-sensitive targets in a suppression of enemy air defenses mission. Because the UAVs are Automated decision support tools that provide more local, as opposed to global, visual recommendations can produce better performance in multiple UAV scheduling.