Pilot performance evaluation of simulated flight approach and landing manoeuvres using quantitative assessment tools P ARCHANA HEBBAR and ABHAY A PASHILKAR * CSIR-National Aerospace Laboratories, Bangalore 560017, India e-mail: p_archana@nal.res.in; apash@nal.res.in MS received 6 January 2016; revised 26 July 2016; accepted 27 September 2016 Abstract. This research work examines the application of different statistical and empirical analysis methods to quantify pilot performance. A realistic approach and landing flight scenario is executed using the reconfig- urable flight simulator at National Aerospace Laboratories and both subjective and quantitative measures are applied to the pilot performance data. Simulations were repeated for different difficult landing conditions like landing with degraded visibility, with crosswinds, with degraded aircraft handling qualities and with emergency conditions. Relative assessment of the different applicable metrics is made and significance of task difficulties on pilot performance is investigated. Changes in the pilot’s control strategy with respect to primary and secondary tasks are also discussed in detail. Results indicate that analysing pilot’s control strategy together with his/her deviations from predetermined flight profile provides a means to quantify pilot performance. Keywords. Pilot in the loop simulations; cognitive workload; tracking errors; control strategy. 1. Introduction Any new aircraft technology, be it fly by wire control system design, advanced glass cockpit design or finalizing head up/down display symbologies, is developed with the primary intention to increase aircraft/pilot safety and to ease pilot to accomplish his/her task smoothly. The current practice is to evaluate the usability of the new technology/ algorithm through pilot opinion. Although these commonly used subjective methodologies may be fast and are non- intrusive, the judgements rely highly on personal opinions, his/her mood and the surrounding environment at that instant. Hence these methods alone may not be always reliable. Use of quantitative tools and techniques along with the subjective ratings can give more meaningful judgements. Further to subjective and performance based methods, pilot workload can also be measured through physiological measures. Research on assessment of mental stress using heart rate variability [1], respiration, electroencephalogram signals [2] and eye related signals [3–5] suggests that these physiological parameters do show signatures of cognitive workload when the subject is under stress. However, accuracy of judgement based on these methods basically depends on the measuring instruments. Also, these instru- ments have to be non-intrusive to pilot’s primary flying tasks. In this paper, we restrict our study to subjective and performance based pilot evaluation only. The realistic study discussed in this paper attempts to prove the concept of quantitative performance assessment. Piloted simulations are carried out using reconfigurable design flight simulator at National Aerospace Laboratories (NAL), Bangalore, India for real life flight task scenarios. The objective of the study is to suggest potential quanti- tative measures available in the literature to analyse pilot performance. These results are compared with pilot’s sub- jective opinions. The implication of task difficulties on pilot’s performance is studied. Also, changes in pilot’s control strategy with respect to task priority are examined. There are many research activities carried out on objective pilot performance evaluation. Ververs and Wickens [6] examined the benefits of display location; display clutter and display intensity on pilot performance in a cruise flight environment using simple statistical analysis methods like mean absolute error, root mean square error (RMSE) and response time delay. Experiments were con- ducted wherein pilots flew a flight segment while observing changes in heading, airspeed and altitude. Pilots were directed to press a button on the joystick when any ran- domly appearing external event is detected. Deviations from the flight path and response time to detect sudden events were measured using the statistical methods men- tioned above. Experiments were repeated for different display locations, clutter and image contrast ratios. Smith and Caldwell [7] used RMSE to study pilot fatigue. Their experiments were extensive and the flight profile con- sisted of many turns and climbs before performing an instru- ment landing. Each flight task was for 1 h duration wherein *For correspondence 405 Sa ¯dhana ¯ Vol. 42, No. 3, March 2017, pp. 405–415 Ó Indian Academy of Sciences DOI 10.1007/s12046-017-0613-0