Trust Observations in Validation Exercises F. Amato ∗ , M. Felici † , P. Lanzi ∗ , G. Lotti ∗ , L. Save ∗ and A. Tedeschi ∗ ∗ Deep Blue S.r.l. Piazza Buenos Aires, 20 00198 Roma - Italy URL: http://www.dblue.it/ Email: {paola.lanzi, giulia.lotti, luca.save, alessandra.tedeschi}@dblue.it † School of Informatics, The University of Edinburgh 10 Crichton Street, Informatics Forum Edinburgh EH8 9AB, UK Email: mfelici@staffmail.ed.ac.uk Abstract—This paper is concerned with an operational ac- count of trust. It reports our experience in observing different trust aspects during a validation session for the assessment of a new tool and relevant operational concepts in the Air Traffic Management (ATM) domain. Despite the fact that trust is yet an elusive concept, our results show how monitoring trust can support the validation of alternative system settings and their operational aspects. This paper reports our experimental work on observing trust during validations exercises. Moreover, it provides new insights about the nature and the investigation of trust. Keywords-Air Traffic Management; Operational Validation; Empirical Analysis; Trust I. I NTRODUCTION Ongoing developments in the Air Traffic Management (ATM) domain involve the implementations and deploy- ments of new technologies and operational concepts, which change ATM practices. The ATM 2000+ Strategic Agenda [1] and the Single European Sky ATM Research (SESAR) Initiative [2] are concerned with a structural revision of ATM processes, a new ATM concept and a system approach for the ATM Network. This requires ATM services to go through significant structural, operational and cultural changes that will contribute towards SESAR. Validation activities are crucial for the development of new technologies and rel- evant operational concepts. They are often concerned with different Key Performance Indicators (KPIs), such as safety, efficiency, and so on, that are critical within the ATM domain. Validation activities involve subsequent evaluations of technologies with respect to different scenarios. In the ATM context, Air Traffic Controllers (ATCOs) exercise with technologies in order to validate system features [3]. This is useful to assess how new technologies and the operational concepts they implement affect work practices. Trust is a concept that has been recognized to be critical for the acceptance and adoption of new technologies. Re- search highlights that trust is critical in the automation of various human activities (e.g. see [4] for an account of trust in the ATM domain). Moreover, trust may interact (that is, either support or affect) other critical aspects (e.g. safety, risk perception, user acceptability, etc.). Unfortunately, our understanding how trust relates to other critical dimensions of technologies is still patchy. Most research is still debating on a generally accepted account of trust (and other relevant concepts like trustworthiness) [5]. This paper is concerned with an operational account of trust. Our main interest is to investigate how trust can sup- port validation activities and investigations. Despite any the- oretical account of trust, we are seeking to acquire an under- standing of trust during operational validation sessions. Our assumption is that trust provides a convenient and alternative viewpoint of analysis that can be combined together with other operational aspects (e.g. situational awareness, team- work, workload, etc.). This paper is structured as follows. Section II highlights the criticality of trust within the ATM domain. Section III describes the ATM validation case study. Section IV introduces our observational approach to trust adopted during a validation session. Section V discusses our trust observations. In particular, it points out different trust aspects. The discussion is supported by qualitative and quantitative analyses carried out during the validation session. Section VI, finally, draws some concluding remarks. II. TRUST IN AIR TRAFFIC MANAGEMENT Trust and trustworthiness [6] capture many diverse aspects that are becoming as critical and relevant as many other aspects (e.g. safety, security, dependability, etc.) for system design and assessment [7], [8]. Research on trust drawn from multidisciplinary domains highlights a continuing debate on its definition and its nature [5], [6], [9], [10]. Yet, it is difficult to derive a definitive and widely-accepted definition of trust. One conclusive remark is that trust is a complex concept. Unveiling its complexity requires us to understand subtle interactions among different aspects of trust (e.g. trust and trustworthiness). Trust is a concept that has been investigated in different disciplines (e.g. economics, social 2011 Fifth International Conference on Secure Software Integration and Reliability Improvement 978-0-7695-4453-3/11 $25.00 © 2011 IEEE DOI 10.1109/SSIRI.2011.26 216