Performance evaluation of judgemental directional exchange rate predictions B Andrew C. Pollock a, * , Alex Macaulay a,1 , Mary E. Thomson b,2 , Dilek O ¨ nkal c,3 a Division of Mathematics, School of Computing and Mathematical Sciences, Glasgow Caledonian University, Cowcaddens Road, Glasgow G4 0BA, UK b Department of Psychology, Glasgow Caledonian University, Cowcaddens Road, Glasgow G4 0BA, UK c Faculty of Business Administration, Bilkent University, 06800 Bilkent, Ankara, Turkey Abstract A procedure is proposed for examining different aspects of performance for judgemental directional probability predictions of exchange rate movements. In particular, a range of new predictive performance measures is identified to highlight specific expressions of strengths and weaknesses in judgemental directional forecasts. Proposed performance qualifiers extend the existing accuracy measures, enabling detailed comparisons of probability forecasts with ex-post empirical probabilities that are derived from changes in the logarithms of the series. This provides a multi-faceted evaluation that is straightforward for practitioners to implement, while affording the flexibility of being used in situations where the time intervals between the predictions have variable lengths. The proposed procedure is illustrated via an application to a set of directional probability exchange rate forecasts for the US Dollar/Swiss Franc from 23/7/96 to 7/12/99 and the findings are discussed. D 2005 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved. Keywords: Accuracy; Exchange rate; Forecasting; Judgement; Probability 1. Introduction Exchange rate movements are primarily affected by expectational elements arising from market senti- ment. These manifest themselves in optimism, pessi- mism and varying degrees of uncertainty in the minds of market participants. Analysts’ judgements of perceived market sentiment, as well as their responses to the uncertainties attributable to political and economic events, play fundamental roles in their forecasts of currency movements (Larson & Madura, 2001). In particular, there exist significant bindividual 0169-2070/$ - see front matter D 2005 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.ijforecast.2004.12.006 B Earlier versions of this paper were presented at the 22nd International Symposium on Forecasting, Dublin, Ireland (June 2002), and the 30th Meeting of the EURO Working Group on Financial Modelling, Capri, Italy (May 2002). The authors are grateful to the participants, whose comments significantly con- tributed to the present paper. * Corresponding author. Tel.: +44 141 331 3613; fax: +44 141 331 3608. E-mail addresses: a.c.pollock@gcal.ac.uk (A.C. Pollock)8 abma@gcal.ac.uk (A. Macaulay)8 mwi@gcal.ac.uk (M.E. Thomson)8 onkal@bilkent.edu.tr (D. O ¨ nkal). 1 Tel.: +44 141 331 3052; fax: +44 141 331 3608. 2 Tel.: +44 141 331 3899; fax: +44 141 331 3636. 3 Tel.: +90 312 290 1596; fax: +90 312 266 4958. International Journal of Forecasting 21 (2005) 473 – 489 www.elsevier.com/locate/ijforecast