Journal of Global Optimization 29: 315–334, 2004. 315 © 2004 Kluwer Academic Publishers. Printed in the Netherlands. An Interior Point Heuristic for the Hamiltonian Cycle Problem via Markov Decision Processes VLADIMIR EJOV 1 , JERZY FILAR 1 and JACEK GONDZIO 2 1 School of Mathematics, The University of South Australia, Mawson Lakes, SA 5095, Australia (e-mail: jerzy.filar@unisa.edu.au; vladimir.ejov@unisa.edu.au) 2 School of Mathematics, University of Edinburgh, Edinburgh, Scotland, UK (e-mail: J.Gondzio@ed.ac.uk) (Received 31 July 2003; accepted 7 August 2003) Abstract. We consider the Hamiltonian cycle problem embedded in a singularly perturbed Markov decision process (MDP). More specifically, we consider the HCP as an optimization problem over the space of long-run state-action frequencies induced by the MDP’s stationary policies. We show that Hamiltonian cycles (if any) correspond to the global minima of a suitably constructed indefinite quadratic programming problem over the frequency space. We show that the above indefinite quadratic can be approximated by quadratic functions that are ‘nearly convex’ and as such suitable for the application of logarithmic barrier methods. We develop an interior-point type algorithm that involves an arc elimination heuristic that appears to perform rather well in moderate size graphs. The approach has the potential for further improvements. Key words. Hamiltonian cycles, interior point methods, Markov decision processes, non-convex optimization. 1. Introduction This paper is a continuation of a line of research [4, 7, 10, 11, 12] which aims to exploit the tools of controlled Markov decision chains (MDP’s) 1 to study the properties of a famous problem of combinatorial optimization: the Hamiltonian Cycle Problem (HCP). More specifically, the present paper provides evidence that computationally effective algorithms for determining Hamiltonicity can be developed based on this approach. As such it can also be viewed as a continuation of the numerical experiments begun in Andramanov et al. [4]. In this paper, we consider the following version of the Hamiltonian cycle problem: given a directed graph, find a simple cycle that contains all vertices of thegraph Hamiltoniancycle HC orprovethatHCdoesnotexist. With respect to this property—Hamiltonicity—graphs possessing HC are called Hamiltonian. Next we shall, briefly, differentiate between our approaches and some of the best known ‘classical’ approaches to the HCP. Many of the successful classical approaches of discrete optimisation focus on solving a linear programming ‘relaxation’ followed by heuristics that prevent the 1 The acronym MDP stems from the alternative name of Markov decision processes.