Appl Math Optim 40:377–392 (1999) © 1999 Springer-Verlag New York Inc. Envelopes of Sets of Measures, Tightness, and Markov Control Processes ∗ J. Gonz´ alez-Hern´ andez 1 and O. Hern´ andez-Lerma 2 1 Departamento de Probabilidad y Estad´ ıstica, IIMAS-UNAM, Apartado Postal 20-726, M´ exico D.F. 01000, M´ exico 2 Departamento de Matem´ aticas, CINVESTAV-IPN, Apartado Postal 14-740, M´ exico D.F. 07000, M´ exico ohernand@math.cinvestav.mx Abstract. We introduce upper and lower envelopes for sets of measures on an arbitrary topological space, which are then used to give a tightness criterion. These concepts are applied to show the existence of optimal policies for a class of Markov control processes. Key Words. Envelopes of measures, Tightness criteria, (Discrete-time) Markov control processes. AMS Classification. 93E20, 28C15. 1. Introduction A well-known fact in control theory is that a large class of optimal control problems (deterministic or stochastic, in discrete or continuous time—see [2], [4], [9]–[11], [17], and [21–[23]) can be transformed into minimization problems over sets of measures. In this case, the control problem is typically reduced to the form: minimize  X cd µ subject to µ ∈ M( X ), (1.1) where M( X ) is a set of measures on some space X , and c denotes the control problem’s running cost. Moreover, under mild assumptions on c and X , and endowing M( X ) with ∗ This research was partially supported by the Consejo Nacional de Ciencia y Tecnolog´ ıa (CONACYT) Grant 3115P-E9608. The work of the first author was also supported by a CONACYT scholarship.