Constrained Continuous-Time Markov Control Processes with Discounted Criteria Xianping Guo # and One ´simo Herna ´ndez-Lerma * Departamento de Matema ´ticas, CINVESTAV-IPN, Me ´xico D.F., Mexico ABSTRACT In this paper we study constrained continuous-time Markov control processes with a denumerable state space and unbounded reward/cost and transition rates. The criterion to be maximized is an expected discounted reward, and the constraint is imposed on an expected discounted cost. We give conditions that ensure the existence of constrained-optimal policies. We also show that a constrained-optimal policy may be a stationary policy or a randomized stationary policy that randomizes between two stationary policies which differ in at most one state. Our results are illustrated with a controlled queueing system. 379 DOI: 10.1081/SAP-120019291 0736-2994 (Print); 1532-9356 (Online) Copyright q 2003 by Marcel Dekker, Inc. www.dekker.com # Permanent address: School of Mathematics and Computational Sciences, Zhongshan University, Guangzhou 510275, P.R. China. * Correspondence: One ´simo Herna ´ndez-Lerma, Departamento de Matema ´ticas, CINVESTAV-IPN, A.Postal 14-740, Me ´xico D.F. 07000, Mexico; Fax: (52-55) 5747-3876; E-mail: ohernand@math.cinvestav.mx. STOCHASTIC ANALYSIS AND APPLICATIONS Vol. 21, No. 2, pp. 379–399, 2003 MARCEL DEKKER, INC. • 270 MADISON AVENUE • NEW YORK, NY 10016 ©2003 Marcel Dekker, Inc. All rights reserved. This material may not be used or reproduced in any form without the express written permission of Marcel Dekker, Inc.