J. oflnequal. & Appl., 2001, Vol. 6, pp. 519-545 Reprints available directly from the publisher Photocopying permitted by license only (C) 2001 OPA (Overseas Publishers Association)N.V. Published by license under the Gordon and Breach Science Publishers imprint, a member of the Taylor & Francis Group. Viscosity Solutions of Two Classes of Coupled Hamilton-Jacobi-Bellman Equations* MINGQING XlAO a’t and TAMER BA,AR b aDepartment of Mathematics, University of California, Davis, One Shields Ave., Davis, CA 956161USA; bCoordinated Science Laboratory, University of Illinois at Urbana-Champaign, 1308 West Main Street, Urbana, IL 618011USA (Received 12 August 1999; In final form 27 January 2000) This paper studies viscosity solutions of two sets of linearly coupled Hamilton-Jacobi- Bellman (HJB) equations (one for finite horizon and the other one for infinite horizon) which arise in the optimal control of nonlinear piecewise deterministic systems where the controls could be unbounded. The controls enter through the system dynamics as well as the transitions for the underlying Markov chain process, and are allowed to depend on both the continuous state and the current state of the Markov chain. The paper establishes the existence and uniqueness of viscosity solutions for these two sets of HJB equations, whose Hamiltonian structures are different from the standard ones. Keywords and Phrases: Viscosity solutions; Coupled Hamilton-Jacobi-Bellman equations; Piecewise deterministic systems AMS Mathematics Subject Classifications 1991: 49L25, 35B37, 93E20, 90D25, 60F10 1. INTRODUCTION This paper studies viscosity solutions of first order, linearly coupled partial differential equations of the following types, where f is a subset * Research supported in part by the U.S. Department of Energy under Grant DOE- DEFG-02-97ER13939, and in part by a University of Illinois Fellowship. Corresponding author, e-mail: xiao@math.ucdavis.edu 519