Optimal control of uncertain stochastic systems subject to total variation distance uncertainty

Abstract

This paper is concerned with optimization of uncertain stochastic systems, in which uncertainty is described by a total variation distance constraint between the measures induced by the uncertain systems and the measure induced by the nominal system, while the payoff is a linear functional of the uncertain measure. Robustness at the abstract setting is formulated as a minimax game, in which the control seeks to minimize the payoff over the admissible controls while the uncertainty aims at maximizing it over the total variation distance constraint. It is shown that the maximizing measure in the total variation distance constraint exists, while the resulting payoff is a linear combination of L 1 and L ∞ norms. Further, the maximizing measure is characterized by a linear combination of a tilted measure and the nominal measure, giving rise to a payoff which is a nonlinear functional on the space of measures to be minimized over the admissible controls. The abstract formulation and results are subsequently applied to continuous-time uncertain stochastic controlled systems, in which the control seeks to minimize the payoff while the uncertainty aims to maximize it over the total variation distance constraint. The minimization over the admissible controls of the nonlinear functional payoff is addressed by developing a generalized principle of optimality or dynamic programming equation satisfied by the value function. Subsequently, it is proved that the value function satisfies a Hamilton-Jacobi-Bellman (HJB) equation. It is also shown that the value function is also a viscosity solution of the HJB equation. Finally, the linear quadratic case is studied, and it is shown that the infinity norm of a quadratic payoff is well defined and finite. Throughout the paper the formulation and conclusions are related to previous work found in the literature. © 2012 Society for Industrial and Applied Mathematics.