Infinite horizon discounted dynamic programming subject to total variation ambiguity on conditional distribution
Date
2016ISBN
978-1-5090-1837-6Publisher
Institute of Electrical and Electronics Engineers Inc.Source
2016 IEEE 55th Conference on Decision and Control, CDC 20162016 IEEE 55th Conference on Decision and Control, CDC 2016
Pages
2010-2015Google Scholar check
Metadata
Show full item recordAbstract
We analyze the infinite horizon minimax discounted cost Markov Control Model (MCM), for a class of controlled process conditional distributions, which belong to a ball, with respect to total variation distance metric, centered at a known nominal controlled conditional distribution with radius R ϵ [0, 2], in which the minimization is over the control strategies and the maximization is over conditional distributions. Through our analysis (i) we derive a new discounted dynamic programming equation, (ii) we show the associated contraction property, and (iii) we develop a new policy iteration algorithm. Finally, the application of the new dynamic programming and the corresponding policy iteration algorithm are shown via an illustrative example. © 2016 IEEE.