Centralized Versus Decentralized Optimization of Distributed Stochastic Differential Decision Systems with Different Information Structures-Part I: A General Theory

Abstract

Decentralized optimization of distributed stochastic dynamical systems with two or more controls of the decision makers (DMs) has been an active area of research for over half a century. Although, such decentralized optimization problems are often formulated utilizing static team and person-by-person (PbP) optimality criteria, the corresponding static team theory results have not been extended to dynamical systems. In this first part of the two-part paper, we derive team and PbP optimality conditions for distributed stochastic differential systems, when the controls of the DMs generate actions based on different information structures. The necessary conditions are given by a Hamiltonian System described by coupled backward and forward stochastic differential equations (SDEs) and a conditional Hamiltonian, conditioned on the information structures available to the controls of the DMs. The sufficient conditions state that PbP optimality implies team optimality, if the Hamiltonian is convex in the state and/or actions spaces of the controls of the DMs. We show existence of relaxed team optimal strategies, when the information structures are not affected by the controls of the DMs. Throughout the paper we discuss similarities to analogous optimality conditions of centralized decision or control of stochastic systems, and we note a connections to mean field stochastic optimal control problems. © 1963-2012 IEEE.