Optimal Estimation via Nonanticipative Rate Distortion Function and Applications to Time-Varying Gauss--Markov Processes

Abstract

In this paper, we develop finite-time horizon causal filters for general processes taking values in Polish spaces using the nonanticipative rate distortion function ($NRDF$). Subsequently, we apply the $NRDF$ to design optimal filters for time-varying vector-valued Gauss--Markov processes, subject to a mean-squared error ($MSE$) distortion. Unlike the classical Kalman filter design, the developed filters based on the $NRDF$ are characterized parametrically by a dynamic reverse-waterfilling optimization problem obtained via Karush--Kuhn--Tucker conditions. We develop algorithms that provide, in general, tight upper bounds to the optimal solution to the dynamic reverse-waterfilling optimization problem subject to a total and per-letter $MSE$ distortion constraint. Under certain conditions, these algorithms produce the optimal solutions. Further, we establish a universal lower bound on the total and per-letter $MSE$ of any estimator of a Gaussian random process. Our theoretical framework is demonstrated via simple examples.