A Pontryagin Maximum Principle on the Belief Space for Continuous-Time Stochastic Optimal Control with Discrete Observations

Overview

Abstract: This talk presents joint work with Christian Bayer, Saifeddine Ben Naamia, Erik von Schwerin, and Raúl Tempone on continuous-time stochastic optimal control under partial observations available only at discrete time instants. We formulate the problem on the space of beliefs, using the controller’s posterior distribution as the state variable, and derive a Pontryagin maximum principle that accounts for continuous evolution between observations and Bayesian jump updates at observation times. A key result links the adjoint process to the gradient of the value functional on the belief space, connecting Pontryagin conditions with dynamic programming on the space of probability measures and yielding a predict–update structure related to the Zakai and Kushner-Stratonovich filtering equations.
We also propose a particle-based numerical scheme for the coupled forward (filter) and backward (adjoint) system: particle filtering represents the belief, while regression approximates the adjoint, enabling computation of near-optimal controls under partial information. Numerical examples in linear and nonlinear settings illustrate the approach and highlight the benefits of actively controlling the observation process.