Lagrangian Relaxation for Continuous-Time Stochastic Optimal Control of Power Systems Including Wind Power, Storage Capacity, and a Cascade of Hydropower Systems with Time Delays

Overview

Abstract 

This work considers a short-term, continuous-time stochastic setting characterized by a coupled power supply system controlled exclusively by a single provider and comprising a cascade of hydropower systems (dams), fossil fuel power stations, a storage capacity modeled by a single large battery, and stochastic wind power generation. Cascaded hydropower generators introduce time-delay effects in the state dynamics, making it impossible to use classical dynamic programming in its standard Markovian form. We address this issue by introducing a novel Lagrangian relaxation technique over continuous-time constraints, constructing a nearly optimal policy efficiently. To retain tractability in the stochastic setting, we restrict the Lagrangian multipliers to a finite-dimensional Markovian approximation class depending on time and the current wind state. This approach yields a concave, nonsmooth dual optimization problem to recover the optimal Lagrangian multipliers, which is numerically solved using a limited memory bundle method. At each step of the dual optimization, we need to solve an optimization subproblem. Given the current values of the Lagrangian multipliers, the time delays are no longer active, and we can solve a corresponding nonlinear Hamilton–Jacobi–Bellman (HJB) partial differential equation (PDE) for the optimization subproblem. The HJB PDE is solved by a hybrid numerical scheme combining a monotone upwind finite-difference discretization in the deterministic state variables with a low-order collocation treatment of the stochastic wind variable, coupled through an operator-splitting time-stepping procedure. To handle the infinite-dimensional nature of the Lagrange multipliers, we design an adaptive refinement strategy to control the duality gap. Furthermore, we use a penalization technique for the constructed admissible primal solution to smooth the controls while achieving a sufficiently small duality gap. Numerical results based on the Uruguayan power system demonstrate the efficiency of the proposed mathematical models and numerical approach.