Policy Gradient for s-Rectangular Robust Markov Decision Processes

01/31/2023
by   Navdeep Kumar, et al.
0

We present a novel robust policy gradient method (RPG) for s-rectangular robust Markov Decision Processes (MDPs). We are the first to derive the adversarial kernel in a closed form and demonstrate that it is a one-rank perturbation of the nominal kernel. This allows us to derive an RPG that is similar to the one used in non-robust MDPs, except with a robust Q-value function and an additional correction term. Both robust Q-values and correction terms are efficiently computable, thus the time complexity of our method matches that of non-robust MDPs, which is significantly faster compared to existing black box methods.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset