Worrisome Properties of Neural Network Controllers and Their Symbolic Representations

07/28/2023
by   Jacek Cyranka, et al.
0

We raise concerns about controllers' robustness in simple reinforcement learning benchmark problems. We focus on neural network controllers and their low neuron and symbolic abstractions. A typical controller reaching high mean return values still generates an abundance of persistent low-return solutions, which is a highly undesirable property, easily exploitable by an adversary. We find that the simpler controllers admit more persistent bad solutions. We provide an algorithm for a systematic robustness study and prove existence of persistent solutions and, in some cases, periodic orbits, using a computer-assisted proof methodology.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset