Training verified learners with learned verifiers
share
This paper proposes a new algorithmic framework,predictor-verifier training,to train neural networks that areverifiable,i.e., networks thatprovablysatisfysome desired input-output properties. The key idea is to simultaneously train twonetworks: apredictornetwork that performs the task at hand,e.g., predictinglabels given inputs, and averifiernetwork that computes a bound on how wellthe predictor satisfies the properties being verified. Both networks can be trainedsimultaneously to optimize a weighted combination of the standard data-fittingloss and a term that bounds the maximum violation of the property. Experimentsshow that not only is the predictor-verifier architecture able to train networks toachieve state of the artverifiedrobustness to adversarial examples with muchshorter training times (outperforming previous algorithms on small datasets likeMNIST and SVHN), but it can also be scaled to produce the first known (to thebest of our knowledge) verifiably robust networks for CIFAR-10.
READ FULL TEXT