ErfAct and PSerf: Non-monotonic smooth trainable Activation Functions
share
An activation function is a crucial component of a neural network that introduces non-linearity in the network. The state-of-the-art performance of a neural network depends on the perfect choice of an activation function. We propose two novel non-monotonic smooth trainable activation functions, called ErfAct and PSerf. Experiments suggest that the proposed functions improve the network performance significantly compared to the widely used activations like ReLU, Swish, and Mish. Replacing ReLU by ErfAct and PSerf, we have 5.21 5.04 dataset, 2.58 network in CIFAR10 dataset, 1.0 precision (mAP) on SSD300 model in Pascal VOC dataset.
READ FULL TEXT