Infrared and Visible Image Fusion with ResNet and zero-phase component analysis
share
In image fusion task, feature extraction and processing are keys for fusion algorithm. Not only traditional feature extraction methods, deep learning-based methods are also applied into image fusion field to extract features. However, most of them use deep features directly which without feature processing. And this will lead the fusion performance degradation in some cases. In this paper, a novel fusion framework which based on deep features and zero-phase component analysis(ZCA) is proposed. Firstly, the residual network(ResNet) is used to extract the deep features from source images. Then ZCA and l_1-norm are utilized to normalize the deep features and obtain initial weight maps. And the final weight maps are obtained by initial weight maps and soft-max operation. Finally, the fused image is reconstructed by weight maps and source images. Compare with the existing fusion methods, experimental results demonstrate that our algorithm achieves better performance in both objective assessment and visual quality. And the code of our fusion algorithm is available at https://github.com/exceptionLi/imagefusion_resnet50
READ FULL TEXT