Current Convolutional Neural Networks (CNNs) does not explicitly capture and select diverse image features. Rather follow an indirect approach of increasing the networks' depth or width, which significantly increase the computational cost of the models. Inspired by biological visual system, this paper proposes a Multi-Kernel Attention Convolutional Network (MKA-Net ), which enables any feed-forward CNNs to explicitly capture and select diverse informative features to efficiently boost CNNs' performance. MKA-Net infers attention from the intermediate feature map by first using multiple sizes of kernels to capture diverse features then exploit neighboring feature-map relationship to adaptively select the most informative features. MKA-Net incurs negligible computational overhead and is designed to be easily integrated with any CNN architecture. We extensively evaluated the proposed MKA-Net module on benchmark datasets, including CIFAR100, SVHN, and ImageNet, with various CNN architectures. The experimental results show our approach provides a significant performance improvement with very minimal computational overhead.