Research regarding the problem of defective insulator recognition on power distribution networks retains an open interest, due to the significant role insulators play to maintain quality service delivery. Most existing methods detect insulators by rectangular bounding box but do not perform segmentation down to instance pixel-level. In this paper, we propose an automated end-to-end framework enabled by attention mechanism to enhance recognition of defective insulators. Using natural industry dataset of images acquired by unmanned aerial vehicle (UAV), pixel-level recognition is formulated into two computer vision tasks; object detection and instance segmentation. We increase the capabilities of our chosen model by leveraging a lightweight but effective three-branch attention structure integrated into the backbone network as an add-on module. Specifically, we exploit cross-dimensional interactions to build an efficient computation of attention weights across channels of the backbone network to achieve gains in detection performance for defective insulators up to about +2.0 points compared to our base model, at negligible overhead cost. Moreover, we implement a training scheme to improve segmentation performance while demonstrating segmentation superiority over traditional segmentation approaches.