Weight Transfer, Composition and Measurement in Detector Ablation Studies
Abstract
The recent literature in weapon detection improves the YOLO family detectors by frequently using attention modules and modern loss functions. We tested different attention modules (CBAM, Coordinate Attention and their combination), loss functions (EIoU, WIoU v3 and MPDIoU) and a cost-sensitive classification loss. We experimented with a run at only one changed component and also inserted two controls commonly omitted in this field of research. The study is based on YOLOv8n model and the Sohas handheld object detection dataset containing six classes, out of which two are categorized as weapons. We explicitly tested the initialisation procedure. Adding a module to the YOLO backbone causes the renumbering of all layers after it, and since the pretrained weights are matched by parameter name this renaming can mean sending part of the checkpoint to the wrong place without any warning. Moving a zero parameter identity module across every valid index shows that 3.6\% of COCO weights stay correctly positioned for a backbone insertion against 64.6\% at the detection head. Coordinate Attention improves the mAP50-95 from 69.95 to 71.21, a gain of 1.26 points, with four of six per-class APs improved, but addition of CBAM degrades its performance. The combination of CBAM and Coordinate Attention reaches 68.97, which is 0.98 below the baseline and 0.04 below CBAM alone. The cost-sensitive classification loss, built from a weapon versus non-weapon cost matrix, was also evaluated. We trained the same modified code path with a cost matrix of ones, checked to be identical to the stock loss in value and gradient. It still lost 0.86 mAP50-95, so the cost-sensitive result is measured against this run and not the baseline. Against it, the weapon-level alarm rate over 537 weapon images rises from 94.04\% to 96.28\%, pistol and knife AP gain 2.91 and 1.88 points, and smartphones misread as pistols drop from 11 to 4. Aggregate mAP50-95 falls 0.47 points because four out of six classes belong to the harmless object category and mAP scores a missed pistol and a missed banknote the same. Two properties of the Sohas dataset are also studied which affect each published result based on it. 6.7\% of images carry unannotated extra instances (calculated from a 120 image sample), and 14.47\% of test images are similar to video frames of training images.