DCM-SAM: Defect-Conditioned Mixture of LoRA Experts for NPU-Deployed AM Defect Segmentation
Md Mushfiqur Rahaman ⋅ Mahedi Hasan ⋅ Imtiaz Ahmed ⋅ Srinjoy Das
Abstract
Defect inspection of metal additive manufacturing (AM) parts relies on X-ray computed tomography (XCT), where the pores and inclusions that decide part quality span only a few pixels and labelled scans are scarce. Segmentation foundation models such as Segment Anything (SAM) can close this data gap, but inspection has to run at the machine, which places a foundation-model-scale backbone and full-resolution input on an embedded accelerator at once. We study what this takes on a Qualcomm Hexagon NPU. Of SAM's three backbones, only ViT-B runs there at $1024^2$: ViT-L and ViT-H compile but cannot allocate memory, because their attention activations, not their weights, exceed the device's limit, so quantizing the weights does not help. We build DCM-SAM on ViT-B, a defect-conditioned mixture of LoRA experts in which one frozen SAM backbone carries a separate Conv-LoRA expert bank and mask decoder for each defect class, trained without prompts while updating only 4.4% of the parameters per head. Adapting the encoder pushes it past the memory limit again, and a numerically identical rewrite of its attention brings it back: the complete model then runs in FP16 at $1024^2$ entirely on the NPU, with masks within 0.01% of pixels of the FP32 reference on the slices we checked, although latency at that resolution could not be profiled on this device. Trained only on synthetic slices, DCM-SAM reaches 64.2% pore IoU on real NIST scans, or 58.8% without one specimen that needs its own post-processing; the post-processing settings are calibrated per split. Code: https://github.com/MushfiqShovon/DCM-SAM.
Chat is not available.
Successful Page Load