Adaptive Multi-Teacher Distillation for Adversarially Robust Network Intrusion Detection

Deep learning-based network intrusion detection systems (NIDS) achieve high accuracy on benchmark traffic. However small, protocol-valid perturbations to flow features can let malicious traffic evade detection while remaining functionally unchanged [1]. Adversarial training is the standard defense [2]. But retraining a deployed detector against every new attack is impractical in production. Knowledge distillation has been proposed as a lighter-weight alternative [3, 4]. Existing approaches, however, distill from a single adversarially-trained teacher. This transfers robustness only against the attack that teacher was trained on. We introduce AMT-KD (Adaptive Multi-Teacher Knowledge Distillation). It trains three teachers instead of one, each hardened against one attack family (FGSM, I-FGSM, or PGD), and transfers their combined robustness into a single student that is never itself adversarially retrained. Each teacher's contribution is re-weighted at every distillation step by the cosine similarity between teacher and student logits. So the student leans on whichever teacher currently agrees with it most, rather than on a fixed or equal blend. We evaluate on the CIC-IDS-2017, CIC-IDS-2018, and CIC-DDoS-2019 intrusion benchmarks across CNN, C-LSTM, and LSTM architectures (nine dataset-architecture configurations) under adaptive white-box FGSM/I-FGSM/PGD attacks. The student improves adversarial macro-F1 over the undefended baseline in 7 of 9 configurations, and the size of that gain is predicted almost entirely by the size of the baseline's own vulnerability ($r=0.940$ Fig. 1). The clearest case is CIC-IDS-2018, where undefended baselines are severely vulnerable (recall driven to 0.000 for C-LSTM). There the student gains up to 0.47 macro-F1 at a clean-accuracy cost below 0.002. On CIC-IDS-2017, where the constrained adversary reaches the attack signature only partially, the framework recovers roughly half of what the attack takes. The exception is CIC-DDoS-2019, whose attack signature lies largely outside the perturbable feature subspace. Baselines there are already near-immune, so distillation only costs macro-F1 (0.045 to 0.162) without a compensating gain, which is exactly what the correlation predicts for a low-vulnerability baseline. In short, baseline degradation under attack is measurable pre-deployment, so whether AMT-KD is worth applying can be decided in advance. A second finding cuts against the mechanism single-teacher distillation defenses assume [4]: teacher specialisation does not occur. Across all 27 teachers, the one hardened on attack family X is strongest against X in only 7 of 27 cases, close to the 9 of 27 expected by chance alone. This makes sense because FGSM, I-FGSM, and PGD are all $l_{\infty}$ gradient-sign attacks in the same subspace, so robustness transfers between them regardless of which one a teacher trained on. What actually varies between teachers is convergence quality, not attack coverage. Adversarial training is unreliable, especially for recurrent architectures, so some teachers simply converge better than others. The student tracks its best teacher far more closely than the teacher mean, even exceeding it in two configurations (Fig. 2). AMT-KD, then, is best understood not as an aggregator of complementary specialists, but as a mechanism robust to the failure of any individual teacher.