Statistic-Space Perturbation Steering for Large Language Models

Inference-time steering can adapt the behavior of a large language model without parameter updates, but existing methods often couple behavioral measurements with fixed steering directions or implicitly defined optimization objectives, making it difficult to independently specify the desired behavioral state and determine the smallest intervention needed to reach it. We introduce a general framework that formulates inference-time steering as explicit target matching in a compact, representative-statistic space. Given a desired target in statistic space, our method estimates the local response of the statistic to perturbations of selected activation coordinates and computes a regularized minimum-cost intervention that moves the statistic toward that target. The response can be obtained analytically for differentiable statistics or approximated using a locally faithful differentiable surrogate, enabling the framework to support both learned behavioral probes and mechanistic internal-state measurements. We evaluate the approach in two complementary settings: truthfulness steering on TruthfulQA using an exact attention-head probe statistic, and hallucination mitigation in retrieval-augmented generation using a ReDeEP-based mechanistic statistic. Our targeted inverse controller achieves the highest MC1 and MC2 scores and comparable hallucination-score reduction among the baselines, while reducing selected-head modification by 28.9% and 41.5%, respectively.