Transformer MLP Transplants Depend on the Receiving State
Muhammed Nehan
Abstract
Activation patching often moves an internal activation from one context to another. We ask a basic question: if we apply the same transformer MLP displacement in two receiving states, does it have the same effect? We measure effect by the change in the model's next-token distribution and predict it with a local sensitivity score computed at the receiver. In a direct crossed-receiver experiment on Qwen3-8B and Mistral-7B, the same fixed natural displacement produces strongly different effects across receivers; the receiver-side score ranks these effects with median within-displacement Spearman correlation $0.858$--$0.928$ across six model-by-data replications. Some receivers are generically fragile, but after removing fixed-displacement and receiver effects the correlation remains $0.504$--$0.675$, above a matched local-magnitude baseline at $0.284$--$0.442$. The destination is therefore part of the intervention: the perturbation alone does not determine its effect; the receiving state matters.
Chat is not available.
Successful Page Load