Two Lenses, Two Inference Modes: Comparing Autoregressive and Diffusion Language Models

The same language model can reach an answer through autoregressive (AR) or diffusion (DLM) inference, creating a controlled way to study how the inference procedure changes internal computation. Across 11 tasks, we compare both modes in the same 3B and 8B Nemotron Labs Diffusion checkpoints while holding all model components fixed. Separate validation prompts identify late layers where the Logit Lens and a shared Tuned Lens reliably recover each mode's final layer preferences. In these regions, DLM states more closely match their own final answer distributions. Separately attenuating the second to last MLP update at the answer position changes the final correct versus competing answer margin more under AR at both scales. The results separate what can be read from an internal state from the causal influence of a specific late update