Higher BBQ Scores, More Unnecessary Abstention: An Item-Level Audit of Bias Steering

Activation steering changes a model's hidden activations during inference without updating its weights. Prior work has used activation steering to improve scores on social-bias benchmarks. But a higher score may hide a trade-off: the model may choose "cannot be determined" more often both when a passage gives no answer and when it gives a clear answer. We examine these changes using the Bias Benchmark for Question Answering (BBQ), which includes questions both with and without enough evidence to answer. We reproduce a published religion-steering experiment on Mistral-7B-Instruct and compare answers to the same questions before and after steering. Overall accuracy rises from 51.5% to 57.8%, close to the published result. On ambiguous questions, steering makes the model choose the correct "cannot be determined" answer more often instead of guessing. On disambiguated questions, unnecessary abstention rises from 9.3% to 37.3%. Of the 224 questions answered correctly before steering, 63 (28.1%) receive "cannot be determined" after steering. This increase occurs both when the passage supports the stereotype-consistent answer and when it supports the other answer. A gender vector also shows a similar pattern: gains on ambiguous questions alongside losses on answerable ones. Bias-steering evaluations should therefore report both correct abstention when evidence is missing and unnecessary abstention when evidence is present, alongside overall accuracy.