Adaptive Content-Contrastive Decoding for Representation-Robust Tabular Reasoning

Large language models (LLMs) have shown strong performance on tabular reasoning tasks, but their predictions can be affected by different types of bias. We identify four potential sources of bias: positional bias, value anchoring bias, parametric prior bias, and question-grounding bias. The effect of these biases varies across inputs, making a fixed correction strategy ineffective for all samples. We address this problem with Content-Contrastive Decoding (CCD), which constructs negative inputs by removing different types of table or question content and uses the prediction gap between the original and negative inputs to reduce the effect of bias. We introduce four bias-motivated negative types, each designed to expose a different source of bias. To select the appropriate negative type and contrastive strength for each input, we train a lightweight selector using the frozen LLM's hidden states. The selector makes this choice at the sample level without modifying the base LLM or requiring additional backbone forward passes beyond standard single-negative contrastive decoding. Across five models and four tabular reasoning datasets, CCD consistently outperforms the best fixed negative strategy. In particular, on LLaMA-3 with TabMWP, where the best fixed negative decreases accuracy by 6.20\%, CCD improves accuracy by 3.25\%, recovering 9.45 percentage points. These results show that selecting the negative input and contrastive strength for each sample can provide more reliable bias mitigation than using a fixed strategy.