Teaching Tabular Learners to See Curvature: A Model-Agnostic Differential Feature Layer

Tabular learners consume flat features, yet a task’s predictive structure resides in the conditional response surface’s local differential geometry (curvature signature), unexposed by flat columns. We diagnosed if learners capture this signature. A study revealed a systematic blind spot: linear models are structurally blind; tree ensembles corrupt second- order signals; and foundation models only partially capture first-order, not second-order. Motivated, we propose CAFA, a model-agnostic layer distilling a smooth surrogate’s curvature signature into ordinary features for any learner. On 45 TabArena-Lite datasets, CAFA-based representations frequently outperform baselines. Promising early results, but partial, highlight differential geometry’s importance for tabular representation.