Semantic Priors Meet Statistical Evidence: Robust Forests for Few-Shot Tabular Learning

Few-shot tabular learning is challenging because conventional tree-based methods suffer from unstable split selection and leaf estimation when labeled examples are scarce. Although feature engineering based on Large Language Models (LLMs) provides useful semantic priors, it can be brittle when table semantics are incomplete or numerical evidence is unstable. This raises two key challenges: achieving semantic-numerical robustness and calibrating multiple weak signals under sparse supervision. Therefore, we present LEAF, a train-time LLM-guided forest that treats LLM-derived features and split preferences as calibratable priors rather than direct supervision. During tree induction, LEAF aligns semantic guidance with task-specific statistical evidence, including teacher-informed predictive signals and data-dependent partition statistics, through a unified split-scoring rule. At prediction time, it further stabilizes local estimates by blending sparse empirical observations with teacher-informed support at each leaf. Experiments on few-shot tabular classification and regression datasets show that LEAF consistently outperforms strong conventional and LLM-based baselines, improving classification performance by 1.4 points and reducing regression error by 0.149 over the strongest previous method. Robustness analyses further demonstrate that LEAF is more stable than prior LLM-based feature engineering methods under missing table semantics and numerical perturbations induced by distributional variation across tabular feature spaces and schema descriptions.