Same Information, More Chances: Representation Multiplicity in Column-Subsampled Tree Ensembles
Om Lala
Abstract
Redundant predictors are known to affect feature-importance estimates in tree ensembles, but their effect on fitted predictions under stochastic feature subsampling is less well characterized. Exact feature duplication is a representation change that adds columns without adding conditional information. We evaluate this intervention in a completed synthetic Random Forest grid and an OpenML-CC18 audit covering 11 numeric binary-classification tasks and 2460 deduplicated real-data cells. At clone-family share $\rho\approx0.50$, median prediction disagreement is 0.008443 and median probability displacement is 0.01172, with median absolute AUC movement of 3.23e-04. Disabling column subsampling attenuates disagreement sharply for XGBoost and LightGBM, and varying the feature-subsampling fraction gives the expected dose response. Signal and independently generated noise families have similar median disagreement, but grouped permutation AUC drop is 0.03057 for signal and approximately zero for noise. Duplicate-column effects are concentrated near low-margin predictions.
Chat is not available.
Successful Page Load