Rashomon Sets Across Tree Ensembles: Predictive Multiplicity in Gradient Boosting and Random Forests

Many classifiers can be nearly indistinguishable in validation loss yet disagree on who receives which prediction. This predictive multiplicity is usually studied within one hypothesis class, leaving a practical question unresolved: does the choice of a standard tree-ensemble family systematically change multiplicity? We compare gradient-boosted trees (GBM) and random forests (RF) on 15 binary-classification datasets using repeated train/validation/test splits, equal hyperparameter-search budgets, class-relative near-best loss bands, seed-only candidate pools, and exactly matched test sizes. Across 71 valid split pairs, dataset-averaged hard-label disagreement is 0.0200 for GBM versus 0.0101 for RF—a 1.98× ratio. GBM is higher on 14 of 15 datasets (exact paired Wilcoxon p = 1.83 × 10⁻⁴; bootstrap 95% confidence interval for the mean gap [0.00625, 0.01376]), while mean accuracy and AUC differ by only 0.16 and 0.005 percentage points. Disagreement concentrates sharply near the reference model’s decision boundary. Importance-rank agreement is also lower for GBM, but top-five feature overlap is not, yielding a deliberately mixed interpretability result. Sensitivity analyses over set size, complete-split datasets, and leave-one-dataset-out re-estimation preserve the primary conclusion. The study shows that model-family choice can materially alter prediction arbitrariness even when conventional predictive performance barely moves.