When Simpler Models Win: Evaluating Machine Learning Generalization for Climate-Aware 5G Signal Prediction

Machine learning models deployed in physical infrastructure must operate under environmental conditions that can change substantially over time, potentially affecting both the underlying system and model reliability. This study evaluates the generalization of machine learning models for predicting 5G signal strength under atmospheric variability using real-world measurements of signal strength, temperature, atmospheric pressure, and relative humidity collected at 30-minute intervals. Six approaches—Linear Regression, Decision Tree, Random Forest, XGBoost, Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU)—were compared across monthly atmospheric regimes. Model rankings varied substantially over time. October represented a difficult prediction regime, with LSTM and GRU achieving only weak positive R² values of 0.099 and 0.091, respectively, while conventional models produced negative R² values. Decision Tree achieved R² = 0.899 in November, whereas Linear Regression performed best under December harmattan conditions with R² = 0.882, outperforming Decision Tree (0.828), Random Forest (0.802), XGBoost (0.702), GRU (0.443), and LSTM (0.346). These findings show that greater model complexity does not necessarily yield greater predictive reliability and that performance under a single environmental distribution can provide an incomplete picture of real-world generalization. The study motivates climate-aware and temporally stratified evaluation of machine learning models for communication infrastructure and other environmentally coupled systems.