In this paper, we present preliminary results on improving out-of-domain weather prediction and uncertainty estimation as part of the \texttt{Shifts Challenge on Robustness and Uncertainty under Real-World Distributional Shift} challenge. Our preliminary results show that by leveraging an ensemble of Bayesian models and thoughtful;y splitting the training set, we can achieve more robust and accurate results than standard libraries. We quantify our predictions using several metrics and propose several future lines of inquiry and experimentation to boost performance.