Multi-Modal Observational Data For Weather Forecasting

Existing AI weather forecasting systems have made strong progress in the last few years, but focus on predicting re-analysis targets (e.g., ERA5). AI forecasting systems share the same shortcomings of the re-analysis process including high computation cost, known non-physical artifacts, and oversampling in particular regions of the globe. In order to facilitate the development of end-to-end weather forecasting methods which bypass the need for re-analysis at operation time, we propose constructing a dataset of multi-modal weather observations containing weather stations measurements, microwave and infrared sounder results, and geospatial imagery. This dataset would help extend the results from other domains (e.g., computer vision, natural language) to weather forecasting by leveraging state-of-the-art multi-modal techniques, advancing both ML and weather forecasting. New methods enabled by this dataset will help reduce the train-serve discrepancy, and improve the operational usefulness of AI-driven weather forecasting.