As our society moves toward a decarbonized energy system, we need to improve our ability to model, predict, and understand power system behavior and dynamics.     The balance between generation and demand on short time scales is reflected by the power grid frequency, making it central to the control of power grids.    Hence, an accurate understanding and forecasting of power grid frequency could ease the planning of control actions and thus improve system stability and help save costs.     Whether deep learning approaches can provide forecasts of the highly resolved and noisy time series, as they are present in the case of power grid frequency, remains an open question.     In this paper, we find that the Temporal Fusion Transformer (TFT) is able to outperform baseline models, while a comparably simple multilayer perceptron is not.    By reducing the time resolution of the frequency time series,  we investigate and quantify the trade-off between the energy consumption and prediction performance % or forecasting accuracy?    of the TFT.    Furthermore, the inclusion of additional exogenous variables (e.g. calendar features, load, or generation) further improves the performance of the TFT.    Utilizing the TFT's inherent interpretability, we identify the forecasted load ramp, the current hour, and the current month as the most relevant features.