Deforestation of the Amazon rainforest is a major contributor to climate change, as it is a crucial precipitation regulator, as well as a large natural carbon reserve. While there have been efforts to create real-time algorithms for deforestation detection, they are oftentimes not accurate or interpretable. We leverage multiple input signals, such as satellite imagery, time-series of deforestation indices and scalar measures, to create a single deep learning model that is both interpretable and accurate. We employ a novel dataset with millions of annotated images of the Brazilian Amazon to train our model, as well as class activation mappings to investigate the added value of interpretability in this context.