The necessity to use non-invasive and fast methods for the diagnosis of pulmonary diseases is more pressing than ever with the emergence of the SARS-CoV-2. Given their documented performances on image classification tasks, deep learning algorithms constitute promising approaches that can rely on abundant data to achieve the detection of pulmonary diseases from chest X-rays. However, due to privacy issues and some patients refusing to disclose their chest X-rays, data can be scarce and hence prevent deep learning algorithms from achieving their optimal capacities. In this paper, we use a dataset containing a total of 270 training samples and 36 training samples with 4 classes (normal, bacterial pneumonia, viral pneumonia and COVID-19). Using the efficientnet-b5 architecture as the backbone, we investigate, by observing the model performances, if transfer learning can be used to overcome data scarcity when training a deep leaning model for a pulmonary disease detection task.