Deep learning models for medical image classification are typically trained to predict pre-determined radiological findings and cannot incorporate a novel disease at test time efficiently. Retraining an entirely new classifier is often out of question due to insufficient novel data or lack of compute/base disease data. Thus, learning fast adapting models is essential. Few-shot learning has shown promise for efficient adaptation to new classes at test time, but literature revolves primarily around single-label natural images distributed over a large number of different classes. However, this setting differs notably from the medical imaging domain, where images are multilabel, of fewer total categories, and retention of base label predictions is desired. In this paper, we study incremental few-shot learning for low- and multilabel medical image data to address the problem of learning a novel disease with few finetuning samples while retaining knowledge over base findings. We show strong performance on incrementally learned novel disease labels for chest X-rays with strong performance retention on base classes.