MOSInversion: Knowledge distillation-based incremental learning in organ segmentation using DeepInversion

Abstract

Despite recent advancements in multi-organ segmentation (MOS) of medical images, existing models are limited in terms of extending their capability to unseen classes. Incremental learning has been proposed to enable models to learn new classes progressively, possibly using multiple datasets from different institutions. In this setting, models easily experience performance degradation on previously learned classes i.e., catastrophic forgetting. Although many methods have been proposed to mitigate this issue, applying them to medical imaging applications like multi-organ segmentation is not easy due to the large memory requirement when used for 3D medical data such as CT scans or the need for additional training of a generator for image synthesis. In this paper, we propose an incremental learning framework that leverages diverse synthetic images to retain the knowledge learned from previously seen data. We design MOSInversion to generate the synthetic images by utilizing a pre-trained model from the previous step. MOSInversion generates diverse images by using segmentation masks so that we can manipulate the shape, location, and size of organs. We evaluate our proposed method using three abdominal CT datasets (FLARE21, MSD, and KiTS19) and achieve state-of-the-art accuracy.