InstaSAM: Instance-Aware Segment Any Nuclei Model with Point Annotations

Abstract

Weakly supervised nuclei segmentation methods have been proposed to simplify the demanding labeling process by primarily depending on point annotations. These methods generate pseudo labels for training based on given points, but their accuracy is often limited by inaccurate pseudo labels. Even though there have been attempts to improve performance by utilizing power of foundation model e.g., Segment Anything Model (SAM), these approaches require more precise guidance (e.g., box), and lack of ability to distinguish individual nuclei instances. To this end, we propose InstaSAM, a novel weakly supervised nuclei instance segmentation method that utilizes confidence of prediction as a guide while leveraging the powerful representation of SAM. Specifically, we use point prompts to initially generate rough pseudo instance maps and fine-tune the adapter layers in the image encoder. To exclude unreliable instances, we selectively extract segmented cells with high confidence from pseudo instance segmentation and utilize these for the training of binary segmentation and distance maps. Owing to their shared use of the image encoder, the binary map, distance map, and pseudo instance map benefit from complementary updates. Our experimental results demonstrate that our method significantly outperforms state-of-the-art methods and is robust in few-shot, shifted point, and cross-domain settings. The code will be available upon publication. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.