# FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentation

**Authors:** Luca Sestini, Benoit Rosa, Elena De Momi, Giancarlo Ferrigno, Nicolas Padoy  
**arXiv:** [2202.08141](https://arxiv.org/abs/2202.08141)  
**Date:** 2022-02-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2202.08141v1/

## Key points

- Automatic surgical instrument segmentation in endoscopic images is crucial for computer-assisted applications in minimally invasive surgery.
- Current approaches rely on manual annotation for ground-truth supervision signals, which can be expensive and time-consuming.
- FUN-SIS is a fully unsupervised approach for binary surgical instrument segmentation.
- It trains a per-frame segmentation model using unlabelled endoscopic videos and leverages implicit motion information and instrument shape-priors as shape templates of realistic segmentation masks.
- Shape-priors can be obtained from various sources, including recycling existing annotations from other datasets.
- FUN-SIS uses a generative-adversarial approach that performs unsupervised instrument segmentation using optical flow images during training.
- The obtained instrument masks are used as pseudo labels to train the per-frame segmentation model.
- A learning from noisy labels architecture is developed to handle the noise introduced by these pseudo labels and extract clean supervision signals from them.
- The proposed method is validated on three surgical datasets, including the MICCAI 2017 EndoVis Robotic Instrument Segmentation Challenge dataset.
- FUN SIS achieves fully unsupervised surgical instrument segmentation results comparable to state-of-the-art fully supervised approaches.
- This research demonstrates the potential of leveraging large amounts of unlabelled data produced in minimally invasive surgery through an innovative unsupervised approach.
- By reducing reliance on manual annotation and utilizing implicit motion information and shape priors, FUN SIS offers an efficient and cost-effective solution for surgical instrument segmentation in computer-assisted applications.

## Summary

In the field of minimally invasive surgery, automatic surgical instrument segmentation in endoscopic images is a crucial component for computer-assisted applications. However, current state-of-the-art approaches heavily rely on manual annotation for ground-truth supervision signals which can be expensive and time-consuming to collect at a large scale. To address this limitation, the authors propose FUN-SIS, a Fully UNsupervised approach for binary Surgical Instrument Segmentation. FUN-SIS trains a per-frame segmentation model using completely unlabelled endoscopic videos. Instead of relying on manual annotations, the approach leverages implicit motion information and instrument shape-priors as shape templates of realistic segmentation masks. These shape-priors can be obtained from various sources including recycling existing annotations from other datasets. The authors introduce a novel generative-adversarial approach that performs unsupervised instrument segmentation using optical flow images during training. The obtained instrument masks are then used as pseudo labels to train a per frame segmentation model. To handle the noise introduced by these pseudo labels, the authors develop a learning from noisy labels architecture that extracts clean supervision signals from them. The proposed method is validated on three surgical datasets including the MICCAI 2017 EndoVis Robotic Instrument Segmentation Challenge dataset. The results show that FUN SIS achieves fully unsupervised surgical instrument segmentation results that are comparable to state of the art fully supervised approaches. This research demonstrates the potential of leveraging large amounts of unlabelled data produced in minimally invasive surgery through an innovative unsupervised approach. By reducing the reliance on manual annotation and utilizing implicit motion information and shape priors, FUN SIS offers an efficient and cost effective solution for surgical instrument segmentation in computer assisted applications.