Segment Anything Model: automatic mask generation using transformers 🤗 library

This notebook demonstrates how to use the Segment Anything Model (SAM) to automatically generate segementation masks on any image. The model was released by Meta AI in the paper Segment Anything Model. The original source code can be found here

The mask-generation pipeline, freshly released for SAM, creates a gris of 1024 which are feed in a batch of points_per_batch to the model. The examples are inspired from the original notebook of the authors.

Utility functions

Run the cells below to import the needed utility functions for displaying the masks!

Model loading

Use the from_pretrained method on the SamForMaskGeneration class to load the model from the Hub! For the sake of this demonstration we will use the vit-huge checkpoint.

Load the example image

Generate the masks

Let's automatically generate the masks on the image! For that simply pass the raw image into the generator

The line above you take ~7 seconds on Google Colab 1xNVIDIA-T4, now let's see the resulting segmentation masks.

Batch of images

You can feed both urls and raw images. Here is an example: