Main Content

extractEmbeddings

Extract image embeddings from Medical Segment Anything Model (MedSAM) encoder

Since R2024b

    Description

    embeddings = extractEmbeddings(medsam,I) extracts the embeddings of the input medical image I from the encoder of a Medical Segment Anything Model (MedSAM), medsam, by running a forward pass on the image encoder.

    Note

    This functionality requires Deep Learning Toolbox™, Computer Vision Toolbox™, and the Medical Imaging Toolbox™ Model for Medical Segment Anything Model. You can install the Medical Imaging Toolbox Model for Medical Segment Anything Model from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

    example

    Examples

    collapse all

    Create a Medical Segment Anything Model (MedSAM) object.

    medsam = medicalSegmentAnythingModel;

    Load a heart ultrasound image into the workspace. Visualize the image.

    img = imread("heartUltrasoundImage.png"); 
    figure
    imshow(img,[])

    Extract the image embeddings from the heart ultrasound image.

    embeddings = extractEmbeddings(medsam,img);

    Visualize some of the embeddings, which capture the features of the image.

    figure
    montage(embeddings(:,:,1:6),DisplayRange=[])

    Input Arguments

    collapse all

    Medical Segment Anything Model, specified as a medicalSegmentAnythingModel object.

    Image or batch of images from which to extract embeddings, specified as a 2-D numeric matrix, 3-D numeric array, or 4-D numeric array depending on the type and number of images.

    Type of ImageNumber of ImagesData Format
    Grayscale imageSingle 2-D image2-D numeric matrix of size H-by-W
    Grayscale imageBatch of B 2-D images4-D numeric array of size H-by-W-by-1-by-B
    RGB imageSingle 2-D image3-D numeric array of size H-by-W-by-3
    RGB imageBatch of B 2-D images4-D numeric array of size H-by-W-by-3-by-B

    H and W are the height and width, respectively, of the input image I. To run the segmentation on a GPU, specify the input image I explicitly as a gpuArray (Parallel Computing Toolbox).

    Data Types: single | double | int8 | int16 | int32 | uint8 | uint16 | uint32

    Output Arguments

    collapse all

    Image embeddings extracted from the Medical Segment Anything Model encoder, returned as a 64-by-64-by-256 or 64-by-64-by-256-by-B numeric array, depending on the type and number of input images.

    Number of ImagesEmbeddings Format
    Single grayscale or RGB image64-by-64-by-256 array
    Batch of B grayscale or RGB images64-by-64-by-256-by-B array

    The image embeddings are representations of corresponding input images that encode the spatial information, features, and context of the input images.

    Extended Capabilities

    Version History

    Introduced in R2024b

    Go to top of page