bigimageDatastore

(Not recommended) Datastore to manage blocks of big image data

Since R2019b

The bigimageDatastore object is not recommended. Use the blockedImageDatastore object instead. For more information, see Compatibility Considerations.

Description

A bigimageDatastore object manages a collection of image blocks that belong to one or more bigimage objects. A bigimageDatastore is analogous to an imageDatastore, which manages a collection of unrelated images.

Creation

Syntax

bigds = bigimageDatastore(images)

bigds = bigimageDatastore(images,levels)

bigds = bigimageDatastore(images,levels,Name,Value)

bigds = bigimageDatastore(images,'BlockLocationSet',blockLocationSet)

bigds = bigimageDatastore(images,'BlockLocationSet',blockLocationSet,Name,Value)

Description

Create Datastore that Reads Blocks Over Entire Image

bigds = bigimageDatastore(images) creates a datastore that manages a collection of image blocks at the finest resolution level of one or more bigimage objects, Images.

bigds = bigimageDatastore(images,levels) creates a datastore that manages a collection of image blocks of one or more bigimage objects, Images, at the specified resolution levels, Levels.

bigds = bigimageDatastore(images,levels,Name,Value) also uses name-value pairs to set one or more Properties except for BlockLocationSet. You can specify multiple name-value pairs. Enclose each property name in quotes.

Example: bigimageDatastore(bigimg,3,'BlockSize',[128 128],'IncompleteBlocks','pad') creates a datastore that reads blocks of size 128-by-128 at resolution level 3 from big image bigimg and zero-pads partial edge blocks.

Create Datastore that Reads Blocks at Specified Locations

bigds = bigimageDatastore(images,'BlockLocationSet',blockLocationSet) creates a datastore that reads blocks from bigimage objects, Images, using the resolution level, block size, and block positions specified by BlockLocationSet.

bigds = bigimageDatastore(images,'BlockLocationSet',blockLocationSet,Name,Value) also uses name-value pairs to set one or more of the BorderSize, IncompleteBlocks, PadMethod, and ReadSize properties. You can specify multiple name-value pairs. Enclose each property name in quotes.

Example: bigimageDatastore(bigimg,'BlockLocationSet',bls,'ReadSize',4) creates a datastore that reads four blocks at a time from big images bigimg according to the position, block size, and resolution level specified by bls.

Properties

BlockLocationSet — Block locations

blockLocationSet object

Block locations, specified as a blockLocationSet object.

BlockOffsets — Block offsets

1-by-2 vector of positive integers

Block offsets, specified as 1-by-2 vector of positive integers of the form [numrows numcols].

The default value is equal to BlockSize. To overlap blocks during calls to read, specify a smaller value. To add a gap between blocks, specify a larger value.

BlockSize — Block size

1-by-2 vector of positive integers

Block size of read data, specified as a 1-by-2 vector of positive integers of the form [numrows numcols]. The default value is equal to the BlockSize property of the first big image in Images at the first resolution level in Levels.

BorderSize — Border size

[0 0] (default) | 1-by-2 vector of nonnegative integers

Border size, specified as a 1-by-2 vector of nonnegative integers of the form [m n]. The function adds m rows above and below each block and n columns to the left and right of each block with data from the neighboring blocks. For blocks that lie on the edge of an image, data is padded according to IncompleteBlocks. By default, the datastore does not add a border to blocks.

Images — Big images

b-element vector of bigimage objects

Big images that supply blocks for the bigimageDatastore, specified as a b-element vector of bigimage objects. To read different resolution levels from the same big image, specify the same image multiple times in the vector.

IncompleteBlocks — Method to handle edge blocks

"same" (default) | "exclude" | "pad"

Method to handle edge blocks that are smaller than BlockSize, specified as one of these values.

Value	Meaning
"same"	Return data of the same size as the edge block.
"exclude"	Do not include edge blocks in calls to read.
"pad"	Pad incomplete blocks to the same size as BlockSize using the pad method specified by PadMethod.

Levels — Resolution level

positive integer | b-element vector of positive integers

Resolution level of blocks from each big image in Images, specified as a positive integer scalar or a b-element vector of positive integers. If you specify a scalar value, then all big images supply blocks to the datastore at the same resolution level.

Data Types: double

PadMethod — Pad method

numeric scalar | string scalar | "replicate" | "symmetric"

Pad method of incomplete edge blocks, specified as one of these values. By default, the datastore pads numeric blocks with 0 and categorical blocks with missing.

Value	Meaning
numeric scalar	Pad numeric array with elements of constant value.
string scalar	Pad categorical array with the specified class in the Classes property of the underlying bigimage.
"replicate"	Pad by repeating border elements of array.
"symmetric"	Pad array with mirror reflections of itself.

ReadSize — Number of blocks to return

1 (default) | positive integer

Number of blocks to return in each call to read, specified as a positive integer.

Object Functions

combine	Combine data from multiple datastores
countEachLabel	(Not recommended) Count number of pixel labels for each class of bigimageDatastore object
hasdata	Determine if data is available to read
numpartitions	Number of datastore partitions
partition	(Not recommended) Partition bigimageDatastore
preview	Preview subset of data in datastore
read	(Not recommended) Read data from bigimageDatastore
readRelative	(Not recommended) Read neighboring block from bigimageDatastore using relative position
reset	Reset datastore to initial state
shuffle	Shuffle data in datastore
transform	Transform datastore
isPartitionable	Determine whether datastore is partitionable
isShuffleable	Determine whether datastore is shuffleable

Examples

Create Big Image Datastore and Specify Block Size

Create a bigimage using a modified version of image "tumor_091.tif" from the CAMELYON16 data set. The original image is a training image of a lymph node containing tumor tissue. The original image has eight resolution levels, and the finest level has resolution 53760-by-61440. The modified image has only three coarse resolution levels. The spatial referencing of the modified image has been adjusted to enforce a consistent aspect ratio and to register features at each level.

bim = bigimage('tumor_091R.tif');

Display the default block size of the bigimage at each resolution level. The block size is a 2-element vector of the form [numrows, numcols].

t = table((1:3)',bim.BlockSize,'VariableNames',["Level" "Block Size"]);
disp(t)

    Level     Block Size 
    _____    ____________

      1      1024    1024
      2      1024    1024
      3      1024    1024

Display the bigimage by using the bigimageshow function.

bigimageshow(bim);

Create a bigimageDatastore at resolution level 1. Specify a nondefault block size. Set the datastore to read four blocks at a time.

bimds = bigimageDatastore(bim,2,'BlockSize',[512 512],'ReadSize',4)

bimds = 
  bigimageDatastore with properties:

            ReadSize: 4
          BorderSize: [0 0]
           PadMethod: 0
              Images: [1x1 bigimage]
              Levels: 2
           BlockSize: [512 512]
        BlockOffsets: [512 512]
    IncompleteBlocks: 'same'
    BlockLocationSet: [1x1 blockLocationSet]

Read one batch of data from the datastore. Notice that the third block is a partial edge block and has a smaller size than interior blocks. Display the returned image patches as a montage. The montage displays the third block with a thicker border because the width of the block is smaller than the width of the complete blocks.

blocks = read(bimds)

blocks=4×1 cell array
    {512x512x3 uint8}
    {512x512x3 uint8}
    {512x316x3 uint8}
    {512x512x3 uint8}

montage(blocks,'Size',[1 bimds.ReadSize],'BorderSize',5,'BackgroundColor','k');

Read the next batch of data from the datastore and display the returned image patches as a montage. The montage displays partial blocks with a thicker border because the dimensions of the blocks are smaller than the dimensions of the full block.

blocks = read(bimds)

blocks=4×1 cell array
    {512x512x3 uint8}
    {512x316x3 uint8}
    {226x512x3 uint8}
    {226x512x3 uint8}

montage(blocks,'Size',[1 bimds.ReadSize],'BorderSize',5,'BackgroundColor','k');

Read the last batch of data from the datastore. The read operation returns a partial batch that contains the only remaining patch. Display the patch.

blocks = read(bimds)

blocks = 1x1 cell array
    {226x316x3 uint8}

montage(blocks,'Size',[1 bimds.ReadSize],'BorderSize',5,'BackgroundColor','k');

Create Big Image Datastore and Specify Mask

Create a bigimage using a modified version of image "tumor_091.tif" from the CAMELYON16 data set. The original image is a training image of a lymph node containing tumor tissue. The original image has eight resolution levels, and the finest level has resolution 53760-by-61440. The modified image has only three coarse resolution levels. The spatial referencing of the modified image has been adjusted to enforce a consistent aspect ratio and to register features at each level.

bim = bigimage("tumor_091R.tif");

Display the bigimage by using the bigimageshow function.

h = bigimageshow(bim);

Create a mask at the coarsest resolution level, retaining the original spatial referencing information.

clevel = bim.CoarsestResolutionLevel;
imcoarse = getFullLevel(bim,clevel);
stainMask = ~imbinarize(im2gray(imcoarse));
bmask = bigimage(stainMask,"SpatialReferencing",bim.SpatialReferencing(clevel));

Specify the location of blocks to read from the bigimage by using the selectBlockLocations function. Set the block size as 256-by-256 pixels. Select blocks that are at least 75% within the ROI defined by the mask by specifying the "InclusionThreshold" name-value pair argument. By default, selectBlockLocations selects blocks from the finest resolution level of the big image.

t = 0.75;
blockSize = [256 256];
blockSet = selectBlockLocations(bim,"BlockSize",blockSize, ...
    "Masks",bmask,"InclusionThreshold",t);

Create a bigimageDatastore that reads four blocks at a time from the locations specified by the blockLocationSet.

bimds = bigimageDatastore(bim,"BlockLocationSet",blockSet,"ReadSize",4);

To preview which patches are read by the datastore, display the mask over the original bigimage using the same block size and inclusion threshold. The overlay highlights in green the patches that are at least 75% within the ROI defined by the mask.

showmask(h,bmask,"BlockSize",blockSize,"InclusionThreshold",t)

Read the first batch of data from the datastore and display the returned image patches as a montage. The content of these patches matches the green blocks of the overlay.

blocks = read(bimds);
montage(blocks,"Size",[1 bimds.ReadSize],"BorderSize",5,"BackgroundColor","k");

Read Big Image Blocks From Specified Block Locations

Create a bigimage using a modified version of image "tumor_091.tif" from the CAMELYON16 data set. The original image is a training image of a lymph node containing tumor tissue. The original image has eight resolution levels, and the finest level has resolution 53760-by-61440. The modified image has only three coarse resolution levels. The spatial referencing of the modified image has been adjusted to enforce a consistent aspect ratio and to register features at each level.

bim = bigimage('tumor_091R.tif');

Display the entire bigimage at the finest resolution level.

bshow = bigimageshow(bim);

Specify four [x y] block locations from the finest level. The first two blocks overlap in the vertical direction. The second two blocks are adjacent horizontally.

xyLocations = [ ...
    2800 1300; ...
    2800 1400; ...
    1500 2400; ...
    1800 2400];
blockSize = [300,300];

All blocks are from the same image. Specify the image number as 1 for all blocks.

imageNumber = [1 1 1 1]';

Create a blockLocationSet object that stores block locations.

locationSet = blockLocationSet(imageNumber,xyLocations,blockSize);

Create a bigimageDatastore object that reads blocks from big image bim at locations specified by the blockLocationSet object.

bimds = bigimageDatastore(bim,'BlockLocationSet',locationSet);

Read two blocks at a time from the datastore and display them in a montage.

bimds.ReadSize = 2;
while hasdata(bimds)
    figure
    blocks = read(bimds);
    montage(blocks,'BorderSize',5,'BackgroundColor','b');
end

References

[1] Bejnordi, Babak Ehteshami, Mitko Veta, Paul Johannes van Diest, Bram van Ginneken, Nico Karssemeijer, Geert Litjens, Jeroen A. W. M. van der Laak, et al. “Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer.” JAMA 318, no. 22 (December 12, 2017): 2199–2210. https://doi.org/10.1001/jama.2017.14585.

Version History

Introduced in R2019b

R2021a: bigimageDatastore is not recommended

Starting in R2021a, the bigimageDatastore object, which reads data from bigimage objects, is no longer recommended. Instead, use the blockedImageDatastore object, which reads data from blockedImage objects. The blockedImage object offers several advantages including extension to N-D processing, a simpler interface, and custom support for reading and writing nonstandard image formats.

Code Updates

Update all instances of the bigimageDatastore object.

Discouraged Usage	Recommended Replacement
This example creates a bigimageDatastore object. bim = bigimage('tumor_091R.tif'); bimds = bigimageDatastore(bim);	Here is equivalent code, replacing the bigimageDatastore object with the new blockedImageDatastore object. bim = blockedImage('tumor_091R.tif'); bimds = blockedImageDatastore(bim);

The blockedImageDatastore object supports some different properties and functions than the bigimageDatastore object. For example, the blockedImageDatastore object does not support the BlockOffset and IncompleteBlocks properties, and it adds support for the TotalNumBlocks property. The blockedImageDatastore object does not support the readRelative function.

R2020b: The Mask and InclusionThreshold properties of bigimageDatastore are not recommended

Starting in R2020b, the Mask and InclusionThreshold properties of bigimageDatastore are no longer recommended. Instead, first specify a mask and inclusion threshold as input arguments to the selectBlockLocations function. Then, specify the resulting blockLocationSet object as an input to bigimageDatastore.

The table shows some typical usages of bigimageDatastore and how to update your code to use selectBlockLocations.

Not Recommended	Recommended
b = bigimageDatastore(bigimg,1,'Masks',m)	bls = selectBlockLocations(bigimg,'Masks',m); b = bigimageDatastore(bigimg,'BlockLocationSet',bls)
b = bigimageDatastore(bigimg,3,'Masks',m,'InclusionThreshold',t)	bls = selectBlockLocations(bigimg,'Levels',3, ... 'Masks',m,'InclusionThreshold',t); b = bigimageDatastore(bigimg,'BlockLocationSet',bls)