YOLO V2 with greyscale FITS images (telling me it's expecting rgb)

Hello,

I am trying to train a YOLO v2 networks using a fits format of image. I have modified the 'imageDatastore' function to read them instead of more traditional data formats.

[table1]=read_write_fits(1,'path_to_dir/');
traintbl=table1(:,'VAR');
imdsTrain = imageDatastore(table1{:,'imagefilename'},'ReadFcn',@fitsreadTwo);
bldsTrain = boxLabelDatastore(traintbl);
trainData = combine(imdsTrain, bldsTrain);

'fitsreadTwo' is present due to the issue that i'm having. when i attempt to use the 'yoloyv2Layers' function, it's telling me that it's expecting [height width 3] images despite mine being in greyscale 11 bit. When I input the following code and attempt to put [Height Width 1](greyscale) the yolov2layers function is telling me It's expecting [height width 3]. How is it coming up with what it's expecting? Why can I not modify it to have the correct input size? I've even tried swapping out the first layer of the network, but that creates compile errors down the line. Any help here or general guidelines will be spectacular. Thanks.

numAnchors = 7;
[anchorBoxes,~] = estimateAnchorBoxes(trainData,numAnchors);
featureExtractionNetwork = resnet50;
layers = [imageInputLayer([height width 1],'Name','input_1') ];
featureLayer = 'activation_40_relu';
lgraph=layerGraph(featureExtractionNetwork);
inputSize = [height width 3];
lgraph_rl = replaceLayer(lgraph,'input_1',layers);
numClasses = 1;
lgraph2 = yolov2Layers(inputSize,numClasses,anchorBoxes,lgraph_rl,featureLayer);

Here is my fix for the problem: I make the greyscale image into an rgb image with the following code:

function [image]=fitsreadTwo(filename)
%the purpose of this script is to read a fits image and store it as an RGB
%even tho it is greyscale. 
image=fitsread(filename);
map=hsv(8200);
image=ind2rgb(image,map);

Answer 1

[Srivardhan Gadila]

As per the documentation of yolov2layers, the Input Argument imageSize should be specified as one of these values:

• Two-element vector of form [H W] - For a grayscale image of size H-by-W
• Three-element vector of form [H W 3] - For an RGB color image of size H-by-W

Along with the above information the documentation page of yolov2layers says that

"yolov2Layers uses a pretrained neural network as the base network to which it adds a detection subnetwork required for creating a YOLO v2 object detection network. Given a base network, yolov2Layers removes all the layers succeeding the feature layer in the base network and adds the detection subnetwork. The detection subnetwork comprises of groups of serially connected convolution, ReLU, and batch normalization layers. The YOLO v2 transform layer and YOLO v2 output layer are added to the detection subnetwork. If you specify the name-value pair 'ReorgLayerSource', the YOLO v2 network concatenates the output of reorganization layer with the output of feature layer." And you can refer to creating a custom YOLO v2 network layer-by-layer at Create YOLO v2 Object Detection Network.

Hence in order to use the network for grayscale input the pretrained network which is given to the yolov2layers function should also take grayscale input i.e, it's imageInputLayer must have the number of channels as 1. As a workaround you can make use of the ColorPreprocessing property of the augmentedImageDatastore.

[Ryan Comeau]

Yes, I've been unclear, my bad. I've tried this already using the regular @fitsread instead of the modified one in the above question. I obtain the following compile error when specifying:

[H W] to the function:

Error using yolov2Layers>iReplaceImageInputLayer (line 295)

Input size mismatch. Size of input to this layer [2560 2560 1] is different from the expected input size [2560 2560 3].

Error in yolov2Layers (line 112)

lgraph = iReplaceImageInputLayer(lgraph,params.ImageSize);

Error in main (line 60)

lgraph2 = yolov2Layers(inputSize,numClasses,anchorBoxes,lgraph_rl,featureLayer);

Here is the code that yields this compile error:

[table1]=read_write_fits(1,'/path_to_dir');
traintbl=table1(:,'VAR');
imdsTrain = imageDatastore(table1{:,'imagefilename'},'ReadFcn',@fitsread);
bldsTrain = boxLabelDatastore(traintbl);
trainData = combine(imdsTrain, bldsTrain);
% %test set

numAnchors = 7;
[anchorBoxes,~] = estimateAnchorBoxes(trainData,numAnchors);
featureExtractionNetwork = resnet50;
featureLayer = 'activation_40_relu';
lgraph=layerGraph(featureExtractionNetwork);
inputSize = [2560 2560];
numClasses = 1;
lgraph2 = yolov2Layers(inputSize,numClasses,anchorBoxes,lgraph,featureLayer);

options = trainingOptions('adam',...
    'InitialLearnRate',0.01,...
    'LearnRateSchedule','piecewise', ...
    'LearnRateDropFactor',0.01, ...
    'LearnRateDropPeriod',1, ...
    'Verbose',true,...
    'MiniBatchSize',4,...
    'MaxEpochs',2,...
    'Shuffle','never',...
    'VerboseFrequency',1,...
    'CheckpointPath',tempdir);

[detector,info] = trainYOLOv2ObjectDetector(trainData,lgraph2,options);

Could this be because I'm using a transfer learning technique? Why is it expecting an image size?

[Srivardhan Gadila]

@Ryan Comeau I have updated the answer with more information.

[Ryan Comeau]

Excellent, thank you.

[ceng wang]

@Ryan Comeau Could you share that how did you fix the issue please? Any words would be appreciated. Thanks a lot.