How to detect and count kiwi fruit on a tree?

I need to make a matlab program that can count kiwi fruits on a tree. How can I do this? Are there are any examples, codes, tutorials? Ccan you please help me?

Answer 1

[Image Analyst]

Click on the fruit tag on the right hand side of this page. You'll probably see examples where the Color Thresholder App on the Apps tab of the tool ribbon is used.

Do your images look like this:

Try LAB color space to segment out brown fruit. Anything else will be sky or leaves. Then you can get an idea of the harvest potential by computing the area fraction of the brown. You can calibrate by harvesting a few scenes and making a calibration curve relating area fraction to number of individual fruits, if you insist on an individual fruit count (which is actually probably not needed).

% Demo by Image Analyst, May, 2021.
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clearvars;
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 16;
fprintf('Beginning to run %s.m ...\n', mfilename);

%-----------------------------------------------------------------------------------------------------------------------------------
% Read in image.
folder = [];
baseFileName = 'kiwis.jpg';
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
	% The file doesn't exist -- didn't find it there in that folder.
	% Check the entire search path (other folders) for the file by stripping off the folder.
	fullFileNameOnSearchPath = baseFileName; % No path this time.
	if ~exist(fullFileNameOnSearchPath, 'file')
		% Still didn't find it.  Alert user.
		errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
		uiwait(warndlg(errorMessage));
		return;
	end
end
rgbImage = imread(fullFileName);
[rows, columns, numberOfColorChannels] = size(rgbImage)
% Display the test image full size.
subplot(2, 2, 1);
imshow(rgbImage, []);
axis('on', 'image');
caption = sprintf('Original Image : "%s"', baseFileName);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Set up figure properties:
% Enlarge figure to full screen.
hFig1 = gcf;
hFig1.Units = 'Normalized';
hFig1.WindowState = 'maximized';
% Get rid of tool bar and pulldown menus that are along top of figure.
% set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
hFig1.Name = 'Demo by Image Analyst';

%--------------------------------------------------------------------------------------------------------
% Threshold image.
[mask, maskedRGBImage] = createMask(rgbImage);

% Display mask image.
subplot(2, 2, 2);
imshow(mask, []);
axis('on', 'image');
title('Initial Color Segmentation Mask', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hold on;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Display masked image of kiwis
subplot(2, 2, 3);
imshow(maskedRGBImage, []);
axis('on', 'image');
title('Initial Color Segmentation Mask', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hold on;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Display masked image of leaves and sky
% Mask the image using bsxfun() function to multiply the mask by each channel individually.  Works for gray scale as well as RGB Color images.
leaves = bsxfun(@times, rgbImage, cast(~mask, 'like', rgbImage));
subplot(2, 2, 4);
imshow(leaves, []);
axis('on', 'image');
title('Leaves and Sky', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hold on;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

fprintf('Done running %s.m\n', mfilename);

function [BW,maskedRGBImage] = createMask(RGB)
%createMask  Threshold RGB image using auto-generated code from colorThresholder app.
%  [BW,MASKEDRGBIMAGE] = createMask(RGB) thresholds image RGB using
%  auto-generated code from the colorThresholder app. The colorspace and
%  range for each channel of the colorspace were set within the app. The
%  segmentation mask is returned in BW, and a composite of the mask and
%  original RGB images is returned in maskedRGBImage.

% Auto-generated by colorThresholder app on 22-May-2021
%------------------------------------------------------


% Convert RGB image to chosen color space
I = rgb2lab(RGB);

% Define thresholds for channel 1 based on histogram settings
channel1Min = 1.176;
channel1Max = 79.447;

% Define thresholds for channel 2 based on histogram settings
channel2Min = -4.567;
channel2Max = 31.611;

% Define thresholds for channel 3 based on histogram settings
channel3Min = -5.773;
channel3Max = 70.934;

% Create mask based on chosen histogram thresholds
sliderBW = (I(:,:,1) >= channel1Min ) & (I(:,:,1) <= channel1Max) & ...
	(I(:,:,2) >= channel2Min ) & (I(:,:,2) <= channel2Max) & ...
	(I(:,:,3) >= channel3Min ) & (I(:,:,3) <= channel3Max);
BW = sliderBW;

% Initialize output masked image based on input image.
maskedRGBImage = RGB;

% Set background pixels where BW is false to zero.
maskedRGBImage(repmat(~BW,[1 1 3])) = 0;

end

[Mohandis Bencaus]

Thank you for helping sir, how can i count kiwis, also how can i use histogram, filtering, noise in this code.

[Image Analyst]

@Mohandis Bencaus, there are functions for all those. Just look up histogram(), imnoise(), and filtering in the documentation. Of course there are a bazillion types of filters so you'll have to figure out exactly how you want to filter it. There is no kiwi fruit filter in any MATLAB toolbox so you'll have to design your own.

[Mohandis Bencaus]

@Image Analyst Hi sir, i couldnt figured out how can count kiwifruits yets can you help me please.

[Image Analyst]

Like I said:

"Then you can get an idea of the harvest potential by computing the area fraction of the brown. You can calibrate by harvesting a few scenes and making a calibration curve relating area fraction to number of individual fruits, if you insist on an individual fruit count (which is actually probably not needed)."

So that means you count them manually for a bunch of images and make up a formula that gives you the count if you give it the area fraction. You can count them manually with ginput() or drawpoint(). Make sure you train it with enough images to span the amount of fruit you expect to see, anywhere from none or one up to nearly the entire image.

[Mohandis Bencaus]

@Image Analyst

Sir i tried something but not works well, it count wrong, for some images it counts 0 always. Can you look

RGB2 = imread('kiwis6.jpg');
cform = makecform('srgb2lab');
lab_he = applycform(RGB2,cform);
ab = double(lab_he(:,:,2:3));
nrows = size(ab,1);
ncols = size(ab,2);
ab = reshape(ab,nrows*ncols,2);
nColors = 3;
[cluster_idx, cluster_center]=kmeans(ab,nColors,'distance','sqEuclidean','Replicates',3);
pixel_labels = reshape(cluster_idx,nrows,ncols);
imshow(pixel_labels,[]);
segmented_images = cell(1,3);
rgb_label = repmat(pixel_labels,[1 1 3]);
for k = 1:nColors
 color = RGB2;
 color(rgb_label ~= k) = 0;
 segmented_images{k} = color;
end

imshow(segmented_images{2}), title('objects in cluster 2');

sekil = segmented_images{2};
hy=fspecial('sobel');
hx=hy;
Iy=imfilter(double(sekil),hy,'replicate');
Ix=imfilter(double(sekil),hx,'replicate');
gradmag=sqrt(Ix.^6+Iy.^6);
figure, imshow(gradmag);

imwrite(gradmag,'sekil.jpg');
sekil1 = imread("sekil.jpg");

se=strel('ball',17,6);
Io=imopen(sekil1,se);
figure, imshow(Io);

r = Io(:, :, 1);
g = Io(:, :, 2);
b = Io(:, :, 3);

justGreen = g;

bw = justGreen > 50;
imagesc(bw);
colormap(gray);
% bw = justGreen > 10;
% imagesc(bw);
% colormap(gray);

bw = imfill(bw,'holes');
SE =strel('disk', 20);   
bw2 = imerode(bw,SE);
[L,num] = bwlabel(bw2); 
display(num)

[Image Analyst]

What you need to do:

1. First, show me the code where you had the user manually count the fruit on the training images.
2. Next show me the table you generated.

You should have a table like

Image #, # manually counted fruit, Area fraction from IA
1        100                       .129
2        149                       .152
3        239                       .205
etc.

Show it to me. Then to get the conversion from area fraction to count, do

% Visualize your training data in a scatter plot.
scatter(areaFraction, count, 'filled');
% Make a quadratic formula to convert area fraction to count.
coefficients = polyfit(areaFraction, counts, 2);
% Now you can plug in some arbitrary area fraction and get the count:
estimatedCount = coefficients(3) * thisAreaFraction ^2 + ...
    coefficients(2) * thisAreaFraction + coefficients(1)

[Mohandis Bencaus]

@Image Analyst I am sorry , i am little new on image processing and matlab. I used different tecqinue in this program above. I mixed coin counter program's codes with kiwi fruit detection but it did not work well.

How can i generate the table and calculate fraction ?, i am sorry again sir, i am really confused. How should i modify the code you sent as answer to calculate fraction and generate table. I have really no idea.

[Mohandis Bencaus]

@Image Analyst sir how can i calculate area fraction of the brown and create table like you said. I have to make this matlab program.

[Image Analyst]

Try this manual counting program. Adapt as needed.

% Demo to manually count items in an image by clicking on them.
clc;	% Clear command window.
fprintf('Running %s.m ...\n', mfilename);
clear;	% Delete all variables.
close all;	% Close all figure windows except those created by imtool.
workspace;	% Make sure the workspace panel is showing.

% Specify the folder where the files live.
myFolder = pwd; %'C:\Users\yourUserName\Documents\My Pictures';
% Check to make sure that folder actually exists.  Warn user if it doesn't.
if ~isfolder(myFolder)
	errorMessage = sprintf('Error: The following folder does not exist:\n%s\nPlease specify a new folder.', myFolder);
	uiwait(warndlg(errorMessage));
	myFolder = uigetdir(); % Ask for a new one.
	if myFolder == 0
		% User clicked Cancel
		return;
	end
end
% Get a list of all files in the folder with the desired file name pattern.
filePattern = fullfile(myFolder, 'kiwi*.jp*'); % Change to whatever pattern you need.
theFiles = dir(filePattern);
for k = 1 : length(theFiles)
	baseFileName = theFiles(k).name;
	fullFileName = fullfile(theFiles(k).folder, baseFileName);
	fprintf(1, 'Now reading %s\n', fullFileName);
	% Now do whatever you want with this file name,
	% such as reading it in as an image array with imread()
	imageArray = imread(fullFileName);
	imshow(imageArray);  % Display image.
	drawnow; % Force display to update immediately.
	g = gcf;
	g.WindowState = 'maximized';
	
	again = true;
	count = 1;
	x = [];
	y = [];
	while again
		if count == 1
			% Give user instructions.
			promptMessage = sprintf('Click on one kiwi');
			titleBarCaption = 'Continue?';
			buttonText = questdlg(promptMessage, titleBarCaption, 'Continue', 'Quit', 'Continue');
			if contains(buttonText, 'Quit', 'IgnoreCase', true)
				count = 0;
				break; % or break or continue.
			end
		end
		p = drawpoint('Color', 'y');
		x(count) = p.Position(1);
		y(count) = p.Position(2);
		caption = sprintf('Counted %d so far', count);
		title(caption, 'FontSize', 18);
		
		promptMessage = sprintf('Locate another kiwi?');
		titleBarCaption = 'Continue?';
		buttonText = questdlg(promptMessage, titleBarCaption, 'Continue', 'Quit', 'Continue');
		if contains(buttonText, 'Quit', 'IgnoreCase', true)
			again = false;
			break; % or break or continue.
		end
		count = count + 1;
	end
	% Save the x and y for this particular image.
	xy{k} = [x(:), y(:)];
end
message = sprintf('You counted %d items in this image', count);
uiwait(helpdlg(message));
fprintf('Done running %s.m\n', mfilename);

Answer 2

[DGM]

Let us assume the subject is a spherical red kiwi tree in an evenly illuminated vacuum.

You might begin your attempt by choosing a color space within which the color properties of the objects of interest are most easily seperable from those of the background. If you have Image Processing Toolbox, see the Color Thresholder app. The rest is simply logical comparison.

inpict = imread('tree2.png');

labpict = rgb2lab(inpict);

% Define thresholds based on observation of histogram
Lrange = [50 70];
Arange = [70 79];
Brange = [60 70];

% Create mask based on a logical combination of thresholds
mask = (labpict(:,:,1) >= Lrange(1) ) & (labpict(:,:,1) <= Lrange(2)) & ...
	(labpict(:,:,2) >= Arange(1) ) & (labpict(:,:,2) <= Arange(2)) & ...
	(labpict(:,:,3) >= Brange(1) ) & (labpict(:,:,3) <= Brange(2));

% identify connected groups and return just the count
[~,numberoffruits] = bwlabel(mask)

The script identifies 11 fruits in the image.

It's worth pointing out that there are actually 20 fruits on the tree. As this illustrates, practical applications often have trivial complications which require non-trivial efforts to accomodate. Your task may be far more challenging than this oversimplified example.

[Mohandis Bencaus]

@DGM Thank you for explaining. How can apply this to kiwifruit on tree.