background subtraction to detect human

I have images from a fence in which

1: image A fence background with no human

2: fence climbed by human.

i tried image subtract but its not clearing out the comman background properly.

Any idea how to detect or even see the human only image with background subtracted

thanks

[Matt J]

We don't have your images.

[tren]

well here are the samples

[tren]

[tren]

somehow need to segment out the humans in the static background... Dont want to use DNN or CNN for this job.

Any advice please

thanks

[Matt J]

Embedding the images in your post will make changes to them. It's better if you attach the images in a .mat file so we can work with the same data variables that you are working with.

[Matt J]

The images from both files are identical, and neither one looks like the ones you display above. Also, they contain lots of extraneous variables that we don't need. Please just put both images in a single .mat file and leave out the other stuff. Both should be able to fit in a single 1.2 MB file.

[tren]

Sorry about that,

please now check the two files

thanks

Answer 1

[Matt J]

Something like this, perhaps?

load data

[r1,b1,g1]=imsplit(img1);
[r2,b2,g2]=imsplit(img2);
mask = imopen( abs(r1-r2)>50 & abs(b1-b2)>50 & abs(g1-g2)>50 , strel('disk',2));

stats=regionprops(mask, 'SubarrayIdx');

for i=1:numel(stats)
  mask(stats(i).SubarrayIdx{:})=1;
end

maskedImg=uint8(double(img2).*mask);
immontage({img2,maskedImg});

[tren]

Thanks for you reply,

The code is a bit static and not generic; not working on different images.

[Matt J]

Naturally, it isn't. You only gave us one example pair of images.

If you cannot use my example to find a more robust threshold rule, it's a job that probably wil require a DNN.

[Matt J]

You might also try this version below. However, we really need more test examples, so we can see what conditions we can rely on in the images.

load data

f=@(z)stdfilt(z);
D=imfeature(img1-img2,f);
 
r=regionprops((D>max(D(:))/4),'Subarrayidx');

mask=false(size(D));
for i=1:numel(r)
  mask(r(i).SubarrayIdx{:})=1;
end

mask=bwpropfilt(mask,'FilledArea',1);

maskedImg=uint8(double(img2).*mask);
immontage({img2,maskedImg});

function D=imfeature(img,f)

 img=imopen(img,strel('disk',3));
 [r,b,g]=imsplit(img);

 r=f(r); g=f(g); b=f(b);

 D=vecnorm(cat(3,r,g,b),2,3);

end

Answer 2

[Image Analyst]

If the common background is not zero then the images are not registered (aligned) or the brightness (pixel values) of the background are not the same. Note that the camera angle has shifted lsightly so that the poles are not in the same position, and it's later in the day so that the fence shadow is shorter/narrower, and the position of the razor wire has also shifted slightly. If the camera angle is supposed to be fixed, then you can set a fixed mask to exclude things like the razor wire.

Background subtraction is not the best way, but if you do, make sure you use imabsdiff intead of a regular subtraction. Then threshold and mask.

Try this:

% Demo by Image Analyst
% Initialization steps:
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 16;
fprintf('Beginning to run %s.m at %s...\n', mfilename, datetime('now','TimeZone','local','Format','HH:mm:ss'));

%--------------------------------------------------------------------------------------------------------
%    READ IN FIRST TEST IMAGE
folder = [];
baseFileName = 'Fence1.png';
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
    % The file doesn't exist -- didn't find it there in that folder.
    % Check the entire search path (other folders) for the file by stripping off the folder.
    fullFileNameOnSearchPath = baseFileName; % No path this time.
    if ~exist(fullFileNameOnSearchPath, 'file')
        % Still didn't find it.  Alert user.
        errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
        uiwait(warndlg(errorMessage));
        return;
    end
end
rgbImage1 = imread(fullFileName);
%--------------------------------------------------------------------------------------------------------
% Display the image.
subplot(2, 2, 1);
imshow(rgbImage1);
impixelinfo;
axis('on', 'image');
title('First image - early in the day', 'FontSize', fontSize, 'Interpreter', 'None');
% Maximize window.
g = gcf;
g.WindowState = 'maximized';
g.Name = 'Demo by Image Analyst';
g.NumberTitle = 'off';
drawnow;

%--------------------------------------------------------------------------------------------------------
%    READ IN SECOND TEST IMAGE
folder = [];
baseFileName = 'Fence2.png';
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
    % The file doesn't exist -- didn't find it there in that folder.
    % Check the entire search path (other folders) for the file by stripping off the folder.
    fullFileNameOnSearchPath = baseFileName; % No path this time.
    if ~exist(fullFileNameOnSearchPath, 'file')
        % Still didn't find it.  Alert user.
        errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
        uiwait(warndlg(errorMessage));
        return;
    end
end
rgbImage2 = imread(fullFileName);
%--------------------------------------------------------------------------------------------------------
% Display the image.
subplot(2, 2, 2);
imshow(rgbImage2);
impixelinfo;
axis('on', 'image');
title('Second image - later in the day', 'FontSize', fontSize, 'Interpreter', 'None');

%--------------------------------------------------------------------------------------------------------
%    CONSTRUCT THE DIFFERENCE IMAGE
diffImage = imabsdiff(rgbImage1, rgbImage2);
% Convert to grayscale
diffImage = max(diffImage, [], 3);
% Increase contrast so we can see it.
diffImage = imadjust(diffImage);
%--------------------------------------------------------------------------------------------------------
% Display the image.
subplot(2, 2, 3);
imshow(diffImage, []);
impixelinfo;
axis('on', 'image');
title('Difference Image', 'FontSize', fontSize, 'Interpreter', 'None');

%--------------------------------------------------------------------------------------------------------
% THRESHOLD THE DIFFERENCE IMAGE TO CREATE A MASK.
lowThreshold = 85;
highThreshold = 255;
% [lowThreshold, highThreshold] = threshold(83, 255, diffImage);
mask = diffImage >= lowThreshold & diffImage <= highThreshold;
% Get rid of differences touching the border due to some kind of
% frame enclosing the original images.
mask = imclearborder(mask);
% There is lots of clutter due to misalignment and shadows so take just the largest one.
mask = bwareafilt(mask, 1);
% Display the image.
subplot(2, 2, 4);
imshow(mask, []);
impixelinfo;
axis('on', 'image');
title('Mask Image', 'FontSize', fontSize, 'Interpreter', 'None');
fprintf('Done running %s.m at %s...\n', mfilename, datetime('now','TimeZone','local','Format','HH:mm:ss'));

[tren]

Thanks Image Analyst,

your code is working but can you please let me know the details of how to control object extraction.

For example in another image when i applied the code, it masked out the actual target.

please see the attached image where yellow highlighted person was the target but the mask removed it.

Can you please guide as to how to control the masking?

thanks

[Image Analyst]

Explain what you want better. I got a mask of the different part. You can use this as a mask against the original image if you want, but I'm sure that is not the final information you want. What do you want? You can get things like area and centroid from regionprops.

The image you just attached is nothing like the original image. If you have the Computer Vision toolbox, there is a forebround detector, but what is "foreground" is a judgment call.

[tren]

I want only to get the human out of the picture, just as in the last attached image. The yellow part is the human ob fence top. I want to just get this part to decide whether there is human or not on fence.

How can I mask out a particular info? Is it through thresholding?

[Image Analyst]

This is what you attached last:

It's a completely different image. I don't know of any algorithm that will find humans in every possible scene, especially if they are partially obscured or wearing camouflage. You're taking on too tough a task especially for a novice in image processing. I suggest you try to narrow down the types of images you use if you're going to use traditional methods. You might have to use Deep Learning but with that you'll have to have lots (hundreds or thousands) of images where you've told it where the human is so that you can train a network.