How to map detected Braille dots (and their XY coordinates) into letters ( 0's 1's matrices of 3*2)?

Hello everyone,

I am working on a project to detect Braille dots in an image and should convert them to text letters. I could detect the dots and get their X,Y coordinates. Now, I am looking for a way to obtain a matrix of 01s representing dots or white spaces to later interpret them into letters, so for example if I found the letter F on the image, the program maps it into a matrix of

[ 1 1

1 0

0 0 ]

And then it maps that matrix to the character 'F' and prints it out.

Find my code below.

img1 = rgb2gray(imread("braille2.JPG"));
img2 = rgb2gray(imread("dot2.JPG"));
corr = normxcorr2(img2,img1);
surf(corr), shading flat
sizeA = size(img2,1);
sizeB = size(img2,2);
t = 0.75;
figure(1)
imshow(img1); hold on;
i = 1;
j = 1;
x_values = zeros(sizeA, 1);
y_values = zeros(1, sizeB);
coordinates = cell(sizeA,sizeB);
while max(max(corr)) >= t
    [r,c] = find(ismember(corr,max(max(corr))));
     corr((r - floor((sizeA)/2)):(r + floor((sizeA)/2)),(c-floor((sizeB)/2)):(c+floor((sizeB)/2))) = 0;
     img1((r - floor((sizeA)/2)),(c-floor((sizeB)/2))) = 255;
     plot(c-floor((sizeB)/2),(r - floor((sizeA)/2)),".",'markersize', 16, "LineWidth",2);
     x_values(i) = c-floor((sizeB)/2);
     y_values(i) = r - floor((sizeA)/2);
     i = i + 1;
end
XYcoordinates = [x_values(:), y_values(:)];
XYcoordinates = sortrows(XYcoordinates);
disp(XYcoordinates);

Find the images below. Thanks in advance.

Answer 1

[KSSV]

You can make your all required matrices into a 3D. And compare the obtained matrix out of it (read about ismember), get the index and get the respective alphabet .

a = [ 1 1
 1 0 
 0 0 ] ;   % say this is the matrix obtained from dots 
idx = 6 ;  % say this the index of matrix a from your framed 3D matrices 
% get alphabet 
char(idx+'A'-1)

ans = 'F'

[Adham Khalifa]

Yes, I was thinking about that. I am just not sure how to "obtain" a matrix out of what I have. I have the XY coordinates of all the dots. How to make a matrix out of that? Thanks.

[KSSV]

You should be having predefined patterns right? Like you said

[ 1 1
 1 0 
 0 0 ]

corresponds to F.

[Adham Khalifa]

I am sorry, I am a bit confused. All what I have as you see is just the XY coordinates of the dots. How to convert those coordinates to a matrix of 01s, so that LATER I can check if it corresponds with any predefined matrices of letters?