Reverse operation of horzcat, reshape and dec2bin code I have written.

Hello, I have been writing a very simple way of encoding text as part of a level 5 course I am undertaking currently. I have reached a point where I am happy with the encoded data but I have become a bit of a car crash trying to reverse the operation. The trouble is; I am a complete novice on both code and Matlab in general. My course is a communications course and not really code focused. I chose this as a project as I found is really interesting but I do seem to have underrestimated the complexity of the task somewhat.

The elements I am struggling with are:

 chr = ( secret_message );
 group_len = 8;
 len = numel( chr );
 d = rem( len, group_len );
 chr = horzcat( chr, repmat( char(255), 1, group_len-d ) );
 chr = reshape( chr, 8,[] );
 secret_message3 = chr(:,1:195)
 secret_messageF = chr(1:length(secret_message3))
 binX = dec2bin(secret_messageF,8) 

This takes an already encoded message and rearranges the layout before changing to binary.

Any help is greatly appreciated. I need to be able to get back to the top of this code at the ('secret_message') in its original format, which was ASCII.

Kind Regards

[dpb]

Well, think we would need more about what is the real problem here. W/o knowing the encryption, not sure how we would be expected to unencrypt, nor do you give much of a hint as to just where your "car crash" is...

[darren Clipson]

Hello dbp,

thanks for you comment.

The 'car crash' is actually in my head trying to process the inverse operation because of lack of knowledge and experience. I can't really apologise for that.

I understand the concatination effect from using horzcat. At this stage, my 1x1560 string becomes 8x195 array.

As for the code, well it could be anything arriving into this section I have posted. I am struggling to understand how to reverse the 'reshape' and 'horzcat' functions. I am not asking to unencrypt fully as there is much more to the code I have written but I already have those sections working. When I intoduce this code here, I have issues but I like what it does.

[DGM]

In order to know how to reverse (e.g. reshape) operations, one needs to know what they're trying to undo. As you said yourself, the prior code could be anything. It should stand to reason then that so is the current status of any possible answer.

If you'd rather avoid revealing your code, but need to understand how to reverse the transformation, you might try to come up with a minimal working example that is succinct and generalized enough that you're comfortable sharing it.

Answer 1

[Walter Roberson]

chr = reshape( chr, 8,[] ); 

chr will be 8 by something

secret_message3 = chr(:,1:195)

secret_message3 will be 8 by 195 (or an error if chr was not at least 195 columns)

secret_messageF = chr(1:length(secret_message3))

length(X) is defined as:

SZ = size(X);
if any(SZ == 0)
    LENGTH = 0
else
    LENGTH = max(SZ)
end

so it is 0 if the array is empty in any dimension, and otherwise is the longest dimension no matter which one that is .

With secret_messageF being 8 x 195, the longest dimension is 195, so length(secret_messageF) is 195.

So you are taking chr(1:195) which is indexing a 2D array with a single dimension. That operation is well defined in MATLAB, and means that you will get the first 195 consecutive memory locations out of the array. Arrays are stored "down" first, so A(1,1) then A(2,1) then A(3,1), A(4,1), A(5,1), A(6,1), A(7,1), A(8,1), A(1,2), A(2,2), A(3,2) and so on. chr has 8 rows, so the first 195 entries in memory is the first 24 full columns plus another 3 from the 25th column.

It seems pretty unlikely that that would be what you would want.

[darren Clipson]

Hi, thanks for the reply. I have made some progress actually over the last 24 hours. It does seem I need to spend a lot of time understanding the process more step by step.

Answer 2

[darren Clipson]

Ok,

What my code does is imports a text file and uses basic ASCII advancement using a key that is selectable by the user. I have removed some steps in the key application stage but I think you should be able to understand what is happening. At the output from the key advancement stage, there is a single 1 x 1560 string of ASCII characters. It is at this stage the rehape takes place and the concatination

[Walter Roberson]

So you do a Caesar Cypher on the upper case letters (and getting the Caesar Cypher wrong unless the key is restricted to be an integer in the range 0 to 26)

Then you take the encyphered text and pad it out to a multiple of 8 characters long. You reshape to 8 rows, using code that will break if group length is ever changed to anything other than 8.

You then index by the length of a variable that is not defined in the code, getting back a column vector. But why? And why did you bother to do the reshape into 8 rows if you are immediately going to extract a column vector?

Does the 8 of the group length have anything to do with 8 bits to the byte? Where do you convert to binary?

[darren Clipson]

Hi Walter,

Sorry for the late reply; I had some time working away from home this week.

To answer your questions:

The Ceasar Cypher does not only look aty the range 0-26, I just omitted that part as I wanted to just give you the idea of what is happening. I was initially sorting the characters based on upper case and lower case of the ASCII table and ignoring non-printable characters.

I changed it to consider all characters using the following:

for n=1:length(secret_message)
    
   % if 65<=secret_message(n) && 90>=secret_message(n)
    if 0<=secret_message(n) && 255>=secret_message(n)    
        if secret_message(n)+key >90
            
            secret_message(n)=secret_message(n)+key-26;
        else            
           secret_message(n)=secret_message(n)+key;
           
        end

The output of this is a single row string.

Once again, thank you for taking the time to read, and assist.

Daz

[Walter Roberson]

What does your code do if secret_message(n)+key-26 is still greater than 127 ? or if secret_message(n)+key < 0?

chr = horzcat( chr, repmat( char(255), 1, group_len-d ) );

Okay, chr gets padded if neeed and afterwards is a multiple of group_len long.

chr = reshape( chr, 8,[] ); 

And then it gets reshaped to 8 rows, even if the group_len is not a multiple of 8.

secret_message3 = chr(:,1:length(chr))

length(char) is 0 if chr is empty, and is otherwise the longer dimension. For for example if chr had been padded to 40 characters long, and got reshaped to 8 x 5, then length() would be 8. But if chr had been padded to 400 characters long and got reshaped to 8 x 50 then length() would be 50.

In the case where the number of elements of chr is less than 64, then chr(:,1:length(chr)) would be chr(:,1:8) which would break the code.

In the case where the number of elements in chr is at least 64, then chr(:,1:length(chr)) would be chr(:,1:size(chr,2)) which would lead chr unchanged. What is the point?

    chr =permute( chr, [2,1] );

Since chr is 2 dimensional, that would more commonly be done as

    chr = chr.';
    

Then you have

    secret_messageF =double(secret_messageF);

but if secret_messageF is not already double(), if it is uint8() or char() then

    secret_message(n)+key-26

the addition of the key might overflow the datatype. For example ￮ (U+FFEE) + 26 would overflow char() . You therefore either need to pay more attention to your order of operations, or else you need to double() much earlier than this in your code.

    binX = dec2bin(secret_messageF,8)

dec2bin() does not char whether the message is an integer data type or char() or double(), so it is not clear why you would have just done the double(), at least now.

So you took the secret_messageF that you had carefully arranged to be 8 x something, and then transposed to something x 8, and you convert it to binary with a minimum of 8 bits per entry. The bits of individual entries are arranged as rows; the rows are in order of linear indexing.

Suppose you had

    'ABCDEFGHIJKLMNOP'

then your reshape() by 8 would have gotten

    'AI'
    'BJ'
    'CK'
    'DL'
    'EM'
    'FN'
    'GO'
    'HP'

and the permute would have modified that to

    'ABCDEFGH'
    'IJKLMNOP'

and dece2bin would be

    
    '01000001'            %A
    '01001001'            %I
    '01000010'            %B
    '01001010'            %J
    '01000011'            %C
    '01001011'            %K
    '01000100'
    '01001100'
    '01000101'
    '01001101'
    '01000110'
    '01001110'
    '01000111'
    '01001111'
    '01001000'
    '01010000'

... What's that all about? What is the point of that padding to a multiple of group_len and then reshaping to 8 rows (even if group_len is something different), and then transposing, and then dec2bin()? Why are you shuffling the order like that? It isn't that you can't do that, but you did not put in any comments to explain why you do that, and people trying to read your code are left trying to figure out if the value of 8 for group_len and for the hard-coded reshape() and for the bits per entry for the binary are just coincidence or have some deeper meaning.

... and what you are you going to do if your original characters include characters in positions greater than 255, so that they need more than 8 bits?

[darren Clipson]

It's going to take some time going through your answer and understanding.

Your question: 'What does your code do if secret_message(n)+key-26 is still greater than 127 ? or if secret_message(n)+key < 0?'

To answer that: I am not sure entirely but I am happy to print the encoded 'secret_message' with characters above 127. The only characters I want to exclude are 00-31 but currently my 'secret_message' is a simple text file with no non-printable characters.

I have characters such as underscores in the inported text file. I was under the impression Matlab would use the extended ASCII table 0 - 255 and this would be ok?

[Walter Roberson]

Suppose the input file contains

恭喜发财

and the user chooses a key of 7918. What is your code going to do?

As UTF16-BE that file would be hex 60 6D 55 9C 53 D1 8D 22. As UTF-8 the file would be hex E6 81 AD E5 96 9C E5 8F 91 E8 B4 A2

[darren Clipson]

I have implimented a menu for the key selection, currently it only allows key selection upto a value of 16.

In the case on an input file not following the ASCII format, the code will surely not function.

[Walter Roberson]

Is your plan to fix the code to handle larger character range? Or is your plan to put in error checking to reject such files?

Please remind us which version of the Caeser Cypher you are using:

1. All values except upper and lower case English letters are left alone, and only letters are encyphered? Upper-case letters are encyphered to upper-case letters and lower-case letter are encyphered to lower-case letters? Or
2. All values are encyphered regardless of range, so 'z' with key 1 would become '{'

[darren Clipson]

I am trying to cypher all values, with the exception of 00 -127. This is proving difficult in itself. In fact any key over a value of 5 returns character errors.

This manifests when characters at the end of the upper case are used (X,Y,Z ), or characters actually within the 91 - 97 'bucket', such as underscores as they are incorrenctly handled. I can either return correctly for the upper case letters, or the special characters inside the 91-97 locations, but not both.

[Walter Roberson]

I suggest you have a look at https://www.mathworks.com/matlabcentral/answers/466636-caesar-cyphor-encryption-problem?s_tid=srchtitle

[darren Clipson]

That’s super helpful! Thanks so much for your time and help Walter. I have progressed quite a lot today with my understanding :)