MATLAB Answers

Hash function for Matlab struct

251 views (last 30 days)
Is there a function to gererate a single Hash value from a whole struct? For background information: I am storing all Preferences for an algorithm in a struct. Before processing, I'd like to check, whether this cobination has been processed before without comparing all the single settings.
Thanks in advance!

  1 Comment

Rik
Rik on 1 Mar 2017
This is why I love this forum and the FEX. I need to do exactly this, and as I need to include the hash in a filename, it is not possible to compare the structs.

Sign in to comment.

Accepted Answer

Jan
Jan on 16 Mar 2011
Edited: Jan on 1 Mar 2017
[EDITED] See FEX: DataHash for a complete version.
If you calculate a hash value for a struct, each field must be processed also. Therefore I do not see a big advantage for using a hash for comparing. I suggest using a simple copy of the struct and ISEQUAL.
But if you really want a struct Hash:
  • EDITED: Consider non numeric values:
  • EDITED (30-Mar-2011): Bugs fixed, LOGICAL, empty arrays, shape of data
  • EDITED (31-Mar-2011); Function handles
function H = DataHash(Data)
Engine = java.security.MessageDigest.getInstance('MD5');
H = CoreHash(Data, Engine);
H = sprintf('%.2x', H); % To hex string
function H = CoreHash(Data, Engine)
% Consider the type of empty arrays:
S = [class(Data), sprintf('%d ', size(Data))];
Engine.update(typecast(uint16(S(:)), 'uint8'));
H = double(typecast(Engine.digest, 'uint8'));
if isa(Data, 'struct')
n = numel(Data);
if n == 1 % Scalar struct:
F = sort(fieldnames(Data)); % ignore order of fields
for iField = 1:length(F)
H = bitxor(H, CoreHash(Data.(F{iField}), Engine));
end
else % Struct array:
for iS = 1:n
H = bitxor(H, CoreHash(Data(iS), Engine));
end
end
elseif isempty(Data)
% No further actions needed
elseif isnumeric(Data)
Engine.update(typecast(Data(:), 'uint8'));
H = bitxor(H, double(typecast(Engine.digest, 'uint8')));
elseif ischar(Data) % Silly TYPECAST cannot handle CHAR
Engine.update(typecast(uint16(Data(:)), 'uint8'));
H = bitxor(H, double(typecast(Engine.digest, 'uint8')));
elseif iscell(Data)
for iS = 1:numel(Data)
H = bitxor(H, CoreHash(Data{iS}, Engine));
end
elseif islogical(Data)
Engine.update(typecast(uint8(Data(:)), 'uint8'));
H = bitxor(H, double(typecast(Engine.digest, 'uint8')));
elseif isa(Data, 'function_handle')
H = bitxor(H, CoreHash(functions(Data), Engine));
else
warning(['Type of variable not considered: ', class(Data)]);
end
If the struct contains large arrays, James Tursa's TYPECAST implementation will save processing time, because it does not create a deep copy: typecast-c-mex-function

  2 Comments

Jan
Jan on 3 May 2011
NOTE: The shown function replies the same hash for "struct('a', 1, 'b', 2)" and "struct('a', 2, 'b', 1)"! Using BITXOR does not consider the order of data. The fieldnames are not considered also. A better (and faster!) method is to use just Engine.update. See the FEX submission: http://www.mathworks.com/matlabcentral/fileexchange/31272-datahash
Stephen Cobeldick
Stephen Cobeldick on 2 Mar 2017
I accepted this answer as it clearly answers the question.

Sign in to comment.

More Answers (3)

Benjamin Bechtel
Benjamin Bechtel on 16 Mar 2011
Thanks to both of you. Maybe I should specify the problem. I'm not just looking for a way to compare structs but for a short identifier for a full struct. The idea is, that I can store the results for specific settings in a file with the ID as name (e.g. hex of the MD5). This way I can see from the filelist, if this parameter-set has been processed without opening each file or making a seperate list.
I like the idea to use the java-class. However, not all of the elements are numeric, so the typecast doesn't work.
Any idea how to get a (more or less) unique id for a struct?
Regards, Benni

  2 Comments

Jan
Jan on 16 Mar 2011
What types do the fields have? It would be helpful, if you post such details... I'll update my function to catch cells also.
Jan
Jan on 2 Mar 2017
This answer has been accepted by John BG. I have no idea why he prefers this question asked by the author. Therefore I've unaccepted this answer.

Sign in to comment.


Francois Rongère
Francois Rongère on 28 Mar 2011
Hello,
Did you find an answer to your question because i am also interested in that feature.
I also have all my simulation parameters in a structure (arrays, scalar values, function handles, strings, cell arrays...) and i would like to refer to one simulation run by its unique identifier based on the structure. The goal is also to put those identifier in a light database in order to check wether a simulation has already been run or not...
Kind regards,
François.

  8 Comments

Show 5 older comments
Francois Rongère
Francois Rongère on 31 Mar 2011
Yes you are right for the overloaded names...
I don't really know if this is frequently needed but I think I am not the only one having this kind of preoccupation. When you have to launch hundreds of simulations with slight modifications in the input parameters as it is done in an optimization process, it is very useful to have a mean to name each simulation with a unique identifier. It allows to store informations in databases, acces back to specific results, data mining...
As using an incremental number is not consistent through the use of the same code on several machines, it is of use to base the name on the input parameters. Your code is an answer for this kind of problem.
Then yes I think it could help people if you post it on FEX !
Francois Rongère
Francois Rongère on 5 Apr 2011
I can tell you now that your code and modifications are part of my phd code... Thank you (you are in comments of my source code).
Jan
Jan on 5 Apr 2011
You are welcome. It seems to be more important for you than for Benjamin ;-)

Sign in to comment.


Arindam Bose
Arindam Bose on 9 Feb 2013
Well thanks a ton Jan Simon. Probably you never know you have helped me a lot in several projects. Wherever I go and search for something in this community, I find you. Now my problem is how can i decode this MD5 hash. Is it possible to get back the original main text from the generated hash? Can you help me please. I don't have much knowledge of JAVA.

  2 Comments

Walter Roberson
Walter Roberson on 9 Feb 2013
No, MD5 hashes are irreversible. They are also "hashes", so by definition many different original items "hash" to the same value.
Jan
Jan on 10 Feb 2013
@Arindam: I'm glad that some of my postings have been useful for others. Then it has been worth to spend the time.
MD5 creates a 8 byte hash (128 bit) for the input. Therefore input with more than 8 must create the same hash as another input with less than 8 byte, a so called collision. In consequence you can only reconstruct an input reliably, when you now that it is shorter. Beside the time-consuming brute force attacks, you find a lot of research about cracking MD5 faster. You find some explanations at http://en.wikipedia.org/wiki/MD5 .
Why do you need to get the clear text of an MD5 sum? Because this conflicts with the purpose and design of a hash sum, there could be a better method to get what you need.

Sign in to comment.

Tags