RBM - linear to binary layer

I am working in Matlab and developing a project which is focused on deep learning. Since I have to apply this technique to images, I thought that the usual binary-binary units are not appropriate since the images are gray scale and not binarized. So I tried to follow the code prof. Hinton provided on his website to write my implementation of a linear-to-binary rbm. In my idea this should be the first layer of the autoencoder: the following ones can be binary-to-binary.
So these are some lines of the code I wrote (the variables' names are quite self-explicative):
poshidprobs = sigmoid(data*vishid + repmat(hidbiases, numcases, 1));
batchposhidprobs(:,:,batch)=poshidprobs;
posprods = data' * poshidprobs; % a sort of correlation
poshidact = sum(poshidprobs);
posvisact = sum(data);
%%%%%%%%%END OF POSITIVE PHASE %%%%%%%%%%%%%%%%
poshidstates = poshidprobs > rand(numcases,numhid);
%%%%%%%%%START NEGATIVE PHASE %%%%%%%%%%%%%%%%%
% linear reconstruction
negdata = poshidstates*vishid' + repmat(visbiases,numcases,1);
% hidden units update
neghidprobs = sigmoid(negdata*vishid + repmat(hidbiases,numcases,1));
negprods = negdata'*neghidprobs;
neghidact = sum(neghidprobs);
negvisact = sum(negdata);
%%%%%%%%%END OF NEGATIVE PHASE %%%%%%%%%%%%%%%%%
err= sum(sum( (data-negdata).^2 ));
errsum = err + errsum;
%%%%%%%%%UPDATE WEIGHTS AND BIASES %%%%%%%
vishidinc = momentum*vishidinc + ...
epsilonw*( (posprods-negprods)/numcases - weightcost*vishid);
visbiasinc = momentum*visbiasinc + (epsilonvb/numcases)*(posvisact-negvisact);
hidbiasinc = momentum*hidbiasinc + (epsilonhb/numcases)*(poshidact-neghidact);
vishid = vishid + vishidinc;
visbiases = visbiases + visbiasinc;
hidbiases = hidbiases + hidbiasinc;
Does it make sense to you? Can you spot any mistake? The reason why I am asking is because the reconstruction error stays very high, like 500000 or so. So I was asking myself (and the community) whether this is normal or it indicates some problems and mistakes.

Answers (0)

Categories

Find more on Mathematics and Optimization in Help Center and File Exchange

Products

Tags

Asked:

on 5 May 2015

Edited:

on 5 May 2015

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!