Applying Dirichlet BCs using speye - error

I'm working on an FE-code in matlab. To apply Dirichlet BCs I using the following code:

K(:, dirichlet) = 0;
K(dirichlet, :) = 0;
K(dirichlet, dirichlet) = speye(size(dirichlet, 1));

where K is sparse and dirichlet are the DOFs where to apply Dirichlet BCs. I've been testing the code and increasing the modelsize using a linear hexahedral element. The code runs fine untill 54^3 elements (K: 499125x499125, 39e6nnz, dirichlet: 9075). At this point, the code takes 1.38sec to run.

Then I increase to 55^3 (K: 526848x526848, 41e6nnz, dirichlet: 9408) and the last line does not finish anymore. I checked memory, no problems there at all with over 4GB of RAM still available.

Any ideas as of why this is happening?