Clear Filters
Clear Filters

pre-allocating disk space for matfile objects containing cell arrays of large sparse matrices

1 view (last 30 days)
I have a custom function that consumes O(n^2) RAM in intermediate steps and generates very sparse outputs. The current strategy is to slice inputs into RAM-fitting blocks and assemble the output from each block: https://www.mathworks.com/matlabcentral/answers/1742830-slicing-matrices-to-fit-in-memory
Now I am looking at temporarily store these outputs on disk pointed by a matfile object to make room for processing of the next slice. According to this it is preferrable to pre-allocate disk for matfile to achieve better performance: https://www.mathworks.com/help/matlab/import_export/troubleshooting-file-size-increases-unexpectedly-when-growing-an-array.html
However I am not sure how to do it for my case. Should I just use this?
TempMAT.A(i,j)=mat2cell(spalloc(m,n,ceil(SparsityEstimate*m*n)),m);
%A{i,j} is a m-by-n sparse matrix with a estimated sparasity of SparsityEstimate

Answers (1)

Prateekshya
Prateekshya on 9 Nov 2023
Hi Yi-xiao,
As per my understanding, you want to improve the performance for large matrices. The piece of code that you have given looks alright. Since you are dealing with sparse matrices, there is a way to maintain sparse matrices in MATLAB. For more information, you can also have a look at this link: https://in.mathworks.com/help/matlab/ref/sparse.html
I hope it helps!

Products


Release

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!