Reading and plotting synchrotron data

Good morning,

I get a lot of data by synchrotron measurements (type: *.dat). The structure of the data is as follows: One file containes the temperatures, this is already manually imported as one column with 512 rows. Now I have for all temperature-steps one seperate *.dat-file with three columns: first is the theta-value (x), the second is my intensity (y) and the third is the error (all together of course 512 files!). I need the for the plot the second column only of all files as one final matrix. Later the temperature will be the x-value, the theta-value will be the y and the intensity the z-value. That means that I am looking for a loop to import from all *.dat-files only the second column, the theta-value I will import seperately (handish). As result I need a 512x20000-matrix for the plot. How can I write this import-loop?

Many thanks for your help,

Thomas

Answer 1

[dpb]

Where does the 20000 come from? Is that how many observations/measurements are in each of the 512 files, I guess?

Doesn't seem difficult, what's the file-naming convention and storage arrangement? Presuming there's a root name for each set or each set is saved in a folder of its own, simply iterating over the result of

nT=512; nO=20000;                                       % sizes; be better to set dynamically
data(nT,nO)=0;                                          % preallocate to size
d=dir(fullfile('WorkingDirectory','FileNameX*.dat'));
for i=1:numel(d)
  data(:,i)=readmatrix(fullfile(d(i).Folder,d(i).Name),'Range','B:B');
end

I'd be inclined to test the performance in comparison to just read the full file and save only the desired column over the specific one-column-only read operation -- and also to read/save all the data going to need in the first operation instead of making multiple passes over the same files.

>> 512*20000/1024/1024*8
ans =
         78.13
>>

is "only" about 80 MB as double, that's just not all that much data any more.