Resampling a Data Array for LSTM Training

Hi,

I am preparing data for training an LSTM network with a sequenceInputLayer. The data is from ANSYS which uses a variable step solver. The time steps in the captured data varying from 1.0E-04 sec to 9.0E-09 sec (average of 5.4E-05 sec). The 4 data sets are imported as approx. 594x7 doubles. I have had help on another post regarding using the interp1 function to help resize the sequences lengths.

My questions is can I resample the data to say 5.0E-05 sec using the resample function? And can the resample function be used on the array as well as a single observation?

I have included the code below as to my progress in using the resample function.

Thanks in advance,

Patrick

%Data Resampling
%Import Chemkin Data from Folder Location - Data captured with Variable Step Solver - Average step size = 5.38E-05
dataraw_03 = xlsread('Data_Ethanol_Raw\export_Ethanol_20%_440t_0.3.csv'); % 574x7 double
dataraw_04 = xlsread('Data_Ethanol_Raw\export_Ethanol_20%_440t_0.4.csv'); % 594x7 double
dataraw_05 = xlsread('Data_Ethanol_Raw\export_Ethanol_20%_440t_0.5.csv'); % 595x7 double
dataraw_06 = xlsread('Data_Ethanol_Raw\export_Ethanol_20%_440t_0.6.csv'); % 596x7 double

%Extract Observations from Array for Resampling
time_03         = dataraw_03(:,1); % Column 1 all rows from first data set
volume_03       = dataraw_03(:,2); % Column 2 all rows from first data set
crankAngle_03   = dataraw_03(:,3); % Column 3 all rows from first data set
temperature_03  = dataraw_03(:,4); % Column 4 all rows from first data set
pressure_03     = dataraw_03(:,5); % Column 5 all rows from first data set
co2_03          = dataraw_03(:,6); % Column 6 all rows from first data set
no_03           = dataraw_03(:,7); % Column 7 all rows from first data set

%Resample Data for LSTM SequenceInputLayer with a target sample rate of 5.0E-05 sec or 20 KHz
timeInterval          = 5.0e-5; % Target sample rate [s]
timeInterval_min      = 9.0e-8; % Smallest sample rate captured from ANSYS variable rate solver [s]

fs                    = 1/timeInterval;     % Target sample frequency of 20,000 [Hz]
f_down                = 1/timeInterval_min; % Downsampling frequency [Hz]

resampledvolume_03    = resample(volume_03,fs,f_down); % Resampled the volume @ 5.0E-05 sec

Answer 1

[Ben]

Hi Patrick,

In principle I think you can use Signal Processing resample - however you may want to verify this resample behaves as you expect. For example I tried this following toy example that demonstrates this type of signal resampling may not always be appropriate:

% Get timeseries from a variable step solver.

[t,y] = ode45(@vdp1,[0,1],[1;0]);

% Resample to a fixed rate of 1e4

samplerate = 1e4;

[yresample,tresample] = resample(y,t,samplerate);

% For comparison compute a timeseries resample

% - this uses interpolation according to getinterpmethod(ts);

% - This was 'linear' for me, so should match interp1

ts = timeseries(y,t);

tsresample = resample(ts,0:samplerate:1);

% Plot everything to compare

plot(t,y(:,1));

hold on;

plot(tresample,yresample(:,1));

plot(tsresample.Time,tsresample.Data(:,1));

legend(["Original","Signal Resample","Linear Interplation"]);

In this example the signal processing resample introduces some error at the boundaries. Unfortunately I don't know enough about signal processing to say whether there are ways to avoid this behaviour, and perhaps for your problem this is actually more appropriate.

In the above I have used the syntax

[yresample,tresample] = resample(y,t,samplerate)

where y is my original data, t is the sample times of y, and samplerate is my desired new sampling rate. I think this syntax should work for your case, though if each of your variables are sampled at different times you may need to call resample independently on each with their given sampling times. Also it is worth getting the 2nd output tresample above as this call to resample actually includes samples at times tresample>max(t).

Hope that helps,

Ben

[PB75]

Hi Ben,

Thanks for taking the time to answer my question.

I used the Signal Analyser app to first plot the raw signal from ANSYS (with the variable time steps), and then resampled to approximately the same sample frequency (49KHz). Then resampled to slighty less than that frequency, but I could not reduce some noise at the beginning and end of the signals, so I had to go down another route. I found the following code on a seperate help post interp1. Which not only resized all the data sets to the same sequence length, but allowed to specify the time step, see below code, as it may help others who are struggling with resampling and resizing data imported from other software such as ANSYS.

% Extract Time from imported data
time_01        = dataraw_01(:,1);

% Use interp1 function with an equally spaced time vector, 50KHz Frequency or 2.0E-05 sec
startTime = min(time_01);
stopTime = max(time_01);
time_equally_spaces = startTime:0.00002:stopTime;

datarawRs_01 = interp1(time_01, squeeze(dataraw_01), time_equally_spaces);

Thanks again,

Patrick