How does interp1 work?

I have a matlab script that interpolates to get a new value. So there are two sets of values unique voltage and unique energy. Then there is interpolated voltage which we create by programming

interpolatedVoltage = (3.5:0.01:3.8)';

Then I created interpolatedEnergy from these data using interpolation.

interpolatedEnergy = interp1(uniqueVoltage, uniqueEnergy, interpolatedVoltage, 'linear', 'extrap');

This is how the program works. The program works on 16 sets of data, i.e 16 sets of unique voltage and unique energy to create 16 sets of interpolated energy.The interpolated voltage is same for all 16 sets which we get from the above line of code. Out of 16sets 15 of them are ok, one set is giving some weird values and I am not sure why. So I will share two sets of data, one from one of the correct 15 sets and other the wrong set.

First correct set:

uniqueVoltage = (3.53066775849519 3.53067035022426 .... 3.84681597611049 3.84682177949044)

This is a 9901x1 double array i cant post all the 9901 values here so just giving you an idea.

unique energy = (96.7962090699804 96.7891484740611 96.8032655767983 96.7820835900533 96.8103181481226 96.7750145267493 .................. 0.000557256488020200 0.000728223962312157 0.000106855162461192 0.000749595056905748 0.000429031265910740 0.00113614410430565 0.000407660141488397 0.00111467573818605 0.000364917826072193 0.000386288986801839 0.00154579966056023 0.00152442844457245)

This is also 9901x1 double. Now using the interp1 script we get

interpolatedEnergy = (157.984164074966 130.741363074253 103.498562073598 76.2557610729498 71.6954943796960 64.3493476570002 60.1380167348793 55.5485757108256 50.6952125525250 45.5301211956612 40.1799493512170 34.7333113979099 29.4825451524645 24.5155873829590 19.9779904161105 15.8415077213954 12.0602433779688 8.65701363815505 5.52719951670294 2.64684289203054 0 -2.50079371842751 -4.76824379342714 -6.84830779926775 -8.72701313741007 -10.4254851017059 -12.0755442961229 -13.6163368207559 -15.1435677773727 -16.6638070391892 -18.1993380906739)

These are all the values, which is a 31x1 double. This is correct.

Now I will give you the wrong ones.

uniqueVoltage = (3.54267857142857 3.54267857142857 ..........3.87370393374741 3.87370734126984)

this is a 12128x1 double.

And similarily

uniqueEnergy = (106.670748749771 106.672279230429 106.669217303301 106.667685154779 .......... 0.000715131192799702 0.000984430538650193 0.000828113707411072 0.000737727801595998)

which is also a 12128x1 double.

and then we get interpolated Energy.

interpolatedEnergy = (-73542349778.8076 -56310669477.8076 -39078989176.8076 -21847308875.8076 -4615628575.12006 70.5205235746552 62.4922409998601 58.1823784683028 53.5935342110583 48.2852823614578 42.7415626377397 37.0069009768616 31.3589515760930 26.0188892296641 21.0859020947200 16.6046767621578 12.6165119246077 9.03573575904217 5.80849576891250 2.75148953663250 0 -2.54632248339794 -4.89544569350973 -7.00713946971490 -8.91955065513943 -10.7003376130882 -12.3868240569588 -14.0218867348210 -15.7049306967095 -17.4390479657499 -19.2372152385485)

which is also 31x1 double.

But as you can see from the first 5 values, they are completely off. I dont know why. For all other 15 data its ok. Not everyone starts at 157 like what I shared, but it is close, 120, 100, 96 and values like that are the remaining 15 values. But this one which is wrong is giving such a huge negative values in the beginning and then jumbing to 70. So anyone who knows interpolation very well or the function interp1 very well and how it works, can you please explain why is this happening with this one data. Thank you very much.

[Stephen23]

@Govind Sankar Madhavan Pillai Ramachandran Nair: please upload your data by clicking the paperclip button.

"I have a matlab script that interpolates to get a new value."

Your very first sentence in combination with the code hints what the problem is: you think that you are interpolating data, but in actual fact you are not, you are extrapolating data. As the Wikipedia article on extrapolation states "...extrapolation is subject to greater uncertainty and a higher risk of producing meaningless results"

Source: https://en.wikipedia.org/wiki/Extrapolation

Which matches exactly what you are doing.

But if you upload your actual data in a MAT file then we might be able to help you do something more "expected".

[Govind Sankar Madhavan Pillai Ramachandran Nair]

As requested I have uploaded two .mat files which contains the same data as above and from their names you can understand one contains the correct interpolatedEnergy and other contains the wrong one. Both contains also interpolatedVoltage, UniqueVoltage and UniqueEnergy with all the values. Hope this helps. Thank you.

[Stephen23]

"one set is giving some weird values and I am not sure why"

https://en.wikipedia.org/wiki/Garbage_in,_garbage_out

S = load('correctFile.mat')

S = struct with fields:
     interpolatedEnergy: [31x1 double]
    interpolatedVoltage: [31x1 double]
           uniqueEnergy: [9901x1 double]
          uniqueVoltage: [9901x1 double]

CuE = S.uniqueEnergy;
CuV = S.uniqueVoltage;
CiV = S.interpolatedVoltage;
CiE = interp1(CuV,CuE,CiV, 'linear','extrap');
plot(CiV,CiE,'-xg', CuV,CuE,'-.b')

Does that green curve really look like it is extending the blue line? Not really...

And what is that blue triangle mess on the left? Lets zoom in and take a closer look:

plot(CiV,CiE,'-xg', CuV,CuE,'-.b')
xlim([3.528,3.548])
ylim([88,100])

So even your "correct" data is rubbish (it seems to include multiple spikes with practically infinite slope), and interpolating/extrapolating such rubbish is basically like playing a game of chance.

Now lets view the "incorrect" data:

S = load('incorrectFile.mat')

S = struct with fields:
     interpolatedEnergy: [31x1 double]
    interpolatedVoltage: [31x1 double]
           uniqueEnergy: [12128x1 double]
          uniqueVoltage: [12128x1 double]

IuE = S.uniqueEnergy;
IuV = S.uniqueVoltage;
%IiV = S.interpolatedVoltage;
%IiE = interp1(IuV,IuE,IiV, 'linear','extrap');
plot(IuV,IuE,'-.b')

Lets again zoom in on that mess on the left:

plot(IuV,IuE,'-.b')
xlim([3.542,3.546])
ylim([104,107])

Without even extrapolating we can see that your data is a mess. The values that you are extrapolating are distingished by no more than binary floating point noise (just as John D'Errico explained several hours ago):

format long G
IuV(1:2)

ans = 2×1

3.54267857142857 3.54267857142857

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

diff(IuV(1:2))

ans = 
      8.88178419700125e-16

IuE(1:2)

ans = 2×1

106.670748749771 106.672279230429

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

diff(IuE(1:2))

ans = 
       0.00153048065783423

Getting meaningful/useful extrapolation results relies on understanding your data: e.g. plotting it, knowing what its limits are (e.g. measurement precision & noise), knowing what model it represents, how to ensure that the data is sufficiently smooth, how to clean it of any spurious data points, what numeric methods can robustly process it. You do not seem to have made any attempt to clean your data, you seem to have simply thrown all of your data at INTERP1 and expected it to give a meaningful output. Sorry, that is not a effective approach to extrapolating.

Understanding extrapolation would help. Verify it by doing it on paper: you will find that INTERP1 is (most likely) correctly extrapolating the (rubbish) data that you are providing it with.

I suspect that you actually want some kind of smoothing algorithm, and that you (incorrectly) thought that interpolation/extrapolation is the same thing as smoothing:

https://www.mathworks.com/help/curvefit/smoothing-data.html

Or a smoothing spline might be what you are looking for:

https://www.mathworks.com/help/curvefit/smoothing-splines.html

Or perhaps fitting some polynomial with a few degrees of freedom:

https://www.mathworks.com/help/matlab/math/polynomial-curve-fitting.html

Of course, such polynomials famously also blow up very quickly outside of the data range (and potentially within it too), so they require understanding that data, etc. just as interpolation/extrapolation does. In any case, simply throwing all of your data at functions without a good understanding of that data... won't get you very far.

Answer 1

[John D'Errico]

Why not post the actual data you used that caused the problem, instead of only a tiny fragment of it?

You certainly CAN post that data. USE A .MAT FILE. Save the data in a .mat file. Attach it to your question, using the paper clip button. Is there a good reason why you want to make it more difficult to get help? As it turns out, you have posted ALMOST sufficient information, but not enough for us to show you what actually did happen.

Anyway, you are extrapolating, as you told interp1 to do. Need I point out the Mark Twain diatribe on the dangers of extrapolation?

interpolatedEnergy = interp1(uniqueVoltage, uniqueEnergy, ...
    interpolatedVoltage, 'linear', 'extrap');

Even though the interpolant is a linear one, what do you expect?

The data you have has TINY variations between consecutive elements in the independet variable, thus uniqueVoltage.

Look at what you have. The first two (voltage) data points lie at:

3.54267857142857 3.54267857142857

Do you see they are so close together, that while apparently unique, they are indistinguishable down to 15 displayed digits? How about the energies there?

106.670748749771 106.672279230429

Now you are asking interp1 to EXTRAPOLATE out as far as 3.5. Given a line that has essentially infinite slope between those two points, what do you expect?

I.e., delta x is zero, but the change in y is not zero. What is the slope of a line between two points? (delta y)/delta x).

[Walter Roberson]

"In the space of one hundred and seventy-six years the Lower Mississippi has shortened itself two hundred and forty-two miles. That is an average of a trifle over one mile and a third per year. Therefore, any calm person, who is not blind or idiotic, can see that in the Old Oolitic Silurian Period,' just a million years ago next November, the Lower Mississippi River was upwards of one million three hundred thousand miles long, and stuck out over the Gulf of Mexico like a fishing-rod. And by the same token any person can see that seven hundred and forty-two years from now the Lower Mississippi will be only a mile and three-quarters long, and Cairo and New Orleans will have joined their streets together, and be plodding comfortably along under a single mayor and a mutual board of aldermen. There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact."

[Govind Sankar Madhavan Pillai Ramachandran Nair]

As requested I have uploaded two .mat files which contains the same data as above and from their names you can understand one contains the correct interpolatedEnergy and other contains the wrong one. Both also contain interpolatedVoltage, UniqueVoltage and UniqueEnergy with all the values. Hope this helps. Thank you.

[John D'Errico]

(That is my all-time favorite quote about mathematics. Well, one of them.)

load incorrectFile.mat
whos

  Name                         Size            Bytes  Class     Attributes

  ans                          1x41               82  char                
  interpolatedEnergy          31x1               248  double              
  interpolatedVoltage         31x1               248  double              
  uniqueEnergy             12128x1             97024  double              
  uniqueVoltage            12128x1             97024  double              

format long g
diff(uniqueVoltage(1:5))

ans = 4×1

1.0e+00 * 8.88178419700125e-16 1.06056949804589e-05 1.20656370654615e-05 4.11773758424161e-06

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

The first two elements of uniqueVoltage are virtually identical, ALMOST. They differ by roughly one part in 1e-16. What is the slope of that first line segment?

Slope = diff(uniqueEnergy(1:2))/diff(uniqueVoltage(1:2))

Slope = 
             1723168030080

The slope is positive, but essentially infinite, because we divided a non-zero by something that is essentially zero.

uniqueVoltage(1)

ans = 
          3.54267857142857

interpolatedVoltage(1)

ans = 
                       3.5

Now, extrapolate downwards by a significant amount. The result will be hugely, immensely negative. Remember, interp1 will extrapolate based on those last to data points. What else can it do?

We can write the equation of the line segment it will use as...

syms x
y(x) = vpa(simplify(uniqueEnergy(1) + Slope*(x - uniqueVoltage(1))),20)

y(x) = 

Yes That is the "equation" of that first line segment. And that is what interp1 will use. YOU TOLD IT TO EXTRAPOLATE LINEARLY!

Now, what happens when you decide to evaluate that mess at a voltage of 3.5, which is SERIOUSLY outside of the range of your data? Yes, I know it does not seem like a lot. But when your curve has essentially infinite slope...

vpa(y(3.5))

ans = 

You cannot be remotely surprised.