Interpolation when data are not strictly monotonic

Hi,

I have the following data points:

X = [102   106   108   109   111   112   113   115   115   116   116   116];
Y = [10    10    10    11    11    13    14    15    17    18    20    113];

I used interp1 for 1D data interpolation, however it fails because data points are not strictly monotonic (problematic point: 116,113). I was wondering how to tackle this problem by either 1- removing the problematic data point(s) or 2- make the interpolation work with some modification or smoothing on data.

Thanks in advance,

Answer 1

[Star Strider]

The solution is to add very small values to the duplicates to create a monotonically-increasing vector in ‘X’:

X = [102   106   108   109   111   112   113   115   115   116   116   116];
Y = [10    10    10    11    11    13    14    15    17    18    20   113];

duplicates = find(diff([0 X]) == 0);                            % Find Duplicates
X(duplicates) = X(duplicates) + [1:length(duplicates)]*1E-8;    % Add Small Increasing Values To Duplicates

xi = [105  110  114];                                           % Desired ‘X’ Values
yi = interp1(X, Y, xi);                                         % Interpolated ‘Y’ Values

[Ive J]

Thanks for your help. Yes, this addresses the problem; however, it resembles the smoothing spline in fitting and cannot capture the final point (the problematic one: 116,113). I think I should add a condition to check if it converges or not.

[John D'Errico]

This is the wrong solution when using splines.

[Ive J]

Dear John, The image belongs to interp1 not splines, Both cannot capture the final point though. Do you think is there a better approach for such situations?

[Star Strider]

What do you mean by ‘capture’?

What do you want to do?

[Ive J]

Sorry for being vague by that word. I just meant that I want to "have" the final point in the interpolated data (maybe something like piecewise interpolation?).

[John D'Errico]

The question is, what do you intend when you ask for interpolation?

if you wish for a function y(x), then it makes no sense to try to interpolate that function at x = 116. From this sample data, y might take on any value, from the set {18, 20, 113}. Which is it?

y is simply not a function of x there.

So you must either decide to do some form of data reduction to eliminate the replicates, or you need to use a tool that can interpolate along the path provided. But if you do choose to interpolate points along a path, then you cannot use that as a predictive function, y(x). Again, for a given x, y appears to take on more than one value.

[Star Strider]

I’m still not sure what you want, but the final point is there if you want it:

xi = [105  110  114  X(end)];                                   % Desired ‘X’ Values
yi = interp1(X, Y, xi);                                         % Interpolated ‘Y’ Values

xi =
          105          110          114          116

yi =
           10           11         14.5          113

[John D'Errico]

If your goal is interpolation along a general path in the (x,y) plane, then do this:

xy = interparc(100,X,Y,'pchip');
plot(X,Y,'ro',xy(:,1),xy(:,2),'b-')

Again, you can find interparc on the FEX.

But you can then not interpolate y(x), because which of those y values should it return?

[Ive J]

The point is that I am trying to follow some data points (here X,Y) from the start point (X(1),Y(1)) to the final point (X(end),Y(end)). This would let me to reproduce any point between these discrete data, similar to what functions like plot() and line() do. In another word, I am trying to walk on data points and connect them so that the final function resembles these data points.

[John D'Errico]

Then you need to use either cscvn, or interparc (or some similar tool that can work with some general path in multiple dimensions.) That is exactly what I did with the example call to interparc. I generated 100 (equally spaced) points along that path.

You CANNOT predict a value of y for a given x, because which y value should be chosen when there are replicates?

The tools I have indicated are able to predict intermediate points along the indicated path.

[Ive J]

So, I assume there is a "tradeoff" between my objective and what those tools do. What if I (as mentioned in my comment) break down my points in two segments where the data points go in vertical direction and simply use x = 116 (from y: 18 to 113) ?

[John D'Errico]

And that is what interparc does. Well, not exactly what it truly does internally, but effectively, that is what it does.

In your scheme, you can never predict a single value of y for a given x. So classic interpolation can never work, so a tool that predicts y(x). But you don't need it.

Answer 2

[John D'Errico]

OK, the problem is you have a function with different y values for the same x. How can interp1 predict when there are more than one y value? Interp1 applies ONLY to single values functions, so for any x, only one y.

If this is actually a general path in two dimensions, then use tools like cscvn, or my interparc (as found on the file exchange.)

But if your goal is to produce a function that predicts y as a function of x, then you need to make the data monotonic in x. Why is it a bad thing to increment the x values by a tiny amount? If you will be using a spline interpolant, it will introduce HUGE amounts of ringing into the function. Having points so close to each other, with widely varying y values does nastiness to a spline fit.

Of course, if you are doing only linear interpolation, then the increment trick will work.

Instead, you are generally best off by choosing some way to reduce the replicates into a single point.

You might choose to take the average value, or perhaps the median of replicates. Or perhaps just the first or last among the reps. Or, the max or min. Lots of ways you could resolve the issue. But for any x, only ONE y.

A simple tool to solve these problems is my consolidator tool , also found on the file exchange. It was designed to solve exactly this class of problems, for spline fits.

X = [102 106 108 109 111 112 113 115 115 116 116 116];
Y = [10   10  10  11  11  13  14  15  17  18  20 113];

Note that consolidator can work in multiple dimensions, so it requires column vectors.

[xc,yc] = consolidator(X',Y',@mean);
[xc,yc]
ans =
          102           10
          106           10
          108           10
          109           11
          111           11
          112           13
          113           14
          115           16
          116       50.333

[xc,yc] = consolidator(X',Y',@median);
[xc,yc]
ans =
   102    10
   106    10
   108    10
   109    11
   111    11
   112    13
   113    14
   115    16
   116    20

As you can see, once consolidator is done, the curves can now be happily interpolated. It is pretty efficient, and it even takes a tolerance.

[Ive J]

Dear John,

Thanks for your response and remarkable points on splines. However, I need a function to follow the behavior of my real data, meaning that I need the final point 113,116 in the final function/interpolation/fit. It seems that it cannot be done with one interpolation, and maybe I need to break down my data into two segments: 1- the monotonic section and 2- the final section which is a vertical line.

[John D'Errico]

If your goal is interpolation along a general path in the (x,y) plane, then do this:

xy = interparc(100,X,Y,'pchip');
plot(X,Y,'ro',xy(:,1),xy(:,2),'b-')

Again, you can find interparc on the FEX.

See that the above interpolation did indeed generate 100 equally spaced points along that path.

You can still not use tools like that to interpolate y(x) here, because which of those y values should it return where x is replicated?

[Ive J]

This is the best tool I can use!

It's perfect! Deeply appreciate your time :)

[John D'Errico]

Note that if you use the default method for interparc, it will be fastest, and considerably so. The default method does linear interpolation between consecutive pairs of points along the path. This may be adequate for your purposes. Those who want smoother curves must use one of the alternative methods in interparc.

[Ive J]

As my last question, since I intend to use interparc in my work, is it enough to include its license within my final codes (for subsequent submitting to a journal) or I should do some more?

Regards.

[John D'Errico]

Anything I post on the FEX is there for all to use freely. If you will publish in a journal, they will often wish to see a citation. For that purpose, we some years ago choose a couple of style that seemed to make sense. The page I've linked has examples of both, as applied to one of my other codes - gridfit.

http://blogs.mathworks.com/community/2010/12/13/citing-file-exchange-submissions/