Main Content

Handling Offsets and Trends in Data

When to Detrend Data

Detrending is removing means, offsets, or linear trends from regularly sampled time-domain input-output data signals. This data processing operation helps you estimate more accurate linear models because linear models cannot capture arbitrary differences between the input and output signal levels. The linear models you estimate from detrended data describe the relationship between the change in input signals and the change in output signals.

For steady-state data, you should remove mean values and linear trends from both input and output signals.

For transient data, you should remove physical-equilibrium offsets measured prior to the excitation input signal.

Remove one linear trend or several piecewise linear trends when the levels drift during the experiment. Signal drift is considered a low-frequency disturbance and can result in unstable models.

You should not detrend data before model estimation when you want:

  • Linear models that capture offsets essential for describing important system dynamics. For example, when a model contains integration behavior, you could estimate a low-order transfer function (process model) from nondetrended data. For more information, see Process Models.

  • Nonlinear black-box models, such as nonlinear ARX or Hammerstein-Wiener models. For more information, see Nonlinear Model Identification.

    Tip

    When signals vary around a large signal level, you can improve computational accuracy of nonlinear models by detrending the signal means.

  • Nonlinear ODE parameters (nonlinear grey-box models). For more information, see Estimate Nonlinear Grey-Box Models.

To simulate or predict the linear model response at the system operating conditions, you can restore the removed trend to the simulated or predicted model output using the retrend command.

For more information about handling drifts in the data, see the chapter on preprocessing data in System Identification: Theory for the User, Second Edition, by Lennart Ljung, Prentice Hall PTR, 1999.

Alternatives for Detrending Data in App or at the Command-Line

You can detrend data using the System Identification app and at the command line using the detrend command.

Both the app and the command line let you subtract the mean values and one linear trend from steady-state time-domain signals.

However, the detrend command provides the following additional functionality (not available in the app):

  • Subtracting piecewise linear trends at specified breakpoints. A breakpoint is a time value that defines the discontinuities between successive linear trends.

  • Subtracting arbitrary offsets and linear trends from transient data signals.

  • Saving trend information to a variable so that you can apply it to multiple data sets.

As an alternative to detrending data beforehand, you can specify the offsets levels as estimation options and use them directly with the estimation command.

For example, suppose your data has an input offset, u0, and an output offset, y0. There are two ways to perform a linear model estimation (say, a transfer function model estimation) using this data:

  • Using detrend:

    T=getTrend(data)
T.InputOffset = u0;
T.OutputOffset = y0;
datad = detrend(data, T);

model = tfest(datad, np);

  • Specify offsets as estimation options:

    opt = tfestOptions('InputOffset',u0, 'OutputOffset', y0);

model = tfest(data, np, opt)

    The advantage of this approach is that there is a record of offset levels in the model in model.Report.OptionsUsed. The limitation of this approach is that it cannot handle linear trends, which can only be removed from the data by using detrend.

Next Steps After Detrending

After detrending your data, you might do the following:

Related Topics