You can specify tolerances in the Baseline Criteria or Equivalence Criteria sections of baseline and equivalence test cases. You can specify relative, absolute, leading, and lagging tolerances for a signal comparison. Leading and lagging tolerances allow you to compensate for differences in time between signals. The units for tolerances are seconds.
To learn about how tolerances are calculated, see How the Simulation Data Inspector Compares Data.
To modify a tolerance, select the signal name in the criteria table, double-click the tolerance value, and enter a new value.
If you modify a tolerance after you run a test case, rerun the test case to apply the new tolerance value to the pass/fail results.
Specify a tolerance when the difference between results falls in a range you consider acceptable. Suppose that your model under test uses a particular solver. Solvers are sometimes updated from one release to the next, and new solvers also become available. If you use an updated solver or change solvers, you can specify an acceptable tolerance for differences between your baseline and later tests. Leading and lagging tolerances allow you to re-evaluate criteria if there are differences in time, for example, due to solver the data is off by .04 seconds you can shift it left or right to account for this.
Generate the baseline for the
sf_car model, which uses the
Open the model
Open the Test Manager and create a test file named
Compare. In the test case, set the system under test to
Select the signal to log. Under Simulation
Outputs, click Add. In the model,
shift_logic output signal. In the Signal
Selection dialog box, select the check box next to
shift_logic and click
Save the baseline. Under Baseline Criteria, click
Capture. Set the file format to
MAT. Name the baseline
solver_baseline and click
After you capture the baseline MAT-file, the model runs and the baseline criteria appear in the table. Each default tolerance is 0.
Suppose that you want to use a different solver with your model. You run a test to compare results using the new solver with the baseline.
In the model, change the solver to
In the Test Manager, with the
Solver Compare test
file selected, click Run.
In the Results and Artifacts pane, notice that the test failed.
Expand the results of the failed test. Under Baseline
Criteria Result, select the
The Comparison tab shows where the difference occurred.
Zoom the comparison chart where the results diverged. The comparison signal changes ahead of the baseline, that is, it leads the baseline signal.
You can use leading and lagging tolerances to allow for slight offsets in time between the simulation and baseline data. Suppose that your team determines that a tolerance the size of the simulation step size (.04 seconds in this case) is acceptable. In the Test Manager, set a leading tolerance value. Use a leading tolerance for the signal whose change occurs ahead of your baseline. Use a lagging tolerance for a signal whose change occurs after your baseline.
You can preview how the tolerance value affects the test to see if the test passes with the specified tolerance. Then set the tolerance on the baseline criteria and rerun the test.
Preview whether the tolerance you want to use causes the test to pass.
With the result signal selected, in the property box, set
Leading Tolerance to
When you change this value, the status changes to show that the failed tests pass.
When you are satisfied with the tolerance value, enter it in the
baseline criteria so you can rerun the test and save the new pass-fail
result. In the Test Browser pane, select the test
case in the
Solver Compare test.
Under Baseline Criteria, change the
Leading Tol value for the
solver_baseline.mat file to
By default, each signal inherits this value from the baseline file. You can override the value for each signal.
Run the test again. The test passes.
To store the tolerance value and the passed test with the test file, save the test file.