Batch Equivalence Testing of Multiple Components

This example shows how to create equivalence test cases and test harnesses for multiple components in batch mode using sltest.testmanager.createTestForComponent. Coverage collection and report generation are also included in the example.

The sltestCruiseControl model used in this example has Atomic Subsystem blocks, Virtual Subsystems blocks & Model Reference Blocks .

Note that this example is configured only for Windows machines.

Open the Model

topModel = "sltestCruiseControl";
load_system(topModel);

Generate the Code

The equivalence tests in this example compare normal mode code and generated code. Embedded Coder is required to generate the code for the model. The model has been configured with appropriate C code generation and coder mapping settings for Windows 64-bit systems. You can change these settings, if desired, before generating the code.

slbuild(topModel);

### Starting serial model reference code generation build.
### Checking status of model reference code generation target for model 'sltestCruiseControlMode' used in 'sltestCruiseControl'.
### Model reference code generation target (sltestCruiseControlMode.c) for model sltestCruiseControlMode must be recompiled because genCodeOnly setting changed.
### Starting build procedure for: sltestCruiseControlMode
### Generating code and artifacts to 'Model specific' folder structure
### Code for the model reference code generation target for model sltestCruiseControlMode is up to date because no functional changes were found in referenced model.
### Saving binary information cache.
### Skipping makefile generation and compilation because C:\Users\dschwart\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\dschwart.Bdoc23a.j2130035.1\simulinktest-ex89766869\slprj\ert\sltestCruiseControlMode\sltestCruiseControlMode_rtwlib.lib is up to date
### Successful completion of code generation for: sltestCruiseControlMode
### Model reference code generation target for sltestCruiseControlMode is up to date.
### Checking status of model reference code generation target for model 'sltestDriverSwRequest' used in 'sltestCruiseControl'.
### Model reference code generation target (sltestDriverSwRequest.c) for model sltestDriverSwRequest must be recompiled because genCodeOnly setting changed.
### Starting build procedure for: sltestDriverSwRequest
### Generating code and artifacts to 'Model specific' folder structure
### Code for the model reference code generation target for model sltestDriverSwRequest is up to date because no functional changes were found in referenced model.
### Saving binary information cache.
### Skipping makefile generation and compilation because C:\Users\dschwart\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\dschwart.Bdoc23a.j2130035.1\simulinktest-ex89766869\slprj\ert\sltestDriverSwRequest\sltestDriverSwRequest_rtwlib.lib is up to date
### Successful completion of code generation for: sltestDriverSwRequest
### Model reference code generation target for sltestDriverSwRequest is up to date.
### Simulink cache artifacts for 'sltestCruiseControlMode' were created in 'C:\Users\dschwart\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\dschwart.Bdoc23a.j2130035.1\simulinktest-ex89766869\sltestCruiseControlMode.slxc'.
### Simulink cache artifacts for 'sltestDriverSwRequest' were created in 'C:\Users\dschwart\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\dschwart.Bdoc23a.j2130035.1\simulinktest-ex89766869\sltestDriverSwRequest.slxc'.
### Starting build procedure for: sltestCruiseControl
### Generating code and artifacts to 'Model specific' folder structure
### Generating code into build folder: C:\Users\dschwart\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\dschwart.Bdoc23a.j2130035.1\simulinktest-ex89766869\sltestCruiseControl_ert_rtw
### Generated code for 'sltestCruiseControl' is up to date because no structural, parameter or code replacement library changes were found.
### Saving binary information cache.
### Skipping makefile generation and compilation because C:\Users\dschwart\OneDrive - MathWorks\Documents\MATLAB\ExampleManager\dschwart.Bdoc23a.j2130035.1\simulinktest-ex89766869\sltestCruiseControl.exe is up to date
### Successful completion of code generation for: sltestCruiseControl

Build Summary

0 of 3 models built (3 models already up to date)
Build duration: 0h 0m 11.953s

Specify the Components to Test

The example tests all of the atomic subsystems that have nonreusable function packaging in the generated code.

componentsToTest = find_system(topModel,...
    "BlockType","SubSystem",...
    "TreatAsAtomicUnit","on",...
    "RTWSystemCode","Nonreusable function");

To improve traceability of testing artifacts, such as test case and test harness names, customize the default names of the created harnesses. The names will use the component name instead of the owning model name.

sltest.harness.setHarnessCreateDefaults("Name","$Component$_Harness");

Create the Test Cases and Test Harnesses in Batch Mode

Use the sltest.testmanager.createTestForComponent API to create multiple test cases and test harnesses at the same time. Using createTestForComponent, you specify to create a test file and the test filename. You also specify the top model, components to test, and the test type and simulations settings. For the test inputs, you specify to use Simulink Design Verifier to automatically generate the inputs in Microsoft Excel format. If you have other existing inputs, you can add them to the test cases and test both those inputs and the generated inputs.

[tc, status] = ...
    sltest.testmanager.createTestForComponent(...
        "CreateTestFile",true,...
        "TestFile","myB2BTestsDemoEx.mldatx",...
        "TopModel",topModel,...
        "Component",componentsToTest,...
        "TestType","equivalence",...
        "Simulation1Mode","Normal",...
        "Simulation2Mode","Software-in-the-Loop (SIL)",...
        "SLDVTestGeneration","on",...
        "CreateExcelFile",true);

Test Execution

The createTestForComponent API generated the test file, test harnesses, and test input signals in the current working directory. Now, save the harnesses attached to the model, and save the test file.

tf = sltest.testmanager.TestFile("myB2BTestsDemoEx.mldatx");
tf.saveToFile;

Enable coverage collection for test file and run the tests.

cov = getCoverageSettings(tf);
cov.RecordCoverage = true;
cov.MetricSettings = "dcmr";
tf.saveToFile;

resultSet = tf.run;
sltest.testmanager.exportResults(resultSet,"myB2BResults.mldatx");
sltestmgr;

Generate a report with coverage and equivalence test results.

sltest.testmanager.report(...
    resultSet,"myB2BResultsReport.pdf",...
    "IncludeCoverageResult",true,...
    "IncludeSimulationSignalPlots",true,...
    "IncludeComparisonSignalPlots",true,...
    "IncludeTestResults",0,...
    "IncludeSimulationMetadata",true);

Clean Up

bdclose({topModel,'sltestDriverSwRequest','sltestCruiseControlMode'})
sltest.testmanager.clear
sltest.testmanager.clearResults
sltest.testmanager.close