Run Tests for Variant Models Using Variant Configurations

Since R2024a

This example shows how to author and run simulation-based tests for your variant model by specifying variant configurations as input to the Simulink Test™ programmatic interface.

The Simulink Test programmatic interface helps you to create test scripts and functions and to run tests from the command-line.

Variant elements in Simulink®, including variant blocks and variant parameters, enable you to represent multiple design alternatives within a single model. For such a model, you can define variant configurations to represent combinations of variant choices across the model hierarchy. These configurations comprise a set of variant control variables and their corresponding values, allowing you to activate specific variants within the model hierarchy.

In the Simulink Test™ programmatic interface, you can specify the variant configuration to use when running a test case. You can also run the same test case for different variant configurations in the model by creating test case iterations. This workflow offers these benefits:

• Using variant configurations scales well for models with large number of variant control variables. If you do not create variant configurations for your model, you must write additional scripts to set the value of variant control variables and use those variables as input to create test iterations.

• Using variant configurations provides traceability from the test results to the the variant configuration used for each test case or iteration.

Note: You must install the Variant Manager for Simulink® support package using Add-On Explorer to create variant configurations for a model.

If your model does not have variant configurations defined, refer to these examples for the testing workflow:

• Verify and Validate Variant Models with Code Compile Activation Time

• Verify and Validate Variant Models with Startup Activation Time

Explore Example Model

Open the model slexVariantReducer.

The model has two named variant configurations created using Variant Manager. The configurations are stored in a variant configuration data object, slexVariantReducer_config, that is associated with the model.

modelName = "slexVariantReducer";
open_system(modelName);

View the variant configurations in slexVariantReducer_config:

vcd = Simulink.VariantManager.getConfigurationData(modelName);
cellfun(@(name)(fprintf("%s\n", name)), {vcd.Configurations(:).Name})

config1
config2

Create New Test File and Test Iterations

Create a new test file with default test suite and test case. Then, modify the test case settings using the setProperty method, add test iterations, and use the setTestParam method to set the iteration parameters.

% Create a test file with default test suite
testFileName = "slexVariantReducer_TestFile";
testFile = sltest.testmanager.TestFile(testFileName);

% Get the default test suite object from the test file
testSuites = getTestSuites(testFile);

% Get the test case object
testCase = getTestCases(testSuites);

% Assign the system under test to the test case
setProperty(testCase,"Model",modelName);

% Set a default variant configuration for the test case.
setProperty(testCase,"VariantConfiguration","config1");

% Set VariantConfiguration property for first test iteration
testIteration1 = sltestiteration;
testIteration1.setTestParam("VariantConfiguration","config2");

% Add the iteration object to the test case
addIteration(testCase,testIteration1);

% Use the default variant configuration from the test case for the second test iteration
testIteration2 = sltestiteration;

% Add the iteration object to the test case
addIteration(testCase,testIteration2);

% Run the test case with iterations
testResults = run(testCase);

Generate Test Results and Generate Reports

Access the test case result data, test iteration result data, and generate reports from the results.

% Get test results
testCaseResults = testResults.getTestCaseResults;

% Get test iteration results
testIterationResults = testCaseResults.getIterationResults;

% Generate report of test specifications and test results
sltest.testmanager.TestSpecReport(testCase, "testSpec.pdf", "LaunchReport", true);
sltest.testmanager.report(testResults, "testResult.pdf", "IncludeTestResults", 1);

View Variant Configuration in Test Manager

In the Simulink Test Manager (Simulink Test) user interface, in the Test Browser pane, you can see the variant configuration used in the test case and test iterations. When you select a test case for which a variant configuration is applicable, the Parameter Overrides (Simulink Test) section and the Iterations (Simulink Test) section show the configurations used. After a test case has finished running in the Test Manager, the test case result becomes available in the Results and Artifacts pane. In the Results and Artifacts pane, when you click a test case or test iteration, the variant configuration appears in the Simulation Metadata and Test Overrides sections, if applicable to the test case.