Creating and Executing Test Cases
This example shows how to use Simulink® Design Verifier™ functions to log input signals, create a harness model, generate test cases for missing coverage, merge harness models, and execute test cases.
The example starts by logging input signals to the component that implements the controller in its parent model and creating harness model for the controller from that logged data. You use Simulink Design Verifier to find a new test case that achieves the missing coverage. Then you merge the first harness model with the harness model generated after the Simulink Design Verifier analysis. Finally, you capture all test cases and execute the controller with those test cases in simulation mode and Software-In-the-Loop (SIL) mode, and compare the results using CGV API.
Check Product Availability
This example requires a valid Stateflow® license. To demonstrate test execution in Software-In-the-Loop (SIL) mode it also requires valid Simulink® Coder™ and Embedded Coder™ licenses.
if ~license('test','Stateflow') return; end canUseSIL = license('test','Real-Time_Workshop') && ... license('test','RTW_Embedded_Coder');
Logging Input Signals to the Component and Creating the Harness Model
The slvnvdemo_powerwindow
model contains a power window controller and a low-order plant model. The component control
is a Model block that references the model slvnvdemo_powerwindow_controller
, which implements the controller with a Stateflow® chart.
To create a harness model for the controller with the signals that simulate the controller in the plant model, first log the input signals and then invoke harness generation with that logged data.
open_system('slvnvdemo_powerwindow'); load_system('slvnvdemo_powerwindow_controller'); loggedSignalsPlant = ... sldvlogsignals('slvnvdemo_powerwindow/power_window_control_system/control'); harnessModelFilePath = ... sldvmakeharness('slvnvdemo_powerwindow_controller',loggedSignalsPlant); [~,harnessModel] = fileparts(harnessModelFilePath);
Measuring the Coverage with Logged Signals
Use the cvtest
(Simulink Coverage) and cvsim
(Simulink Coverage) functions to measure the model coverage achieved for the controller model slvnvdemo_powerwindow_controller
with the logged signals that are captured in the harness model.
The cvhtml
(Simulink Coverage) function produces a report that indicates that 40% Decision, 35% Condition, and 10% MCDC coverage is achieved by simulating the test cases captured from the closed-loop model.
test = cvtest(harnessModel); test.modelRefSettings.enable = 'On'; test.modelRefSettings.excludeTopModel = 1; covDataFromLoggedSignals = cvsim(test); cvhtml('Coverage with Logged Test Cases',covDataFromLoggedSignals);
Finding Test Cases for Missing Coverage
Before you can use existing coverage data during test generation, the data must be saved to a coverage data file(.cvt). You can use the existing coverage data by specifying the coverage data path in the Coverage data file parameter and setting the Ignore objectives satisfied in existing coverage data parameter to on
in the Test Generation pane of Simulink Design Verifier configuration parameters.
As you can see in the report, Simulink Design Verifier restricts test generation to the coverage objectives that are not covered in the existing coverage file. Notice that 8 coverage objectives in the Stateflow chart slvnvdemo_powerwindow_controller
;|control| are proven to be unsatisfiable. This indicates unnecessary redundant logic that cannot be tested.
cvsave('existingCovFromLoggedSignals',covDataFromLoggedSignals); opts = sldvoptions; opts.IgnoreCovSatisfied = 'on'; opts.CoverageDataFile = 'existingCovFromLoggedSignals.cvt'; opts.ModelCoverageObjectives = 'MCDC'; opts.TestSuiteOptimization = 'LongTestcases'; opts.SaveHarnessModel = 'on'; opts.ModelReferenceHarness = 'on'; opts.MaxProcessTime = 500; [status, fileNames] = sldvrun('slvnvdemo_powerwindow_controller',opts,true); [~, newHarnessModel] = fileparts(fileNames.HarnessModel); open_system(newHarnessModel);
Merging Test Cases from Harness Models
Now use sldvmergeharness
to combine generated test cases with logged test case. The command takes a list of harness models as arguments.
sldvmergeharness(harnessModel, newHarnessModel);
Logging Test Cases of the Harness Model
In order to programmatically execute the model slvnvdemo_powerwindow_controller
with the test cases captured in the merged harness model, first use the sldvlogsignals
function to obtain the input values of all test cases in the necessary data format.
loggedSignalsMergedHarness = sldvlogsignals(harnessModel); disp(loggedSignalsMergedHarness);
LoggedTestUnitInfo: [1x1 struct] TestCases: [1x2 struct]
Execute the Model in Simulation Mode with CGV API
Use the sldvruncgvtest
function to execute the model slvnvdemo_powerwindow_controller
in simulation mode, with test cases captured from the harness model.
runopts = sldvruntestopts('cgv'); disp(runopts); runopts.cgvConn = 'sim'; cgvSim = sldvruncgvtest('slvnvdemo_powerwindow_controller',... loggedSignalsMergedHarness,runopts);
DefectChecker: 'off' testIdx: [] allowCopyModel: 0 cgvCompType: 'topmodel' cgvConn: 'sim' Starting execution: ComponentType: topmodel Connectivity: sim InputData: /tmp/Bdoc24b_2679053_1496143/tpa73589a2/sldv-ex48508823/cgv_runtest/slvnvdemo_powerwindow_controller/slvnvdemo_powerwindow_controller_cgv_input_tc_1.mat End CGV execution: status completed. Starting execution: ComponentType: topmodel Connectivity: sim InputData: /tmp/Bdoc24b_2679053_1496143/tpa73589a2/sldv-ex48508823/cgv_runtest/slvnvdemo_powerwindow_controller/slvnvdemo_powerwindow_controller_cgv_input_tc_2.mat End CGV execution: status completed.
Execute the Model in Software-In-the-Loop (SIL) Mode with CGV API
Now use the sldvruncgvtest
function to execute the model slvnvdemo_powerwindow_controller
in SIL mode, with the same test cases.
if canUseSIL runopts.cgvConn = 'sil'; else % When SIL is not possible, the example runs another simulation. runopts.cgvConn = 'sim'; end cgvSil = sldvruncgvtest('slvnvdemo_powerwindow_controller',... loggedSignalsMergedHarness,runopts);
Starting execution: ComponentType: topmodel Connectivity: sil InputData: /tmp/Bdoc24b_2679053_1496143/tpa73589a2/sldv-ex48508823/cgv_runtest/slvnvdemo_powerwindow_controller/slvnvdemo_powerwindow_controller_cgv_input_tc_1_1.mat ### Starting build procedure for: slvnvdemo_powerwindow_controller ### Successful completion of build procedure for: slvnvdemo_powerwindow_controller Build Summary Top model targets: Model Build Reason Status Build Duration =================================================================================================================================== slvnvdemo_powerwindow_controller Information cache folder or artifacts were missing. Code generated and compiled. 0h 0m 10.867s 1 of 1 models built (0 models already up to date) Build duration: 0h 0m 11.366s ### Preparing to start SIL simulation ... Building with 'gcc'. MEX completed successfully. ### Starting SIL simulation for component: slvnvdemo_powerwindow_controller ### Application stopped ### Stopping SIL simulation for component: slvnvdemo_powerwindow_controller End CGV execution: status completed. Starting execution: ComponentType: topmodel Connectivity: sil InputData: /tmp/Bdoc24b_2679053_1496143/tpa73589a2/sldv-ex48508823/cgv_runtest/slvnvdemo_powerwindow_controller/slvnvdemo_powerwindow_controller_cgv_input_tc_2_1.mat ### Starting build procedure for: slvnvdemo_powerwindow_controller ### Successful completion of build procedure for: slvnvdemo_powerwindow_controller Build Summary Top model targets: Model Build Reason Status Build Duration =============================================================================================================== slvnvdemo_powerwindow_controller Generated code was out of date. Code generated and compiled. 0h 0m 9.2045s 1 of 1 models built (0 models already up to date) Build duration: 0h 0m 9.6419s ### Preparing to start SIL simulation ... Building with 'gcc'. MEX completed successfully. ### Starting SIL simulation for component: slvnvdemo_powerwindow_controller ### Application stopped ### Stopping SIL simulation for component: slvnvdemo_powerwindow_controller End CGV execution: status completed.
Compare Results of Simulation and SIL Modes
The sldvruncgvtest
returns a cgv.CGV
object after running tests. Use the CGV API to compare the results of executions in simulation and SIL modes for each test case designed in the harness model and show that they are equal.
for i=1:length(loggedSignalsMergedHarness.TestCases) simout = cgvSim.getOutputData(i); silout = cgvSil.getOutputData(i); [matchNames, ~, mismatchNames, ~ ] = ... cgv.CGV.compare(simout, silout); fprintf('\nTest Case(%d): %d Signals match, %d Signals mismatch', ... i, length(matchNames), length(mismatchNames)); end
Test Case(1): 4 Signals match, 0 Signals mismatch Test Case(2): 4 Signals match, 0 Signals mismatch
Clean Up
To complete the example, close all models.
close_system(harnessModel,0); close_system(newHarnessModel,0); close_system('slvnvdemo_powerwindow',0); close_system('slvnvdemo_powerwindow_controller',0);