Main Content

MATLAB Test Manager

Manage MATLAB tests and results for projects

Since R2023a

expand all in page

Description

Use the MATLAB Test Manager to manage tests and test results for projects. You can create and run tests and test suites, view results and failure diagnostics, collect coverage, and view coverage reports. View and verify linked requirements.

To add tests to the test manager:

  1. Add a test file to the project. For more information, see Manage Project Files.

  2. Add a Test label to the file. For more information, see Add Labels to Project Files.

  3. Add tests to the test file. For more information, see Ways to Write Unit Tests.

MATLAB Test Manager with 14 passed tests and 1 failed test.

Open the MATLAB Test Manager App

Open a project, then use one of these approaches to open the app:

  • MATLAB® Toolstrip: In the Project tab, in the Tools menu, under Apps, click the MATLAB Test Manager.

  • MATLAB command prompt: Enter matlabTestManager.

Examples

expand all

Open Live Script

Open the MATLABShortestPath project.

openProject("MATLABShortestPath");

Open the MATLAB Test Manager.

matlabTestManager

Run all tests in the project by clicking the Run button run_16.png. Filter the results to only show the failed tests by clicking the Failed button in the test results summary.

Open Live Script

Open the MATLABShortestPath project.

openProject("MATLABShortestPath");

Open the MATLAB Test Manager.

matlabTestManager

Open the Test Suite Manager by clicking the drop-down to the left of the Run button run_16.png and selecting Manage Custom Test Suites. Click New to create a new test suite. Set the Name field to InputTests.

Select Tags and enter InputTests in the text box. Click Save, then click Close.

In the MATLAB Test Manager, select the test suite by setting the drop-down to the left of the Run button run_16.png to InputTests. Run the tests in the test suite by clicking the Run button run_16.png.

Open Live Script

Open the MATLABShortestPath project.

openProject("MATLABShortestPath");

Open the MATLAB Test Manager.

matlabTestManager

Click the Coverage button coverage_16.png, then select Enable code coverage and set the metric level to MC/DC.

Run all tests in the project by clicking the Run button run_16.png. The Statement button shows the statement coverage percentage. To view the coverage summary for other coverage metrics, point to the Statement coverage button.

To view the coverage report, click the Statement button, or click the Report button coverage-report_16.png and, under Code Coverage Reports, select the coverage report.

Since R2024b

If you have Embedded Coder®, you can collect code coverage for generated C/C++ code that executes when you run equivalence tests.

To collect code coverage:

  1. Ensure that your equivalence tests use SIL or PIL verification. For more information, see Collect Coverage for Generated C/C++ Code in Equivalence Tests.

  2. Enable generated code coverage by clicking the Code Coverage button and selecting Enable generated code coverage. To configure the test manager to automatically open the generated code coverage report when you run equivalence tests, select Automatically open code coverage report.

  3. Set the coverage metric level to the highest level of coverage that you want to include in the code coverage report.

    Enable generated code coverage is selected in the MATLAB Test Manager.

    When you select a metric level, the report includes the metrics up to and including the selected level. For example, if you select Condition, the report includes the statement, decision, and condition coverage metrics.

  4. Run the equivalence tests.

    • Run all tests in the project by setting the drop-down menu to the left of the Run button to All Tests in Current Project. Then, click the Run button .

    • Run specific equivalence tests by right-clicking the test or test file and selecting Run selected.

  5. To open the code coverage report, click the Report button , then, under Generated Code Coverage Reports, select the report. The MATLAB Test Manager stores the five most recent generated code coverage reports.

    For more information, see Collect Coverage for Generated C/C++ Code in Equivalence Tests.

You can customize your test run by using buttons in the toolstrip:

  • Parallel execution — To use parallel execution, click the Parallel button .

  • Strict checks — To treat warnings as errors, enable strict checks. Click the Settings button , then, in the Test Run Settings dialog box, under Strictness, select Apply strict checks when running tests.

  • Output Detail — To specify the detail level for reported events, click the Settings button , then, in the Test Run Settings dialog box, under Output Detail, clear Use Default and set the output detail level.

  • Logging Level — To specify the maximum level at which logged diagnostics are included by the plugin instance, click the Settings button , then, in the Test Run Settings dialog box, under Logging Level, clear Use Default and set the logging level.

  • Display test failure diagnostics in the Command Window — To display test failure diagnostics in the MATLAB Command Window, next to the Run button , click the Expand icon and select Enable logged diagnostics.

Note

The MATLAB Test Manager shares test settings with the MATLAB Editor and Test Browser. For more information, see Comparison of Test Settings.

Since R2024b

You can view test and coverage results from the five most recent previous runs.

To view a previous run, click the button to the left of the Run button and, under Previous Runs, select a run.

The mouse points to a previous run of All Tests in Current Project.

The MATLAB Test Manager opens the test suite, displays the results, and lists the date and time that the suite ran.

The MATLAB Test Manager shows the test results from a previous run.

You can view reports associated with the run by clicking the Report button . Then:

  • To view the coverage report associated with the run, under Code Coverage Reports, select the report.

  • To view the generated code coverage report for equivalence tests that generated C/C++ code as a static library and executed in SIL or PIL mode, under Generated Code Coverage Reports, select the report.

  • To generate a report from the test results in the previous run, under Test Result Reports, click Generate Test Result Report.

To navigate to the current test results, click the button to the left of the Report button and select All Tests in Current Project or a custom test suite.

You can add and remove columns in the Test Details table. Click the Column button and select or clear columns.

Note

To add the Requirements column, you must have Requirements Toolbox™ installed.

By default, the MATLAB Test Manager lists tests in the Test Details table by ascending load order for each test file. To change how the table lists the tests, point to a column, then click the Sort Up button . To reverse sort, click the Sort Down button . You can also select the column to sort by clicking the three-dot button (since R2024b). The table sorts the tests for each column:

  • Test — Lists the tests by natural sort or by load order. Click the three-dot button and select Test (Natural Sort) or Test (Load Order) to specify the sort type. Sorting by load order lists the tests in the order that the test file defines them. Sorting by natural sort lists the tests in alphabetical order. However, when test names differ only by numbers, natural sort lists the tests in numerical order. For example, natural sort lists the test a2 before the test a11.

  • Diagnostic — Lists the tests alphabetically by diagnostic.

  • Tags — Lists the tests alphabetically by tag name.

  • Requirements — Lists the tests alphabetically by requirement summary.

  • Time — Lists the tests by the test run time length.

Since R2024b

If you have Requirements Toolbox, you can create and delete links from MATLAB tests to requirements.

To create links:

  1. Open the Requirements Editor. On the Apps tab, under Verification, Validation, and Test, click Requirements Editor.

  2. Open a requirement set. For more information, see Create, Open, and Delete Requirement Sets (Requirements Toolbox).

  3. Select a requirement.

  4. In the MATLAB Test Manager, select the test.

  5. Right-click the test and select Requirements > Link to Selection in Requirements Browser.

A test is selected in the is selected in the MATLAB editor.

To delete all links from a test, select the test in the MATLAB Test Manager, then right-click and select Requirements > Delete All Links.

Since R2024b

You can run baseline tests in a project by using the MATLAB Test Manager. For more information about baseline tests, see Create Baseline Tests for MATLAB Code.

The test manager enables you to create baseline data, export test data to the workspace, and update the stored baseline data:

  • To create baseline data for a test that failed because there was no baseline data when it ran, open the failure diagnostics by clicking the hyperlink in the Diagnostic column. Then, click the Create baseline from recorded test data hyperlink to store the actual value as baseline data in a MAT file.

  • To export the actual value and the corresponding baseline data for a failed test to the workspace, open the failure diagnostics by clicking the hyperlink in the Diagnostic column. Then, click the Export test data to workspace hyperlink.

  • To export only the baseline data for a test to the workspace, click the Baseline Data hyperlink in the Data column. To add the Data column to the Test Details table, click the Column button and then select Data.

  • To update the baseline data stored in a MAT file for a failed test, open the failure diagnostics by clicking the hyperlink in the Diagnostic column. Then, click the Update baseline from recorded test data hyperlink. Alternatively, you can select the test in the Test column, and then right-click and select Update baseline data.

Version History

Introduced in R2023a

expand all

See Also

Apps

Topics