Run Impacted Tests Using MATLAB Build Tool

Since R2025a

With MATLAB^® Test™, you can accelerate testing in your automated builds by integrating test impact analysis using the MATLAB build tool. With test impact analysis enabled, the build tool detects changes to tests and exercised source code, finds the tests impacted by those changes, and runs only the impacted tests rather than running the entire test suite. Incremental validation through test impact analysis reduces testing overhead, allowing for faster regression testing and more frequent integrations.

This topic shows how to enable test impact analysis and run only the impacted tests using the MATLAB build tool. For impact-based testing, you must configure a matlab.buildtool.tasks.TestTask instance for test impact analysis and run the task.

What Are Impacted Tests?

Impacted tests are the tests associated with a TestTask instance that are affected by changes to specific file-based task inputs (test, source, and supporting files) since the last successful task run. Finding impacted tests involves detecting changes to these task inputs (change detection) followed by analyzing the impact of the detected changes on the tests (impact analysis).

Change Detection

If you enable test impact analysis, then the build tool detects changes to the following file-based inputs of the task (TestTask instance), which represent code and data, since the last successful task run:

• Tests — Test files to run, specified in the Tests property of the task. In addition to the test files, the build tool also tracks test class folders and test superclasses.

• Source files — Source files under test, specified in the SourceFiles property of the task.

• Supporting files — Supporting files used by the tests, such as test data and helper files, specified in the SupportingFiles property of the task.

The build tool detects changes to an input when you add a new file to or modify an existing file in that input. File deletions are not taken into account.

A task trace is a record of the inputs, outputs, actions, and arguments of a task from its last successful run. If a TestTask instance does not have a trace, then the build tool considers all of its inputs to be changed for the purpose of test impact analysis, which results in all the tests being run. For example, the build tool considers all the test, source, and supporting files to be changed when:

• The task runs for the first time after creation.

• The task runs after its trace is deleted.

• The task does not support incremental builds. For a TestTask instance to support incremental builds, its SourceFiles property must be nonempty and its DisableIncremental property must be false. For more information about incremental builds, see Improve Performance with Incremental Builds.

Impact Analysis

To find the tests that are impacted by changes to code and data since the last successful task run, the task runs a dependency analysis similar to that of the Dependency Analyzer app. Based on this analysis, the task runs only these tests:

• Modified or new tests

• Tests with a modified test class folder or test superclass

• Tests that depend on modified or new source files (specified in the SourceFiles property of the task)

• Tests that depend on modified or new supporting files (specified in the SupportingFiles property of the task)

For more information about dependency analysis and its limitations, see Dependency Analyzer Scope and Limitations.

Enable Test Impact Analysis

By default, a TestTask instance executes all the tests in the test suite when it runs. However, if you have a MATLAB Test license, you can enable test impact analysis by specifying the RunOnlyImpactedTests property or argument of the task as true.

For example, the RunOnlyImpactedTests property of the "test" task in this build file is set to true. Therefore, the task runs only the impacted tests in the current folder and its subfolders and fails the build if any of those tests fail.

function plan = buildfile
import matlab.buildtool.tasks.TestTask

% Create a plan with no tasks
plan = buildplan;

% Add a task to run only the impacted tests
plan("test") = TestTask( ...
    SourceFiles="mySource", ...
    RunOnlyImpactedTests=true);
end

You can also use the RunOnlyImpactedTests task argument to control the task behavior at run time. For example, specify the task argument as false so that the "test" task runs all the tests in the test suite without performing a test impact analysis.

buildtool test(RunOnlyImpactedTests=false)

Note

If you have not specified any source code (the SourceFiles property of the task is empty), then the task runs all the tests regardless of the value of the RunOnlyImpactedTests property or task argument.

Run Impacted Tests

This example shows how to run the tests impacted by changes to a source file by using a TestTask instance whose RunOnlyImpactedTests property is true.

First, open the example and then navigate to the impacted_tests_example folder.

cd impacted_tests_example

The impacted_tests_example folder contains a build file named buildfile.m as well as source and tests folders:

• The source folder contains the DocPolynom.m, BankAccount.m, and AccountManager.m source files.

• The tests folder contains the DocPolynomTest.m and BankAccountTest.m test files.

This dependency graph, created using the Dependency Analyzer app, shows the files in the impacted_tests_example folder and how they relate to each other. The test files depend on the corresponding source files. For example, the DocPolynomTest.m test file depends on the DocPolynom.m file.

This code shows the contents of the build file. The build file contains two built-in tasks:

• The "clean" task deletes outputs and traces of the other tasks in the build file.

• The "test" task runs only the impacted tests in the tests folder and fails the build if any of those tests fail. The task runs a test impact analysis because its RunOnlyImpactedTests property is set to true.

function plan = buildfile
import matlab.buildtool.tasks.CleanTask
import matlab.buildtool.tasks.TestTask

% Create a plan with no tasks
plan = buildplan;

% Add the source folder to the path
addpath("source")

% Add a task to delete outputs and traces
plan("clean") = CleanTask;

% Add a task to run only the impacted tests
plan("test") = TestTask("tests", ...
    SourceFiles="source", ...
    RunOnlyImpactedTests=true);
end

Run the "test" task. The first time you run the task, the build tool considers all the tests to be impacted because no task trace exists. Therefore, the task runs the three tests in the DocPolynomTest.m test file and the five tests in the BankAccountTest.m test file.

buildtool test

** Starting test
Finding impacted tests...
--> Found 8 tests impacted by changes since the last successful run (8 total).
........

Test Summary:
    Total Tests: 8
         Passed: 8
         Failed: 0
     Incomplete: 0
       Duration: 1.5958 seconds testing time.
                 
** Finished test

Run the "test" task again. The build tool skips the task because the task is up to date.

buildtool test

** Skipped test (up-to-date)

Modify one of the source files. For example, append a string to the DocPolynom.m source file. Then, run the "test" task. Because test impact analysis is enabled, the task runs only the tests impacted by the change to the DocPolynom.m source file instead of running all the tests. To better understand the changes, you can specify the -verbosity build option (for example, buildtool test -verbosity Detailed).

writelines("","source/DocPolynom.m",WriteMode="append")
buildtool test

** Starting test
Finding impacted tests...
--> Found 3 tests impacted by changes since the last successful run (8 total).
...

Test Summary:
    Total Tests: 3
         Passed: 3
         Failed: 0
     Incomplete: 0
       Duration: 0.24838 seconds testing time.
                 
** Finished test

Rerun the "test" task after deleting its trace using the "clean" task. The "test" task runs all the tests because they are considered impacted when no task trace exists.

buildtool clean test

** Starting clean
** Finished clean

** Starting test
Finding impacted tests...
--> Found 8 tests impacted by changes since the last successful run (8 total).
........

Test Summary:
    Total Tests: 8
         Passed: 8
         Failed: 0
     Incomplete: 0
       Duration: 0.31257 seconds testing time.
                 
** Finished test

Modify the DocPolynom.m file once more and then run the "test" task by temporarily disabling test impact analysis. When you specify the RunOnlyImpactedTests task argument as false, the task runs all the tests instead of running only the impacted tests.

writelines("","source/DocPolynom.m",WriteMode="append")
buildtool test(RunOnlyImpactedTests=false)

** Starting test
........

Test Summary:
    Total Tests: 8
         Passed: 8
         Failed: 0
     Incomplete: 0
       Duration: 0.30211 seconds testing time.
                 
** Finished test