Analyze Failed Test Results

This example shows how to identify and rerun failed tests.

Create an Incorrect Test Method

Using the SolverTest test case, add a method, testBadRealSolution. This test, based on testRealSolution, calls the quadraticSolver function with inputs 1,3,2, but tests the results against an incorrect solution, [2,1].

function testBadRealSolution(testCase)
    actSolution = quadraticSolver(1,3,2);
    expSolution = [2,1];
    testCase.verifyEqual(actSolution,expSolution)
end

Run New Test Suite

Save the updated SolverTest class definition and rerun the tests.

quadTests = matlab.unittest.TestSuite.fromClass(?SolverTest);
result1 = run(quadTests);

Running SolverTest
..
================================================================================
Verification failed in SolverTest/testBadRealSolution.

    ---------------------
    Framework Diagnostic:
    ---------------------
    verifyEqual failed.
    --> The values are not equal using "isequaln".
    --> Failure table:
                Index    Actual    Expected    Error    RelativeError
                _____    ______    ________    _____    _____________
            
                1        -1        2           -3       -1.5         
                2        -2        1           -3         -3         
    
    Actual Value:
            -1    -2
    Expected Value:
             2     1

    ------------------
    Stack Information:
    ------------------
    In C:\work\SolverTest.m (SolverTest.testBadRealSolution) at 19
================================================================================
.
Done SolverTest
__________

Failure Summary:

     Name                            Failed  Incomplete  Reason(s)
    =============================================================================
     SolverTest/testBadRealSolution    X                 Failed by verification.

Analyze Results

The output tells you SolverTest/testBadRealSolution failed. From the Framework Diagnostic you see the following:

    Actual Value:
            -1    -2
    Expected Value:
             2     1

At this point, you must decide if the error is in quadraticSolver or in your value for expSolution.

Correct Error

Edit the value of expSolution in testBadRealSolution:

expSolution = [-1 -2];

Rerun Tests

Save SolverTest and rerun only the failed tests.

failedTests = quadTests([result1.Failed]);
result2 = run(failedTests)

Running SolverTest
.
Done SolverTest
__________


result2 = 

  TestResult with properties:

          Name: 'SolverTest/testBadRealSolution'
        Passed: 1
        Failed: 0
    Incomplete: 0
      Duration: 0.0108
       Details: [1x1 struct]

Totals:
   1 Passed, 0 Failed, 0 Incomplete.
   0.010813 seconds testing time.

Alternatively, you can rerun failed tests using the (rerun) link in the test results.