onFailure

Class: matlab.unittest.TestCase
Namespace: matlab.unittest

Dynamically add diagnostics for test failures

expand all in page

Syntax

onFailure(testcase,failureDiag)
onFailure(testcase,failureDiag,'IncludingAssumptionFailures',tf)

Description

onFailure(testcase,failureDiag) adds diagnostics for test failures. If a test fails, then the testing framework executes the diagnostics. By default, these diagnostics execute upon verification failures, assertion failures, fatal assertion failures, and uncaught exceptions.

example

onFailure(testcase,failureDiag,'IncludingAssumptionFailures',tf) indicates if the testing framework also executes diagnostics upon assumption failures. To execute diagnostics upon assumption failures also, set tf to true.

Input Arguments

expand all

Instance of test case, specified as a matlab.unittest.TestCase object.

Diagnostic information to display upon a failure, specified as a character vector, string array, function handle, or array of matlab.automation.diagnostics.Diagnostic instances.

Example: @() disp('Failure Detected')

Example: matlab.unittest.diagnostics.ScreenshotDiagnostic

React to assumption failures, specified as false (logical 0) or true (logical 1). By default, this value is false and the testing framework executes diagnostics upon verification failures, assertion failures, fatal assertion failures, and uncaught exceptions. However, the framework does not execute diagnostics upon assumption failures. To execute additional diagnostics upon assumption failures, specify this value as true.

Examples

expand all

Open Script

In your current folder, create the SampleOnFailureTest test class.

classdef SampleOnFailureTest < matlab.unittest.TestCase
    methods (TestMethodSetup)
        function addFailureDiag(testCase)
            testCase.onFailure('Failure Detected')
        end
    end
    methods (Test)
        function verificationFailTest(testCase)
            testCase.onFailure(@()disp(datetime))
            testCase.verifyEqual(42,13)
        end
        function passingTest(testCase)
            testCase.assertTrue(true)
        end
        function assumptionFailTest(testCase)
            testCase.assumeEmpty(rand(2))
        end
        function assertionFailTest(testCase)
            act = randi(100,1,15);
            floor = randi(100,1,15);
            f = figure;
            plot(1:length(act),act,1:length(floor),floor)
            legend('actual','floor')
            testCase.addTeardown(@close,f)
            import matlab.unittest.diagnostics.FigureDiagnostic
            testCase.onFailure(FigureDiagnostic(f,'Formats','png'))
            testCase.assertGreaterThan(act,floor)
        end
    end
end

At the command prompt, run the tests. The SampleOnFailureTest class has these results:

  • The diagnostic message 'Failure Detected' is displayed for each test with a verification, assertion, or fatal assertion failure, because addFailureDiag calls onFailure in a TestMethodSetup block.

  • The verificationFailTest test adds another diagnostic upon failure that displays the current date and time.

  • The assumptionFailTest test fails by assumption. Therefore, the 'Failure Detected' message is not displayed.

  • The assertionFailTest test plots the data. If the test fails, the testing framework saves the plot.

results = runtests('SampleOnFailureTest');

Running SampleOnFailureTest

================================================================================
Verification failed in SampleOnFailureTest/verificationFailTest.
    ---------------------
    Framework Diagnostic:
    ---------------------
    verifyEqual failed.
    --> The numeric values are not equal using "isequaln".
    --> Failure table:
            Actual    Expected    Error     RelativeError  
            ______    ________    _____    ________________
        
              42         13        29      2.23076923076923
    
    Actual Value:
        42
    Expected Value:
        13
    ----------------------
    Additional Diagnostic:
    ----------------------
    Failure Detected
    ----------------------
    Additional Diagnostic:
    ----------------------
       05-Feb-2021 12:13:36
    
    
    ------------------
    Stack Information:
    ------------------
    In C:\work\MyExamples\SampleOnFailureTest.m (SampleOnFailureTest.verificationFailTest) at 10
================================================================================
..
================================================================================
SampleOnFailureTest/assumptionFailTest was filtered.
================================================================================
.
================================================================================
Assertion failed in SampleOnFailureTest/assertionFailTest and it did not run to completion.
    ---------------------
    Framework Diagnostic:
    ---------------------
    assertGreaterThan failed.
    --> Each element must be greater than each corresponding element of the minimum value array.
        
        Failing Indices:
             3     8    10    11    13    15
    
    Actual Value:
      Columns 1 through 13
    
        44    59    29     8    93    66    31    55    84    52    18    97    23
    
      Columns 14 through 15
    
        95     2
    Minimum Value (Exclusive):
      Columns 1 through 13
    
         3    23    32     5    43     1    17    76    62    91    51    33    26
    
      Columns 14 through 15
    
        11   100
    ----------------------
    Additional Diagnostic:
    ----------------------
    Failure Detected
    ----------------------
    Additional Diagnostic:
    ----------------------
    Figure saved to:
    --> C:\work\Temp\Figure_20a2887e-85de-4398-8aa4-26ef8d3d921d.png
    ------------------
    Stack Information:
    ------------------
    In C:\work\MyExamples\SampleOnFailureTest.m (SampleOnFailureTest.assertionFailTest) at 27
================================================================================
.
Done SampleOnFailureTest
__________

Failure Summary:

     Name                                      Failed  Incomplete  Reason(s)
    =======================================================================================
     SampleOnFailureTest/verificationFailTest    X                 Failed by verification.
    ---------------------------------------------------------------------------------------
     SampleOnFailureTest/assumptionFailTest                X       Filtered by assumption.
    ---------------------------------------------------------------------------------------
     SampleOnFailureTest/assertionFailTest       X         X       Failed by assertion.

Tips

  • To add a diagnostic for each test in your class, call onFailure from a method in the TestMethodSetup block.

Version History

Introduced in R2017b

See Also

|