Measure Fast Executing Test Code

Performance tests that execute too quickly for MATLAB^® to measure accurately are filtered with an assumption failure. With the keepMeasuring method, the testing framework can measure significantly faster code by automatically determining the number of times to iterate through code and measuring the average performance.

In your current working folder, create a class-based test, PreallocationTest.m, that compares different methods of preallocation. Since the test methods include qualifications, use the startMeasuring and stopMeasuring methods to define boundaries for the code you want to measure.

classdef PreallocationTest < matlab.perftest.TestCase
    methods(Test)
        function testOnes(testCase)
            testCase.startMeasuring
            x = ones(1,1e5);
            testCase.stopMeasuring
            testCase.verifyEqual(size(x),[1 1e5])
        end
        
        function testIndexingWithVariable(testCase)
            import matlab.unittest.constraints.IsSameSetAs
            testCase.startMeasuring
            id = 1:1e5;
            x(id) = 1;
            testCase.stopMeasuring
            testCase.verifyThat(x,IsSameSetAs(1))
        end
        
        function testIndexingOnLHS(testCase)
            import matlab.unittest.constraints.EveryElementOf
            import matlab.unittest.constraints.IsEqualTo
            testCase.startMeasuring
            x(1:1e5) = 1;
            testCase.stopMeasuring
            testCase.verifyThat(EveryElementOf(x),IsEqualTo(1))
        end
        
        function testForLoop(testCase)
            testCase.startMeasuring
            for i=1:1e5
                x(i) = 1;
            end
            testCase.stopMeasuring
            testCase.verifyNumElements(x,1e5)
        end
    end
end

Run PreallocationTest as a performance test. Two tests are filtered because the measurements are too close to the precision of the framework.

results = runperf('PreallocationTest');

Running PreallocationTest
........
================================================================================
PreallocationTest/testOnes was filtered.
    Test Diagnostic: The MeasuredTime should not be too close to the precision of the framework.
Details
================================================================================
.. .......... ....
================================================================================
PreallocationTest/testIndexingOnLHS was filtered.
    Test Diagnostic: The MeasuredTime should not be too close to the precision of the framework.
Details
================================================================================
...... ..
Done PreallocationTest
__________

Failure Summary:

     Name                                 Failed  Incomplete  Reason(s)
    ==================================================================================
     PreallocationTest/testOnes                       X       Filtered by assumption.
    ----------------------------------------------------------------------------------
     PreallocationTest/testIndexingOnLHS              X       Filtered by assumption.

To instruct the framework to automatically loop through the measured code and average the measurement results, modify PreallocationTest to use a keepMeasuring-while loop instead of startMeasuring and stopMeasuring.

classdef PreallocationTest < matlab.perftest.TestCase
    methods(Test)
        function testOnes(testCase)
            while(testCase.keepMeasuring)
                x = ones(1,1e5);
            end
            testCase.verifyEqual(size(x),[1 1e5])
        end
        
        function testIndexingWithVariable(testCase)
            import matlab.unittest.constraints.IsSameSetAs
            while(testCase.keepMeasuring)
                id = 1:1e5;
                x(id) = 1;
            end
            testCase.verifyThat(x,IsSameSetAs(1))
        end
        
        function testIndexingOnLHS(testCase)
            import matlab.unittest.constraints.EveryElementOf
            import matlab.unittest.constraints.IsEqualTo
            while(testCase.keepMeasuring)
                x(1:1e5) = 1;
            end
            testCase.verifyThat(EveryElementOf(x),IsEqualTo(1))
        end
        
        function testForLoop(testCase)
            while(testCase.keepMeasuring)
                for i=1:1e5
                    x(i) = 1;
                end
            end
            testCase.verifyNumElements(x,1e5)
        end
    end
end

Rerun the tests. All the tests complete.

results = runperf('PreallocationTest');

Running PreallocationTest
.......... .......... .......... ..
Done PreallocationTest
__________

View the results.

sampleSummary(results)

ans =

  4×7 table

                       Name                       SampleSize       Mean       StandardDeviation       Min          Median         Max    
    __________________________________________    __________    __________    _________________    __________    __________    __________

    PreallocationTest/testOnes                        4         3.0804e-05       1.8337e-07        3.0577e-05    3.0843e-05    3.0953e-05
    PreallocationTest/testIndexingWithVariable        4         0.00044536       1.7788e-05        0.00042912    0.00044396    0.00046441
    PreallocationTest/testIndexingOnLHS               4         5.6352e-05       1.8863e-06        5.5108e-05    5.5598e-05    5.9102e-05
    PreallocationTest/testForLoop                     4          0.0097656       0.00018202         0.0096181     0.0097065      0.010031