coder.perfCompare

Compare execution times of MATLAB code and code generated using multiple configuration objects

Since R2024b

Syntax

t = coder.perfCompare(fcnName,numOutputs,runtimeArgs)

t = coder.perfCompare(fcnName,numOutputs,runtimeArgs,coderConfigs)

t = coder.perfCompare(___,Name=Value)

Description

t = coder.perfCompare(fcnName,numOutputs,runtimeArgs) compares the execution times of the MATLAB^® functions specified by fcnName with the generated MEX code. Use runtimeArgs to specify the run-time input arguments used to compare the performance, and numOutputs to specify the number of output arguments.

• By default, coder.perfCompare uses a MEX configuration object with IntegrityChecks and ResponsivenessChecks properties set to false.

• coder.perfCompare compares the performance for multiple runs and returns the median of execution times.

• coder.perfCompare function uses internal heuristics to determine the number of runs.

• coder.perfCompare excludes the timing overhead incurred by the data transfer between MATLAB and generated code execution.

t = coder.perfCompare(fcnName,numOutputs,runtimeArgs,coderConfigs) compares the execution times of MATLAB code and the code generated using configuration objects specified by coderConfigs. For build types lib and dll, generated code is executed using software-in-the-loop (SIL) or processor-in-the-loop (PIL) simulation, which requires an Embedded Coder^® license.

t = coder.perfCompare(___,Name=Value) specifies additional options using name-value arguments.

Examples

Compare Execution Times of MATLAB and Generated MEX Functions

Compare the execution times for the MATLAB® function integralExpLn and its generated MEX function.

Define the MATLAB® function integralExpLn.

function out = integralExpLn(xmin, xmax)
    f = @(x)x.^5.*exp(-x).*sin(x);
    out = integral(f, xmin, xmax, 'RelTol',1e-10);
end

Measure and compare the execution times of integralExpLn and its generated MEX function.

t = coder.perfCompare("integralExpLn",1,{0,Inf})

==== Running (integralExpLn, MATLAB) ====
- Running MATLAB script.
TimingResult with 961 Runtime Sample(s)

Statistical Overview:
   mean = 5.20e-04 s    max = 3.09e-02 s     sd = 1.02e-03 s
 median = 4.50e-04 s    min = 3.76e-04 s   90th = 5.21e-04 s

==== Running (integralExpLn, Codegen Mex) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 3780 Runtime Sample(s)

Statistical Overview:
   mean = 1.32e-04 s    max = 1.69e-03 s     sd = 5.03e-05 s
 median = 1.27e-04 s    min = 1.21e-04 s   90th = 1.38e-04 s

t=1×2 table
                        MATLAB                    Codegen Mex            
                     _____________    ___________________________________

                     Runtime (sec)    Runtime (sec)    Speedup wrt MATLAB
                     _____________    _____________    __________________
                                                                         
    integralExpLn       0.00045        0.00012678            3.5495      

Compare Execution Times of MATLAB Function and Two Generated MEX Functions

Compare the execution time of the MATLAB® function linearInterp with the execution times of two MEX functions generated using separate code configuration objects.

Define the MATLAB function linearInterp.

function out = linearInterp(x,v,xq)
    out = interp1(x,v,xq,"linear");
end

Define input arguments and code generation configurations.

x = 0:pi/1000:2*pi;
v = sin(x);
xq = 0:pi/2000:2*pi;

cfgFastMex = coder.config('mex');
cfgFastMex.IntegrityChecks = false;
cfgFastMex.ResponsivenessChecks = false;

cfgSIMDMex = coder.config('mex');
cfgSIMDMex.IntegrityChecks = false;
cfgSIMDMex.ResponsivenessChecks = false;
cfgSIMDMex.SIMDAcceleration = 'Full';

coder.perfCompare("linearInterp",1,{x,v,xq},{cfgFastMex,cfgSIMDMex})

==== Running (linearInterp, MATLAB) ====
- Running MATLAB script.
TimingResult with 3227 Runtime Sample(s)

Statistical Overview:
   mean = 1.55e-04 s    max = 1.22e-02 s     sd = 5.39e-04 s
 median = 1.04e-04 s    min = 9.30e-05 s   90th = 1.33e-04 s

==== Running (linearInterp, Coder Config 1) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 2983 Runtime Sample(s)

Statistical Overview:
   mean = 1.68e-04 s    max = 9.01e-03 s     sd = 2.53e-04 s
 median = 1.37e-04 s    min = 6.80e-05 s   90th = 1.80e-04 s

==== Running (linearInterp, Coder Config 2) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 4938 Runtime Sample(s)

Statistical Overview:
   mean = 1.01e-04 s    max = 9.17e-03 s     sd = 2.39e-04 s
 median = 8.39e-05 s    min = 5.56e-05 s   90th = 1.20e-04 s

                       MATLAB                  Coder Config 1                         Coder Config 2           
                    _____________    ___________________________________    ___________________________________
                    Runtime (sec)    Runtime (sec)    Speedup wrt MATLAB    Runtime (sec)    Speedup wrt MATLAB
                    _____________    _____________    __________________    _____________    __________________
    linearInterp      0.000104        0.00013736           0.75712           8.3857e-05            1.2402      

Compare Execution Times of MEX Functions Using Different Code Generation Configurations

Compare the execution times of the MEX functions generated from the MATLAB® functions sum1, sum2, and sum3, each with two different code generation configurations.

Define MATLAB® functions sum1, sum2, and sum3.

function out = sum1(in)
    out = sum(in,2);
end

function out = sum2(in)
    [nOut, nSum] = size(in);
    out = coder.nullcopy(zeros(nOut,1));
    for ii = 1: nOut
        outLocal = 0;
        for jj = 1:nSum
            outLocal = outLocal + in(ii,jj);
        end
    end
    out(ii) = outLocal;
end

function out = sum3(in)
    [nOut, nSum] = size(in);
    out = zeros(nOut,1);
    for ii = 1:nSum
        for jj = 1:nOut
            out(jj) = out(jj) + in(jj, ii);
        end
    end
end

Define code generation configurations.

cfg1 = coder.config('mex');
cfg1.IntegrityChecks = false;
cfg1.ResponsivenessChecks = false;

cfg2 = coder.config('mex');
cfg2.IntegrityChecks = false;
cfg2.ResponsivenessChecks = false;
cfg2.EnableAutoParallelization = true;

Define inputs, and measure and compare execution times for the six generated MEX functions (two for each sum function).

 
rng(42);
x = rand(1e4, 1e4);
 
rSum = coder.perfCompare({'sum1','sum2','sum3'},1,{x},{cfg1,cfg2}, ...
                ConfigNames={'Default','Autopar'},CompareWithMATLAB=false)

==== Running (sum1, Default) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 9.05e-02 s    max = 9.79e-02 s     sd = 3.79e-03 s
 median = 8.89e-02 s    min = 8.65e-02 s   90th = 9.62e-02 s

==== Running (sum1, Autopar) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 9.14e-02 s    max = 9.46e-02 s     sd = 1.84e-03 s
 median = 9.14e-02 s    min = 8.88e-02 s   90th = 9.39e-02 s

==== Running (sum2, Default) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 1.05e-01 s    max = 1.08e-01 s     sd = 2.19e-03 s
 median = 1.04e-01 s    min = 1.03e-01 s   90th = 1.08e-01 s

==== Running (sum2, Autopar) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 1.16e-01 s    max = 1.93e-01 s     sd = 2.86e-02 s
 median = 1.05e-01 s    min = 1.01e-01 s   90th = 1.63e-01 s

==== Running (sum3, Default) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 9.14e-02 s    max = 9.56e-02 s     sd = 2.72e-03 s
 median = 9.15e-02 s    min = 8.72e-02 s   90th = 9.53e-02 s

==== Running (sum3, Autopar) ====
- Generating code and building MEX.
- Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 8.99e-02 s    max = 9.73e-02 s     sd = 2.80e-03 s
 median = 8.91e-02 s    min = 8.79e-02 s   90th = 9.40e-02 s

rSum=3×2 table
               Default                     Autopar               
            _____________    ____________________________________

            Runtime (sec)    Runtime (sec)    Speedup wrt Default
            _____________    _____________    ___________________
                                                                 
    sum1      0.088894         0.091365             0.97295      
    sum2       0.10388          0.10461             0.99308      
    sum3      0.091535         0.089105              1.0273      

==== Running (sum1, Default) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 6.53e-02 s    max = 6.70e-02 s     sd = 1.00e-03 s
 median = 6.54e-02 s    min = 6.40e-02 s   90th = 6.67e-02 s

==== Running (sum1, Autopar) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 12 Runtime Sample(s)

Statistical Overview:
   mean = 4.46e-02 s    max = 4.93e-02 s     sd = 2.70e-03 s
 median = 4.36e-02 s    min = 4.15e-02 s   90th = 4.84e-02 s

==== Running (sum2, Default) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 5.40e-01 s    max = 5.56e-01 s     sd = 9.38e-03 s
 median = 5.40e-01 s    min = 5.27e-01 s   90th = 5.53e-01 s

==== Running (sum2, Autopar) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 10 Runtime Sample(s)

Statistical Overview:
   mean = 8.43e-02 s    max = 9.70e-02 s     sd = 6.77e-03 s
 median = 8.40e-02 s    min = 7.44e-02 s   90th = 9.49e-02 s

==== Running (sum3, Default) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 12 Runtime Sample(s)

Statistical Overview:
   mean = 4.36e-02 s    max = 4.55e-02 s     sd = 1.08e-03 s
 median = 4.37e-02 s    min = 4.14e-02 s   90th = 4.50e-02 s

==== Running (sum3, Autopar) ====
Perf Compare INFO: Generating code and building MEX.
Perf Compare INFO: Running MEX.
TimingResult with 12 Runtime Sample(s)

Statistical Overview:
   mean = 4.26e-02 s    max = 4.47e-02 s     sd = 1.16e-03 s
 median = 4.23e-02 s    min = 4.09e-02 s   90th = 4.42e-02 s


rSum =

  3×2 table

               Default                     Autopar               
            _____________    ____________________________________

            Runtime (sec)    Runtime (sec)    Speedup wrt Default
            _____________    _____________    ___________________
                                                                 
    sum1       0.06535         0.043552             1.5005       
    sum2        0.5395         0.083972             6.4248       
    sum3      0.043654         0.042331             1.0313       

Input Arguments

fcnName — Names of entry-point functions

character vector | cell array of character vectors or strings | string array

Names of entry-point functions for code generation and MATLAB execution, specified as a character vector, cell array of character vector or strings, or string array. When fcnName specifies multiple entry-point functions, they must all have the same number of input and output arguments, and the same input argument data types.

numOutputs — Number of outputs

nonnegative integer

Number of outputs the functions return, specified as a nonnegative integer.

runtimeArgs — Run-time arguments

cell array

Run-time input arguments to fcnName, specified as a cell array.

coderConfigs — Code generation configuration objects

cell array of coder.CodeConfig objects | cell array of coder.MexCodeConfig objects | cell array of coder.EmbeddedCodeConfig objects

Code generation configurations objects. To create these objects, use coder.config.

coder.config("exe") is not supported with coder.perfCompare.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: t = coder.perfCompare("sum1",1,{x},{cfg1},ConfigNames={"Mex"},CompareWithMATLAB=false);

ConfigNames — Names of output table columns

cell array of character vectors | cell array of strings

Names of the output table columns, specified as cell array of character vectors or cell array of strings.

Example: t = coder.perfCompare({'sum1','sum2'},1,{x},{cfg1,cfg2},ConfigNames={"config1","config2"});

CompareWithMATLAB — Option to compare execution time of generated code with MATLAB

true (default) | false

Option to compare execution time of generated code with MATLAB, specified as one of the values in this table.

Value	Description
true(default)	Compare the execution time of the MATLAB function with the different versions of the generated code.
false	Compare the execution times of the different versions of the generated code with each other.

Example: t = coder.perfCompare('sum1',1,{x},CompareWithMATLAB=false);

Output Arguments

t — Execution times

table

Execution times in seconds, returned as a table. The table also shows execution time speedup of each function relative to the first column. If you specify a single configuration and multiple functions, coder.perfCompare calculates the speedup relative to the run time of the first function. The table shows the medians of run times for each entry.

Tips

• To achieve consistent results, make sure no other compute-intensive processes are running on the machine where the MATLAB or the generated function is timed.

Version History

Introduced in R2024b