simulate

Simulate expected shortfall (ES) test statistics

Syntax

ebts = simulate(ebts)

ebts = simulate(ebts,Name,Value)

Description

ebts = simulate(ebts) performs a simulation of ES test statistics. The simulate function simulates portfolio outcomes according to the distribution assumptions indicated in the esbacktestbysim object, and calculates all the supported test statistics under each scenario. The simulated test statistics are used to estimate the significance of the ES backtests.

ebts = simulate(ebts,Name,Value) adds optional name-value pair arguments.

Examples

Simulate ES Test Statistics

Create an esbacktestbysim object and run a simulation of 1000 scenarios.

load ESBacktestBySimData
rng('default'); % for reproducibility
ebts = esbacktestbysim(Returns,VaR,ES,"t",...
       'DegreesOfFreedom',10,...
       'Location',Mu,...
       'Scale',Sigma,...
       'PortfolioID',"S&P",...
       'VaRID',["t(10) 95%","t(10) 97.5%","t(10) 99%"],...
       'VaRLevel',VaRLevel);

The unconditional and minBiasAbsolute tests report 1000 scenarios (see the Scenarios column in the report).

unconditional(ebts) 

ans=3×10 table
    PortfolioID        VaRID        VaRLevel    Unconditional    PValue    TestStatistic    CriticalValue    Observations    Scenarios    TestLevel
    ___________    _____________    ________    _____________    ______    _____________    _____________    ____________    _________    _________

       "S&P"       "t(10) 95%"        0.95         accept        0.093       -0.13342         -0.16252           1966          1000         0.95   
       "S&P"       "t(10) 97.5%"     0.975         reject        0.031       -0.25011          -0.2268           1966          1000         0.95   
       "S&P"       "t(10) 99%"        0.99         reject        0.008       -0.57396         -0.38264           1966          1000         0.95   

minBiasAbsolute(ebts)

ans=3×10 table
    PortfolioID        VaRID        VaRLevel    MinBiasAbsolute    PValue    TestStatistic    CriticalValue    Observations    Scenarios    TestLevel
    ___________    _____________    ________    _______________    ______    _____________    _____________    ____________    _________    _________

       "S&P"       "t(10) 95%"        0.95          accept         0.062      -0.0014247       -0.0015578          1966          1000         0.95   
       "S&P"       "t(10) 97.5%"     0.975          reject         0.029      -0.0026674       -0.0023251          1966          1000         0.95   
       "S&P"       "t(10) 99%"        0.99          reject         0.005      -0.0060982       -0.0039004          1966          1000         0.95   

Run a second simulation with 5000 scenarios using the simulate function. Rerun the unconditional and minBiasAbsolute tests using the updated esbacktestbysim object. Notice that the tests now show 5,000 scenarios along with updated p-values and critical values.

ebts = simulate(ebts,'BlockSize',10000,'NumScenarios',5000,'TestList',["conditional","unconditional","quantile","minBiasAbsolute","minBiasRelative"]);   
unconditional(ebts) 

ans=3×10 table
    PortfolioID        VaRID        VaRLevel    Unconditional    PValue    TestStatistic    CriticalValue    Observations    Scenarios    TestLevel
    ___________    _____________    ________    _____________    ______    _____________    _____________    ____________    _________    _________

       "S&P"       "t(10) 95%"        0.95         accept        0.0952      -0.13342         -0.17352           1966          5000         0.95   
       "S&P"       "t(10) 97.5%"     0.975         reject        0.0456      -0.25011         -0.24318           1966          5000         0.95   
       "S&P"       "t(10) 99%"        0.99         reject         0.009      -0.57396         -0.38608           1966          5000         0.95   

minBiasAbsolute(ebts,"TestLevel",0.99)

ans=3×10 table
    PortfolioID        VaRID        VaRLevel    MinBiasAbsolute    PValue    TestStatistic    CriticalValue    Observations    Scenarios    TestLevel
    ___________    _____________    ________    _______________    ______    _____________    _____________    ____________    _________    _________

       "S&P"       "t(10) 95%"        0.95          accept         0.0622     -0.0014247       -0.0021797          1966          5000         0.99   
       "S&P"       "t(10) 97.5%"     0.975          accept          0.026     -0.0026674       -0.0032702          1966          5000         0.99   
       "S&P"       "t(10) 99%"        0.99          reject          0.006     -0.0060982       -0.0054814          1966          5000         0.99   

Input Arguments

ebts — esbacktestbysim object

object

esbacktestbysim (ebts) object, which contains a copy of the given data (the PortfolioData, VarData, ESData, and Distribution properties) and all combinations of portfolio ID, VaR ID, and VaR levels to be tested. For more information on creating an esbacktestbysim object, see esbacktestbysim.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: ebts = simulate(ebts,'NumScenarios',1000000,'BlockSize',10000,'TestList','conditional')

'NumScenarios' — Number of scenarios to simulate

1000 (default) | positive integer

Number of scenarios to simulate, specified using the comma-separated pair consisting of 'NumScenarios' and a positive integer.

Data Types: double

'BlockSize' — Number of scenarios to simulate in single simulation block

1000 (default) | positive integer

Number of scenarios to simulate in a single simulation block, specified using the comma-separated pair consisting of 'BlockSize' and a positive integer.

Data Types: double

'TestList' — Indicator for which test statistics to simulate

["conditional","unconditional","quantile","minBiasAbsolute","minBiasRelative"] (default) | character vector with value of 'conditional', 'unconditional', 'quantile', 'minBiasAbsolute', or 'minBiasRelative' | string with value of "conditional", "unconditional", "quantile", "minBiasAbsolute", or "minBiasRelative"

Indicator for which test statistics to simulate, specified as the comma-separated pair consisting of 'TestList' and a cell array of character vectors or a string array with the value conditional, unconditional, quantile, minBiasAbsolute or minBiasRelative.

Data Types: char | cell | string

Output Arguments

ebts — Updated esbacktestbysim object

object

esbacktestbysim (ebts), returned as an updated object. After running simulate, the updated esbacktestbysim object stores the simulated test statistics, which are used to calculate p-values and generate test results.

For more information on an esbacktestbysim object, see esbacktestbysim.

More About

Simulation of Test Statistics and Significance of the Tests

The VaR and ES models assume that for each period t, the portfolio outcomes X_t have a cumulative probability distribution P_t.

Under the assumption that the distributions P_t are correct (the null hypothesis), test statistics are simulated by:

• Simulating M scenarios of N observations each, for example, $$X^s=(X_1^s,\mathrm{...},X_t^s,\mathrm{...},X_N^s)$$, with $$X_t^s~P_t$$, t = 1,…,N, and s = 1,…,M.

• For each simulated scenario X^s, compute the test statistic of interest Z^s = Z(X^s), s = 1,…,M.

• The resulting M simulated test statistic values Z¹,…,Z^M from a distribution of the test statistic assuming the probability distributions P_t are correct.

The p-value is defined as the proportion of scenarios for which the simulated test statistic is smaller than the test statistic evaluated at the observed portfolio outcomes: $$Z^{obs}=Z(X_1,\mathrm{...}{X}_N)$$:

$$P_{value}=\frac{1}{M}{\displaystyle \sum _{s=1}^{M}I(Z^s\le Z^{obs})}$$

where I(Z^s ≤ Z^obs) is an indicator function with a value of 1 if Z^s ≤ Z^obs, and 0 otherwise. If P_test is 1 minus the test confidence level, the test result is to ‘reject’ if $$P_{value}<P_{test}$$.

The critical value is defined as the minimum simulated test statistic Z^critwith a p-value greater than or equal to P_test.

References

[1] Acerbi, C., and B. Szekely. Backtesting Expected Shortfall. MSCI Inc. December, 2014.