conditionalDE

Conditional Du-Escanciano (DE) expected shortfall (ES) backtest

Syntax

TestResults = conditionalDE(ebtde)

[TestResults,SimTestStatistic] = conditionalDE(___,Name,Value)

Description

TestResults = conditionalDE(ebtde) runs the conditional expected shortfall (ES) backtest by Du and Escanciano [1]. The conditional test supports critical values by large-scale approximation and by finite-sample simulation.

[TestResults,SimTestStatistic] = conditionalDE(___,Name,Value) specifies options using one or more name-value pair arguments in addition to the input argument in the previous syntax.

Examples

Create an esbacktestbyde Object and Run a ConditionalDE Test

Create an esbacktestbyde object for a t model with 10 degrees of freedom and 2 lags, and then run a conditionalDE test.

load ESBacktestDistributionData.mat
    rng('default'); % For reproducibility
    ebtde = esbacktestbyde(Returns,"t",...
       'DegreesOfFreedom',T10DoF,...
       'Location',T10Location,...
       'Scale',T10Scale,...
       'PortfolioID',"S&P",...
       'VaRID',["t(10) 95%","t(10) 97.5%","t(10) 99%"],...
       'VaRLevel',VaRLevel);
    conditionalDE(ebtde,'NumLags',2)

ans=3×13 table
    PortfolioID        VaRID        VaRLevel    ConditionalDE      PValue      TestStatistic    CriticalValue    AutoCorrelation    Observations    CriticalValueMethod    NumLags    Scenarios    TestLevel
    ___________    _____________    ________    _____________    __________    _____________    _____________    _______________    ____________    ___________________    _______    _________    _________

       "S&P"       "t(10) 95%"        0.95         reject        3.2121e-09       39.113           5.9915            0.11009            1966          "large-sample"          2          NaN         0.95   
       "S&P"       "t(10) 97.5%"     0.975         reject        1.6979e-07       31.177           5.9915           0.087348            1966          "large-sample"          2          NaN         0.95   
       "S&P"       "t(10) 99%"        0.99         reject        9.1526e-05       18.598           5.9915           0.076814            1966          "large-sample"          2          NaN         0.95   

Input Arguments

ebtde — esbacktestbyde object

object

esbacktestbyde object, which contains a copy of the data (the PortfolioData, VarData, and ESData properties) and all combinations of portfolio ID, VaR ID, and VaR levels to be tested. For more information on creating an esbacktestbyde object, see esbacktestbyde.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: TestResults = conditionalDE(ebtde,'CriticalValueMethod','simulation','NumLags',10,'TestLevel',0.99)

'CriticalValueMethod' — Method to compute critical values, confidence intervals, and p-values

'large-sample' (default) | character vector with values of 'large-sample' or 'simulation' | string with values of "large-sample" or "simulation"

Method to compute critical values, confidence intervals, and p-values, specified as the comma-separated pair consisting of 'CriticalValueMethod' and a character vector or string with a value of 'large-sample' or 'simulation'.

Data Types: char | string

'NumLags' — Number of lags in conditionalDE test

1 (default) | positive integer

Number of lags in the conditionalDE test, specified as the comma-separated pair consisting of 'NumLags' and a positive integer.

Data Types: double

'TestLevel' — Test confidence level

0.95 (default) | numeric value between 0 and 1

Test confidence level, specified as the comma-separated pair consisting of 'TestLevel' and a numeric value between 0 and 1.

Data Types: double

Output Arguments

TestResults — Results

table

Results, returned as a table where the rows correspond to all combinations of portfolio ID, VaR ID, and VaR levels to be tested. The columns correspond to the following:

• 'PortfolioID' — Portfolio ID for the given data

• 'VaRID' — VaR ID for each of the VaR levels

• 'VaRLevel' — VaR level

• 'ConditionalDE'— Categorical array with the categories 'accept' and 'reject', which indicate the result of the conditional DE test

• 'PValue'— P-value of the conditional DE test

• 'TestStatistic'— Conditional DE test statistic

• 'CriticalValue'— Critical value for the conditional DE test

• 'AutoCorrelation'— Autocorrelation for the reported number of lags

• 'Observations'— Number of observations

• 'CriticalValueMethod'— Method used to compute confidence intervals and p-values

• 'NumLags'— Number of lags

• 'Scenarios'— Number of scenarios simulated to get the p-values

• 'TestLevel'— Test confidence level

Note

If you specify CriticalValueMethod as 'large-sample', the function reports the number of 'Scenarios' as NaN.

For the test results, the terms 'accept' and 'reject' are used for convenience. Technically, a test does not accept a model; rather, a test fails to reject it.

SimTestStatistic — Simulated values of the test statistics

numeric array

Simulated values of the test statistics, returned as an NumVaRs-by-NumScenarios numeric array.

More About

Conditional DE Test

The conditional DE test is a one-sided test to check if the test statistic is much larger than zero.

The test statistic for the conditional DE test is derived in several steps. First, define the autocovariance for lag j:

$${\gamma }_j=\frac{1}{N-j}{\displaystyle {\sum }_{t=j+1}^N(H_t-\alpha /2)(H_{t-j}-}\alpha /2)$$

where

• ɑ = 1- VaRLevel.

• H_t is the cumulative failures or violations process: H_t = (α - U_t)I(U_t < α) / α, where I(x) is the indicator function.

• U_t are the ranks or mapped returns U_t = P_t(X_t), where P_t(X_t) = P(X_t | θ_t) is the cumulative distribution of the portfolio outcomes or returns X_t over a given test window t = 1,...N and θ_t are the parameters of the distribution. For simplicity, the subindex t is both the return and the parameters, understanding that the parameters are those used on date t, even though those parameters are estimated on the previous date t-1, or even prior to that.

The exact theoretical mean α/2, as opposed to the sample mean, is used in the autocovariance formula, as suggested in the paper by Du and Escanciano [1].

The autocorrelation for lag j is then

$${\rho }_j=\frac{{\gamma }_j}{{\gamma }_0}$$

The test statistic for m lags is

$$C_{ES}(m)=N{\displaystyle {\sum }_{j=1}^m{\rho }_j^2}$$

Significance of the Test

The test statistic C_ES is a random variable and a function of random return sequences or portfolio outcomes X₁,…,X_N:

$$C_{ES}=C_{ES}(X_1,\mathrm{...},X_N).$$

For returns observed in the test window 1,…,N, the test statistic attains a fixed value:

$$C_{ES}^{obs}=C_{ES}(X_{obs1},\mathrm{...},X_{obsN}).$$

In general, for unknown returns that follow a distribution of P_t, the value of C_ES is uncertain and it follows a cumulative distribution function:

$$P_C(x)=P\left[C_{ES}\le x\right].$$

This distribution function computes a confidence interval and a p-value. To determine the distribution P_C, the esbacktestbyde class supports the large-sample approximation and simulation methods. You can specify one of these methods by using the optional name-value pair argument CriticalValueMethod.

For the large sample approximation method, the distribution P_C is derived from an asymptotic analysis. If the number of observations N is large, the test statistic is approximately distributed as a chi-square distribution with m degrees of freedom:

$$C_{ES}(m)\underset{dist}{\to }{\chi }_m^2=P_C$$

Note that the limiting distribution is independent of α.

If α_test = 1 - test confidence level, then the critical value CV is the value that satisfies the equation

$$1-P_C(CV)={\alpha }_{test}.$$

The p-value is determined as

$$P_{value}1-P_C(C_{ES}^{obs}).$$

The test rejects if p_value < α_test.

For the simulation method, the distribution P_Cis estimated as follows

1. Simulate M scenarios of returns as

   $$X^s=(X_1^s,\mathrm{...},X_N^s),s=1,\mathrm{...},M.$$

2. Compute the corresponding test statistic as

   $$C_{ES}^s=C_{ES}(X_1^s,\mathrm{...},X_N^s),s=1,\mathrm{...},M.$$

3. Define P_C as the empirical distribution of the simulated test statistic values as

   $$P_C=P\left[C_{ES}\le x\right]=\frac{1}{M}I(C_{ES}^s\le x),$$

   where I(.) is the indicator function.

In practice, simulating ranks is more efficient than simulating returns and then transforming the returns into ranks. simulate.

For the empirical distribution, the value of 1-P_C(x) may be different than P[C_ES ≥ x] because the distribution may have nontrivial jumps (simulated tied values). Use the latter probability for the estimation of confidence levels and p-values.

If ɑ_test = 1 - test confidence level, then the critical value of levels CV is the value that satisfies the equation

$$P\left[C_{ES}\ge CV\right]={\alpha }_{test}.$$

The reported critical value CV is one of the simulated test statistic values C^s_ES that approximately solves the preceding equation.

The p-value is determined as

$$p_{value}=P\left[C_{ES}\ge C_{ES}^{obs}\right].$$

The test rejects if p_value < α_test.

References

[1] Du, Z., and J. C. Escanciano. "Backtesting Expected Shortfall: Accounting for Tail Risk." Management Science. Vol. 63, Issue 4, April 2017.

[2] Basel Committee on Banking Supervision. "Minimum Capital Requirements for Market Risk". January 2016 (https://www.bis.org/bcbs/publ/d352.pdf).