risk.validation.proportionOfFailuresTest

Proportion of failures (POF) test

Since R2025b

collapse all in page

    Syntax

    hPOFTest = risk.validation.proportionOfFailuresTest(Probability,NumEvents,NumTrials)
    hPOFTest = risk.validation.proportionOfFailuresTest(Probability,NumEvents,NumTrials,Confidencelevel=confidenceLevel)
    [hPOFTest,Output] = risk.validation.proportionOfFailuresTest(___)

    Description

    hPOFTest = risk.validation.proportionOfFailuresTest(Probability,NumEvents,NumTrials) returns the proportion of failures (POF) test result or results, hPOFTest, for a given set of probabilities, events, and trials. The output is 1 if the test rejects the null hypothesis at the 95% confidence level, or 0 otherwise. Probability contains numeric values that represent quantities such as a VaR level for a portfolio.

    hPOFTest = risk.validation.proportionOfFailuresTest(Probability,NumEvents,NumTrials,Confidencelevel=confidenceLevel) specifies the confidence level for the POF test.

    [hPOFTest,Output] = risk.validation.proportionOfFailuresTest(___) also returns a structure Output that contains a table of summary metrics.

    example

    Examples

    collapse all

    Open Live Script

    Perform a proportion of failures (POF) test to determine whether 5% or 1% is an accurate probability for an event occurring in a single trial, given a sample of 250 trials in which three events occurred.

    [hPOFTest,Output] = risk.validation.proportionOfFailuresTest([0.05,0.01],3,250)
    hPOFTest = 2×1

     1
     0


    Output = struct with fields:
    Results: [2×8 table]


    Output.Results
    ans=2×8 table
    RejectPOFTest     PValue      LogRatio    CriticalValue    ConfidenceLevel    NumEvents    NumTrials    Probability
    _____________    _________    ________    _____________    _______________    _________    _________    ___________

          1          0.0010083     10.812        3.8415             0.95              3           250          0.05    
          0            0.75799    0.09494        3.8415             0.95              3           250          0.01

    The output includes the hypothesis test results, and their corresponding p-values. The test results indicate that enough evidence exists to reject the null hypothesis for the 5%, but not the 1% probability. This conclusion is supported by the small p-value for the test statistics corresponding to 5%.

    Input Arguments

    collapse all

    Probability values, specified as a numeric scalar between 0 and 1, or a vector of values between 0 and 1. For value-at-risk (VaR) models, Probability can be a set of VaR levels for a portfolio or portfolios.

    When Probability is a vector, risk.validation.proportionOfFailuresTest performs a test for each of its elements. If one or both of NumEvents and NumTrials are also vectors, they must be of the same length as Probability.

    Example: 0.95

    Data Types: single | double

    Number of events observed, specified as a numeric vector of nonnegative integers. For VaR models, NumEvents contains the number of VaR exceptions.

    When NumEvents is a vector, risk.validation.proportionOfFailuresTest performs a test for each of its elements. If one or both of Probability and NumTrials are also vectors, they must be of the same length as NumEvents.

    Example: 30

    Data Types: single | double

    Number of trials, specified as a numeric vector of positive integers. For VaR models, NumTrials contains the number of time periods in the backtesting window.

    When NumTrials is a vector, risk.validation.proportionOfFailuresTest performs a test for each of its elements. If one or both of Probability and NumEvents are also vectors, they must be of the same length as NumEvents.

    Example: 250

    Data Types: single | double

    Confidence level of the hypothesis test, specified as a numeric scalar in the range (0,1).

    Output Arguments

    collapse all

    Hypothesis test results, returned as a positive integer scalar or vector of positive integers.

    • A value of 1 rejects the null hypothesis at the specified confidence level.

    • A value of 0 fails to reject the null hypothesis at the specified confidence level.

    Output metrics, returned as a structure with a single field Results. Results contains a table with the following columns:

    • RejectPOFTest — Hypothesis test results, returned as a numeric scalar or vector.

    • PValue — p-values for the POF tests, returned as a numeric scalar or vector.

    • LogRatio — Test statistics for the hypothesis tests, returned as a numeric scalar or vector.

    • CriticalValue — Critical values for the hypothesis tests, returned as a numeric scalar or vector.

    • ConfidenceLevel — Confidence level for the hypothesis tests, returned as a numeric scalar or vector. You specify the confidence level using the confidenceLevel input argument.

    • NumEvents — Number of events, returned as a positive integer or vector of positive integers. You specify this value using the NumEvents input argument.

    • NumTrials — Number of trials, returned as a positive integer or vector of positive integers. You specify this value using the NumTrials input argument.

    • Probability — Probability of an event, returned as a positive scalar or vector of positive scalars. You specify this value using the Probability input argument.

    For more information about the POF test and its corresponding statistics, see More About.

    More About

    collapse all

    Alternative Functionality

    Function

    You can use the varbacktest object function pof to perform a proportion of failures test on a timeseries of portfolio outcomes. pof allows you to perform proportion of failures tests on multiple timeseries with different VaR confidence levels.

    References

    [1] Kupiec, P. "Techniques for Verifying the Accuracy of Risk Management Models." Journal of Derivatives. Vol. 3, 1995, pp. 73–84.

    Version History

    Introduced in R2025b

    See Also

    | | |

    Topics