risk.validation.generalizedAreaUnderCurveTest

Generalized area under curve test

Since R2026a

Syntax

hGAUCTest = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade)

hGAUCTest = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade,ConfidenceLevel=confidenceLevel)

[hGAUCTest,Output] = risk.validation.generalizedAreaUnderCurveTest(___)

Description

hGAUCTest = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade) returns the result of a generalized area under curve (gAUC) test, which compares scores and responses in a baseline and target portfolio. The output is 1 if the test rejects the null hypothesis at the 95% confidence level, or 0 otherwise.

hGAUCTest = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade,ConfidenceLevel=confidenceLevel) specifies the confidence level for the gAUC test.

[hGAUCTest,Output] = risk.validation.generalizedAreaUnderCurveTest(___) also returns a structure Output that contains summary metrics.

Examples

Apply Generalized AUC Test of Loss Given Default Data

Load the loss given default (LGD) data, which contains the expected and realized LGD grades for 200 defaulted customers, as well as the year that the recovery process was closed.

lgdData = readtable("LGDRatingGradeData.csv")

lgdData=200×5 table
    ExpectedLGD    ExpectedLGDGrade    RealizedLGD    RealizedLGDGrade    RecoveryClosedYear
    ___________    ________________    ___________    ________________    __________________

      0.21067              4             0.11221              3                  2023       
      0.68432              8             0.90534             11                  2023       
       0.4815              6             0.35169              5                  2023       
      0.86389             10             0.91321             11                  2023       
      0.85797             10             0.84803             10                  2023       
      0.29985              4             0.24432              4                  2023       
      0.30411              5                   0              1                  2023       
      0.88206             10             0.65058              8                  2023       
        1.054             12             0.73065              9                  2023       
      0.96353             11             0.61439              8                  2023       
       0.3936              5            0.061432              2                  2023       
      0.55109              7             0.59951              7                  2023       
      0.75181              9             0.55973              7                  2023       
      0.78397              9             0.68201              8                  2023       
      0.40728              6             0.27822              4                  2023       
      0.61732              8             0.38996              5                  2023       
      ⋮

The ExpectedLGD and RealizedLGD table variables contain values from 1 to 12, which correspond to 12 rating grades. The values in ExpectedLGDGrade were calculated based on the borrower's characteristics before the default event and are predictions for the rating grade at the end of the recovery period. RealizedLGDGrade contains the actual grades of the portfolios at the end of the observation period.

Create a baseline portfolio table from observations whose recovery processes were closed in 2023 and a target portfolio table from observations whose recovery processes were closed in 2024.

baselinePortfolio = lgdData(lgdData.RecoveryClosedYear==2023,:);
targetPortfolio = lgdData(lgdData.RecoveryClosedYear==2024,:);

Extract the predicted and actual grades from each portfolio.

BaselineExpectedGrade = baselinePortfolio.ExpectedLGDGrade;
BaselineRealizedGrade = baselinePortfolio.RealizedLGDGrade;
TargetExpectedGrade = targetPortfolio.ExpectedLGDGrade;
TargetRealizedGrade = targetPortfolio.RealizedLGDGrade;

Perform the generalized AUC test for the grades.

[hGAUCTest,Output] = risk.validation.generalizedAreaUnderCurveTest(BaselineExpectedGrade,BaselineRealizedGrade,TargetExpectedGrade,TargetRealizedGrade)

hGAUCTest = logical
   1

Output = struct with fields:
                        RejectTest: 1
                            PValue: 0.0031
                     TestStatistic: 2.7421
                     CriticalValue: 1.6449
            BaselineGeneralizedAUC: 0.8390
    BaselineGeneralizedAUCVariance: 3.3724e-04
              TargetGeneralizedAUC: 0.7431
      TargetGeneralizedAUCVariance: 0.0012
                   ConfidenceLevel: 0.9500

The output includes the result of the hypothesis test and the p-value, among other statistics. The small p-value indicates that enough evidence exists to reject the null hypothesis that the gAUC for the baseline portfolio is smaller than the gAUC for the target portfolio.

Input Arguments

BaselineExpectedGrade — Predicted baseline portfolio grades

nonnegative vector

Predicted baseline portfolio grades, specified as a nonnegative vector. BaselineExpectedGrade must be of the same size as BaselineRealizedGrade. The baseline portfolio is sometimes called the initial portfolio.

Data Types: single | double

BaselineRealizedGrade — Actual baseline portfolio grades

nonnegative vector

Actual baseline portfolio grades, specified as a nonnegative vector. BaselineRealizedGrade must be of the same size as BaselineExpectedGrade. The baseline portfolio is sometimes called the initial portfolio.

Data Types: single | double

TargetExpectedGrade — Expected target portfolio grades

nonnegative vector

Expected target portfolio grades, specified as a nonnegative vector. TargetExpectedGrade must be of the same size as TargetRealizedGrade. The target portfolio is sometimes called the current portfolio.

Data Types: single | double

TargetRealizedGrade — Actual target portfolio grades

nonnegative vector

Actual target portfolio grades, specified as a nonnegative vector. TargetRealizedGrade must be of the same size as TargetExpectedGrade. The target portfolio is sometimes called the current portfolio.

Data Types: single | double

confidenceLevel — Confidence level

0.95 (default) | numeric scalar in the range (0,1)

Confidence level of the hypothesis test, specified as a numeric scalar in the range (0,1).

Output Arguments

hGAUCTest — Hypothesis test result

logical vector containing values of 1 or 0

Hypothesis test results, returned as a logical vector.

• A value of 1 rejects the null hypothesis at the specified confidence level.

• A value of 0 fails to reject the null hypothesis at the specified confidence level.

Output — Output metrics

structure

Output metrics, returned as a structure with the following fields:

• RejectTest — Logical scalar indicating whether the null hypothesis was rejected. This field represents the same values as hGAUCTest.

• PValue — p-value for the hypothesis test returned as a scalar in the range [0,1]. A small value indicates that the null hypothesis might not be valid.

• TestStatistic — Value of the test statistic for the hypothesis test, returned as a numeric scalar.

• CriticalValue — Minimum test statistic at which the test rejects the null hypothesis for the given probability and confidence level, returned as a numeric scalar.

• BaselineGeneralizedAUC — gAUC for the baseline portfolio, returned as a numeric scalar.

• BaselineGeneralizedAUCVariance — Estimated variance of the gAUC for the baseline portfolio, returned as a numeric scalar.

• TargetGeneralizedAUC — gAUC for the target portfolio, returned as a numeric scalar.

• TargetGeneralizedAUCVariance — Estimated variance of the gAUC for the target portfolio, returned as a numeric scalar.

• TargetTrials — Number of observations in the target portfolio, returned as a numeric scalar.

• ConfidenceLevel — Confidence level for the hypothesis test, returned as a numeric scalar.

For more information about the gAUC test and its corresponding statistics, see More About.

More About

Current gAUC vs. gAUC at Initial Validation Test

The current gAUC vs. gAUC at initial validation test [1], also called the gAUC test, requires you to input predicted and actual values for a baseline portfolio and a target portfolio. The null hypothesis is that the gAUC for the baseline portfolio is smaller than the gAUC for the target portfolio.

The gAUC is given by

$$\text{gAUC}=\frac{D_{ER}+1}{2},$$

where D_ER is the Somers' D metric for the expected and realized portfolio grades. For more information about Somers' D, see risk.validation.somersD.

The test statistic for the gAUC test is given by

$$S=\frac{\text{BaselineGeneralizedAUC-TargetGeneralizedAUC}}{\sqrt{\text{TargetGeneralizedAUCVariance}}}$$

where BaselineGeneralizedAUC and TargetGeneralizedAUC are the gAUCs for the baseline and target portfolios, respectively, and TargetGeneralizedAUCVariance is the estimated variance for the target portfolio. The software estimates the gAUCs and the target gAUC variance using the equations described in Section 3.2 of [1].

The p-value for S is equal to $$1-F(S)$$, where F is the cumulative distribution function (cdf) for the standard normal distribution.

References

[1] European Central Bank, “Instructions for reporting the validation results of internal models.” February, 2019. https://www.bankingsupervision.europa.eu/activities/internal_models/shared/pdf/instructions_validation_reporting_credit_risk.en.pdf.

Version History

Introduced in R2026a