GPR number of kernel parameter

Hi,all

I just use linear regression app to calculated 5-fold cross validation, (bayesian optimaztion). However, finally only one kernel scale provided.

the reviewer's comment:

As for the GPR model no-isotropic kernel function, the number of kernel scale is usually equal to the number of variables (xi). However, the authors mentioned in the Supporting Information that only one kernel scale was obtained. If it is such a case, the isotropic kernel function may be actually used. The similar issue also exists in the employed SVM model. The authors should check them again.

from the code, there are only two parameters

regressionGP = fitrgp(...
    predictors, ...
    response, ...
    'BasisFunction', 'none', ...
    'KernelFunction', 'exponential', ...
    'KernelParameters', [0.1832775367419647 4.728824320552641], ...
    'Sigma', 1.618591169127487, ...
    'Standardize', true);

how to find other kernel scale?

Thank you!

Answer 1

[Maneet Kaur Bagga]

Hi,

As per my understanding it seems that you have an exponential kernel function with two parameters specified in the "KernelParameters" argument. Assuming you have a dataset with several input features, you want to fit a Gaussian Process Regression (GPR) model with an anisotropic kernel function that allows for a different length scale for each feature.

One of the possible workaround for this could be to use the squared exponential function with "Automatic Relevance Determination", which allows for anisotropic scaling known as "ardsquaredexponential". It will give you a separate length scale for each input feature.

• The KernelFunction is set to "ardsquaredexponential", which is the ARD version of the squared exponential kernel function.
• The KernelParameters will include a vector of initial length scales "initialLengthScales" which should have the same number of elements as there are features in your dataset. The last parameter in KernelParameters is the signal variance (which was the second parameter in your original kernel parameters).
• After fitting the model, "optimizedLengthScales" will contain the optimized length scales for each feature. These values are learned during the model fitting process and indicate the relative importance of each feature.

Please refer to the following code snippet for better understanding:

% Assuming 'predictors' is your matrix of input features and 'response' is the output
numFeatures = size(predictors, 2); % Number of features

% Initialize the length scales (kernel scales) for each feature
% You can start with ones, or use any other heuristic or data-driven initialization
initialLengthScales = ones(1, numFeatures);

% Fit the GPR model with an anisotropic ARD kernel
regressionGP = fitrgp(...
    predictors, ...
    response, ...
    'BasisFunction', 'none', ...
    'KernelFunction', 'ardsquaredexponential', ... % ARD kernel for anisotropic scaling
    'KernelParameters', [initialLengthScales, 4.728824320552641], ... % Initial length scales for each feature
    'Sigma', 1.618591169127487, ...
    'Standardize', true);

% After training, you can extract the optimized length scales
optimizedLengthScales = regressionGP.KernelInformation.KernelParameters(1:end-1);

Please refer to the following MathWorks Documentation for further understanding:

https://in.mathworks.com/help/stats/kernel-covariance-function-options.html

Hope this helps!