I need to run Optimization using models I generated in Regression Learner

Hi, all!

I desperately need help. As the title says, I generated multiple surrogate models on Regression Learner; now I need to use these models in a gradient-based optimization process.

I've tried Export>Generate Function & Export>Export Model / Export Model For Deployment. None of them work, I don't understand how anything works. If I can convert the surrogate models into matlab functions, these matlab functions are supposed to go into the 'select problem data>Objective function'? How do I define lower/upper bounds with just one value for each when the surrogate models have multiple inputs?? How do I define non-linear constraints??

I put an image just to explain my problem better.

Thank you....

[dpb]

What tool are you trying to use? I've never done anything with the apps; I've do everything at the command line so would have to try to go look at the doc..

[Roberto Louis]

Sorry to reply late. I'm using Regression Learner in the "Machine Learning and Deep Learning" section. I tried to Optimization tool in the "Math, Statistics, and Optimization" section, but its no longer available and Matlab keeps on redirecting me to the Live Editor.

Answer 1

[Hitesh]

Hi Roberto !

I am assuming that you are able to generate multiple surrogate models using the Regression Learner in the Machine Learning and Deep Learning Toolbox. Once you trained the model, click on Generate function option to get the model function in workspace. Make sure to save the model and verify that the model is able to predict the response.

Below functions are required to minimized the predicted response and optimize the input variable within the non-linear constraints:

• Objective and Constraints Definition:The "objectiveFunction" computes the average predicted response from two regression models using input features, while "nonLinearConstraints" defines inequality constraints on the inputs, ensuring they satisfy specific conditions.
• Bounds and Initial Guess: Specify the lower and upper bounds for the input variables and provide an initial guess for optimization.
• Optimization Execution: Use an appropriate optimization function (such as ga, fmincon, or fminunc) to optimize the model based on your objective function and obtain the optimized values.
• Refer to the documentation: Regression Learner App - MATLAB & Simulink (mathworks.com)

Please refer to the below code for minimizing the predicted response using optimization of input variable within the upper and lower bound.

%% Objective Function for Multiple Models
function objValue = objectiveFunction(x, model1, model2)
    % x is a vector of inputs [Feature1, Feature2]
    % model1, model2 are regression models trained earlier
    
    % Create a table with the input vector
    inputData = table(x(1), x(2), 'VariableNames', {'Feature1', 'Feature2'});
    
    % Predict the response using both models
    predictedValue1 = model1.predictFcn(inputData);
    predictedValue2 = model2.predictFcn(inputData);
    
    % Combine predictions (e.g., average of predictions)
    objValue = (predictedValue1 + predictedValue2) / 2;
end


%% Non-linear Constraints
function [c, ceq] = nonLinearConstraints(x)
    % Non-linear inequality constraints
    c = [x(1)^2 + x(2)^2 - 25;  % Example: x1^2 + x2^2 <= 25
         x(1) + x(2) - 10];     % Example: x1 + x2 <= 10
    
    % No equality constraints
    ceq = [];
end

%% Optimization Process
% Initial guess for the inputs
x0 = [5, 2.5];  % Example initial guess for [Feature1, Feature2]

% Define lower and upper bounds for the inputs
lb = [0, 0];  % Lower bounds for [Feature1, Feature2]
ub = [10, 5]; % Upper bounds for [Feature1, Feature2]

% Set optimization options
opts = optimoptions('fmincon', 'Display', 'iter');

% Call the optimization function
[xOpt, fval] = fmincon(@(x) objectiveFunction(x, trainedModel1, trainedModel2), x0, [], [], [], [], lb, ub, @nonLinearConstraints, opts);

% Display the optimal inputs and the corresponding objective value
disp('Optimal Inputs:');
disp(xOpt);
disp('Objective Value at Optimal Inputs:');
disp(fval);

[Roberto Louis]

Thank you for this!

It took a lot of trial and error but I was able to adopt this code to my project.