# Bellman Equation with two independent variables

25 views (last 30 days)
Laura Freitas on 15 Oct 2021
Answered: Harsh Mahalwar on 4 Mar 2024
Imagine I have a bellman equation with two independent variables:
V(a,b) = max{u(a,b)+βV(a',b')}
where the maximization is with respect to both a and b, β is the discount rate and a' is the future value of a.
What is the code to get matlab to solve this equation?

Harsh Mahalwar on 4 Mar 2024
Hi Laura,
From what I can gather, you are looking for an implementation of Bellman equation with 2 independent variables in MATLAB.
V(a, b) = max{u(a, b) + βV(a’, b’)};
Here’s a snippet of code that tries to implement this Bellman equation in MATLAB:
beta = 0.9; % Discount rate
A = linspace(0, 10, 50);
B = linspace(0, 10, 50);
% directions for future states (You can add or remove values from here)
dir = [1 0; -1 0; 0 1; 0 -1];
Here, I have created 2 example matrices (A and B) of size 1x50 each. For this example, I have added 4 directions to dir. This will help us to traverse the reward function V.
function V = bellmanEQwith2Vars(beta, A, B, dir)
% Sample utility function
u = @(x, y) x + y;
V = zeros(length(A) + 2, length(B) + 2);
% For this example we are going to do 10 iterations
for iter = 1:10
for i = 1:length(A)
for j = 1:length(B)
for k = 1:length(dir)
V(i + 1, j + 1) = max(V(i + 1, j + 1), u(A(i), B(j)) ...
+ beta * V(i + dir(k, 1) + 1, j + dir(k, 2) + 1));
end
end
end
end
end
Change the utility function(u) according to your needs.
I hope this help, thanks!