Clear Filters
Clear Filters

Bellman Equation with two independent variables

25 views (last 30 days)
Imagine I have a bellman equation with two independent variables:
V(a,b) = max{u(a,b)+βV(a',b')}
where the maximization is with respect to both a and b, β is the discount rate and a' is the future value of a.
What is the code to get matlab to solve this equation?

Answers (1)

Harsh Mahalwar
Harsh Mahalwar on 4 Mar 2024
Hi Laura,
From what I can gather, you are looking for an implementation of Bellman equation with 2 independent variables in MATLAB.
V(a, b) = max{u(a, b) + βV(a’, b’)};
Here’s a snippet of code that tries to implement this Bellman equation in MATLAB:
beta = 0.9; % Discount rate
A = linspace(0, 10, 50);
B = linspace(0, 10, 50);
% directions for future states (You can add or remove values from here)
dir = [1 0; -1 0; 0 1; 0 -1];
Here, I have created 2 example matrices (A and B) of size 1x50 each. For this example, I have added 4 directions to dir. This will help us to traverse the reward function V.
function V = bellmanEQwith2Vars(beta, A, B, dir)
% Sample utility function
u = @(x, y) x + y;
V = zeros(length(A) + 2, length(B) + 2);
% For this example we are going to do 10 iterations
for iter = 1:10
for i = 1:length(A)
for j = 1:length(B)
for k = 1:length(dir)
V(i + 1, j + 1) = max(V(i + 1, j + 1), u(A(i), B(j)) ...
+ beta * V(i + dir(k, 1) + 1, j + dir(k, 2) + 1));
end
end
end
end
end
Change the utility function(u) according to your needs.
Please use the following link to learn more about bellman equations and dynamic programming:
I hope this help, thanks!

Categories

Find more on Mathematics in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!