How to make Markov Chain model from sequence of data in MATLAB?

Markov Chain model considers only 1-step transition probabilities i.e. probability distribution of next state depends only on current state and not on previous state. I have a sequence and from that I have to make Markov Chain Model in MATLAB. I am using equation given below to make Markov Chain Model.

So probability matrix becomes as follows

Answer 1

[Shantanu Dixit]

Hi Vijay,

Assuming you already have an observed sequence of states 'X' and a known number of unique states 'S', you can build a Markov Chain transition probability matrix using MATLAB math functionalities like 'zeros': https://www.mathworks.com/help/matlab/ref/zeros.html

The idea is to count how often each transition from state 'i' to state 'j' occurs and then normalize it to get probabilities. Here's a simple implementation:

function P = buildMarkovChain(X, S)
    Nij = zeros(S, S);    % Counts of transitions from state i to state j
    Ni = zeros(1, S);     % Counts of transitions starting from state i

    for t = 1:length(X)-1
        i = X(t);        
        j = X(t+1);       
        Nij(i, j) = Nij(i, j) + 1;
        Ni(i) = Ni(i) + 1;
    end

    % Transition probability matrix
    P = zeros(S, S);      
    for i = 1:S
        if Ni(i) > 0
            P(i, :) = Nij(i, :) / Ni(i);  % Normalize row i to get probabilities
        end
    end
end

Each row of the resulting matrix 'P' gives the probability distribution of transitioning from a given state to all possible next states. The logic inside the loop builds a frequency count for transitions, and the second loop turns those counts into probabilities using basic element-wise operations.

Hope this helps!