Markov Chain Models
A discrete state-space Markov process, or Markov chain, is represented by a directed graph and described by a right-stochastic transition matrix P. The distribution of states at time t + 1 is the distribution of states at time t multiplied by P. The structure of P determines the evolutionary trajectory of the chain, including asymptotics.
For an overview of the Markov chain analysis tools, see Markov Chain Modeling.
Functions
Topics
- Discrete-Time Markov Chains
Markov chains are discrete-state Markov processes described by a right-stochastic transition matrix and represented by a directed graph.
- Markov Chain Modeling
The
dtmcclass provides basic tools for modeling and analysis of discrete-time Markov chains. The class supports chains with a finite number of states that evolve in discrete time with a time-homogeneous transition structure. - Create and Modify Markov Chain Model Objects
Create a Markov chain model object from a state transition matrix of probabilities or observed counts, and create a random Markov chain with a specified structure.
- Visualize Markov Chain Structure and Evolution
Visualize the structure and evolution of a Markov chain model by using
dtmcplotting functions. - Work with State Transitions
This example shows how to work with transition data from an empirical array of state counts, and create a discrete-time Markov chain (
dtmc) model characterizing state transitions. - Determine Asymptotic Behavior of Markov Chain
Compute the stationary distribution of a Markov chain, estimate its mixing time, and determine whether the chain is ergodic and reducible.
- Compare Markov Chain Mixing Times
Compare the estimated mixing times of several Markov chains with different structures.
- Identify Classes in Markov Chain
Programmatically and visually identify classes in a Markov chain.
- Simulate Random Walks Through Markov Chain
Generate and visualize random walks through a Markov chain.
- Compute State Distribution of Markov Chain at Each Time Step
Compute and visualize state redistributions, which show the evolution of the deterministic state distributions over time from an initial distribution.
