Markov Models

Discrete-time Markov chains and state-space models

Econometrics Toolbox™ supports modeling and analyzing discrete-time Markov models. These models describe stochastic processes that assume states xt in a state space X, subject to the Markov property, which requires the distribution of xt + 1 to be independent of the process history before reaching state xt.

Discrete state-space Markov processes, or Markov chains, are represented by a directed graph and described by a right-stochastic transition matrix P. The distribution of states at time t+1 is the distribution of states at time t multiplied by P. The structure of P determines the evolutionary trajectory of the chain, including asymptotics.

Continuous state-space Markov processes, or state-space models, allow for trajectories through a continuous state space. The underlying Markov process is typically unobserved. Supplemental observation equations describe the evolution of measurable characteristics of a system, dependent on the Markov process.

  • Markov Chain Models
    Discrete state-space processes characterized by transition matrices
  • State-Space Models
    Continuous state-space processes characterized by state and observation equations