Main Content

Bayesian linear regression model with samples from prior or posterior distributions

The Bayesian linear regression
model object `empiricalblm`

contains samples from the prior
distributions of *β* and
*σ*^{2}, which MATLAB^{®} uses to characterize the prior or posterior distributions.

The data likelihood is $$\prod _{t=1}^{T}\varphi \left({y}_{t};{x}_{t}\beta ,{\sigma}^{2}\right)},$$ where
*ϕ*(*y _{t}*;

You can create a Bayesian linear regression model with an empirical prior directly
using `bayeslm`

or `empiricalblm`

.
However, for empirical priors, estimating the posterior distribution requires that the
prior closely resemble the posterior. Hence, empirical models are better suited for
updating posterior distributions estimated using Monte Carlo sampling (for example,
semiconjugate and custom prior models) given new data.

`estimate`

For semiconjugate, empirical, or custom prior models, `estimate`

estimates the posterior distribution using Monte Carlo
sampling. That is, `estimate`

characterizes the posterior
distribution by a large number of draws from that distribution.
`estimate`

stores the draws in the `BetaDraws`

and `Sigma2Draws`

properties of the returned Bayesian linear
regression model object. Hence, when you estimate `semiconjugateblm`

, `empiricalblm`

, `customblm`

, `lassoblm`

, `mixconjugateblm`

, and `mixconjugateblm`

model objects, `estimate`

returns an
`empiricalblm`

model object.

If you want to update an estimated posterior distribution using new data, and you
have draws from the posterior distribution of *β* and
*σ*^{2}, then you can create an
empirical model using `empiricalblm`

.

creates a Bayesian linear
regression model object (`PriorMdl`

= empiricalblm(`NumPredictors`

,'`BetaDraws`

',BetaDraws,'`Sigma2Draws`

',Sigma2Draws)`PriorMdl`

)
composed of `NumPredictors`

predictors and an
intercept, and sets the `NumPredictors`

property. The
random samples from the prior distributions of *β* and
*σ*^{2},
`BetaDraws`

and `Sigma2Draws`

,
respectively, characterize the prior distributions.
`PriorMdl`

is a template that defines the prior
distributions and the dimensionality of *β*.

sets properties (except
`PriorMdl`

= empiricalblm(`NumPredictors`

,'`BetaDraws`

',BetaDraws,'`Sigma2Draws`

',Sigma2Draws,`Name,Value`

)`NumPredictors`

) using name-value pair arguments.
Enclose each property name in quotes. For example,
`empiricalblm(2,'`

specifies the random samples from the prior
distributions of `BetaDraws`

',BetaDraws,'`Sigma2Draws`

',Sigma2Draws,'Intercept',
false)*β* and
*σ*^{2} and specifies a
regression model with 2 regression coefficients, but no
intercept.

`estimate` | Estimate posterior distribution of Bayesian linear regression model parameters |

`simulate` | Simulate regression coefficients and disturbance variance of Bayesian linear regression model |

`forecast` | Forecast responses of Bayesian linear regression model |

`plot` | Visualize prior and posterior densities of Bayesian linear regression model parameters |

`summarize` | Distribution summary statistics of standard Bayesian linear regression model |

After implementing sampling importance resampling to sample from the posterior distribution,

`estimate`

,`simulate`

, and`forecast`

compute the*effective sample size*(*ESS*), which is the number of samples required to yield reasonable posterior statistics and inferences. Its formula is$$ESS=\frac{1}{{\displaystyle {\sum}_{j}{w}_{j}^{2}}}.$$

If

*ESS*<`0.01*NumDraws`

, then MATLAB throws a warning. The warning implies that, given the sample from the prior distribution, the sample from the proposal distribution is too small to yield good quality posterior statistics and inferences.If the effective sample size is too small, then:

Increase the sample size of the draws from the prior distributions.

Adjust the prior distribution hyperparameters, and then resample from them.

Specify

`BetaDraws`

and`Sigma2Draws`

as samples from*informative*prior distributions. That is, if the proposal draws come from nearly flat distributions, then the algorithm can be inefficient.

The `bayeslm`

function can create any supported prior model object for Bayesian linear regression.