How to calculate confidence interval of parameter estimated by global optimization--pattern search

5 views (last 30 days)
Hi everyone,
I recently used pattern search to find the global minimal. Though the algorithm worked very well to find me a solution, I do not know how to estimate the confidence interval for these predicted values. Different from local lsqcurvefit which offers output can be used to calculate confidence interval directly, pattern search does not that output.
So I would like to know normally how do people handle this after they get their global minimal solution? How do they perform a uncertainty test on their prediction? And do they generally perform any other statistical analysis/inference on it? Any suggestion will be highly appreciated!
Rui

Answers (2)

Matt J
Matt J on 8 Jun 2013
Edited: Matt J on 10 Jun 2013
It's usually pretty hard to get statistical information about a solution for general problems. I imagine most people just measure statistical variation of the solution by running repeated simulations.
The methods you're talking about usually assume that all of the following are true of the problem
  1. It is unconstrained and differentiable
  2. It uses a least squares objective function
  3. The residuals are Gaussian distributed.
If that is your situation, you could compute the Jacobian at the solution yourself. Or, once you've found the solution, you can feed it to LSQCURVEFIT as an initial point and use its Jacobian output in the usual way. LSQCURVEFIT should stop in 1 iteration, since your solution is already optimal.
  5 Comments
Matt J
Matt J on 10 Jun 2013
Edited: Matt J on 10 Jun 2013
What I meant was, computing statistical variation based on data repetitions that are not i.i.d. sounds problematic.
Rui
Rui on 10 Jun 2013
Matt, that's true. We have 7 compounds. Data set (concentration time course, several data points) for each compound is from a different paper. Our model has a few compound-specific but known parameters, and a few parameters same across compounds, which we try to estimate. These parameters are estimated by minimizing errors between pooled simulation and pooled data with a global algorithm. Can we still use bootstrap safely under this situation?

Sign in to comment.


Shashank Prasanna
Shashank Prasanna on 10 Jun 2013
There are no analytic approximations for CIs for pattern search similar to linear regression. The best alternative as you note is to use a resampling technique. Of course you are further away from an ideal world if you violate MattJ points.
BOOTCI is probably your best bet:
  7 Comments
Matt J
Matt J on 11 Jun 2013
Then time concentration curves are randomly picked according to a uniform distribution. According to the curves chosen, corresponding simulations are made from our model.
If each simulated trial is randomized based on the same uniform distribution, then yes, it does sound i.i.d.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!