Hi,
I'm running a matrix of regressions (say, 10*50 regressions, same original dataset, different approaches).
Beside simply compare the R-square of the regressions, I also wish to know how are the residuals distributed since the residuals are important to further investigations. I wish the residuals to be convergent and quite well distributed so that it means the original dataset and fitted curve are closely linked (probability of deviation to be minimized).
Since there are lots of them, the plotting methodology are not suitable (histogram, qqplot, etc.). I wish there are some statistics that can evaluate how good the fitted distributions are so that I can pick the 'good models' and use it as a good approach to the orignal dataset.
With human comparison, I feel the fitted distribution is likely t-localization-scale distributed. I tried coftest, kstest and adtest. However, they all reject the null hypothesis (even when R-square is 0.96). I would like to know how can I change the criteria of null hypothesis? Furthermore, the p-value are miserably low. I'm deeply confused with it.
Here are a comparison of some of the distributions:
As you can see the data1 and data2 are quite well distributed as they have the shape while data3 and data4 are not-so-convergent. I tried to compare loglikelihood but even data4 has some 1.0e+07 level of high loglikelihood. What am I supposed to do?