A Comparison of Parameter Estimations of Multiple Regression Model with and without Multicollinearity A Comparison of Parameter Estimations
Article Sidebar
Main Article Content
Abstract
The objective of this research is to compare the parameter estimation methods, including ordinary least-squares method, weight least-squares method, Bayesian method and Markov Chain Monte Carlo method. The multiple regression model consists of independent variable and two independent variables which is considered with and without multicollinearity. The criterion of the best efficiency is investigated by minimum of the average mean square errors. In this research, the data is generated from R program when the independent variables with multicollinearity are simulated from the multivariate normal distribution at the level correlation 0.3, 0.6, and 0.9. In the same time, the independent variables without multicollinearity are simulated from the normal distribution. The dependent variable is approximated by the coefficient of multiple regression model multiply by with the independent variable and plus with the error that randomized from the normal distribution following the multiple regression model. The sample sizes are 5, 30 and 50. The results are found that the Bayesian method presents the minimum of average mean square errors at the sample sizes 5. However, when the sample size value is increased, the best efficiency method is weight least-squares method when the independent variables are presented with and without multicollinearity.
Article Details
References
Sereewattananukul, P., 2012, Comparison of the Estimation Methods for the Multiple Linear Regression Model with Heteroscedasticity Error, Master Thesis, Chulalongkorn University, Bangkok, 13 p. (in Thai)
Koop, G., 2003, Bayesian Econometrics, John Wiley, England, 137 p.
Gilks, W., Richarden, S. and Speigelhalter, D., 1996, Markov Chain Monte Carlo in Practice, Interdisciplinary, Chapman & Hall, Suffolk.
Gelfand, A., Hills, S., Racine-Poon, A. and Smith, A., 1990, Illustration of Bayesian Inference in Normal Data Models using Gibbs Sampling, J. Am. Stat. Assoc. 85: 972-985.
Mekbunditkul, T., 2003, Parameters-Estimation of Polynomial Regression Models with Errors in Independent Variables, Master Thesis, Chulalongkorn University, Bangkok, 40 p. (in Thai)
Bunyamas, T., 2014, Comparison of the Estimation Methods for the Multiple Linear Regression with Nonconstant Variance Error From Lognormal and Gamma Distribution Master Thesis, Chulalongkorn University, Bangkok, 24 p. (in Thai)
Sae-ui, P., Supapakorn, T. and Payakkapong, P., 2016. A Comparison of Parameter Estimations of Bayesian, Ordinary Least Square and Parametric Bootstrap Methods for Simple Linear Regression Model, Thai Sci. Technol. 24(3): 363-369. (in Thai)
Diana, S. and Tanty, P. H., 2018. Linear Regression Model Using Bayesian Approach for Energy Performance of Residential Building. Pro. Com. Sc. 135: 671–677.
Willett, J.D. and Singer, J. D. 1988, Another Cautionary Note About R2: Its Use in Weighted Least-Squares Regression Analysis, J. Am. Stat. 42(3): 236-238.
