A Comparison of Parameter Estimations of Multiple Regression Model with and without Multicollinearity A Comparison of Parameter Estimations

Abstract

The objective of this research is to compare the parameter estimation methods, including ordinary least-squares method, weight least-squares method, Bayesian method and Markov Chain Monte Carlo method. The multiple regression model consists of independent variable and two independent variables which is considered with and without multicollinearity. The criterion of the best efficiency is investigated by minimum of the average mean square errors. In this research, the data is generated from R program when the independent variables with multicollinearity are simulated from the multivariate normal distribution at the level correlation 0.3, 0.6, and 0.9. In the same time, the independent variables without multicollinearity are simulated from the normal distribution. The dependent variable is approximated by the coefficient of multiple regression model multiply by with the independent variable and plus with the error that randomized from the normal distribution following the multiple regression model. The sample sizes are 5, 30 and 50. The results are found that the Bayesian method presents the minimum of average mean square errors at the sample sizes 5. However, when the sample size value is increased, the best efficiency method is weight least-squares method when the independent variables are presented with and without multicollinearity.