A Study of the Effectiveness of Model Selection Criteria for Multiple Regression Model
Keywords:
model selection criterion, multiple regression model, efficiencyAbstract
The study aims to compare the effectiveness of the ten model selection criteria for multiple regression model, namely, AIC, BIC, HQIC, AICc, AICu, HQICc, KIC, KICcC, KICcSB, and KICcHM. The conditions for simulation were differences in sample size, number of parameters in the model, regression coefficient, and error variance. The results of a small sample case showed that if the true model is difficult to identify, the appropriate criteria are AIC, HQIC, AICc, and HQICc. If the true model is easy to identify, the appropriate criteria are AICu and KICc. In a medium sample case, if the true model is difficult to identify, the appropriate criteria are AIC and AICc. If the true model is easy to identify, the appropriate criteria are AICu and KICc. For the large sample case, an appropriate criterion is BIC. It was also found that when the error variance increased, the efficiency of all model selection criteria decreased. Therefore, the error variance should be checked after model construction because it affects the model selection criteria. For the small sample size, model selection criteria have low accuracy, but it is more accurate in a larger sample.
References
Akaike, H. 1973. Information Theory and an Extension of the Maximum Likelihood Principle, pp. 267-281. In B.N. Petrov and F. Csaki, eds. 2nd International Symposium on Information Theory. Akademiai Kiado, Budapest.
Akaike, H. 1974. A New Look at the Statistical Model Identification. IEEE Transactions on Automatic Control 19(6): 716-723.
Cavanaugh, J.E. 1999. A Large-Sample Model Selection Criterion Based on Kullback’s Symmetric Divergence. Statistics and Probability Letters 42(4): 333-343.
Cavanaugh, J.E. 2004. Criteria for Linear Model Selection Based on Kullback’s Symmetric Divergence. Australian and New Zealand Journal of Statistics 46(2): 257-274.
Hafidi, B. and Mkhadri, A. 2006. A Corrected Akaike Criterion Based on Kullback’s Symmetric Divergence: Applications in Time Series, Multiple and Multivariate Regression. Computational Statistics and Data Analysis 50(6): 1524-1550.
Hannan, E.J. and Quinn, B.G. 1979. The Determination of the Order of an Autoregression. Journal of the Royal Statistical Society Series B 41(2): 190-195.
Hurvich, C.M. and Tsai, C.L. 1989. Regression and Time Series Model Selection in Small Samples. Biometrika 76(2): 297-307.
Keerativibool, W. 2014a. Unifying the Derivations of Kullback Information Criterion and Corrected Versions. Thailand Statistician Journal of Thai Statistical Association 12(1): 37-53.
Keerativibool, W. 2014b. Study on the Penalty Functions of Model Selection Criteria. Thailand Statistician Journal of Thai Statistical Association 12(2): 161-178.
Keerativibool, W. and Siripanich, P. 2017. Comparison of the Model Selection Criteria for Multiple Regression Based on Kullback-Leibler’s Information. Chiang Mai Journal of Science 44(2): 699-714.
McQuarrie, A.D.R. and Tsai, C.L. 1998. Regression and Time Series Model Selection. World Scientific, Singapore.
McQuarrie, A.D.R., Shumway, R.H. and Tsai, C.L. 1997. The Model Selection Criterion AICu. Statistics and Probability Letters 34(3): 285-292.
Montgomery, D.C., Peck, E.A. and Vining, G.G. 2006. Introduction to Linear Regression Analysis. 4th ed. John Wiley & Sons, New York.
Neath, A. and Cavanaugh, J.E. 1997. Regression and Time Series Model Selection Using Variants of the Schwarz Information Criterion. Communication in Statistic-Theory and Method 26(3): 559-580.
Sangthong, M. 2019. A Study of the Effectiveness of Model Selection Criteria for Multilevel Analysis. Burapha Science Journal 24(1): 156-169. (in Thai)
Schwarz, G. 1978. Estimating the Dimension of a Model. The Annals of Statistics 6(2): 461-464.
Seghouane, A.K. and Bekara, M. 2004. A Small Sample Model Selection Criterion Based on Kullback’s Symmetric Divergence. IEEE Transactions on Signal Processing 52(12): 3314-3323.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Rajamangala University of Technology Srivijaya Research Journal
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The content and information in the article published in Journal of Rajamangala University of Technology Srivijaya It is the opinion and responsibility of the author of the article. The editorial journals do not need to agree. Or share any responsibility.
