Abstract

It is well known that model averaging techniques help to reduce variability of the estimates and could lead to a good compromise in model selection when it is not a priori known which model is superior. In this paper, we consider a model averaging setup while allowing the number of parameters to diverge in the limit. Specifically, we develop a valid asymptotic theory of averaging estimator under the assumption of the ratio of number of covariates being non-negligible to the sample size in the limit, which is the assumption conventionally used in the random matrix theory. Our "many-covariates" asymptotic theory, valid both in homo- and heteroskedastic case, is motivated by the usage of nowadays popular large datasets that could potentially invalidate limiting results built under the standard asymptotics.

We develop our estimator under the adjusted-to-many-covariates local-to-zero assumption that creates a classical bias-variance trade-off in each of candidate models. Due to the data-dependent weights, the resulting estimator has a non-normal limiting distribution that makes inference based on the t-test infeasible. However, we show that a properly recentered and normalized statistic with a robust-to-many-covariates variance estimator has a standard normal distribution that makes it possible to conduct correct inference based on the confidence intervals.