For the last two decades, high-dimensional data and methods have proliferated
throughout the literature. The classical technique of linear regression, however, has not lost
its touch in applications. Most high-dimensional estimation techniques can be seen as variable
selection tools which lead to a smaller set of variables where classical linear regression
technique applies. In this paper, we prove estimation error and linear representation bounds
for the linear regression estimator uniformly over (many) subsets of variables. Based on deterministic
inequalities, our results provide “good” rates when applied to both independent
and dependent data. These results are useful in correctly interpreting the linear regression
estimator obtained after exploring the data and also in post model-selection inference. All
the results are derived under no model assumptions and are non-asymptotic in nature.