Welcome to our explanation of the Gauss-Markov Theorem. This fundamental result in statistics states that under certain assumptions, the Ordinary Least Squares estimator is the Best Linear Unbiased Estimator, often abbreviated as BLUE. In linear regression, we fit a line through data points to minimize the sum of squared errors. The Gauss-Markov theorem provides theoretical justification for using this method.
For the Gauss-Markov theorem to hold, five key assumptions must be met. First, the model must be linear in parameters. Second, the data must be randomly sampled from the population. Third, exogeneity requires that the error terms have a conditional mean of zero. Fourth, homoscedasticity means the error terms have constant variance across all observations. Finally, there should be no perfect multicollinearity among independent variables. When these assumptions are satisfied, the OLS estimator is guaranteed to be the Best Linear Unbiased Estimator.
Let's break down what it means for the OLS estimator to be BLUE - the Best Linear Unbiased Estimator. 'Best' means it has the minimum variance among all unbiased linear estimators, making it the most precise. The blue curve shows the OLS estimator with lower variance centered at the true parameter value. 'Linear' means the estimator is a linear function of the dependent variable. 'Unbiased' means the expected value of the estimator equals the true parameter value, as shown by the blue and green distributions centered at the true value. The red distribution is biased because its center is shifted away from the true parameter. The Gauss-Markov theorem guarantees that under the assumptions we discussed, no other linear unbiased estimator can have lower variance than OLS.
Let's examine the mathematical formulation of the Gauss-Markov theorem. The linear regression model is expressed as y equals X beta plus epsilon, where y is the vector of dependent variables, X is the matrix of independent variables, beta is the vector of coefficients to be estimated, and epsilon is the vector of error terms. The Ordinary Least Squares estimator is given by the formula beta-hat equals X-transpose-X inverse times X-transpose-y. This formula minimizes the sum of squared residuals. Under the Gauss-Markov assumptions, this estimator has the minimum variance among all linear unbiased estimators. The derivation involves multiplying both sides of the original equation by X-transpose and using the exogeneity assumption that the expected value of X-transpose times epsilon equals zero.
To summarize what we've learned about the Gauss-Markov theorem: It provides the theoretical foundation for using Ordinary Least Squares in linear regression. Under five key assumptions—linearity, random sampling, exogeneity, homoscedasticity, and no perfect multicollinearity—OLS is guaranteed to be the Best Linear Unbiased Estimator. This means that among all linear estimators that don't systematically over or underestimate the true parameters, OLS has the minimum variance, making it the most precise. Importantly, the theorem doesn't require the errors to be normally distributed, although normality is needed for hypothesis testing and confidence intervals. This result is fundamental to statistical theory and explains why OLS is so widely used in empirical research across many disciplines.