A Primer On Maximum Likelihood Estimation

Nigel Da Costa Lewis

The method of maximum likelihood is a general method for obtaining estimators, such as the mean or standard deviation, from a sample. It is used to estimate the parameters of a statistical or probability model. In order to apply the method, we require a sample of observations and a postulated statistical model. The objective of this chapter is to describe the principle underlying maximum likelihood estimation and provide details of a numerical procedure for maximising the likelihood when an analytical solution is not available. We begin by introducing the likelihood equation and its application. This is followed by a detailed discussion of the score vector and information matrix, used to obtain estimates of the parameters and standard errors, respectively. This is followed by analysis of a numerical algorithm known as the Newton–Raphson method. Finally, we outline the application of maximum likelihood estimation for the case of linear regression.

THE LIKELIHOOD EQUATION

We have already seen that the normal distribution f(x) depends on the mean, μ, and standard deviation, σ. Given a sample of data, we can use maximum likelihood to obtain estimates of both of these parameters