Maximum Likelihood Estimation

Choose the parameter that makes the observed data most probable by maximizing the (log-)likelihood function.

1. The idea

Given data $x_{1}, \dots, x_{n}$ assumed drawn i.i.d. from a distribution with unknown parameter $θ$ , the likelihood $L (θ) = \prod_{i} f (x_{i}; θ)$ measures how probable the observed sample is under each candidate $θ$ . The MLE $θ^$ maximizes this.

Taking the log converts the product into a sum, which is easier to differentiate: maximize $ℓ (θ) = \sum_{i} lo g f (x_{i}; θ)$ instead. Since $lo g$ is increasing, the maximizer is unchanged.

For several common distributions the MLE has a clean closed form: Bernoulli's MLE for $p$ is the sample mean; Exponential's MLE for $λ$ is the reciprocal of the sample mean; Uniform $(0, θ)$ 's MLE for $θ$ is the sample maximum.

2. Why it matters

MLE is the default estimation framework in quant research, and interviewers expect you to derive standard MLEs (Bernoulli, exponential, uniform) from first principles quickly.

3. Core mathematics

Likelihood

L (θ) = i = 1 \prod n f (x_{i}; θ)

Log-likelihood

ℓ (θ) = i = 1 \sum n lo g f (x_{i}; θ)

Bernoulli MLE

p^= x ˉ = \frac{1}{n} i \sum x_{i}

Exponential MLE

λ^= \frac{1}{x ˉ}

Uniform(0,θ) MLE

θ^= max (x_{1}, \dots, x_{n})

Boundary maximum, not a calculus stationary point.

4. How to recognize it

Think of this when…

You are given a sample and a parametric family, and asked to 'estimate' a parameter
The distribution is Bernoulli, exponential, or uniform — recognize the closed-form MLE immediately
The support of the distribution depends on the parameter (uniform case) — calculus won't find the max, the boundary will

5. Worked example

Worked example

Observed i.i.d. exponential data $x_{1}, \dots, x_{n}$ with unknown rate $λ$ . Derive the MLE for $λ$ .

1
Write the log-likelihood using the exponential density $f (x; λ) = λ e^{- λ x}$ .
$ℓ (λ) = i = 1 \sum n (lo g λ - λ x_{i}) = n lo g λ - λ i \sum x_{i}$
2
Differentiate with respect to $λ$ and set to zero.
$\frac{d ℓ}{d λ} = \frac{n}{λ} - i \sum x_{i} = 0$
3
Solve for $λ$ .
$λ^= \frac{n}{\sum _{i} x _{i}} = \frac{1}{x ˉ}$
Answer: $λ^= 1/ x ˉ$

6. Common mistakes

Trying to differentiate the uniform likelihood — its maximum is at a boundary, not a stationary point
Forgetting that the exponential MLE is the reciprocal of the sample mean, not the sample mean itself
Skipping the log transform and struggling with the product rule on the raw likelihood
Confusing MLE with method-of-moments — they coincide for Bernoulli and exponential but not always in general

7. Quick check

You observe 7 heads in 10 independent coin flips. What is the MLE for p?

For Uniform(0, θ) data with observed maximum 8.2, what is the MLE for θ?

8. Practice

5 QuantSolve questions use this concept.

Practice this concept

9. Related methods

Lagrange Multipliers

Create a free account to save lesson progress.