Principal Component Analysis

PCA finds the directions of maximum variance in data by solving a constrained optimization whose answer is the top eigenvector of the covariance matrix.

1. The idea

Given a covariance matrix $Σ$ , the first principal component is the direction $v$ (with $∥ v ∥ = 1$ ) along which the projected data has maximum variance: $max_{v} v^{T} Σ v$ subject to $v^{T} v = 1$ .

Using a Lagrange multiplier, the stationarity condition of this problem is exactly $Σ v = λ v$ — the eigenvector equation. Substituting back shows the maximized variance equals $λ$ , so the top eigenvector (paired with the largest eigenvalue) is the first principal component.

Subsequent components are found by repeating the maximization subject to orthogonality with previous components, yielding the remaining eigenvectors in decreasing eigenvalue order. The total variance captured by the first $k$ components is $\sum_{i = 1}^{k} λ_{i} / \sum_{i} λ_{i}$ .

2. Why it matters

PCA is the standard tool for building risk factors from asset returns and for dimension reduction, and interviewers frequently ask you to derive the eigenvector result from the optimization rather than quote it.

3. Core mathematics

Optimization

v max v^{T} Σ v s.t. v^{T} v = 1

Lagrangian

L (v, λ) = v^{T} Σ v - λ (v^{T} v - 1)

Stationarity condition

Σ v = λ v

Variance explained

v^{T} Σ v = λ

the maximum equals the top eigenvalue

4. How to recognize it

Think of this when…

'Find the direction of maximum variance' or 'dominant factor'
Dimension reduction on a set of correlated asset returns
Any derivation asking you to connect an optimization to an eigenvector equation

5. Worked example

Worked example

For $Σ = (4001)$ , find the first principal component and the variance it explains.

1
Eigenvalues of a diagonal matrix are its diagonal entries.
$λ_{1} = 4, λ_{2} = 1$
2
The top eigenvector is the standard basis vector for $λ = 4$ .
$v_{1} = (1, 0)$
3
Fraction of variance explained.
$\frac{4}{4 + 1} = 0.8$
Answer: First PC is (1,0), explaining 80% of total variance.

6. Common mistakes

Forgetting to normalize $v$ to unit length before interpreting it as a direction
Ordering components by eigenvalue magnitude incorrectly (should be descending)
Running PCA on raw (non-standardized) data when variables have very different scales

7. Quick check

The first principal component maximizes:

The variance explained by the first PC equals:

8. Practice

2 QuantSolve questions use this concept.

Practice this concept

9. Related methods

Lagrange Multipliers

Create a free account to save lesson progress.