Orthogonality & Projections

Projecting a vector onto a subspace finds the closest point in that subspace — the geometric heart of least squares and hedging.

1. The idea

The projection of $b$ onto the line spanned by $a$ is $b^= \frac{a \cdot b}{a \cdot a} a$ , the point on that line closest to $b$ in Euclidean distance. The residual $b - b^$ is orthogonal to $a$ .

This generalizes to projecting onto any subspace, e.g. the column space of a matrix $X$ : the least-squares fit $y^= X β^$ is exactly the projection of $y$ onto that column space, with $β^= (X^{T} X)^{- 1} X^{T} y$ .

Projection matrices $P$ satisfy $P^{2} = P$ (idempotent) and $P^{T} = P$ (symmetric, for orthogonal projections), which is a quick way to recognize or verify one.

2. Why it matters

Regression, hedge-ratio computation and best linear approximations are all orthogonal projections, making this one of the most frequently reused ideas in quant interviews.

3. Core mathematics

Projection onto a vector

proj_{a} (b) = \frac{a \cdot b}{a \cdot a} a

Projection onto column space

P = X (X^{T} X)^{- 1} X^{T}

Idempotent property

P^{2} = P

Orthogonal residual

b - proj_{a} (b) ⊥ a

4. How to recognize it

Think of this when…

Least squares regression or 'best linear approximation'
Finding the minimum-variance hedge or closest point in a subspace
Decomposing a vector into parallel and orthogonal components

5. Worked example

Worked example

Project $b = (4, 2)$ onto $a = (1, 0)$ , then find the residual.

1
Apply the projection formula.
$proj_{a} (b) = \frac{4 ( 1 ) + 2 ( 0 )}{1} (1, 0) = (4, 0)$
2
Compute the residual.
$b - proj_{a} (b) = (0, 2)$
3
Check orthogonality: $(0, 2) \cdot (1, 0) = 0$ .
Answer: Projection = (4,0), residual = (0,2), orthogonal to a

6. Common mistakes

Dividing by $∥ a ∥$ instead of $a \cdot a = ∥ a ∥^{2}$ in the projection formula
Forgetting that projection matrices are idempotent, not their own inverse
Projecting onto the wrong subspace (row space vs. column space)

7. Quick check

A projection matrix $P$ always satisfies:

The residual $b - proj_{a} (b)$ is:

8. Practice

4 QuantSolve questions use this concept.

Practice this concept

9. Related methods

This lesson is conceptual; see the topic hub for methods.

Create a free account to save lesson progress.