Euclidean Spaces and Inner Products

An inner product on a real vector space is a function ⟨u, v⟩ that is symmetric, linear in each argument and positive definite (⟨v, v⟩ > 0 for v ≠ 0); it defines the length |v| = √⟨v, v⟩ and the angle through cos θ = ⟨u, v⟩/(|u||v|), and a space with an inner product is called a Euclidean space. In brief: in ℝ³ with the dot product, u = (1, 2, 2) and v = (2, 3, 6) give ⟨u, v⟩ = 20, |u| = 3, |v| = 7, and indeed 20 ≤ 3 · 7 = 21, as the Cauchy–Schwarz inequality demands. The Gram–Schmidt process turns any basis into an orthonormal one.

The ordinary dot product is the prototype. Inner products also exist on spaces of polynomials, matrices and functions, which extends geometry (lengths, angles, projections) to these spaces. This guide gives the axioms and examples, the Cauchy–Schwarz and triangle inequalities, orthogonality and orthonormal bases, Gram–Schmidt orthogonalization and orthogonal projections, with worked examples and a table.

What is an inner product?

A real inner product on V satisfies, for all u, v, w and scalars a: ⟨u, v⟩ = ⟨v, u⟩; ⟨au + w, v⟩ = a⟨u, v⟩ + ⟨w, v⟩; and ⟨v, v⟩ > 0 for every v ≠ 0. The norm is |v| = √⟨v, v⟩ and the distance is |u − v|. Vectors are orthogonal when ⟨u, v⟩ = 0. (For complex spaces, the symmetry is replaced by conjugate symmetry.) On ℝⁿ, every inner product has the form ⟨x, y⟩ = x^TAy with a symmetric positive definite matrix A.

Example 1: a weighted inner product

On ℝ², ⟨x, y⟩ = x₁y₁ + 2x₂y₂ has the matrix diag(1, 2) with positive entries, so it is an inner product. In it, (1, 1) and (2, −1) are orthogonal: 2 − 2 = 0, although their ordinary dot product is 2 − 1 = 1. And |(1, 1)|² = 1 + 2 = 3.

Example 2: an inner product on functions

On the continuous functions on [0, 1], ⟨f, g⟩ = ∫₀¹ f(x)g(x) dx. For f = 1 and g = x: ⟨1, x⟩ = 1/2, |1| = 1 and |x| = √(1/3).

Example 3: an inner product on matrices

On m × n matrices, ⟨A, B⟩ = tr(A^TB) = Σ a_ijb_ij, which gives the Frobenius norm |A|_F = √(Σ a_ij²). For A = [[1, 2], [3, 4]], |A|_F = √30.

What inequalities hold?

The Cauchy–Schwarz inequality: |⟨u, v⟩| ≤ |u||v|, with equality exactly when u and v are dependent. It makes the angle formula meaningful, since the quotient lies between −1 and 1. The triangle inequality |u + v| ≤ |u| + |v| follows from it. The parallelogram law |u + v|² + |u − v|² = 2|u|² + 2|v|² and the Pythagorean theorem for orthogonal vectors, |u + v|² = |u|² + |v|², also hold.

Example 4: Cauchy–Schwarz at work

For u = (1, 2, 2) and v = (2, 3, 6): ⟨u, v⟩ = 20 ≤ 3 · 7 = 21. For the functions of Example 2: |∫₀¹ x dx|² = 1/4 ≤ ∫₀¹ 1 dx · ∫₀¹ x² dx = 1/3 ✓.

What are orthogonal and orthonormal bases?

A set of nonzero pairwise orthogonal vectors is linearly independent. An orthogonal basis consists of such vectors, and it is orthonormal if all have the length 1. In an orthonormal basis e₁, …, e_n, the coordinates of v are simply c_i = ⟨v, e_i⟩, and |v|² = Σc_i² (Parseval). A matrix whose columns are an orthonormal basis of ℝⁿ is orthogonal (Q^TQ = I).

Example 5: coordinates in an orthonormal basis

e₁ = (3/5, 4/5) and e₂ = (−4/5, 3/5) form an orthonormal basis of ℝ². For v = (4, 3): c₁ = 12/5 + 12/5 = 24/5 and c₂ = −16/5 + 9/5 = −7/5. Check: (24/5)² + (7/5)² = 625/25 = 25 = 16 + 9 ✓.

Example 6: orthogonal polynomials

With ⟨f, g⟩ = ∫₋₁¹ fg dx, the polynomials 1, x and x² − 1/3 are pairwise orthogonal: ⟨1, x⟩ = 0, ⟨x, x² − 1/3⟩ = 0 (odd integrand) and ⟨1, x² − 1/3⟩ = 2/3 − 2/3 = 0. These are (multiples of) the first Legendre polynomials.

How does the Gram–Schmidt process work?

Given independent vectors v₁, …, v_k, produce an orthogonal set by subtracting the projections onto the previously constructed vectors: w₁ = v₁ and w_j = v_j − Σ_i<j (⟨v_j, w_i⟩/⟨w_i, w_i⟩) w_i. Dividing each w_j by its length yields an orthonormal set e_j spanning the same subspaces at every stage. This is the basis of the QR factorization.

Example 7: orthonormalizing three vectors

v₁ = (1, 1, 0), v₂ = (1, 0, 1), v₃ = (0, 1, 1). Then w₁ = (1, 1, 0), and e₁ = (1, 1, 0)/√2. w₂ = v₂ − (1/2)w₁ = (1/2, −1/2, 1), with |w₂|² = 3/2, so e₂ = (1, −1, 2)/√6. w₃ = v₃ − (1/2)w₁ − (⟨v₃, w₂⟩/(3/2))w₂ = (0, 1, 1) − (1/2, 1/2, 0) − (1/3)(1/2, −1/2, 1) = (−2/3, 2/3, 2/3), since ⟨v₃, w₂⟩ = 1/2. So e₃ = (−1, 1, 1)/√3. Check orthogonality: (1, −1, 2) · (−1, 1, 1) = −1 − 1 + 2 = 0 and (1, 1, 0) · (−1, 1, 1) = 0 ✓.

How do you project orthogonally?

The orthogonal projection of v onto a subspace U with an orthonormal basis e₁, …, e_k is proj_U v = Σ⟨v, e_i⟩e_i. It is the vector of U closest to v, and v − proj_U v is orthogonal to U. The orthogonal complement U^⊥ consists of all vectors orthogonal to U, and V = U ⊕ U^⊥, so dim U + dim U^⊥ = dim V. The foot of a perpendicular dropped to a line or plane is the orthogonal projection onto it.

Example 8: projections in ℝ³

Project v = (1, 2, 3) onto the line through (1, 1, 1): e = (1, 1, 1)/√3, ⟨v, e⟩ = 6/√3, so the projection is (6/3)(1, 1, 1) = (2, 2, 2). The residual (−1, 0, 1) is orthogonal to (1, 1, 1) ✓. It is also the projection of v onto the plane x + y + z = 0 (the orthogonal complement of the line).

Example 9: best approximation of a function

Approximate f(x) = x² on [0, 1] by a constant, using ⟨f, g⟩ = ∫₀¹ fg dx. The projection onto span{1} is ⟨x², 1⟩/⟨1, 1⟩ = 1/3, the mean value. The squared error is ∫₀¹ (x² − 1/3)² dx = 1/5 − 2/9 + 1/9 = 4/45.

Table of inner products

Where does the concept come from?

The Cauchy–Schwarz inequality dates to Cauchy (1821) for sums and to Bunyakovsky (1859) and Schwarz (1888) for integrals. The abstract inner product space was developed by David Hilbert and his students in the study of integral equations around 1904–1910, and by von Neumann, who formalized Hilbert spaces in 1929–1932 for quantum mechanics. The Gram–Schmidt process is named after Jørgen Pedersen Gram (1883) and Erhard Schmidt (1907), though it was known to Laplace and Cauchy.

Which mistakes are common?

Assuming the dot product is the only one

Orthogonality depends on the inner product. In Example 1, (1, 1) and (2, −1) are orthogonal for the weighted one but not for the standard one.

Forgetting to normalize

The coordinate formula c_i = ⟨v, e_i⟩ holds only for orthonormal bases. For an orthogonal basis, divide by ⟨w_i, w_i⟩.

Subtracting projections onto the original vectors

In Gram–Schmidt, subtract the projections onto the new orthogonal vectors w_i, not onto the original v_i.

Ignoring positivity

A symmetric bilinear form is an inner product only if ⟨v, v⟩ > 0 for all v ≠ 0. The form x₁y₁ − x₂y₂ fails this.