MA1034: Process Modelling using Linear Algebra

Session 08: Gram-Schmidt Process

Prof. Alejandro Ucan-Puc

Tecnológico de Monterrey • Departamento de Matemáticas

Goals of the Session

  • Distinguish between orthogonal and orthonormal bases.
  • Apply the Gram-Schmidt Orthogonalization Algorithm step-by-step.
  • Normalize orthogonal sets to form orthonormal bases for engineering spaces.

Motivation: Uncorrelated Process Metrics

Correlated sensor measurements $v_1 = (2, 1, 1)^T, v_2 = (1, 3, 1)^T, v_3 = (0, 1, 2)^T$ introduce redundancy in process diagnostics.

How can we construct an orthonormal basis $E = \{e_1, e_2, e_3\}$ spanning the exact same space without correlation?

Orthogonal & Orthonormal Bases

Definition: A basis $S = \{v_1, v_2, \dots, v_n\}$ is orthogonal if:

$$\langle v_i, v_j \rangle = 0 \quad \text{for all } i \neq j$$

A basis $S = \{e_1, e_2, \dots, e_n\}$ is orthonormal if it is orthogonal and every vector has unit norm:

$$\|e_i\| = 1 \quad \implies \quad \langle e_i, e_j \rangle = \delta_{ij}$$

The Gram-Schmidt Algorithm

Given a linearly independent basis $\{v_1, v_2, \dots, v_k\}$, construct an orthogonal basis $\{u_1, u_2, \dots, u_k\}$:

  • Step 1: $u_1 = v_1$
  • Step 2: $u_2 = v_2 - \text{proj}_{u_1}(v_2) = v_2 - \frac{\langle v_2, u_1 \rangle}{\langle u_1, u_1 \rangle} u_1$
  • Step 3: $u_3 = v_3 - \text{proj}_{u_1}(v_3) - \text{proj}_{u_2}(v_3) = v_3 - \frac{\langle v_3, u_1 \rangle}{\|u_1\|^2} u_1 - \frac{\langle v_3, u_2 \rangle}{\|u_2\|^2} u_2$

Normalization Step

Divide each orthogonal vector $u_i$ by its length $\|u_i\|$ to obtain the orthonormal basis $\{e_1, e_2, \dots, e_k\}$:

$$e_i = \frac{u_i}{\|u_i\|}, \quad i = 1, 2, \dots, k$$

Course Summary & Applications

Congratulations! You have completed all 8 sessions of MA1034: Linear Algebra in Process Modelling:

  • Vector spaces & subspaces
  • Linear independence & dimension
  • Linear transformations & matrix representations
  • Eigenvalues, eigenvectors & diagonalization
  • Orthogonality & Gram-Schmidt process