• Linear Algebra: #11 Eigenvalues, Eigenspaces, Matrices which can be Diagonalized

Definition
Let f : V → V be a linear mapping of an n-dimensional vector space into itself. A subspace U ⊂ V is called invariant with respect to f if f(U) ⊂ U. That is, f(u) ∈ U for all u ∈ U.


Theorem 28
Assume that the r dimensional subspace U ⊂ V is invariant with respect to f : V → V. Let A be the matrix representing f with respect to a given basis {v₁, . . . , v_n} of V. Then A is similar to a matrix A' which has the following form

Proof
Let {u₁, . . . , u_r} be a basis for the subspace U. Then extend this to a basis {u₁, . . . , u_r, u_r+1, . . . , u_n} of V. The matrix of f with respect to this new basis has the desired form.

Definition
Let U₁, . . . , U_p ⊂ V be subspaces. We say that V is the direct sum of these subspaces if V = U₁+ · · · +U_p, and furthermore if v = u₁+ · · · +u_p such that u_i ∈ U_i, for each i, then this expression for v is unique. In other words, if v = u₁+ · · · +u_p = u₁'+ · · · +u_p' with u_i' ∈ U_i for each i, then u_i = u_i', for each i. In this case, one writes V = U₁ ⊕ · · · ⊕ U_p

This immediately gives the following result:


Theorem 29
Let f : V → V be such that there exist subspaces U_i ⊂ V, for i = 1, . . . , p, such that V = U₁ ⊕ · · · ⊕ U_p and also f is invariant with respect to each U_i. Then there exists a basis of V such that the matrix of f with respect to this basis has the following block form.

where each block A_i is a square matrix, representing the restriction of f to the subspace U_i.

Proof
Choose the basis to be a union of bases for each of the U_i.

A special case is when the invariant subspace is an eigenspace.

Definition
Assume that λ ∈ F is an eigenvalue of the mapping f : V → V. The set {v ∈ V : f(v) = λv} is called the eigenspace of λ with respect to the mapping f. That is, the eigenspace is the set of all eigenvectors (and with the zero vector 0 included) with eigenvalue λ.


Theorem 30
Each eigenspace is a subspace of V.

Proof
Let u, w ∈ V be in the eigenspace of λ. Let a, b ∈ F be arbitrary scalars. Then we have

f(au + bw) = af(u) + bf(w) = aλu + bλw = λ(au + bw). 

Obviously if λ₁ and λ₂ are two different (λ₁ ≠ λ₂ ) eigenvalues, then the only common element of the eigenspaces is the zero vector 0. Thus if every vector in V is an eigenvector, then we have the situation of theorem 29. One very particular case is that we have n different eigenvalues, where n is the dimension of V.


Theorem 31
Let λ₁, . . . , λ_n, be eigenvalues of the linear mapping f : V → V, where  λ_i ≠ λ_j for i ≠ j. Let v₁, . . . , v_n be eigenvectors to these eigenvalues. That is, v_i ≠ 0 and f(v_i) = λ_iv_i, for each i = 1, . . . , n. Then the set {v₁, . . . , v_n} is linearly independent.

Proof
Assume to the contrary that there exist a₁, . . . , a_n, not all zero, with

a₁v₁ + · · · + a_nv_n = 0. 

Assume further that as few of the a_i as possible are non-zero. Let a_p be the first non-zero scalar. That is, a_i = 0 for i < p, and a_p≠ 0. Obviously some other a_k is non-zero, for some k ≠ p, for otherwise we would have the equation 0 = a_pv_p , which would imply that v_p = 0, contrary to the assumption that v_p is an eigenvector. Therefore we have

But, remembering that  λ_i ≠ λ_j for i ≠ j, we see that the scalar term for v_p is zero, yet all other non-zero scalar terms remain non-zero. Thus we have found a new sum with fewer non-zero scalars than in the original sum with the a_is. This is a contradiction.

Therefore, in this particular case, the given set of eigenvectors {v₁, . . . , v_n} form a basis for V. With respect to this basis, the matrix of the mapping is diagonal, with the diagonal elements being the eigenvalues.