• Linear Algebra: #7 Matrix Transformations

Matrices are used to describe linear mappings f : V → W with respect to particular bases of V and W. But clearly, if we choose different bases than the ones we had been thinking about before, then we will have a different matrix for describing the same linear mapping. Later on in these lectures we will see how changing the bases changes the matrix, but for now, it is time to think about various systematic ways of changing matrices — in a purely abstract way.


Elementary Column Operations 
We begin with the elementary column operations. Let us denote the set of all n×m matrices of elements from the field F by M(m × n, F). Thus, if

then it contains n columns which, as we have seen, are the images of the basis vectors of the linear mapping which is being represented by the matrix. So The first elementary column operation is to exchange column i with column j, for i ≠ j. We can write

So this column operation is denoted by S_ij. It can be thought of as being a mapping S_ij : M(m × n, F) → M(m × n, F).

Another way to imagine this is to say that S is the set of column vectors in the matrix A considered as an ordered list. Thus S ⊂ F^m. Then S_ij is the same set of column vectors, but with the positions of the i-th and the j-th vectors interchanged. But obviously, as a subset of F^m, the order of the vectors makes no difference. Therefore we can say that the span of S is the same as the span of S_ij. That is [S] = [S_ij].

The second elementary column operation, denoted S_i(a), is that we form the scalar product of the element a ≠ 0 in F with the i-th vector in S. So the i-th vector

All the other column vectors in the matrix remain unchanged.

The third elementary column operation, denoted S_ij(c) is that we take the j-th column (where j ≠ i) and multiply it with c ≠ 0, then add it to the i-th column. Therefore the i-th column is changed to

All the other columns — including the j-th column — remain unchanged.


Theorem 20

[S] = [S_ij] = [S_i(a)] = [S_ij(c)], where i ≠ j and a ≠ 0 ≠ c. 

Proof
Let us say that S = {v₁, . . . , v_n} ⊂ F^m. That is, v_i is the i-th column vector of the matrix A, for each i. We have already seen that [S] = [S_ij] is trivially true. But also, say v = x₁v₁ + · · · + x_nv_n is some arbitrary vector in [S]. Then, since a ≠ 0, we can write

v = x₁v₁ + · · · + a^—1x_i(av_i) + · · · + x_nv_n .

Therefore [S] ⊂ [S_i(a)]. The other inclusion, [S_i(a)] ⊂ [S] is also quite trivial so that we have [S] = [S_i(a)].

Similarly we can write

Therefore [S] ⊂ [S_ij(c)], and again, the other inclusion is similar.

Let us call [S] the column space (Spaltenraum), which is a subspace of F^m. Then we see that the column space remains invariant under the three types of elementary column operations. In particular, the dimension of the column space remains invariant


Elementary Row Operations
Again, looking at the m×n matrix A in a purely abstract way, we can say that it is made up of m row vectors, which are just the rows of the matrix. Let us call them w₁, . . . , w_m ∈ Fⁿ. That is,

Again, we define the three elementary row operations analogously to the way we defined the elementary column operations. Clearly we have the same results. Namely if R = {w₁, . . . , w_m} are the original rows, in their proper order, then we have [R] = [R_ij] = [R_i(a)] = [R_ij(c)].

But it is perhaps easier to think about the row operations when changing a matrix into a form which is easier to think about. We would like to change the matrix into a step form (Zeilenstufenform).


Definition
The m × n matrix A is in step form if there exists some r with 0 ≤ r ≤ m and indices 1 ≤ j₁< j₂< · · · < j_r ≤ m with a_iji = 1 for all i = 1, . . . , r and a_st = 0 for all s, t with t < j_s or s > j_r. That is:

Theorem 21
By means of a finite sequence of elementary row operations, every matrix can be transformed into a matrix in step form.

Proof
Induction on m, the number of rows in the matrix. We use the technique of “Gaussian elimination”, which is simply the usual way anyone would go about solving a system of linear equations. This will be dealt with in the next section. The induction step in this proof, which uses a number of simple ideas which are easy to write on the blackboard, but overly tedious to compose here (so it`s currently not available here).

Now it is obvious that the row space (Zeilenraum), that is [R] ⊂ Fⁿ, has the dimension r, and in fact the non-zero row vectors of a matrix in step form provide us with a basis for the row space. But then, looking at the column vectors of this matrix in step form, we see that the columns j₁, j₂, and so on up to  j_r are all linearly independent, and they generate the column space.

Definition
Given an m×n matrix, the dimension of the column space is called the column rank; similarly the dimension of the row space is the row rank.

So, using theorem 21 and exercise 6.3, we conclude that:


Theorem 22
For any matrix A, the column rank is equal to the row rank. This common dimension is simply called the rank — written Rank(A) — of the matrix.

Definition
Let A be a quadratic n×n matrix. Then A is called regular if Rank(A) = n, otherwise A is called singular.


Theorem 23
The n × n matrix A is regular ⇔ the linear mapping f : Fⁿ → Fⁿ, represented by the matrix A with respect to the canonical basis of Fⁿ is an isomorphism.

Proof
‘⇒’ If A is regular, then the rank of A — namely the dimension of the column space [S] — is n. Since the dimension of Fⁿ is n, we must therefore have [S] = Fⁿ. The linear mapping f : Fⁿ → Fⁿ is then both an injection (since S must be linearly independent) and also a surjection.

‘⇐’ Since the set of column vectors S is the set of images of the canonical basis vectors of Fⁿ under f, they must be linearly independent. There are n column vectors; thus the rank of A is n.