• Linear Algebra: #17 Scalar Products, Norms, etc.

So now we have arrived at the subject matter which is usually taught in the second semester of the beginning lectures in mathematics — that is in Linear Algebra II— namely, the properties of (finite dimensional) real and complex vector spaces. Finally now, we are talking about geometry. That is, about vector spaces which have a distance function. (The word “geometry” obviously has to do with the measurement of physical distances on the earth.)

So let V be some finite dimensional vector space over ℜ, or ℂ. Let v ∈ V be some vector in V. Then, since V ≅ ℜⁿ, or ℂⁿ, we can write v = ∑ a_je_j, (with j ranging from 1, ... , n) where {e₁, . . . , e_n} is the canonical basis for ℜⁿ or ℂⁿ and a_j ∈ ℜ or ℂ, respectively, for all j. Then the length of v is defined to be the non-negative real number

||v|| = √|a₁|²+ · · · + |a_n|²  . 

Of course, as these things always are, we will not simply confine ourselves to measurements of normal physical things on the earth. We have already seen that the idea of a complex vector space defies our normal powers of geometric visualization. Also, we will not always restrict things to finite dimensional vector spaces. For example, spaces of functions — which are almost always infinite dimensional — are also very important in theoretical physics. Therefore, rather than saying that ||v|| is the “length” of the vector v, we use a new word, and we say that ||v|| is the norm of v. In order to define this concept in a way which is suitable for further developments, we will start with the idea of a scalar product of vectors.

Definition
Let F = ℜ or ℂ and let V, W be two vector spaces over F. A bilinear form is a mapping s : V × W → F satisfying the following conditions with respect to arbitrary elements v, v₁ and v₂ ∈ V, w, w₁ and w₂ ∈ W, and a ∈ F.

1. s(v₁ + v₂, w) = s(v₁, w) + s(v₂, w), 
2. s(av, w) = as(v, w), 
3. s(v, w₁ + w₂) = s(v, w₁) + s(v, w₂) and 
4. s(v, aw) = as(v, w). 

If V = W, then we say that a bilinear form s : V × V → F is symmetric, if we always have s(v₁, v₂) = s(v₂, v₁). Also the form is called positive definite if s(v, v) > 0 for all v ≠ 0.

On the other hand, if F = ℂ and f : V → W is such that we always have

1. f(v₁ + v₂) = f(v₁) + f(v₂) and 
2. f(av) = af(v)

Then f is a semi-linear (not a linear) mapping. (Note: if F = ℜ then semi-linear is the same as linear.)

A mapping s : V × W → F such that

1. The mapping given by s(·, w) : V → F, where v → s(v, w) is semi-linear for all w ∈ W, whereas 
2. The mapping given by s(v, ·) : W → F, where w → s(v, w) is linear for all v ∈ V 

is called a sesqui-linear form.

In the case V = W, we say that the sesqui-linear form is Hermitian (or Euclidean, if we only have F = ℜ), if we always have s(v₁, v₂) = s(v₂, v₁). (Therefore, if F = ℜ, an Hermitian form is symmetric.)

Finally, a scalar product is a positive definite Hermitian form s : V × V → F. Normally, one writes (v₁, v₂), rather than s(v₁, v₂).

Well, these are a lot of new words. To be more concrete, we have the inner products, which are examples of scalar products.


Inner products

Thus, we are considering these vectors as column vectors, defined with respect to the canonical basis of ℂⁿ. Then define (using matrix multiplication)

It is easy to check that this gives a scalar product on ℂⁿ. This particular scalar product is called the inner product.

Remark
One often writes u · v for the inner product. Thus, considering it to be a scalar product, we just have u · v = <u, v>.

This inner product notation is often used in classical physics; in particular in Maxwell’s equations. Maxwell’s equations also involve the “vector product” u × v. However the vector product of classical physics only makes sense in 3-dimensional space. Most physicists today prefer to imagine that physical space has 10, or even more — perhaps even a frothy, undefinable number of — dimensions. Therefore it appears to be the case that the vector product might have gone out of fashion in contemporary physics. Indeed, mathematicians can imagine many other possible vector-space structures as well. Thus I shall dismiss the vector product from further discussion here.

Definition
A real vector space (that is, over the field of the real numbers ℜ), together with a scalar product is called a Euclidean vector space. A complex vector space with scalar product is called a unitary vector space.

Now, the basic reason for making all these definitions is that we want to define the length — that is the norm — of the vectors in V. Given a scalar product, then the norm of v ∈ V — with respect to this scalar product — is the non-negative real number

||v|| = √<v, v> .

More generally, one defines a norm-function on a vector space in the following way.

Definition
Let V be a vector space over ℂ (and thus we automatically also include the case ℜ ⊂ ℂ as well). A function || · || : V → ℜ is called a norm on V if it satisfies the following conditions.

1. ||av|| = |a| ||v|| for all v ∈ V and for all a ∈ ℂ, 
2. ||v₁ + v₂|| ≤ ||v₁|| + ||v₂|| for all v ∈ V (the triangle inequality), and 
3. ||v|| = 0 ⇔ v = 0.

Theorem 46 (Cauchy-Schwarz inequality)
Let V be a Euclidean or a unitary vector space, and let ||v|| = √<v, v> for all v ∈ V. Then we have

|<u, v>| ≤ ||u|| · ||v||

for all u and v ∈ V. Furthermore, the equality |<u, v>| ≤ ||u|| · ||v|| holds if, and only if, the set {u, v} is linearly dependent.

Proof
It suffices to show that |<u, v>|² ≤ <u, u><v, v>. Now, if v = 0, then — using the properties of the scalar product — we have both <u, v> = 0 and <v, v> = 0. Therefore the theorem is true in this case, and we may assume that v ≠ 0. Thus <v, v> > 0. Let

which gives the Cauchy-Schwarz inequality. When do we have equality?

If v = 0 then, as we have already seen, the equality |<u, v>| ≤ ||u|| · ||v|| is trivially true. On the other hand, when v ≠ 0, then equality holds when <u − av, u − av> = 0. But since the scalar product is positive definite, this holds when u − av = 0. So in this case as well, {u, v} is linearly dependent.


Theorem 47
Let V be a vector space with scalar product, and define the non-negative function || · || : V → ℜ by  ||v|| = √<v, v> . Then || · || is a norm function on V.

Proof
The first and third properties in our definition of norms are obviously satisfied. As far as the triangle inequality is concerned, begin by observing that for arbitrary complex numbers z = x + yi ∈ ℂ we have