Orthogonal and Orthonormal Bases in \R^n

Section 24 Orthogonal and Orthonormal Bases in $R^{n}$

Focus Questions

By the end of this section, you should be able to give precise and thorough answers to the questions listed below. You may want to keep these questions in mind to focus your thoughts as you complete the section.

What is an orthogonal set in $R^{n} ?$ What is one useful fact about orthogonal subsets of $R^{n} ?$
What is an orthogonal basis for a subspace of $R^{n} ?$ What is an orthonormal basis?
How does orthogonality help us find the weights to write a vector as a linear combination of vectors in an orthogonal basis?
What is an orthogonal matrix and why are orthogonal matrices useful?

🔗

Subsection Application: Rotations in 3D

🔗

An aircraft in flight, like a plane or the space shuttle, can perform three independent rotations: roll, pitch, and yaw. Roll is a rotation about the axis through the nose and tail of the aircraft, pitch is rotation moving the nose of the aircraft up or down through the axis from wingtip to wingtip, and yaw is the rotation when the nose of the aircraft turns left or right about the axis though the plane from top to bottom. These rotations take place in

3

-space and the axes of the rotations change as the aircraft travels through space. To understand how aircraft maneuver, it is important to know about general rotations in space. These are more complicated than rotations in

2

-space, and, as we will see later in this section, involve orthogonal sets.

🔗

Subsection Introduction

🔗

B = {v_{1}, v_{2}, \dots, v_{m}}

is a basis for a subspace

W

R^{n},

we know that any vector

w

W

can be written uniquely as a linear combination of the vectors in

B .

In the past, the way we have found the coordinates of

x

with respect to

B,

i.e. the weights needed to write a vector

x

as a linear combination of the elements in

B,

has been to row reduce the matrix

[v_{1} v_{2} \dots v_{m} | x]

to solve the corresponding system. This can be a cumbersome process, especially if we need to do it many times. This process also forces us to determine all of the weights at once. For certain types of bases, namely the orthogonal and orthonormal bases, there is a much easier way to find individual weights for this linear combination.

🔗

Recall that two nonzero vectors

u

and

v

R^{n}

are orthogonal if

u \cdot v = 0 .

We can extend this idea to an entire set. For example, the standard basis

S = {e_{1}, e_{2}, e_{3}}

for

R^{3}

has the property that any two distinct vectors in

S

are orthogonal to each other. The basis vectors in

S

make a very nice coordinate system for

R^{3},

where the basis vectors provide the directions for the coordinate axes. We could rotate this standard basis, or multiply any of the vectors in the basis by a nonzero constant, and retain a basis in which all distinct vectors are orthogonal to each other (e.g.,

{[2 0 0]^{T}, [0 3 0]^{T}, [0 0 1]^{T}}

). We define this idea of having all vectors be orthogonal to each other for sets, and then for bases.

🔗

Definition 24.1.

A non-empty subset $S$ of $R^{n}$ is orthogonal if $u \cdot v = 0$ for every pair of distinct vectors $u$ and $v$ in $S .$

🔗

Preview Activity 24.1.

🔗

(a)

Determine if the set $S = {[1 2 1]^{T}, [2 - 1 0]^{T}}$ is an orthogonal set.

🔗

(b)

Orthogonal bases are especially important.

🔗

Definition 24.2.

An orthogonal basis $B$ for a subspace $W$ of $R^{n}$ is a basis of $W$ that is also an orthogonal set.

Let

B = {v_{1}, v_{2}, v_{3}},

where

v_{1} = [1 2 1]^{T},

v_{2} = [2 - 1 0]^{T},

and

v_{3} = [1 2 - 5]^{T} .

🔗

(i)

Explain why $B$ is an orthogonal basis for $R^{3} .$

🔗

(ii)

Suppose $x$ has coordinates $x_{1}, x_{2}, x_{3}$ with respect to the basis $B,$ i.e.

x = x_{1} v_{1} + x_{2} v_{2} + x_{3} v_{3} .

Substitute this expression for $x$ in $x \cdot v_{1}$ and use the orthogonality property of the basis $B$ to show that $x_{1} = \frac{x \cdot v_{1}}{v_{1} \cdot v_{1}} .$ Then determine $x_{2}$ and $x_{3}$ similarly. Finally, calculate the values of $x_{1},$ $x_{2},$ and $x_{3}$ if $x = [1 1 1]^{T} .$

🔗

(iii)

Find components of $x = [1 1 1]^{T}$ by reducing the augmented matrix $[v_{1} v_{2} v_{3} | x] .$ Does this result agree with your work from the previous part?

🔗

Subsection Orthogonal Sets

🔗

We defined orthogonal sets in

R^{n}

and bases of subspaces of

R^{n}

in Definitions 24.1 and Definition 24.2. We saw that the standard basis in

R^{3}

is an orthogonal set and an orthogonal basis of

R^{3}

— there are many other examples as well.

🔗

Activity 24.2.

Let $w_{1} = [\begin{array}{r} - 2 \\ 1 \\ - 1 \end{array}],$ $w_{2} = [\begin{array}{r} 0 \\ 1 \\ 1 \end{array}],$ and $w_{3} = [\begin{array}{r} 1 \\ 1 \\ - 1 \end{array}] .$ In the same manner as in Preview Activity 24.1, we can show that the set $S_{1} = {w_{1}, w_{2}, w_{3}}$ is an orthogonal subset of $R^{3} .$

🔗

(a)

Is the set $S_{2} = {[\begin{array}{r} - 2 \\ 1 \\ - 1 \end{array}], [\begin{array}{r} 0 \\ 1 \\ 1 \end{array}], [\begin{array}{r} 1 \\ 1 \\ - 1 \end{array}], [\begin{array}{r} 1 \\ 2 \\ 0 \end{array}]}$ an orthogonal subset of $R^{3} ?$

🔗

(b)

Suppose a vector $v$ is a vector so that $S_{1} \cup {v}$ is an orthogonal subset of $R^{3} .$ Then $w_{i} \cdot v = 0$ for each $i .$ Explain why this implies that $v$ is in $Nul A,$ where $A = [\begin{array}{rcr} - 2 & 1 & - 1 \\ 0 & 1 & 1 \\ 1 & 1 & - 1 \end{array}] .$

🔗

(c)

Assuming that the reduced row echelon form of the matrix $A$ is $I_{3},$ explain why it is not possible to find a nonzero vector $v$ so that $S_{1} \cup {v}$ is an orthogonal subset of $R^{3} .$

🔗

The example from Activity 24.2 suggests that we can have three orthogonal nonzero vectors in

R^{3},

but no more. Orthogonal vectors are, in a sense, as far apart as they can be. So we might expect that there is no linear relationship between orthogonal vectors. The following theorem makes this clear.

🔗

Theorem 24.3.

Let ${v_{1}, v_{2}, \dots, v_{m}}$ be a set of nonzero orthogonal vectors in $R^{n} .$ Then the vectors $v_{1}, v_{2}, \dots, v_{m}$ are linearly independent.

🔗

Proof.

Let $S = {v_{1}, v_{2}, \dots, v_{m}}$ be a set of nonzero orthogonal vectors in $R^{n} .$ To show that $v_{1},$ $v_{2},$ $\dots,$ $v_{m}$ are linearly independent, assume that

\begin{matrix} (24.1) & x_{1} v_{1} + x_{2} v_{2} + \dots + x_{m} v_{m} = 0 \end{matrix}

for some scalars $x_{1},$ $x_{2},$ $\dots,$ $x_{m} .$ We will show that $x_{i} = 0$ for each $i$ from 1 to $m .$ Since the vectors in $S$ are orthogonal to each other, we know that $v_{i} \cdot v_{j} = 0$ whenever $i \neq j .$ Fix an index $k$ between 1 and $m .$ We evaluate the dot product of both sides of (24.1) with $v_{k}$ and simplify using the dot product properties:

\begin{aligned} v_{k} \cdot (x_{1} v_{1} + x_{2} v_{2} + \dots + x_{m} v_{m}) & = v_{k} \cdot 0 \\ (v_{k} \cdot x_{1} v_{1}) + (v_{k} \cdot x_{2} v_{2}) + \dots + (v_{k} \cdot x_{m} v_{m}) & = 0 \\ (24.2) & x_{1} (v_{k} \cdot v_{1}) + x_{2} (v_{k} \cdot v_{2}) + \dots + x_{m} (v_{k} \cdot v_{m}) & = 0 . \end{aligned}

Now all of the dot products on the left are 0 except for $v_{k} \cdot v_{k},$ so (24.2) becomes

x_{k} (v_{k} \cdot v_{k}) = 0 .

We assumed that $v_{k} \neq 0$ and since $v_{k} \cdot v_{k} = | | v_{k} | |^{2} \neq 0,$ we conclude that $x_{k} = 0 .$ We chose $k$ arbitrarily, so we have shown that $x_{k} = 0$ for each $k$ between 1 and $m .$ Therefore, the only solution to equation (24.1) is the trivial solution with $x_{1} = x_{2} = \dots = x_{m} = 0$ and the set $S$ is linearly independent.

🔗

Subsection Properties of Orthogonal Bases

🔗

Orthogonality is a useful and important property for a basis to have. In Preview Activity 24.1 we saw that if a vector

x

in the span of an orthogonal basis

{v_{1}, v_{2}, v_{3}}

could be written as a linear combination of the basis vectors as

x = x_{1} v_{1} + x_{2} v_{2} + x_{3} v_{3},

then

x_{1} = \frac{x \cdot v_{1}}{v_{1} \cdot v_{1}} .

If we continued that same argument we could show that

x = (\frac{x \cdot v_{1}}{v_{1} \cdot v_{1}}) v_{1} + (\frac{x \cdot v_{2}}{v_{2} \cdot v_{2}}) v_{2} + (\frac{x \cdot v_{3}}{v_{3} \cdot v_{3}}) v_{3} .

🔗

We can apply this idea in general to see how the orthogonality of an orthogonal basis allows us to quickly and easily determine the weights to write a given vector as a linear combination of orthogonal basis vectors. To see why, let

B = {v_{1}, v_{2}, \dots, v_{m}}

be an orthogonal basis for a subspace

W

R^{n}

and let

x

be any vector in

W .

We know that

x = x_{1} v_{1} + x_{2} v_{2} + \dots + x_{m} v_{m}

🔗

for some scalars

x_{1},

x_{2},

\dots,

x_{m} .

Let

1 \leq k \leq m .

Then, using orthogonality of vectors

v_{1}, v_{2}, \dots, v_{m},

we have

v_{k} \cdot x = x_{1} (v_{k} \cdot v_{1}) + x_{2} (v_{k} \cdot v_{2}) + \dots + x_{m} (v_{k} \cdot v_{m}) = x_{k} v_{k} \cdot v_{k} .

🔗

x_{k} = \frac{x \cdot v_{k}}{v_{k} \cdot v_{k}} .

🔗

Thus, we can calculate each weight individually with two simple dot products. We summarize this discussion in the next theorem.

🔗

Theorem 24.4.

Let $B = {v_{1}, v_{2}, \dots, v_{m}}$ be an orthogonal basis for a subspace of $R^{n} .$ Let $x$ be a vector in $W .$ Then

\begin{matrix} (24.3) & x = \frac{x \cdot v_{1}}{v_{1} \cdot v_{1}} v_{1} + \frac{x \cdot v_{2}}{v_{2} \cdot v_{2}} v_{2} + \dots + \frac{x \cdot v_{m}}{v_{m} \cdot v_{m}} v_{m} . \end{matrix}

🔗

Activity 24.3.

Let $v_{1} = [1 0 1]^{T},$ $v_{2} = [0 1 0]^{T},$ and $v_{3} = [0 0 1]^{T} .$ The set $B = {v_{1}, v_{2}, v_{3}}$ is a basis for $R^{3} .$ Let $x = [1 0 0]^{T} .$ Calculate

\frac{x \cdot v_{1}}{v_{1} \cdot v_{1}} v_{1} + \frac{x \cdot v_{2}}{v_{2} \cdot v_{2}} v_{2} + \frac{x \cdot v_{3}}{v_{3} \cdot v_{3}} v_{3} .

Compare to $x .$ Does this violate Theorem 24.4? Explain.

🔗

Subsection Orthonormal Bases

🔗

The decomposition (24.3) is even simpler if

v_{k} \cdot v_{k} = 1

for each

k,

that is, if

v_{k}

is a unit vector for each

k .

In this case, the denominators are all 1 and we don't even need to consider them. We have a familiar example of such a basis for

R^{n},

namely the standard basis

S = {e_{1}, e_{2}, \dots, e_{n}} .

🔗

Recall that

v \cdot v = | | v | |^{2},

🔗

so the condition

v \cdot v = 1

implies that the vector

v

has norm 1. An orthogonal basis with this additional condition is a very nice basis and is given a special name.

🔗

Definition 24.5.

An orthonormal basis $B = {u_{1}, u_{2}, \dots, u_{m}}$ for a subspace $W$ of $R^{n}$ is an orthogonal basis such that $| | u_{k} | | = 1$ for $1 \leq k \leq m .$

🔗

In other words, an orthonormal basis is an orthogonal basis in which every basis vector is a unit vector. A good question to ask here is how we can construct an orthonormal basis from an orthogonal basis.

🔗

Activity 24.4.

🔗

(a)

Let $v_{1}$ and $v_{2}$ be orthogonal vectors. Explain how we can obtain unit vectors $u_{1}$ in the direction of $v_{1}$ and $u_{2}$ in the direction of $v_{2} .$

🔗

(b)

Show that $u_{1}$ and $u_{2}$ from the previous part are orthogonal vectors.

🔗

(c)

Use the ideas from this problem to construct an orthonormal basis for $R^{3}$ from the orthogonal basis $S = {[\begin{array}{r} - 2 \\ 1 \\ - 1 \end{array}], [\begin{array}{r} 0 \\ 1 \\ 1 \end{array}], [\begin{array}{r} 1 \\ 1 \\ - 1 \end{array}]} .$

🔗

In general, we can construct an orthonormal basis

{u_{1}, u_{2}, \dots, u_{m}}

from an orthogonal basis

B = {v_{1}, v_{2}, \dots, v_{m}}

by normalizing each vector in

B

(that is, dividing each vector by its norm).

🔗

Subsection Orthogonal Matrices

🔗

We have seen in the diagonalization process that we diagonalize a matrix

A

with a matrix

P

whose columns are linearly independent eigenvectors of

A .

In general, calculating the inverse of the matrix whose columns are eigenvectors of

A

in the diagonalization process can be time consuming, but if the columns form an orthonormal set, then the calculation is very straightforward.

🔗

Activity 24.5.

Let $u_{1} = \frac{1}{3} [2 1 2]^{T},$ $u_{2} = \frac{1}{3} [- 2 2 1]^{T},$ and $u_{3} = \frac{1}{3} [1 2 - 2]^{T} .$ It is not difficult to see that the set ${u_{1}, u_{2}, u_{3}}$ is an orthonormal basis for $R^{3} .$ Let

A = [u_{1} u_{2} u_{3}] = \frac{1}{3} [\begin{array}{crr} 2 & - 2 & 1 \\ 1 & 2 & 2 \\ 2 & 1 & - 2 \end{array}] .

🔗

(a)

Use the definition of the matrix-matrix product to find the entries of the second row of the matrix product $A^{T} A .$ Why should you have expected the result?

Hint.

How are the rows of $A^{T}$ related to the columns of $A ?$

🔗

(b)

With the result of part (a) in mind, what is the matrix product $A^{T} A ?$ What does this tell us about the relationship between $A^{T}$ and $A^{- 1} ?$ Use technology to calculate $A^{- 1}$ and confirm your answer.

🔗

(c)

Suppose $P$ is an $n \times n$ matrix whose columns form an orthonormal basis for $R^{n} .$ Explain why $P^{T} P = I_{n} .$

🔗

The result of Activity 24.5 is that if the columns of a square matrix

P

form an orthonormal set, then

P^{- 1} = P^{T} .

This makes calculating

P^{- 1}

very easy. Note, however, that this only works if the columns of

P

form an orthonormal basis for

Col P .

You should also note that if

P

is an

n \times n

matrix satisfying

P^{T} P = I_{n},

then the columns of

P

must form an orthonormal set. Matrices like this appear quite often and are given a special name.

🔗

Definition 24.6.

An orthogonal matrix is an $n \times n$ matrix $P$ such that $P^{T} P = I_{n} .$ ⁴¹

🔗

Activity 24.6.

As a special case, we apply the result of Activity 24.5 to a $2 \times 2$ rotation matrix $P = [\begin{array}{cr} \cos (θ) & - \sin (θ) \\ \sin (θ) & \cos (θ) \end{array}] .$

🔗

(a)

Show that the columns of $P$ form an orthonormal set.

🔗

(b)

Use the fact that $P^{- 1} = P^{T}$ to find $P^{- 1} .$ Explain how this shows that the inverse of a rotation matrix by an angle $θ$ is just another rotation matrix but by the angle $- θ .$

🔗

Orthogonal matrices are useful because they satisfy some special properties. For example, if

P

is an orthogonal

n \times n

matrix and

x, y \in R^{n},

then

(P x) \cdot (P y) = (P x)^{T} (P y) = x^{T} P^{T} P y = x^{T} y = x \cdot y .

🔗

This property tells us that the matrix transformation

T

defined by

T (x) = P x

preserves dot products and, hence, orthogonality. In addition,

| | P x | |^{2} = P x \cdot P x = x \cdot x = | | x | |^{2},

🔗

| | P x | | = | | x | | .

This means that

T

preserves length. Such a transformation is called an isometry and it is convenient to work with functions that don't expand or contract things. Moreover, if

x

and

y

are nonzero vectors, then

\frac{P x \cdot P y}{| | P x | | | | P y | |} = \frac{x \cdot y}{| | x | | | | y | |} .

🔗

Thus

T

also preserves angles. Transformations defined by orthogonal matrices are very well behaved transformations. To summarize,

🔗

Theorem 24.7.

Let $P$ be an $n \times n$ orthogonal matrix and let $x, y \in R^{n} .$ Then

$(P x) \cdot (P y) = x \cdot y,$
$| | P x | | = | | x | |,$ and
$\frac{P x \cdot P y}{| | P x | | | | P y | |} = \frac{x \cdot y}{| | x | | | | y | |}$ if $x$ and $y$ are nonzero.

🔗

We have discussed orthogonal and orthonormal bases for subspaces of

R^{n}

in this section. There are reasonable questions that follow, such as

Can we always find an orthogonal (or orthonormal) basis for any subspace of $R^{n} ?$
Given a vector $v$ in $W,$ can we find an orthogonal basis of $W$ that contain $v ?$

🔗

We will answer these questions in subsequent sections.

🔗

Subsection Examples

🔗

What follows are worked examples that use the concepts from this section.

🔗

Example 24.8.

Let $S = {v_{1}, v_{2}, v_{3}},$ where $v_{1} = [1 1 - 4]^{T},$ $v_{2} = [2 2 1]^{T},$ and $v_{3} = [1 - 1 0]^{T} .$

🔗

(a)

Show that $S$ is an orthogonal set.

Solution.

Using the dot product formula, we see that $v_{1} \cdot v_{2} = 0,$ $v_{1} \cdot v_{3} = 0,$ and $v_{2} \cdot v_{3} = 0 .$ Thus, the set $S$ is an orthogonal set.

🔗

(b)

Create an orthonormal set $S^{'} = {u_{1}, u_{2}, u_{3}}$ from the vectors in $S .$

Solution.

To make an orthonormal set $S^{'} = {u_{1}, u_{2}, u_{3}}$ from $S,$ we divide each vector in $S$ by its magnitude. This gives us

u_{1} = \frac{1}{\sqrt{18}} [1 1 - 4]^{T}, u_{2} = \frac{1}{3} [2 2 1]^{T}, and u_{3} = \frac{1}{\sqrt{2}} [1 - 1 0]^{T} .

🔗

(c)

Just by calculating dot products, write the vector $w = [2 1 - 1]^{T}$ as a linear combination of the vectors in $S^{'} .$

Solution.

Since $S^{'}$ is an orthonormal basis for $R^{3},$ we know that

\begin{aligned} w & = (w \cdot u_{1}) u_{1} + (w \cdot u_{2}) u_{2} + (w \cdot u_{3}) u_{3} \\ = \frac{7}{\sqrt{18}} [1 1 - 4]^{T} + \frac{5}{3} [2 2 1]^{T} + \frac{1}{\sqrt{2}} [1 - 1 0]^{T} . \end{aligned}

🔗

Example 24.9.

Let $u_{1} = \frac{1}{\sqrt{3}} [1 1 1]^{T},$ $u_{2} = \frac{1}{\sqrt{2}} [1 - 1 0]^{T},$ and $u_{3} = \frac{1}{\sqrt{6}} [1 1 - 2]^{T} .$ Let $B = {u_{1}, u_{2}, u_{3}} .$

🔗

(a)

Show that $B$ is an orthonormal basis for $R^{3} .$

Solution.

Using the dot product formula, we see that $u_{i} \cdot u_{j} = 0$ if $i \neq j$ and that $u_{i} \cdot u_{i} = 1$ for each $i .$ Since orthogonal vectors are linearly independent, the set $B$ is a linearly independent set with $3$ vectors in a $3$ -dimensional space. It follows that $B$ is an orthonormal basis for $R^{3} .$

🔗

(b)

Let $w = [1 2 1]^{T} .$ Find $[w]_{B} .$

Solution.

Since $B$ is an orthonormal basis for $R^{3},$ we know that

w = (w \cdot u_{1}) u_{1} + (w \cdot u_{2}) u_{2} + (w \cdot u_{3}) u_{3} .

Therefore,

[w]_{B} = [(w \cdot u_{1}) (w \cdot u_{2}) (w \cdot u_{3})]^{T} = {[\frac{4}{\sqrt{3}} - \frac{1}{\sqrt{2}} \frac{1}{\sqrt{6}}]}^{T} .

🔗

(c)

Calculate $| | w | |$ and $| | [w]_{B} | | .$ What do you notice?

Solution.

Using the definition of the norm of a vector we have

\begin{aligned} | | w | | & = \sqrt{1^{2} + 2^{2} + 1^{2}} = \sqrt{6} \\ | | [w]_{| C B} | | & = \sqrt{{(\frac{4}{\sqrt{3}})}^{2} + {(- \frac{1}{\sqrt{2}})}^{2} + {(\frac{1}{\sqrt{6}})}^{2}} = \sqrt{6} . \end{aligned}

So in this case we have $| | w | | = | | [w]_{B} | | .$

🔗

(d)

Show that the result of part (c) is true in general. That is, if $S = {v_{1}, v_{2}, \dots, v_{n}}$ is an orthonormal basis for $R^{n},$ and if $z = c_{1} v_{1} + c_{2} v_{2} + \dots + c_{n} v_{n},$ then

| | z | | = \sqrt{c_{1}^{2} + c_{2}^{2} + \dots + c_{n}^{2}} .

Solution.

Let $S = {v_{1}, v_{2}, \dots, v_{n}}$ be an orthonormal basis for $R^{n},$ and suppose that $z = c_{1} v_{1} + c_{2} v_{2} + \dots + c_{n} v_{n} .$ Then

\begin{aligned} | | z | | & = \sqrt{z \cdot z} \\ (24.4) & = \sqrt{(c_{1} v_{1} + c_{2} v_{2} + \dots + c_{n} v_{n}) \cdot (c_{1} v_{1} + c_{2} v_{2} + \dots + c_{n} v_{n})} . \end{aligned}

Since $S$ is an orthonormal basis for $R^{n},$ it follows that $v_{i} \cdot v_{j} = 0$ if $i \neq j$ and $v_{\cdot} v_{i} = 1 .$ Expanding the dot product in (24.4), the only terms that won't be zero are the ones that involve $v_{i} \cdot v_{i} .$ This leaves us with

\begin{aligned} | | z | | & = \sqrt{(c_{1} v_{1} + \dots + c_{n} v_{n}) \cdot (c_{1} v_{1} + \dots + c_{n} v_{n})} \\ = \sqrt{c_{1} c_{1} (v_{1} \cdot v_{1}) + c_{2} c_{2} (v_{2} \cdot v_{2}) + \dots + c_{n} c_{n} (v_{n} \cdot v_{n})} \\ = \sqrt{c_{1}^{2} + c_{2}^{2} + \dots + c_{n}^{2}} . \end{aligned}

🔗

Subsection Summary

A subset $S$ of $R^{n}$ is an orthogonal set if $u \cdot v = 0$ for every pair of distinct vector $u$ and $v$ in $S .$
Any orthogonal set of nonzero vectors is linearly independent.
A basis $B$ for a subspace $W$ of $R^{n}$ is an orthogonal basis if $B$ is also an orthogonal set.
An orthogonal basis $B$ for a subspace $W$ of $R^{n}$ is an orthonormal basis if each vector in $B$ has unit length.
If $B = {v_{1}, v_{2}, \dots, v_{m}}$ is an orthogonal basis for a subspace $W$ of $R^{n}$ and $x$ is any vector in $W,$ then

$x = \sum_{i = 1}^{m} c_{i} v_{i}$

where $c_{i} = \frac{x \cdot v_{i}}{v_{i} \cdot v_{i}} .$
An $n \times n$ matrix $P$ is an orthogonal matrix if $P^{T} P = I_{n} .$ Orthogonal matrices are important, in part, because the matrix transformations they define are isometries.

🔗

Exercises Exercises

🔗

1.

Find an orthogonal basis for the subspace $W = {[x y z] : 4 x - 3 z = 0}$ of $R^{3} .$

🔗

2.

Let ${v_{1}, v_{2}, \dots, v_{n}}$ be an orthogonal basis for $R^{n}$ and, for some $k$ between 1 and $n,$ let $W = Span {v_{1}, v_{2}, \dots, v_{k}} .$ Show that ${v_{k + 1}, v_{k + 2}, \dots, v_{n}}$ is a basis for $W^{⊥} .$

🔗

3.

Let $W$ be a subspace of $R^{n}$ for some $n,$ and let ${w_{1}, w_{2}, \dots, w_{k}}$ be an orthogonal basis for $W .$ Let $x$ be a vector in $R^{n} .$ and define $w$ as

w = \frac{x \cdot w_{1}}{w_{1} \cdot w_{1}} w_{1} + \frac{x \cdot w_{2}}{w_{2} \cdot w_{2}} w_{2} + \dots + \frac{x \cdot w_{k}}{w_{k} \cdot w_{k}} w_{k} .

🔗

(a)

Explain why $w$ is in $W .$

Hint.

Where are $w_{1},$ $w_{2},$ $\dots,$ $w_{k} ?$

🔗

(b)

Let $z = x - w .$ Show that $z$ is in $W^{⊥} .$

Hint.

Take the dot product of $z$ with $w_{i} .$

🔗

(c)

Explain why $x$ can be written as a sum of vectors, one in $W$ and one in $W^{⊥} .$

Hint.

Use parts (a) and (b).

🔗

(d)

Suppose $x = w + w_{1}$ and $x = u + u_{1},$ where $w$ and $u$ are in $W$ and $w_{1}$ and $u_{1}$ are in $W^{⊥} .$ Show that $w = u$ and $w_{1} = u_{1},$ so that the representation of $x$ as a sum of a vector in $W$ and a vector in $W^{⊥}$ is unique.

Hint.

Collect terms in $W$ and in $W^{⊥} .$

🔗

4.

Use the result of problem Exercise 3 above and that $W \cap W^{⊥} = {0}$ to show that $\dim (W) + \dim (W^{⊥}) = n$ for a subspace $W$ of $R^{n} .$ (See Exercise 13 in Section 12 for the definition of the sum of subspaces.)

🔗

5.

Let $P$ be an $n \times n$ matrix. We showed that if $P$ is an orthogonal matrix, then $(P x) \cdot (P y) = x \cdot y$ for any vectors $x$ and $y$ in $R^{n} .$ Now we ask if the converse of this statement is true. That is, determine the validity of the following statement: if $(P x) \cdot (P y) = x \cdot y$ for any vectors $x$ and $y$ in $R^{n},$ then $P$ is an orthogonal matrix? Verify your answer.

Hint.

Consider $(P e_{i}) \cdot (P e_{j})$ where $e_{t}$ is the $t$ th standard basis vector for $R^{n} .$

🔗

6.

In this exercise we examine reflection matrices. In the following exercise we will show that the reflection and rotation matrices are the only $2 \times 2$ orthogonal matrices. We will determine how to represent the reflection across a line through the origin in $R^{2}$ as a matrix transformation. The setup is as follows. Let $L (θ)$ be the line through the origin in $R^{2}$ that makes an angle $θ$ with the positive $x$ -axis as illustrated in Figure 24.10.

Figure 24.10. Reflecting across a line $L (θ) .$

🔗

(a)

Find a unit vector $u$ in the direction of the line $L (θ) .$

🔗

(b)

Let $v = [\begin{matrix} a \\ b \end{matrix}]$ be an arbitrary vector in $R^{2}$ as represented in Figure 24.10. Determine the components of the vectors ${proj}_{u} v$ and ${proj}_{⊥ u} v .$ Reproduce Figure 24.10 and draw the vectors ${proj}_{u} v$ and ${proj}_{⊥ u} v$ in your figure.

🔗

(c)

The vector labeled $w$ is the reflection of the vector $v$ across the line $L (θ) .$ Write $w$ in terms of $v$ and ${proj}_{u} v .$ Clearly explain your method.

🔗

(d)

Finally, show that the matrix $A$ such that $A v = w$ is given by

A = [\begin{array}{cr} \cos (2 θ) & \sin (2 θ) \\ \sin (2 θ) & - \cos (2 θ) \end{array}] .

The matrix $A$ is the reflection matrix across the line $L (θ) .$ (You will want to look up some appropriate trigonometric identities.)

🔗

7.

In this exercise we will show that the only orthogonal $2 \times 2$ matrices are the rotation matrices $[\begin{array}{cr} \cos (θ) & - \sin (θ) \\ \sin (θ) & \cos (θ) \end{array}]$ and the reflection matrices $[\begin{array}{cr} \cos (θ) & \sin (θ) \\ \sin (θ) & - \cos (θ) \end{array}]$ (see Exercise 6). Throughout this exercise let $a,$ $b,$ $c,$ and $d$ be real numbers such that $M = [\begin{array}{cc} a & b \\ c & d \end{array}]$ is an orthogonal $2 \times 2$ matrix. Let $v_{1} = [\begin{matrix} a \\ c \end{matrix}]$ and $v_{2} = [\begin{matrix} b \\ d \end{matrix}]$ be the columns of $M .$

🔗

(a)

Explain why the terminal point of $v_{1}$ in standard position lies on the unit circle. Then explain why there is an angle $θ$ such that $a = \cos (θ)$ and $c = \sin (θ) .$ What angle, specifically, is $θ ?$ Draw a picture to illustrate.

Hint.

Think polar coordinates.

🔗

(b)

A similar argument to (b) shows that there is an angle $α$ such that $v_{2} = [\begin{matrix} b \\ d \end{matrix}] = [\begin{matrix} \cos (α) \\ \sin (α) \end{matrix}] .$ Given that $M$ is an orthogonal matrix, how must $α$ be related to $θ ?$ Use this result to find the two possibilities for $v_{2}$ as a vector in terms of $\cos (θ)$ and $\sin (θ) .$ (You will likely want to look up some trigonometric identities for this part of the problem.)

Hint.

What properties do the columns of an orthogonal matrix have?

🔗

(c)

By considering the two possibilities from part (c), show that $M$ is either a rotation matrix or a reflection matrix. Conclude that the only $2 \times 2$ orthogonal matrices are the reflection and rotation matrices.

🔗

8.

Suppose $A, B$ are orthogonal matrices of the same size.

🔗

(a)

Show that $A B$ is also an orthogonal matrix.

🔗

(b)

Show that $A^{T}$ is also an orthogonal matrix.

🔗

(c)

Show that $A^{- 1}$ is also an orthogonal matrix.

🔗

9.

Label each of the following statements as True or False. Provide justification for your response.

🔗

(a) True/False.

Any orthogonal subset of $R^{n}$ is linearly independent.

🔗

(b) True/False.

Every single vector set is an orthogonal set.

🔗

(c) True/False.

If $S$ is an orthogonal set in $R^{n}$ with exactly $n$ nonzero vectors, then $S$ is a basis for $R^{n} .$

🔗

(d) True/False.

Every set of three linearly independent vectors in $R^{3}$ is an orthogonal set.

🔗

(e) True/False.

If $A$ and $B$ are $n \times n$ orthogonal matrices, then $A + B$ must also be an orthogonal matrix.

🔗

(f) True/False.

If the set $S = {v_{1}, v_{2}, \dots, v_{n}}$ is an orthogonal set in $R^{n},$ then so is the set ${c_{1} v_{1}, c_{2} v_{2}, \dots, c_{n} v_{n}}$ for any scalars $c_{1},$ $c_{2},$ $\dots,$ $c_{n} .$

🔗

(g) True/False.

If $B = {v_{1}, v_{2}, \dots, v_{n}}$ is an orthogonal basis of $R^{n},$ then so is ${c_{1} v_{1},$ $c_{2} v_{2},$ $\dots,$ $c_{n} v_{n}}$ for any nonzero scalars $c_{1},$ $c_{2},$ $\dots,$ $c_{n} .$

🔗

(h) True/False.

If $A$ is an $n \times n$ orthogonal matrix, the rows of $A$ form an orthonormal basis of $R^{n} .$

🔗

(i) True/False.

If $A$ is an orthogonal matrix, any matrix obtained by interchanging columns of $A$ is also an orthogonal matrix.

🔗

Subsection Project: Understanding Rotations in 3-Space

🔗

Recall that a counterclockwise rotation of

2

-space around the origin by an angle

θ

is accomplished by left multiplication by the matrix

[\begin{array}{cr} \cos (θ) & - \sin (θ) \\ \sin (θ) & \cos (θ) \end{array}] .

Notice that the columns of this rotation matrix are orthonormal, so this rotation matrix is an orthogonal matrix. As the next activity shows, rotation matrices in 3D are also orthogonal matrices.

🔗

Project Activity 24.7.

Let $R$ be a rotation matrix in 3D. A rotation does not change lengths of vectors, nor does it change angles between vectors. Let $e_{1} = [1 0 0]^{T},$ $e_{2} = [0 1 0]^{T},$ and $e_{3} = [0 0 1]^{T}$ be the standard unit vectors in $R^{3} .$

🔗

(a)

Explain why the columns of $R$ form an orthonormal set.

Hint.

How are $R e_{1},$ $R e_{2},$ and $R e_{3}$ related to the columns of $R ?$

🔗

(b)

Explain why $R$ is an orthogonal matrix. What must be true about $det (R) ?$

Hint.

What is $R^{T}$ and what is $det (R^{T} R) ?$

🔗

By Project Activity 24.7 we know that the determinant of any rotation matrix is either

1

- 1 .

Having a determinant of

1

preserves orientation, and we will identify these rotations as being counterclockwise, and we will identify the others with determinant of

- 1

as being clockwise. We will set the convention that a rotation is always measured counterclockwise (as we did in

R^{2}

), and so every rotation matrix will have determinant

1 .

🔗

Returning to the counterclockwise rotation of

2

-space around the origin by an angle

θ

determined by left multiplication by the matrix

[\begin{array}{cr} \cos (θ) & - \sin (θ) \\ \sin (θ) & \cos (θ) \end{array}],

we can think of this rotation in

3

-space as the rotation that keeps points in the

x y

plane in the

x y

plane, but rotates these points counterclockwise around the

z

axis. In other words, in the standard

x y z

coordinate system, with standard basis

e_{1},

e_{2},

e_{3},

our rotation matrix

R

has the property that

R e_{3} = e_{3} .

Now

R e_{3}

is the third column of

R,

so the third column of

R

e_{3} .

Similarly,

R e_{1}

is the first column of

R

and

R e_{2}

is the second column of

R .

Since

R

is a counterclockwise rotation of the

x y

plane space around the origin by an angle

θ

it follows that this rotation is given by the matrix

\begin{matrix} (24.5) & R_{e_{3}} (θ) = [\begin{array}{ccc} \cos (θ) & - \sin (θ) & 0 \\ \sin (θ) & \cos (θ) & 0 \\ 0 & 0 & 1 \end{array}] . \end{matrix}

🔗

In this notation in (24.5), the subscript gives the direction of the line fixed by the rotation and the angle provides the counterclockwise rotation in the plane perpendicular to this vector. This vector is called a normal vector for the rotation. Note also that the columns of

R_{e_{3}} (θ)

form an orthogonal set such that each column vector has norm

1 .

🔗

This idea describes a general rotation matrix

R_{n} (θ)

in 3D by specifying a normal vector

n

and an angle

θ .

For example, with roll, a normal vector points from the tail of the aircraft to its tip. It is our goal to understand how we can determine an arbitrary rotation matrix of the form

R_{n} (θ) .

We can accomplish this by using the rotation around the

z

axis and change of basis matrices to find rotation matrices around other axes. Let

S = {e_{1}, e_{2}, e_{3}}

be the standard basis for

R^{3}

🔗

Project Activity 24.8.

In this activity we see how to determine the rotation matrix around the $x$ axis using the matrix $R_{e_{3}} (θ)$ and a change of basis.

🔗

(a)

Define a new ordered basis $B$ so that our axis of rotation is the third vector. So in this case the third vector in $B$ will be $e_{1} .$ The other two vectors need to make $B$ an orthonormal set. So we have plenty of choices. For example, we could set $B = {e_{2}, e_{3}, e_{1}} .$ Find the change of basis matrix $\underset{S \leftarrow B}{P}$ from $B$ to $S .$

🔗

(b)

Use the change of basis matrix from part (a) to find the change of basis matrix $\underset{B \leftarrow S}{P}$ from $S$ to $B .$

🔗

(c)

To find our rotation matrix around the $x$ axis, we can first change basis from $S$ to $B,$ then perform a rotation around the new $z$ axis using (24.5), then changing basis back from $B$ to $S .$ In other words,

R_{e_{1}} (θ) = \underset{S \leftarrow B}{P} R_{e_{3}} (θ) \underset{B \leftarrow S}{P} .

Find the entries of this matrix $R_{e_{1}} (θ) .$

🔗

IMPORTANT NOTE.

🔗

We could have considered using

B_{1} = {e_{3}, e_{2}, e_{1}}

in Project Activity 24.8 instead of

B = {e_{2}, e_{3}, e_{1}} .

Then we would have

\underset{S \leftarrow B_{1}}{P} = [\begin{array}{ccc} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{array}] .

🔗

The difference between the two options is that, in the first we have

det (\underset{S \leftarrow B_{1}}{P}) = - 1

while

det (\underset{S \leftarrow B}{P}) = 1

in the second. Using

B_{1}

will give clockwise rotations while

B

gives counterclockwise rotations (this is the difference between a left hand system and a right hand system). So it is important to ensure that our change of basis matrix is one with determinant

1 .

🔗

Project Activity 24.9.

Repeat Project Activity 24.9 to find the 3D rotation around the $y$ axis.

🔗

We do one more example to illustrate the process before tackling the general case.

🔗

Project Activity 24.10.

In this activity we find the rotation around the axis given by the line $x = y / 2 = z .$ This line is in the direction of the vector $n = [1 2 1]^{T} .$ So we start with making a unit vector in the direction of $n$ as the third vector in an ordered basis $B .$ The other two vectors need to make $B$ an orthonormal set with $det (\underset{S \leftarrow B}{P}) = 1 .$

🔗

(a)

Find a unit vector $w$ in the direction of $n .$

🔗

(b)

Show that $[2 - 1 0]^{T}$ is orthogonal to the vector $w$ from part (a). Then find a unit vector $v$ that is in the same direction as $[2 - 1 0]^{T} .$

🔗

(c)

Let $v$ be as in the previous part. Now the trick is to find a third unit vector $u$ so that $B = {u, v, w}$ is an orthonormal set. This can be done with the cross product. If $a = [a_{1} a_{2} a_{3}]^{T}$ and $b = [b_{1} b_{2} b_{3}]^{T},$ then the cross product $a \times b$ of $a$ and $b$ is the vector

a \times b = (a_{2} b_{3} - a_{3} b_{2}) e_{1} - (a_{1} b_{3} - a_{3} b_{1}) e_{2} + (a_{1} b_{2} - a_{2} b_{1}) e_{3} .

(You can check that ${a \times b, a, b}$ is an orthogonal set that gives the correct determinant for the change of basis matrix.) Use the cross product to find a unit vector $u$ so that $B = {u, v, w}$ is an orthonormal set.

🔗

(d)

Find the entries of the matrix $R_{w} (θ) .$

🔗

In the next activity we summarize the general process to find a 3D rotation matrix

R_{n} (θ)

for any normal vector

n .

There is a GeoGebra applet at geogebra.org/m/n9gbjhfx that allows you to visualize rotation matrices in 3D.

🔗

Project Activity 24.11.

Let $n = [n_{1} n_{2} n_{3}]^{T}$ be a normal vector (nonzero) for our rotation. We need to create an orthonormal basis $B = {u, v, w}$ where $w$ is a unit vector in the direction of $n$ so that the change of basis matrix $\underset{S \leftarrow B}{P}$ has determinant $1 .$

🔗

(a)

Find, by inspection, a vector $y$ that is orthogonal to $n .$

Hint.

You may need to consider some cases to ensure that $v$ is not the zero vector.

🔗

(b)

Once we have a normal vector $n$ and a vector $y$ orthogonal to $n,$ the vector $z = y \times n$ gives us an orthogonal set ${z, y, n} .$ We then normalize each vector to create our orthonormal basis $B = {u, v, w} .$ Use this process to find the matrix that produces a $45^{\circ}$ counterclockwise rotation around the normal vector $[1 0 - 1]^{T} .$

It isn't clear why such matrices are called orthogonal since the columns are actually orthonormal, but that is the standard terminology in mathematics.

Section 24 Orthogonal and Orthonormal Bases in Rn

Focus Questions

Subsection Application: Rotations in 3D

Subsection Introduction

Definition 24.1.

Preview Activity 24.1.

(a)

(b)

Definition 24.2.

(i)

(ii)

(iii)

Subsection Orthogonal Sets

Activity 24.2.

(a)

(b)

(c)

Theorem 24.3.

Proof.

Subsection Properties of Orthogonal Bases

Theorem 24.4.

Activity 24.3.

Subsection Orthonormal Bases

Definition 24.5.

Activity 24.4.

(a)

(b)

(c)

Subsection Orthogonal Matrices

Activity 24.5.

(a)

(b)

(c)

Definition 24.6.

Activity 24.6.

(a)

(b)

Theorem 24.7.

Subsection Examples

Example 24.8.

(a)

Solution.

(b)

Solution.

(c)

Solution.

Example 24.9.

(a)

Solution.

(b)

Solution.

(c)

Solution.

(d)

Solution.

Subsection Summary

Exercises Exercises

1.

2.

3.

(a)

(b)

(c)

(d)

4.

5.

6.

(a)

(b)

(c)

(d)

7.

(a)

(b)

(c)

8.

(a)

(b)

(c)

9.

Section 24 Orthogonal and Orthonormal Bases in $R^{n}$