The Dot Product in \R^n

Section 23 The Dot Product in $R^{n}$

Focus Questions

By the end of this section, you should be able to give precise and thorough answers to the questions listed below. You may want to keep these questions in mind to focus your thoughts as you complete the section.

What is the dot product of two vectors? Under what conditions is the dot product defined?
How do we find the angle between two nonzero vectors in $R^{n} ?$
How does the dot product tell us if two vectors are orthogonal?
How do we define the length of a vector in any dimension and how can the dot product be used to calculate the length?
How do we define the distance between two vectors?
What is the orthogonal projection of a vector $u$ in the direction of the vector $v$ and how do we find it?
What is the orthogonal complement of a subspace $W$ of $R^{n} ?$

🔗

Subsection Application: Hidden Figures in Computer Graphics

🔗

In video games, the speed at which a computer can render changing graphics views is vitally important. To increase a computer's ability to render a scene, programs often try to identify those parts of the images a viewer could see and those parts the viewer could not see. For example, in a scene involving buildings, the viewer could not see any images blocked by a solid building. In the mathematical world, this can be visualized by graphing surfaces. In Figure 23.1 we see a crude image of a house made up of small polygons (this is how programs generally represent surfaces). On the left in Figure 23.1 we see all of the polygons that are needed to construct the entire surface, even those polygons that lie behind others which we could not see if the surface was solid. On the right in Figure 23.1 we have hidden the parts of the polygons that we cannot see from our view.

🔗

We also see this idea in mathematics when we graph surfaces. Figure 23.2 shows the graph of the surface defined by

f (x, y) = \sqrt{4 - x^{2}}

that is made up of polygons. At left we see all of the polygons and at right only those parts that would be visible from our viewing perspective.

🔗
Figure 23.2. Graphs of $f (x, y) = \sqrt{4 - x^{2}} .$

🔗

By eliminating the parts of the polygons we cannot see from our viewing perspective, the computer program can more quickly render the viewing image. Later in this section we will explore one method for how programs remove the hidden portions of images. This process involves the dot product of vectors.

🔗

Subsection Introduction

🔗

Orthogonality, a concept which generalizes the idea of perpendicularity, is an important concept in linear algebra. We use the dot product to define orthogonality and more generally angles between vectors in

R^{n}

for any dimension

n .

The dot product has many applications, e.g., finding components of forces acting in different directions in physics and engineering. The dot product is also an example of a larger concept, inner products, that we will discuss later. We introduce and investigate dot products in this section.

🔗

We will illustrate the dot product in

R^{2},

but the process we go through will translate to any dimension. Recall that we can represent the vector

v = [\begin{matrix} v_{1} \\ v_{2} \end{matrix}]

as the directed line segment (or arrow) from the origin to the point

(v_{1}, v_{2})

R^{2},

as illustrated in Figure 23.3. Using the Pythagorean Theorem we can then define the length (or magnitude or norm) of the vector

v

R^{2}

| | v | | = \sqrt{v_{1}^{2} + v_{2}^{2}} .

🔗

We can also write this norm as

\sqrt{v_{1} v_{1} + v_{2} v_{2}} .

🔗

The expression under the square root is an important one and we extend it and give it a special name.

🔗
Figure 23.3. A vector in $R^{2}$ from the origin to a point.

🔗

u = [u_{1} u_{2}]^{T}

and

v = [v_{1} v_{2}]^{T}

are vectors in

R^{2},

then we call the expression

u_{1} v_{1} + u_{2} v_{2}

the dot product of

u

and

v,

and denote it as

u \cdot v .

With this idea in mind, we can rewrite the norm of the vector

v

| | v | | = \sqrt{v \cdot v} .

🔗

The definition of the dot product translates naturally to

R^{n}

(see Exercise 5 in Section 5).

🔗

Definition 23.4.

Let $u = [u_{1} u_{2} \dots u_{n}]$ and $v = [v_{1} v_{2} \dots v_{n}]$ be vectors in $R^{n} .$ The dot product (or scalar product ) of $u$ and $v$ is the scalar

u \cdot v = u_{1} v_{1} + u_{2} v_{2} + \dots + u_{n} v_{n} = \sum_{i = 1}^{n} u_{i} v_{i} .

🔗

The dot product then allows us to define the norm (or magnitude or length) of any vector in

R^{n} .

🔗

Definition 23.5.

The norm $| | v | |$ of the vector $v \in R^{n}$ is

| | v | | = \sqrt{v \cdot v} .

🔗

We also use the words magnitude or length as alternatives for the word norm. We can equivalently write the norm of the vector

v = [v_{1} v_{2} \dots v_{n}]^{T}

| | v | | = \sqrt{v_{1}^{2} + v_{2}^{2} + \dots + v_{n}^{2}} .

🔗

We can also realize the dot product as a matrix product. If

u = [u_{1} u_{2} \dots u_{n}]^{T}

and

v = [v_{1} v_{2} \dots v_{n}]^{T},

then

u \cdot v = u^{T} v

🔗

³⁹.

🔗

IMPORTANT NOTE.

🔗

The dot product is only defined between two vectors with the same number of components.

🔗

Preview Activity 23.1.

🔗

(a)

Find $u \cdot v$ if $u = [2 3 - 1 4]^{T}$ and $v = [4 6 7 - 5]^{T}$ in $R^{4} .$

🔗

(b)

The dot product satisfies some useful properties as given in the next theorem.

🔗

Theorem 23.6.

Let $u,$ $v,$ and $w$ be vectors in $R^{n},$ and let $c$ be a scalar. Then

$u \cdot v = v \cdot u$ (the dot product is commutative),
$(u + v) \cdot w = (u \cdot w) + (v \cdot w)$ (the dot product distributes over vector addition),
$(c u) \cdot v = u \cdot (c v) = c (u \cdot v),$
$u \cdot u \geq 0$ with equality if and only if $u = 0,$
$| | c u | | = | c | | | u | | .$

Verification of some of these properties is left to the exercises. Let $u$ and $v$ be vectors in $R^{5}$ with $u \cdot v = - 1,$ $| | u | | = 2$ and $| | v | | = 3 .$

🔗

(i)

Use property (c) of Theorem 23.6 to determine the value of $u \cdot 2 v .$

🔗

(ii)

Use property (b) of Theorem 23.6 to determine the value of $(u + v) \cdot v .$

🔗

(iii)

Use whatever properties of Theorem 23.6 that are needed to determine the value of $(2 u + 4 v) \cdot (u - 7 v) .$

🔗

(c)

At times we will want to find vectors in the direction of a given vector that have a certain magnitude. Let $u = [2 2 1]^{T}$ in $R^{3} .$

🔗

(i)

What is $| | u | | ?$

🔗

(ii)

Show that $| | \frac{1}{| | u | |} u | | = 1 .$

🔗

(iii)

Vectors with magnitude 1 are important and are given a special name.

🔗

Definition 23.7.

A vector $v$ in $R^{n}$ is a unit vector if $| | v | | = 1 .$

We can use unit vectors to find vectors of a given length in the direction of a given vector. Let $c$ be a positive scalar and $v$ a vector in $R^{n} .$ Use properties from Theorem 23.6 to show that the magnitude of the vector $c \frac{v}{| | v | |}$ is $c .$

🔗

Subsection The Distance Between Vectors

🔗

Finding optimal solutions to systems is an important problem in applied mathematics. It is often the case that we cannot find an exact solution that satisfies certain constraints, so we look instead for the “best” solution that satisfies the constraints. An example of this is fitting a least squares line to a set of data. As we will see, the dot product will allow us to find “best” solutions to certain types of problems, where we measure accuracy using the notion of a distance between vectors. Geometrically, we can represent a vector

u

as a directed line segment from the origin to the point defined by

u .

If we have two vectors

u

and

v,

we can think of the length of the difference

u - v

as a measure of how far apart the two vectors are from each other. It is natural, then to define the distance between vectors as follows.

🔗

Definition 23.8.

Let $u$ and $v$ be vectors in $R^{n} .$ The distance between $u$ and $v$ is the length of the difference $u - v$ or

| | u - v | | .

🔗

As Figure 23.9 illustrates, if vectors

u

and

v

emanate from the same initial point, and

P

and

Q

are the terminal points of

u

and

v,

respectively, then the difference

| | u - v | |

is the standard Euclidean distance between the points

P

and

Q .

🔗

Subsection The Angle Between Two Vectors

🔗

Determining a “best” solution to a problem often involves finding a solution that minimizes a distance. We generally accomplish a minimization through orthogonality — which depends on the angle between vectors. Given two vectors

u

and

v

R^{n},

we position the vectors so that they emanate from the same initial point. If the vectors are nonzero, then they determine a plane in

R^{n} .

In that plane there are two angles that these vectors create. We will define the angle between the vectors to be the smaller of these two angles. The dot product will tell us how to find the angle between vectors. Let

u

and

v

be vectors in

R^{n}

and

θ

the angle between them as illustrated in Figure 23.10.

🔗
Figure 23.10. The angle between $u$ and $v$

🔗

Using the Law of Cosines, we have

| | u - v | |^{2} = | | u | |^{2} + | | v | |^{2} - 2 | | u | | | | v | | \cos (θ) .

🔗

Rearranging, we obtain

\begin{aligned} | | u | | | | v | | \cos (θ) & = \frac{1}{2} (| | u | |^{2} + | | v | |^{2} - | | u - v | |^{2}) \\ = \frac{1}{2} (| | u | |^{2} + | | v | |^{2} - (u - v) \cdot (u - v)) \\ = \frac{1}{2} (| | u | |^{2} + | | v | |^{2} - u \cdot u + 2 u \cdot v - v \cdot v) \\ = u \cdot v . \end{aligned}

🔗

So the angle

θ

between two nonzero vectors

u

and

v

R^{n}

satisfies the equation

\begin{matrix} (23.1) & \cos (θ) = \frac{u \cdot v}{| | u | | | | v | |} . \end{matrix}

🔗

Of particular interest to us will be the situation where vectors

u

and

v

are orthogonal (perpendicular).⁴⁰ Intuitively, we think of two vectors as orthogonal if the angle between them is

90^{\circ} .

🔗

Activity 23.2.

🔗

(a)

The vectors $e_{1} = [1 0]^{T}$ and $e_{2} = [0 1]^{T}$ are perpendicular in $R^{2} .$ What is $e_{1} \cdot e_{2} ?$

🔗

(b)

Now let $u$ and $v$ be any vectors in $R^{n} .$

🔗

(i)

Suppose the angle between nonzero vectors $u$ and $v$ is $90^{\circ} .$ What does Equation (23.1) tell us about $u \cdot v ?$

🔗

(ii)

Now suppose that $u \cdot v = 0 .$ What does Equation (23.1) tell us about the angle between $u$ and $v ?$ Why?

🔗

(iii)

Explain why the following definition makes sense.

🔗

Definition 23.11.

Two vectors $u$ and $v$ in $R^{n}$ are orthogonal if $u \cdot v = 0 .$

🔗

(iv)

According to Definition 23.11, to which vectors is $0$ orthogonal? Does this make sense to you intuitively? Explain.

🔗

Activity 23.3.

🔗

(a)

Find the angle between the two vectors $u = [1 3 - 2 5]^{T}$ and $v = [5 2 3 - 1]^{T} .$

🔗

(b)

Find, if possible, two non-parallel vectors orthogonal to $u = [\begin{array}{r} 0 \\ 3 \\ - 2 \\ 1 \end{array}] .$

🔗

Subsection Orthogonal Projections

🔗

When running a sprint, the racers may be aided or slowed by the wind. The wind assistance is a measure of the wind speed that is helping push the runners down the track. It is much easier to run a very fast race if the wind is blowing hard in the direction of the race. So that world records aren't dependent on the weather conditions, times are only recorded as record times if the wind aiding the runners is less than or equal to 2 meters per second. Wind speed for a race is recorded by a wind gauge that is set up close to the track. It is important to note, however, that weather is not always as cooperative as we might like. The wind does not always blow exactly in the direction of the track, so the gauge must account for the angle the wind makes with the track. If the wind is blowing in the direction of the vector

u

in Figure 23.12 and the track is in the direction of the vector

v

in Figure 23.12, then only part of the total wind vector is actually working to help the runners. This part is called the orthogonal projection of the vector

u

onto the vector

v

and is denoted

{proj}_{v} u .

The next activity shows how to find this projection.

🔗
Figure 23.12. The orthogonal projection of $u$ onto $v .$

🔗

Activity 23.4.

Since the orthogonal projection ${proj}_{v} u$ is in the direction of $v,$ there exists a constant $c$ such that ${proj}_{v} u = c v .$ If we determine the value of $c,$ we can find ${proj}_{v} u .$

🔗

(a)

The wind component that acts perpendicular to the direction of $v$ is called the projection of $u$ orthogonal to $v$ and is denoted ${proj}_{⊥ v} u$ as shown in Figure 23.12. Write an equation that involves ${proj}_{v} u,$ ${proj}_{⊥ v} u,$ and $u .$ Then solve that equation for ${proj}_{⊥ v} u .$

🔗

(b)

Given that $v$ and ${proj}_{⊥ v} u$ are orthogonal, what does that tell us about $v \cdot {proj}_{⊥ v} u ?$ Combine this fact with the result of part (a) and that ${proj}_{v} u = c v$ to obtain an equation involving $v,$ $u,$ and $c .$

🔗

(c)

Solve for $c$ using the equation you found in the previous step.

🔗

(d)

Use your value of $c$ to identify ${proj}_{v} u .$

🔗

To summarize:

🔗

Definition 23.13.

Let $u$ and $v$ be vectors in $R^{n}$ with $v \neq 0 .$

The orthogonal projection of $u$ onto $v$ is the vector

$\begin{matrix} (23.2) & {proj}_{v} u = \frac{u \cdot v}{| | v | |^{2}} v . \end{matrix}$
The projection of $u$ orthogonal to $v$ is the vector

${proj}_{⊥ v} u = u - {proj}_{v} u .$

🔗

Activity 23.5.

Let $u = [\begin{matrix} 5 \\ 8 \end{matrix}]$ and $v = [\begin{array}{r} 6 \\ - 10 \end{array}] .$ Find ${proj}_{v} u$ and ${proj}_{⊥ v} u$ and draw a picture to illustrate.

🔗

The orthogonal projection of a vector

u

onto a vector

v

is really a projection of the vector

u

onto the space

Span {v} .

The vector

{proj}_{v} u

is the best approximation to

u

of all the vectors in

Span {v}

in the sense that

{proj}_{v} u

is the closest to

u

among all vectors in

Span {v},

as we will prove later.

🔗

Subsection Orthogonal Complements

🔗

In Activity 23.2 we defined two vectors

u

and

v

R^{n}

to be orthogonal (or perpendicular) if

u \cdot v = 0 .

A use of orthogonality in geometry is to define a plane. A plane through the origin in

R^{3}

is a two dimensional subspace of

R^{3},

and a plane is defined to be the set of all vectors in

R^{3}

that are orthogonal to a given vector (called a normal vector). For example, to find the equation of the plane through the origin in

R^{3}

orthogonal to the normal vector

n = [1 2 - 1]^{T},

we seek all the vectors

v = [x y z]^{T}

such that

v \cdot n = 0 .

🔗

This gives us the equation

x + 2 y - z = 0

🔗

as the equation of this plane. The collection of all vectors that are orthogonal to a given subspace of vectors is called the orthogonal complement of that subspace in

R^{n} .

🔗

Definition 23.14.

Let $W$ be a subspace of $R^{n}$ for some $n \geq 1 .$ The orthogonal complement of $W$ is the set

W^{⊥} = {x \in R^{n} : x \cdot w = 0 for all w \in W} .

🔗

Preview Activity 23.6.

Let $W = Span {[1 - 1]^{T}}$ in $R^{2} .$ Completely describe all vectors in $W^{⊥}$ both algebraically and geometrically.

🔗

There is a more general idea here as defined in Preview Activity 23.6. If we have a set

S

of vectors in

R^{n},

we let

S^{⊥}

(read as “

S

perp”, called the orthogonal complement of

S

) be the set of all vectors in

R^{n}

that are orthogonal to every vector in

S .

In our plane example, the set

S

{n}

and

S^{⊥}

is the plane with equation

x + 2 y - z = 0 .

🔗

Activity 23.7.

We have seen another example of orthogonal complements. Let $A$ be an $m \times n$ matrix with rows $r_{1},$ $r_{2}$ , $\dots,$ $r_{m}$ in order. Consider the three spaces $Nul A,$ $Row A,$ and $Col A$ related to $A,$ where $Row A = Span {r_{1}, r_{2}, \dots, r_{m}}$ (that is, $Row A$ is the span of the rows of $A$ ). Let $x$ be a vector in $Row A .$

🔗

(a)

What does it mean for $x$ to be in $Row A ?$

🔗

(b)

Now let $y$ be a vector in $Nul A .$ Use the result of part (a) and the fact that $A y = 0$ to explain why $x \cdot y = 0 .$ Explain how this verifies $(Row A)^{⊥} = Nul A .$

Hint.

Calculate $A y$ using scalar products of rows of $A$ with $y .$

🔗

(c)

Use $A^{T}$ in place of $A$ in the result of the previous part to show $(Col A)^{⊥} = Nul A^{T} .$

🔗

The activity proves the following theorem:

🔗

Theorem 23.15.

Let $A$ be an $m \times n$ matrix. Then

(Row A)^{⊥} = Nul A and (Col A)^{⊥} = Nul A^{T} .

🔗

To show that a vector is in the orthogonal complement of a subspace, it is not necessary to demonstrate that the vector is orthogonal to every vector in the subspace. If we have a basis for the subspace, it suffices to show that the vector is orthogonal to every vector in that basis for the subspace, as the next theorem demonstrates.

🔗

Theorem 23.16.

Let $B = {w_{1}, w_{2}, \dots, w_{m}}$ be a basis for a subspace $W$ of $R^{n} .$ A vector $v$ in $R^{n}$ is orthogonal to every vector in $W$ if and only if $v$ is orthogonal to every vector in $B .$

🔗

Proof.

Let $B = {w_{1}, w_{2}, \dots, w_{m}}$ be a basis for a subspace $W$ of $R^{n}$ and let $v$ be a vector in $R^{n} .$ Our theorem is a biconditional, so we need to prove both implications. Since $B \subset W,$ it follows that if $v$ is orthogonal to every vector in $W,$ then $v$ is orthogonal to every vector in $B .$ This proves the forward implication. Now we assume that $v$ is orthogonal to every vector in $B$ and show that $v$ is orthogonal to every vector in $W .$ Let $x$ be a vector in $W .$ Then

x = x_{1} w_{1} + x_{2} w_{2} + \dots + x_{m} w_{m}

for some scalars $x_{1},$ $x_{2},$ $\dots,$ $x_{m} .$ Then

\begin{aligned} v \cdot x & = v \cdot (x_{1} w_{1} + x_{2} w_{2} + \dots + x_{m} w_{m}) \\ = x_{1} (v \cdot w_{1}) + x_{2} (v \cdot w_{2}) + \dots + x_{m} (v \cdot w_{m}) \\ = 0 . \end{aligned}

Thus, $v$ is orthogonal to $x$ and $v$ is orthogonal to every vector in $W .$

🔗

Activity 23.8.

Let $W = Span {[\begin{matrix} 1 \\ 1 \\ 0 \end{matrix}], [\begin{matrix} 0 \\ 0 \\ 1 \end{matrix}]} .$ Find all vectors in $W^{⊥} .$

🔗

We will work more closely with projections and orthogonal complements in later sections.

🔗

Subsection Examples

🔗

What follows are worked examples that use the concepts from this section.

🔗

Example 23.17.

Let $ℓ$ be the line defined by the equation $a x + b y + c = 0$ with in $R^{2}$ and let $P = (x_{0}, y_{0})$ be a point in the plane. In this example we will learn how to find the distance from $P$ to $ℓ .$

🔗

(a)

Show that $n = [a b]^{T}$ is orthogonal to the line $ℓ .$ That is, $n$ is orthogonal to any vector on the line $ℓ .$

Solution.

Any vector on the line $ℓ$ is a vector between two points on the line. Let $Q = (x_{1}, y_{1})$ and $R = (x_{2}, y_{2})$ be points on the line $ℓ .$ Then $u = \vec{Q R} = [x_{2} - x_{1} y_{2} - y_{1}]^{T}$ is a vector on line $ℓ .$ Since $Q$ and $R$ are on the line, we know that $a x_{1} + b y_{1} + c = 0$ and $a x_{2} + b y_{2} + c = 0 .$ So $- c = a x_{1} + b y_{1} = a x_{2} + b y_{2}$ and

0 = a (x_{2} - x_{1}) + b (y_{2} - y_{1}) = [a b]^{T} u .

Thus, $n = [a b]^{T}$ is orthogonal to every vector on the line $ℓ .$

🔗

(b)

Let $Q = (x_{1}, y_{1})$ be any point on line $ℓ .$ Draw a representative picture of $P,$ $n$ with its initial point at $P,$ along with $Q$ and $ℓ .$ Explain how to use a projection to determine the distance from $P$ to $ℓ .$

Solution.

A picture of the situation is shown in Figure 23.18. If $v = \vec{P Q},$ then the distance from point $P$ to line $ℓ$ is given by $| | {proj}_{n} v | | .$

Figure 23.18. Distance from a point to a line.

🔗

(c)

Use the idea from part (b) to show that the distance $d$ from $P$ to $ℓ$ satisfies

\begin{matrix} (23.3) & d = \frac{| a x_{0} + b y_{0} + c |}{\sqrt{a^{2} + b^{2}}} . \end{matrix}

Solution.

Recall that $n = [a b]^{T}$ and $v = \vec{P Q} = [x_{1} - x_{0} y_{1} - y_{0}]^{T} .$ Since $a x_{1} + b y_{1} + c = 0,$ we have

\begin{aligned} {proj}_{n} v & = \frac{v \cdot v}{| | n | |^{2}} n \\ = \frac{a (x_{1} - x_{0}) + b (y_{1} - y_{0})}{a^{2} + b^{2}} [a b]^{T} \\ = \frac{a x_{1} + b y_{1} - a x_{0} - b y_{0}}{a^{2} + b^{2}} [a b]^{T} \\ = \frac{- c - a x_{0} - b y_{0}}{a^{2} + b^{2}} [a b]^{T} . \end{aligned}

| | {proj}_{n} v | | = \frac{| a x_{0} + b y_{0} + c |}{a^{2} + b^{2}} \sqrt{a^{2} + b^{2}} = \frac{| a x_{0} + b y_{0} + c |}{\sqrt{a^{2} + b^{2}}} .

🔗

(d)

Use Equation (23.3) to find the distance from the point $(3, 4)$ to the line $y = 2 x + 1 .$

Solution.

Here we have $P = (3, 4),$ and the equation of our line is $2 x - y + 1 = 0 .$ So $a = 2,$ $b = - 1,$ and $c = - 1 .$ Thus, the distance from $P$ to the line is

\frac{| a x_{0} + b y_{0} + c |}{\sqrt{a^{2} + b^{2}}} = \frac{| 2 (3) - (4) + 1 |}{\sqrt{4 + 1}} = \frac{3}{\sqrt{5}} .

🔗

Example 23.19.

Let $a,$ $b,$ and $c$ be scalars with $a \neq 0,$ and let

W = {a x + b y + c z = 0 : x, y, z \in R} .

🔗

(a)

Find two vectors that span $W,$ showing that $W$ is a subspace of $R^{3} .$ (In fact, $W$ is a plane through the origin in $R^{3} .$ )

Solution.

The coefficient matrix for the system $a x + b y + c z = 0$ is $[a b c]^{T} .$ The first column is a pivot column and the others are not. So $y$ and $z$ are free variables and

[x y z]^{T} = {[- \frac{b}{a} y - \frac{c}{a} z, y, z]}^{T} = y {[- \frac{b}{a} 1 0]}^{T} + z {[- \frac{c}{a} 0 1]}^{T} .

So $W = Span {{[- \frac{b}{a} 1 0]}^{T}, {[- \frac{c}{a} 0 1]}^{T}} .$

🔗

(b)

Find a vector $n$ that is orthogonal to the two vectors you found in part (a).

Solution.

If we let $n = [a b c]^{T},$ then

\begin{aligned} n \cdot {[- \frac{b}{a} 1 0]}^{T} & = - b + b = 0 \\ n \cdot {[- \frac{c}{a} 0 1]}^{T} & = - c + c = 0 . \end{aligned}

Thus, $[a b c]^{T}$ is orthogonal to both ${[- \frac{b}{a} 1 0]}^{T}$ and ${[- \frac{c}{a} 0 1]}^{T} .$

🔗

(c)

Explain why ${n}$ is a basis for $W^{⊥} .$

Solution.

Let $u = {[- \frac{b}{a} 1 0]}^{T}$ and $v = {[- \frac{c}{a} 0 1]}^{T} .$ Every vector in $W$ has the form $x u + y v$ for some scalars $x$ and $y,$ and

n \cdot (x u + y v) = x (n \cdot u) + y (n \cdot v) = 0 .

So $n \in W^{⊥} .$ Now we need to verify that ${n}$ spans $W^{⊥} .$ Let $w = [w_{1} w_{2} w_{3}]^{T}$ be in $W^{⊥} .$ Then $w \cdot z = 0$ for every $z \in W .$ In particular, $w \cdot u = 0$ or $- \frac{b}{a} w_{1} + w_{2} = 0,$ and $w \cdot v = 0$ or $- \frac{c}{a} w_{1} + w_{3} = 0 .$ Equivalently, we have $w_{2} = \frac{b}{a} w_{1}$ and $w_{3} = \frac{c}{a} w_{1} .$ So

\begin{aligned} w & = [w_{1} w_{2} w_{3}]^{T} \\ = {[w_{1} \frac{b}{a} w_{1} \frac{c}{a} w_{1}]}^{T} \\ = \frac{1}{a} [a b c]^{T} w_{1} \\ = \frac{w_{1}}{a} n . \end{aligned}

So every vector in $W^{⊥}$ is a multiple of $n,$ and ${n}$ spans $W^{⊥} .$ We conclude that ${n}$ is a basis for $W^{⊥} .$ Thus, the vector $[a b c]^{T}$ is a normal vector to the plane $a x + b y + c z = 0$ if $a \neq 0 .$ The same reasoning works if at least one of $a,$ $b,$ or $c$ is nonzero, so we can say in every case that $[a b c]^{T}$ is a normal vector to the plane $a x + b y + c z = 0 .$

🔗

Subsection Summary

The dot product of vectors $u = [u_{1} u_{2} \dots u_{n}]^{T}$ and $v = [v_{1} v_{2} \dots v_{n}]^{T}$ in $R^{n}$ is the scalar

$u \cdot v = u_{1} v_{1} + u_{2} v_{2} + \dots + u_{n} v_{n} = \sum_{i = 1}^{n} u_{i} v_{i} .$
The angle $θ$ between two nonzero vectors $u$ and $v$ in $R^{n}$ satisfies the equation

$\cos (θ) = \frac{u \cdot v}{| | u | | | | v | |}$

and $0 \leq θ \leq 180 .$
Two vectors $u$ and $v$ are orthogonal if $u \cdot v = 0 .$
The length, or norm, of the vector $u$ can be found as $| | u | | = \sqrt{u \cdot u} .$
The distance between the vectors $u$ and $v$ in $R^{n}$ is $| | u - v | |,$ which is the length of the difference $u - v .$
Let $u$ and $v$ be vectors in $R^{n} .$
- The orthogonal projection of $u$ onto $v$ is the vector
  
  ${proj}_{v} u = \frac{u \cdot v}{| | v | |^{2}} v .$
- The projection of $u$ perpendicular to $v$ is the vector
  
  ${proj}_{⊥ v} u = u - {proj}_{v} u .$
The orthogonal complement of the subspace $W$ of $R^{n}$ is the set

$W^{⊥} = {x \in R^{n} : x \cdot w = 0 for all w \in W} .$

🔗

Exercises Exercises

🔗

1.

For each of the following pairs of vectors, find $u \cdot v,$ calculate the angle between $u$ and $v,$ determine if $u$ and $v$ are orthogonal, find $| | u | |$ and $| | v | |,$ calculate the distance between $u$ and $v,$ and determine the orthogonal projection of $u$ onto $v .$

🔗

(a)

$u = [1 2]^{T},$ $v = [- 2 1]^{T}$

🔗

(b)

$u = [2 - 2]^{T},$ $v = [1 - 1]^{T}$

🔗

(c)

$u = [2 - 1]^{T},$ $v = [1 3]^{T}$

🔗

(d)

$u = [1 2 0]^{T},$ $v = [- 2 1 1]^{T}$

🔗

(e)

$u = [0 0 1]^{T},$ $v = [1 1 1]^{T}$

🔗

2.

Given $u = [2 1 2]^{T},$ find a vector $v$ so that the angle between $u$ and $v$ is $60^{\circ}$ and the orthogonal projection of $v$ onto $u$ has length 2.

🔗

3.

For which value(s) of $h$ is the angle between $[1 1 h]^{T}$ and $[1 2 1]^{T}$ equal to $60^{\circ} ?$

🔗

4.

Let $A = [a_{i j}]$ be a $k \times m$ matrix with rows $r_{1},$ $r_{2},$ $\dots,$ $r_{k},$ and let $B = [b_{1} b_{2} \dots b_{n}]$ be an $m \times n$ matrix with columns $b_{1},$ $b_{2},$ $\dots,$ $b_{n} .$ Show that we can write the matrix product $A B$ in a shorthand way as $A B = [r_{i} \cdot b_{j}] .$

🔗

5.

Let $A$ be an $m \times n,$ $u$ a vector in $R^{n}$ and $v$ a vector in $R^{m} .$ Show that

A u \cdot v = u \cdot A^{T} v .

🔗

6.

Let $u,$ $v,$ and $w$ be vectors in $R^{n} .$ Show that

🔗

(a)

$(u + v) \cdot w = (u \cdot w) + (v \cdot w)$ (the dot product distributes over vector addition)

🔗

(b)

If $c$ is an arbitrary constant, then $(c u) \cdot v = u \cdot (c v) = c (u \cdot v)$

🔗

7.

The Pythagorean Theorem states that if $a$ and $b$ are the lengths of the legs of a right triangle whose hypotenuse has length $c,$ then $a^{2} + b^{2} = c^{2} .$ If we think of the legs as defining vectors $u$ and $v,$ then the hypotenuse is the vector $u + v$ and we can restate the Pythagorean Theorem as

| | u + v | |^{2} = | | u | |^{2} + | | v | |^{2} .

In this exercise we show that this result holds in any dimension.

🔗

(a)

Let $u$ and $v$ be orthogonal vectors in $R^{n} .$ Show that $| | u + v | |^{2} = | | u | |^{2} + | | v | |^{2} .$

Hint.

Rewrite $| | u + v | |^{2}$ using the dot product.

🔗

(b)

Must it be true that if $u$ and $v$ are vectors in $R^{n}$ with $| | u + v | |^{2} = | | u | |^{2} + | | v | |^{2},$ then $u$ and $v$ are orthogonal? If not, provide a counterexample. If true, verify the statement.

🔗

8.

The Cauchy-Schwarz inequality,

\begin{matrix} (23.4) & | u \cdot v | \leq | | u | | ‖ | v | | \end{matrix}

for any vectors $u$ and $v$ in $R^{n},$ is considered one of the most important inequalities in mathematics. We verify the Cauchy-Schwarz inequality in this exercise. Let $u$ and $v$ be vectors in $R^{n} .$

🔗

(a)

Explain why the inequality (23.4) is true if either $u$ or $v$ is the zero vector. As a consequence, we assume that $u$ and $v$ are nonzero vectors for the remainder of this exercise.

🔗

(b)

Let $w = {proj}_{v} u = \frac{u \cdot v}{| | v | |^{2}} v$ and let $z = u - w .$ We know that $w \cdot z = 0 .$ Use Exercise 7 of this section to show that

| | u | |^{2} \geq | | w | |^{2} .

🔗

(c)

Now show that $| | w | |^{2} = \frac{| u \cdot v |^{2}}{| | v | |^{2}} .$

🔗

(d)

Combine parts (b) and (c) to explain why equation (23.4) is true.

🔗

9.

Let $u$ and $v$ be vectors in $R^{n} .$ Then $u,$ $v$ and $u + v$ form a triangle. We should then expect that the length of any one side of the triangle is smaller than the sum of the lengths of the other sides (since the straight line distance is the shortest distance between two points). In other words, we expect that

\begin{matrix} (23.5) & | | u + v | | \leq | | u | | + | | v | | . \end{matrix}

Equation (23.5) is called the Triangle Inequality. Use the Cauchy-Schwarz inequality (Exercise 8) to prove the triangle inequality.

🔗

10.

Let $W$ be a subspace of $R^{n}$ for some $n .$ Show that $W^{⊥}$ is also a subspace of $R^{n} .$

🔗

11.

Let $W$ be a subspace of $R^{n} .$ Show that $W$ is a subspace of $(W^{⊥})^{⊥} .$

🔗

12.

If $W$ is a subspace of $R^{n}$ for some $n,$ what is $W \cap W^{⊥} ?$ Verify your answer.

🔗

13.

Suppose $W_{1} \subseteq W_{2}$ are two subspaces of $R^{n} .$ Show that $W_{2}^{⊥} \subseteq W_{1}^{⊥} .$

🔗

14.

What are ${(R^{n})}^{⊥}$ and ${0}^{⊥}$ in $R^{n} ?$ Justify your answers.

🔗

15.

Label each of the following statements as True or False. Provide justification for your response.

🔗

(a) True/False.

The dot product is defined between any two vectors.

🔗

(b) True/False.

If $u$ and $v$ are vectors in $R^{n},$ then $u \cdot v$ is another vector in $R^{n} .$

🔗

(c) True/False.

If $u$ and $v$ are vectors in $R^{n},$ then $u \cdot v$ is always non-negative.

🔗

(d) True/False.

If $v$ is a vector in $R^{n},$ then $v \cdot v$ is never negative.

🔗

(e) True/False.

If $u$ and $v$ are vectors in $R^{n}$ and $u \cdot v = 0,$ then $u = v = 0 .$

🔗

(f) True/False.

If $v$ is a vector in $R^{n}$ and $v \cdot v = 0,$ then $v = 0 .$

🔗

(g) True/False.

The norm of the sum of vectors is the sum of the norms of the vectors.

🔗

(h) True/False.

If $u$ and $v$ are vectors in $R^{n},$ then ${proj}_{v} u$ is a vector in the same direction as $u .$

🔗

(i) True/False.

The only subspace $W$ of $R^{n}$ for which $W^{⊥} = {0}$ is $W = R^{n} .$

🔗

(j) True/False.

If a vector $u$ is orthogonal to $v_{1}$ and $v_{2},$ then $u$ is also orthogonal to $v_{1} + v_{2} .$

🔗

(k) True/False.

If a vector $u$ is orthogonal to $v_{1}$ and $v_{2},$ then $u$ is also orthogonal to all linear combinations of $v_{1}$ and $v_{2} .$

🔗

(l) True/False.

If $u \neq 0$ and $v$ are parallel, then the orthogonal projection of $v$ onto $u$ equals $v .$

🔗

(m) True/False.

If $u \neq 0$ and $v$ are orthogonal, then the orthogonal projection of $v$ onto $u$ equals $v .$

🔗

(n) True/False.

For any vector $v$ and $u \neq 0,$ $| | {proj}_{u} v | | \leq | | v | | .$

🔗

(o) True/False.

Given an $m \times n$ matrix, $\dim (Row A) + \dim (Row A)^{⊥} = n .$

🔗

(p) True/False.

If $A$ is a square matrix, then the columns of $A$ are orthogonal to the vectors in $Nul A .$

🔗

(q) True/False.

The vectors in the null space of an $m \times n$ matrix are orthogonal to vectors in the row space of $A .$

🔗

Subsection Project: Back-Face Culling

🔗

To identify hidden polygons in a surface, we will utilize a technique called back face culling. This involves identifying which polygons are back facing and which are front facing relative to the viewer's perspective. The first step is to assign a direction to each polygon in a surface.

🔗

Project Activity 23.9.

Consider the polygon $A B C D$ in Figure 23.20. Since a polygon is flat, every vector in the polygon is perpendicular to a fixed vector (which we call a normal vector to the polygon). A normal vector $n$ for the polygon $A B C D$ in Figure 23.20 is shown. In this activity we learn how to find a normal vector to a polygon.

Let $x = [x_{1} x_{2} x_{3}]^{T}$ and $y = [y_{1} y_{2} y_{3}]^{T}$ be two vectors in $R^{3} .$ If $x$ and $y$ are linearly independent, then $x$ and $y$ determine a polygon as shown in Figure 23.20. Our goal is to find a vector $n$ that is orthogonal to both $x$ and $y .$ Let $w = [w_{1} w_{2} w_{3}]^{T}$ be another vector in $R^{3}$ and let $C = [\begin{matrix} w^{T} \\ x^{T} \\ y^{T} \end{matrix}]$ be the matrix whose rows are $w,$ $x,$ and $y .$ Let $C_{i j}$ be the $i j$ th cofactor of $C,$ that is $C_{i j}$ is $(- 1)^{i + j}$ times the determinant of the submatrix of $C$ obtained by deleting the $i$ th row and $j$ th column of $C .$ Now define the vector $x \times y$ as follows:

x \times y = C_{11} e_{1} + C_{12} e_{2} + C_{13} e_{3} .

The vector $x \times y$ is called the cross product of the vectors $x$ and $y .$ (Note that the cross product is only defined for vectors in $R^{3} .$ ) We will show that $x \times y$ is orthogonal to both $x$ and $y,$ making $x \times y$ a normal vector to the polygon defined by $x$ and $y .$

Figure 23.20. Normal vector to a polygon.

🔗

(a)

Show that

x \times y = [\begin{matrix} x_{2} y_{3} - x_{3} y_{2} \\ x_{3} y_{1} - x_{1} y_{3} \\ x_{1} y_{2} - x_{2} y_{1} \end{matrix}] .

🔗

(b)

Use a cofactor expansion of $C$ along the first row and properties of the dot product to show that

det (C) = w \cdot (x \times y) .

🔗

(c)

Use the result of part (b) and properties of the determinant to calculate $x \cdot (x \times y)$ and $y \cdot (x \times y) .$ Explain why $x \times y$ is orthogonal to both $x$ and $y$ and is therefore a normal vector to the polygon determined by $x$ and $y .$

🔗

Project Activity 23.9 shows how we can find a normal vector to a parallelogram — take two vectors

x

and

y

between the vertices of the parallelogram and calculate their cross products. Such a normal vector can define a direction for the parallelogram. There is still a problem, however.

🔗

Project Activity 23.10.

Let $x = [x_{1} x_{2} x_{3}]^{T}$ and $y = [y_{1} y_{2} y_{3}]^{T}$ be any vectors in $R^{3} .$ There is a relationship between $x \times y$ and $y \times x .$ Find and verify this relationship.

🔗

Project Activity 23.10 shows that the cross product is anticommutative, so we get different directions if we switch the order in which we calculate the cross product. To fix a direction, we establish the convention that we always label the vertices of our parallelogram in the counterclockwise direction as shown in Figure 23.20. This way we always use

x

as the vector from vertex

A

to vertex

B

rather than the reverse. With this convention established, we can now define the direction of a parallelogram as the direction of its normal vector.

🔗

Once we have a normal vector established for each polygon, we can now determine which polygons are back-face and which are front-face. Figure 23.21 at left provides the gist of the idea, where we represent the polygons with line segments to illustrate. If the viewer's eye is at point

P

and views the figures, the normal vectors of the visible polygons point in a direction toward the viewer (front-face) and the normal vectors of the polygons hidden from the viewer point away from the viewer (back-face). What remains is to determine an effective computational way to identify the front and back facing polygons.

🔗
Figure 23.21. Left: Hidden faces. Right: Back face culling.

🔗

Project Activity 23.11.

Consider the situation as depicted at right in Figure 23.21. Assume that $A B$ and $R S$ are polygons (rendered one dimensionally here) with normal vectors $n$ at their centers as shown. The viewer's eye is at point $P$ and the viewer's line of vision to the centers $C_{A B}$ and $C_{R S}$ are indicated by the vectors $v .$ Each vector $v$ makes an angle $θ$ with the normal to the polygon.

🔗

(a)

What can be said about the angle $θ$ for a front-facing polygon? What must be true about $v \cdot n$ for a front-facing polygon? Why?

🔗

(b)

What can be said about the angle $θ$ for a back-facing polygon? What must be true about $v \cdot n$ for a back-facing polygon? Why?

🔗

(c)

The dot product then provides us with a simple computational tool for identifying back-facing polygons (assuming we have already calculated all of the normal vectors). We can then create an algorithm to cull the back-facing polygons. Assuming that we the viewpoint $P$ and the coordinates of the polygons of the surface, complete the pseudo-code for a back-face culling algorithm:

for all polygons on the surface do
  calculate the normal vector n using the ______ product for the current polygon
  calculate the center C of the current polygon
  calculate the viewing vector ______
    if ______ then
      render the current polygon
  end if
end for

🔗

As a final comment, back-face culling generally reduces the number of polygons to be rendered by half. This algorithm is not perfect and does not always do what we want it to do (e.g., it may not remove all parts of a polygon that we don't see), so there are other algorithms to use in concert with back-face culling to correctly render objects.

Technically,

u^{T} v

is a

1 \times 1

matrix and not a scalar, but we usually think of

1 \times 1

matrices as scalars.

We use the term orthogonal instead of perpendicular because we will be able to extend this idea to situations where we normally don't think of objects as being perpendicular.

Section 23 The Dot Product in Rn

Focus Questions

Subsection Application: Hidden Figures in Computer Graphics

Subsection Introduction

Definition 23.4.

Definition 23.5.

IMPORTANT NOTE.

Preview Activity 23.1.

(a)

(b)

Theorem 23.6.

(i)

(ii)

(iii)

(c)

(i)

(ii)

(iii)

Definition 23.7.

Subsection The Distance Between Vectors

Definition 23.8.

Subsection The Angle Between Two Vectors

Activity 23.2.

(a)

(b)

(i)

(ii)

(iii)

Definition 23.11.

(iv)

Activity 23.3.

(a)

(b)

Subsection Orthogonal Projections

Activity 23.4.

(a)

(b)

(c)

(d)

Definition 23.13.

Activity 23.5.

Subsection Orthogonal Complements

Definition 23.14.

Preview Activity 23.6.

Activity 23.7.

(a)

(b)

(c)

Theorem 23.15.

Theorem 23.16.

Proof.

Activity 23.8.

Subsection Examples

Example 23.17.

(a)

Solution.

(b)

Solution.

(c)

Solution.

(d)

Solution.

Example 23.19.

(a)

Solution.

(b)

Solution.

(c)

Solution.

Subsection Summary

Exercises Exercises

1.

(a)

(b)

(c)

(d)

(e)

2.

3.

4.

Section 23 The Dot Product in $R^{n}$