Inner Product Spaces

Section 35 Inner Product Spaces

Focus Questions

By the end of this section, you should be able to give precise and thorough answers to the questions listed below. You may want to keep these questions in mind to focus your thoughts as you complete the section.

What is an inner product space?
What is an orthogonal set in an inner product space?
What is an orthogonal basis for an inner product space?
How do we find the coordinate vector for a vector in an inner product space relative to an orthogonal basis for the space?
What is the projection of a vector orthogonal to a subspace and why are such orthogonal projections important?

🔗

Subsection Application: Fourier Series

🔗

In calculus, a Taylor polynomial for a function

f

is a polynomial approximation that fits

f

well around the center of the approximation. For this reason, Taylor polynomials are good local approximations, but they are not in general good global approximations. In particular, if a function

f

has periodic behavior it is impossible to model

f

well globally with polynomials that have infinite limits at infinity. For these kinds of functions, trigonometric polynomials are better choices. Trigonometric polynomials lead us to Fourier series, and we will investigate how inner products allow us to use trigonometric polynomials to model musical tones later in this section.

🔗

Subsection Introduction

🔗

In Section 23 we were introduced to inner products in

R^{n} .

The concept of an inner product can be extended to vector spaces, as we will see in this section. This will allow us to measure lengths and angles between vectors and define orthogonality in certain vector spaces.

🔗

Recall that an inner product on

R^{n}

assigns to each pair of vectors

u

and

v

the scalar

⟨ u, v ⟩ .

Thus, an inner product on

R^{n}

defines a mapping from

R^{n} \times R^{n}

R .

Recall also that an inner product on

R^{n}

is commutative, distributes over vector addition, and respects scalar multiplication, and the inner product of a vector in

R^{n}

by itself is always non-negative and is equal to 0 only when the vector is the zero vector. We will investigate this ideas in vector spaces in Preview Activity 35.1.

🔗

Preview Activity 35.1.

Consider the vector space $P_{2}$ of polynomials of degree less than or equal to $2$ with real coefficients. Define a mapping from $P_{2} \times P_{2}$ to $R$ by

⟨ p (t), q (t) ⟩ = \int_{0}^{1} p (t) q (t) d t .

🔗

(a)

Calculate $⟨ 1, t ⟩ .$

🔗

(b)

If $p (t)$ and $q (t)$ are in $P_{2},$ is it true that

⟨ p (t), q (t) ⟩ = ⟨ q (t), p (t) ⟩ ?

Verify your answer.

🔗

(c)

If $p (t),$ $q (t),$ and $r (t)$ are in $P_{2},$ is it true that

⟨ p (t) + q (t), r (t) ⟩ = ⟨ p (t), r (t) ⟩ + ⟨ q (t), r (t) ⟩ ?

Verify your answer.

🔗

(d)

If $p (t)$ and $q (t)$ are in $P_{2}$ and $c$ is a scalar, is it true that

⟨ c p (t), q (t) ⟩ = c ⟨ p (t), q (t) ⟩ ?

Verify your answer.

🔗

(e)

If $p (t)$ is in $P_{2},$ must it be the case that $⟨ p (t), p (t) ⟩ \geq 0 ?$ When is $⟨ p (t), p (t) ⟩ = 0 ?$

🔗

Subsection Inner Product Spaces

🔗

As we saw in Preview Activity 35.1, we can define a mapping from

P_{2} \times P_{2}

R

that has the same properties of inner products on

R^{n} .

So we can extend the definition of inner product to arbitrary vector spaces.

🔗

Definition 35.1.

An inner product $⟨, ⟩$ on a vector space $V$ is a mapping from $V \times V \to R$ satisfying

$⟨ u, v ⟩ = ⟨ v, u ⟩$ for all $u$ and $v$ in $V,$
$⟨ u + v, w ⟩ = ⟨ u, w ⟩ + ⟨ v, w ⟩$ for all $u,$ $v,$ and $w$ in $V,$
$⟨ c u, v ⟩ = c ⟨ u, v ⟩$ for all $u,$ $v$ in $V$ and all scalars $c,$
$⟨ u, u ⟩ \geq 0$ for all $u$ in $V$ and $⟨ u, u ⟩ = 0$ if and only if $u = 0 .$

An inner product space is a vector space on which an inner product is defined.

🔗

Activity 35.2.

Consider the mapping $⟨, ⟩$ from $P_{1} \times P_{1}$ to $R$ defined by

⟨ p (t), q (t) ⟩ = p^{'} (0) q^{'} (0) .

🔗

(a)

Show that this mapping satisfies the second property of an inner product.

🔗

(b)

Although this mapping satisfies the first three properties of an inner product, show that this mapping does not satisfy the fourth property and so is not an inner product.

🔗

Since inner products in vector spaces are defined in the same way as inner products in

R^{n},

they will satisfy the same properties. Some of these properties are summarized in the following theorem.

🔗

Theorem 35.2.

Let $⟨, ⟩$ be an inner product on $R^{n}$ and let $u, v,$ and $w$ be vectors in $R^{n}$ and $c$ a scalar. Then

$⟨ 0, v ⟩ = ⟨ v, 0 ⟩ = 0$
$⟨ u, c v ⟩ = c ⟨ u, v ⟩$
$⟨ u, v + w ⟩ = ⟨ u, v ⟩ + ⟨ u, w ⟩$
$⟨ u - v, w ⟩ = ⟨ u, w ⟩ - ⟨ v, w ⟩$

🔗

One special inner product is indicated in Preview Activity 35.1. Recall that

C [a, b]

is the vector space of continuous functions on the closed interval

[a, b] .

Let

⟨, ⟩

map from

C [a, b] \times C [a, b]

R

be defined by

⟨ f (x), g (x) ⟩ = \int_{a}^{b} f (x) g (x) d x .

🔗

The verification that this mapping is an inner product is left to the exercises.

🔗

Subsection The Length of a Vector

🔗

We can use inner products to define the length of any vector in an inner product space and the distance between two vectors in an inner product space. The idea comes right from the relationship between lengths of vectors in

R^{n}

and inner products on

R^{n}

(compare to Definition 23.5).

🔗

Definition 35.3.

Let $v$ be a vector in an inner product space. The length of $v$ is the real number

| | v | | = \sqrt{⟨ v, v ⟩} .

🔗

The length of a vector in a vector space is also called magnitude or norm. Just as with inner products on

R^{n}

we can use the notion of length to define unit vectors in inner product spaces (compare to Definition 23.7).

🔗

Definition 35.4.

A vector $v$ in an inner product space is a unit vector if $| | v | | = 1 .$

🔗

We can find a unit vector in the direction of a nonzero vector

v

in an inner product space by dividing by the norm of the vector. That is, the vector

\frac{v}{| | v | |}

is a unit vector in the direction of the vector

v,

provided that

v

is not zero.

🔗

We define the distance between vectors

u

and

v

in an inner product space in the same way we defined distance using the dot product (compare to Definition 23.8).

🔗

Definition 35.5.

Let $u$ and $v$ be vectors in an inner product space. The distance between $u$ and $v$ is the length of the difference $u - v$ or

| | u - v | | .

🔗

Activity 35.3.

The trace (see Definition 19.8) of an $n \times n$ matrix $A = [a_{i j}]$ is the sum of the diagonal entries of $A .$ That is,

trace (A) = a_{11} + a_{22} + \dots + a_{n n} .

If $A$ and $B$ are in the space $M_{n \times n}$ of $n \times n$ matrices with real entries, we define the mapping $⟨, ⟩$ from $M_{n \times n} \times M_{n \times n}$ to $R$ by

⟨ A, B ⟩ = trace (A B^{T}) .

This mapping is an inner product on the space $M_{n \times n}$ called the Frobenius inner product (details are in the exercises). Let $A = [\begin{array}{cc} 1 & 0 \\ 2 & 2 \end{array}]$ and $B = [\begin{array}{rc} - 3 & 4 \\ - 2 & 1 \end{array}]$ in $M_{2 \times 2} .$

🔗

(a)

Find the length of the vectors $A$ and $B$ using the Frobenius inner product.

🔗

(b)

Find the distance between $A$ and $B$ using the Frobenius inner product.

🔗

Subsection Orthogonality in Inner Product Spaces

🔗

We defined orthogonality in

R^{n}

using inner products in

R^{n}

(see Definition 23.11) and the angle between vectors. We can extend those ideas to any inner product space.

🔗

u

and

v

are nonzero vectors in an inner product space, then the angle

θ

between

u

and

v

is such that

\cos (θ) = \frac{⟨ u, v ⟩}{| | u | | | | v | |} .

🔗

and

0 \leq θ \leq π .

This angle is well-defined due to the Cauchy-Schwarz inequality

| ⟨ u, v ⟩ | \leq | | u | | | | v | |

whose proof is left to the exercises.

🔗

With the angle between vectors in mind, we can define orthogonal vectors in an inner product space.

🔗

Definition 35.6.

Vectors $u$ and $v$ in an inner product space are orthogonal if

⟨ u, v ⟩ = 0 .

🔗

Note that this defines the zero vector to be orthogonal to every vector.

🔗

Activity 35.4.

In this activity we use the Frobenius inner product (see Activity 35.3). Let $A = [\begin{array}{cc} 0 & 1 \\ 1 & 0 \end{array}]$ and $B = [\begin{array}{cc} 0 & 1 \\ 0 & 0 \end{array}]$ in $M_{2 \times 2} .$

🔗

(a)

Find a nonzero vector in $M_{2 \times 2}$ orthogonal to $A .$

🔗

(b)

Find the angle between $A$ and $B .$

🔗

Using orthogonality we can generalize the notions of orthogonal sets and bases, orthonormal bases and orthogonal complements we defined in

R^{n}

to all inner product spaces in a natural way.

🔗

Subsection Orthogonal and Orthonormal Bases in Inner Product Spaces

🔗

As we did with inner products in

R^{n},

we define an orthogonal set to be one in which all of the vectors in the set are orthogonal to each other (compare to Definition 24.1).

🔗

Definition 35.7.

A subset $S$ of an inner product space for which $⟨ u, v ⟩ = 0$ for all $u \neq v$ in $S$ is called an orthogonal set.

🔗

As in

R^{n},

an orthogonal set of nonzero vectors is always linearly independent. The proof is similar to that of Theorem 24.3 and is left to the exercises.

🔗

Theorem 35.8.

Let ${v_{1}, v_{2}, \dots, v_{m}}$ be a set of nonzero orthogonal vectors in an inner product space. Then the vectors $v_{1},$ $v_{2},$ $\dots,$ $v_{m}$ are linearly independent.

🔗

A basis that is also an orthogonal set is given a special name (compare to Definition 24.2).

🔗

Definition 35.9.

An orthogonal basis $B$ for a subspace $W$ of an inner product space is a basis of $W$ that is also an orthogonal set.

🔗

Using inner products in

R^{n},

we saw that the representation of a vector as a linear combination of vectors in an orthogonal basis was quite elegant. The same is true in any inner product space. To see this, let

B = {v_{1}, v_{2}, \dots, v_{m}}

be an orthogonal basis for a subspace

W

of an inner product space and let

x

be any vector in

W .

We know that

x = x_{1} v_{1} + x_{2} v_{2} + \dots + x_{m} v_{m}

🔗

for some scalars

x_{1},

x_{2},

\dots,

x_{m} .

1 \leq k \leq m,

then, using inner product properties and the orthogonality of the vectors

v_{i},

we have

⟨ v_{k}, x ⟩ = x_{1} ⟨ v_{k}, v_{1} ⟩ + x_{2} ⟨ v_{k}, v_{2} ⟩ + \dots + x_{m} ⟨ v_{k}, v_{m} ⟩ = x_{k} ⟨ v_{k}, v_{k} ⟩ .

🔗

x_{k} = \frac{⟨ x, v_{k} ⟩}{⟨ v_{k}, v_{k} ⟩} .

🔗

Thus, we can calculate each weight individually with two simple inner product calculations.

🔗

We summarize this discussion in the next theorem (compare to Theorem 24.4).

🔗

Theorem 35.10.

Let $B = {v_{1}, v_{2}, \dots, v_{m}}$ be an orthogonal basis for a subspace of an inner product space. Let $x$ be a vector in $W .$ Then

\begin{matrix} (35.1) & x = \frac{⟨ x, v_{1} ⟩}{⟨ v_{1}, v_{1} ⟩} v_{1} + \frac{⟨ x, v_{2} ⟩}{⟨ v_{2}, v_{2} ⟩} v_{2} + \dots + \frac{⟨ x, v_{m} ⟩}{⟨ v_{m}, v_{m} ⟩} v_{m} . \end{matrix}

🔗

Activity 35.5.

Let $p_{1} (t) = 1 - t,$ $p_{2} (t) = - 2 + 4 t + 4 t^{2},$ and $p_{3} (t) = 7 - 41 t + 40 t^{2}$ be vectors in the inner product space $P_{2}$ with inner product defined by $⟨ p (t), q (t) ⟩ = \int_{0}^{1} p (t) q (t) d t .$ Let $B = {p_{1} (t), p_{2} (t), p_{3} (t)} .$ You may assume that $B$ is an orthogonal basis for $P_{2} .$ Let $z (t) = 4 - 2 t^{2} .$ Find the weight $x_{3}$ so that $z (t) = x_{1} p_{1} (t) + x_{2} p_{2} (t) + x_{3} p_{3} (t) .$ Use technology as appropriate to evaluate any integrals.

🔗

The decomposition (35.1) is even simpler if

⟨ v_{k}, v_{k} ⟩ = 1

for each

k .

Recall that

⟨ v, v ⟩ = | | v | |^{2},

🔗

so the condition

⟨ v, v ⟩ = 1

implies that the vector

v

has norm 1. As with inner products in

R^{n},

an orthogonal basis with this additional condition is given a special name (compare to Definition 24.5).

🔗

Definition 35.11.

An orthonormal basis $B = {v_{1}, v_{2}, \dots, v_{m}}$ for a subspace $W$ of an inner product space is an orthogonal basis such that $| | v_{k} | | = 1$ for $1 \leq k \leq m .$

🔗

B = {v_{1}, v_{2}, \dots, v_{m}}

is an orthonormal basis for a subspace

W

of an inner product space and

x

is a vector in

W,

then (35.1) becomes

\begin{matrix} (35.2) & x = ⟨ x, v_{1} ⟩ v_{1} + ⟨ x, v_{2} ⟩ v_{2} + \dots + ⟨ x, v_{m} ⟩ v_{m} . \end{matrix}

🔗

Recall that we can construct an orthonormal basis from an orthogonal basis by dividing each basis vector by its magnitude.

🔗

Subsection Orthogonal Projections onto Subspaces

🔗

In Section 25 we saw how to project a vector

v

R^{n}

onto a subspace

W

R^{n} .

The same process works for vectors in any inner product space.

🔗

Definition 35.12.

Let $W$ be a subspace of an inner product space $V$ and let $B = {w_{1}, w_{2}, \dots, w_{m}}$ be an orthogonal basis for $W .$ For a vector $v$ in $V,$ the orthogonal projection of $v$ onto $W$ is the vector

{proj}_{W} v = \frac{⟨ v, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} w_{1} + \frac{⟨ v, w_{2} ⟩}{⟨ w_{2}, w_{2} ⟩} w_{2} + \dots + \frac{⟨ v, w_{m} ⟩}{⟨ w_{m}, w_{m} ⟩} w_{m} .

The projection of $v$ orthogonal to $W$ is the vector

{proj}_{W^{⊥}} v = v - {proj}_{W} v .

🔗

The notation

{proj}_{W^{⊥}} v

indicates that we expect this vector to be orthogonal to every vector in

W .

🔗

Activity 35.6.

In Section 25 we showed that in the inner product space $R^{n}$ using the dot product as inner product, if $W$ is a subspace of $R^{n}$ and $v$ is in $R^{n},$ then ${proj}_{W^{⊥}} v$ is orthogonal to every vector in $W .$ In this activity we verify that same fact in an inner product space. That is, assume that $B = {w_{1}, w_{2}, \dots, w_{m}}$ is an orthogonal basis for a subspace $W$ of an inner product space $V$ and $v$ is a vector in $V .$ Follow the indicated steps to show that ${proj}_{W^{⊥}} v$ is orthogonal to every vector in $B .$

🔗

(a)

Let $z$ be the projection of $v$ onto $W .$ Write $z$ in terms of the basis vectors in $B .$

🔗

(b)

The vector $v - z$ is the projection of $v$ orthogonal to $W .$ Let $k$ be between $1$ and $m .$ Use the result of part (a) to show that $v - z$ is orthogonal to $w_{k} .$ Exercise 16 then shows that $v - z$ is orthogonal to every vector in $W .$

🔗

Activity 35.6 shows that the vector

v - w

is orthogonal to vector for

W .

So, in fact,

{proj}_{W^{⊥}} v

is the projection of

v

onto the orthogonal complement of

W,

which will be defined shortly.

🔗

Subsection Best Approximations in Inner Product Spaces

🔗

We have seen, e.g., linear regression to fit a line to a set of data, that we often want to find a vector in a subspace that “best” approximates a given vector in a vector space. As in

R^{n},

the projection of a vector onto a subspace has this important property. That is,

{proj}_{W} v

is the vector in

W

closest to

v

and therefore the best approximation of

v

by a vector in

W .

To see that this is true in any inner product space, we first need a generalization of the Pythagorean Theorem that holds in inner product spaces.

🔗

Theorem 35.13. Generalized Pythagorean Theorem.

Let $u$ and $v$ be orthogonal vectors in an inner product space $V .$ Then

| | u - v | |^{2} = | | u | |^{2} + | | v | |^{2} .

🔗

Proof.

Let $u$ and $v$ be orthogonal vectors in an inner product space $V .$ Then

\begin{aligned} | | u - v | |^{2} & = ⟨ u - v, u - v ⟩ \\ = ⟨ u, u ⟩ - 2 ⟨ u, v ⟩ + ⟨ v, v ⟩ \\ = ⟨ u, u ⟩ - 2 (0) + ⟨ v, v ⟩ \\ = | | u | |^{2} + | | v | |^{2} . \end{aligned}

🔗

Note that replacing

v

with

- v

in the theorem also shows that

| | u + v | |^{2} = | | u | |^{2} + | | v | |^{2}

u

and

v

are orthogonal.

🔗

Now we will prove that the projection of a vector

u

onto a subspace

W

of an inner product space

V

is the best approximation in

W

to the vector

u .

🔗

Theorem 35.14.

Let $W$ be a subspace of an inner product space $V$ and let $u$ be a vector in $V .$ Then

| | u - {proj}_{W} u | | < | | u - x | |

for every vector $x$ in $W$ different from ${proj}_{W} u .$

🔗

Proof.

Let $W$ be a subspace of an inner product space $V$ and let $u$ be a vector in $V .$ Let $x$ be a vector in $W .$ Now

u - x = (u - {proj}_{W} u) + ({proj}_{W} u - x) .

Since both ${proj}_{W} u$ and $x$ are in $W,$ we know that ${proj}_{W} u - x$ is in $W .$ Since ${proj}_{W^{⊥}} u = u - {proj}_{W} u$ is orthogonal to every vector in $W,$ we have that $u - {proj}_{W} u$ is orthogonal to ${proj}_{W} u - x .$ We can now use the Generalized Pythagorean Theorem to conclude that

| | u - x | |^{2} = | | u - {proj}_{W} u | |^{2} + | | {proj}_{W} u - x | |^{2} .

Since $x \neq {proj}_{W} u,$ it follows that $| | {proj}_{W} u - x | |^{2} > 0$ and

| | u - x | |^{2} > | | u - {proj}_{W} u | |^{2} .

Since norms are nonnegative, we can conclude that $| | u - {proj}_{W} u | | < | | u - x | |$ as desired.

🔗

Theorem 35.14 shows that the distance from

{proj}_{W} v

v

is less than the distance from any other vector in

W

v .

{proj}_{W} v

is the best approximation to

v

of all the vectors in

W .

🔗

R^{n}

using the dot product as inner product, if

v = [v_{1} v_{2} v_{3} \dots v_{n}]^{T}

and

{proj}_{W} v = [w_{1} w_{2} w_{3} \dots w_{n}]^{T},

then the square of the error in approximating

v

{proj}_{W} v

is given by

| | v - {proj}_{W} v | |^{2} = \sum_{i = 1}^{n} (v_{i} - w_{i})^{2} .

🔗

{proj}_{W} v

minimizes this sum of squares over all vectors in

W .

As a result, we call

{proj}_{W} v

the least squares approximation to

v .

🔗

Activity 35.7.

The set $B = {1, t - \frac{1}{2}, t^{3} - \frac{9}{10} t + \frac{1}{5}}$ is an orthogonal basis for a subspace $W$ of the inner product space $P_{3}$ using the inner product $⟨ p (t), q (t) ⟩ = \int_{0}^{1} p (t) q (t) d t .$ Find the polynomial in $W$ that is closest to the polynomial $r (t) = t^{2}$ and give a numeric estimate of how good this approximation is.

🔗

Subsection Orthogonal Complements

🔗

If we have a set of vectors

S

in an inner product space

V,

we can define the orthogonal complement of

S

as we did in

R^{n}

(see Definition 23.14).

🔗

Definition 35.15.

The orthogonal complement of a subset $S$ of an inner product space $V$ is the set

S^{⊥} = {v \in V : ⟨ v, u ⟩ = 0 for all u \in S} .

🔗

As we saw in

R^{n},

to show that a vector is in the orthogonal complement of a subspace, it is enough to show that the vector is orthogonal to every vector in a basis for that subspace. The same is true in any inner product space. The proof is left to the exercises.

🔗

Theorem 35.16.

Let $B = {w_{1}, w_{2}, \dots, w_{m}}$ be a basis for a subspace $W$ of an inner product space $V .$ A vector $v$ in $V$ is orthogonal to every vector in $W$ if and only if $v$ is orthogonal to every vector in $B .$

🔗

Activity 35.8.

Consider $P_{2}$ with the inner product $⟨ p (t), q (t) ⟩ = \int_{0}^{1} p (t) q (t) d t .$

🔗

(a)

Find $⟨ p (t), 1 - t ⟩$ where $p (t) = a + b t + c t^{2}$ is in $P_{2} .$

🔗

(b)

Describe as best you can the orthogonal complement of $Span {1 - t}$ in $P_{2} .$ Is $p (t) = 1 - 2 t - 2 t^{2}$ in this orthogonal complement? Is $p (t) = 1 + t - t^{2} ?$

🔗

As was the case in

R^{n},

give a subspace

W

of an inner product space

V,

any vector in

V

can be written uniquely as a sum of a vector in

W

and a vector in

W^{⊥} .

🔗

Activity 35.9.

Let $V$ be an inner product space of dimension $n,$ and let $W$ be a subspace of $V .$ Let $x$ be any vector in $V .$ We will demonstrate that $x$ can be written uniquely as a sum of a vector in $W$ and a vector in $W^{⊥} .$

🔗

(a)

Explain why ${proj}_{W} x$ is in $W .$

🔗

(b)

Explain why ${proj}_{W^{⊥}} x$ is in $W^{⊥} .$

🔗

(c)

Explain why $x$ can be written as a sum of vectors, one in $W$ and one in $W^{⊥} .$

🔗

(d)

Now we demonstrate the uniqueness of this decomposition. Suppose $x = w + w_{1}$ and $x = u + u_{1},$ where $w$ and $u$ are in $W$ and $w_{1}$ and $u_{1}$ are in $W^{⊥} .$ Show that $w = u$ and $w_{1} = u_{1},$ so that the representation of $x$ as a sum of a vector in $W$ and a vector in $W^{⊥}$ is unique. (Hint: What is $W \cap W^{⊥} ?$ )

🔗

We summarize the result of Activity 35.9.

🔗

Theorem 35.17.

Let $V$ be a finite dimensional inner product space, and let $W$ be a subspace of $V .$ Any vector in $V$ can be written in a unique way as a sum of a vector in $W$ and a vector in $W^{⊥} .$

🔗

Theorem 35.17 is useful in many applications. For example, to compress an image using wavelets, we store the image as a collection of data, then rewrite the data using a succession of subspaces and their orthogonal complements. This new representation allows us to visualize the data in a way that compression is possible.

🔗

Subsection Examples

🔗

What follows are worked examples that use the concepts from this section.

🔗

Example 35.18.

Let $V = P_{3}$ be the inner product space with inner product

⟨ p (t), q (t) ⟩ = \int_{- 1}^{1} p (t) q (t) d t .

Let $p_{1} (t) = 1 + t,$ $p_{2} (t) = 1 - 3 t,$ $p_{3} (t) = 3 t - 5 t^{3},$ and $p_{4} (t) = 1 - 3 t^{2} .$

🔗

(a)

Show that the set $B = {p_{1} (t), p_{2} (t), p_{3} (t), p_{4} (t)}$ is an orthogonal basis for $V .$

Solution.

All calculations are done by hand or with a computer algebra system, so we leave those details to the reader.

If we show that the set $B$ is an orthogonal set, then Theorem 35.8 shows that $B$ is linearly independent. Since $\dim (P_{3}) = 4,$ the linearly independent set $B$ that contains four vectors must be a basis for $P_{3} .$ To determine if the set $B$ is an orthogonal set, we must calculate the inner products of pairs of distinct vectors in $B .$ Since $⟨ 1 + t, 1 - 3 t ⟩ = 0,$ $⟨ 1 + t, 3 t - 5 t^{3} ⟩ = 0,$ $⟨ 1 + t, 1 - 3 t^{2} ⟩ = 0,$ $⟨ 1 - 3 t, 3 t - 5 t^{3} ⟩ = 0,$ $⟨ 1 - 3 t, 1 - 3 t^{2} ⟩ = 0,$ and $⟨ 3 t - 5 t^{3}, 1 - 3 t^{2} ⟩ = 0,$ we conclude that $B$ is an orthogonal basis for $P_{3} .$

🔗

(b)

Use 35.10 to write the polynomial $q (t) = t^{2} + t^{3}$ as a linear combination of the basis vectors in $B .$

Solution.

All calculations are done by hand or with a computer algebra system, so we leave those details to the reader.

We can write the polynomial $q (t) = 1 + t + t^{2} + t^{3}$ as a linear combination of the basis vectors in $B$ as follows:

\begin{aligned} q (t) = \frac{⟨ q (t), p_{1} (t) ⟩}{⟨ p_{1} (t), p_{1} (t) ⟩} p_{1} (t) & + \frac{⟨ q (t), p_{2} (t) ⟩}{⟨ p_{2} (t), p_{2} (t) ⟩} p_{2} (t) \\ + \frac{⟨ q (t), p_{3} (t) ⟩}{⟨ p_{3} (t), p_{3} (t) ⟩} p_{3} (t) + \frac{⟨ q (t), p_{4} (t) ⟩}{⟨ p_{4} (t), p_{4} (t) ⟩} p_{4} (t) . \end{aligned}

Now

\begin{array}{ccc} ⟨ q (t), p_{1} (t) ⟩ = \frac{16}{15}, & ⟨ q (t), p_{2} (t) ⟩ = - \frac{8}{15}, & ⟨ q (t), p_{3} (t) ⟩ = - \frac{8}{35}, \\ ⟨ q (t), p_{4} (t) ⟩ = - \frac{8}{15}, & ⟨ p_{1} (t), p_{1} (t) ⟩ = \frac{8}{3}, & ⟨ p_{2} (t), p_{2} (t) ⟩ = 8, \\ ⟨ p_{3} (t), p_{3} (t) ⟩ = \frac{8}{7}, & ⟨ p_{4} (t), p_{4} (t) ⟩ = \frac{8}{5} \end{array}

\begin{aligned} q (t) & = \frac{\frac{16}{15}}{\frac{8}{3}} p_{1} (t) - \frac{\frac{8}{15}}{8} p_{2} (t) - \frac{\frac{8}{35}}{\frac{8}{7}} p_{3} (t) - \frac{\frac{8}{15}}{\frac{8}{5}} p_{4} (t) \\ \frac{2}{5} p_{1} (t) - \frac{1}{15} p_{2} (t) - \frac{1}{5} p_{3} (t) - \frac{1}{3} p_{4} (t) . \end{aligned}

🔗

Example 35.19.

Let $V$ be the inner product space $R^{4}$ with inner product defined by

⟨ [u_{1} u_{2} u_{3} u_{4}]^{T}, [v_{1} v_{2} v_{3} v_{4}]^{T} ⟩ = u_{1} v_{1} + 2 u_{2} v_{2} + 3 u_{3} v_{3} + 4 u_{4} v_{4} .

🔗

(a)

Let $W$ be the plane spanned by $[- 1 1 0 1]^{T}$ and $[6 1 7 1]^{T}$ in $V .$ Find the vector in $W$ that is closest to the vector $[2 0 - 1 3]^{T} .$ Exactly how close is your best approximation to the vector $[2 0 - 1 3]^{T} ?$

Solution.

The vector we're looking for is the projection of $[2 0 - 1 3]^{T}$ onto the plane. A spanning set for the plane is $B = {[- 1 1 0 1]^{T}, [6 1 7 1]^{T}} .$ Neither vector in $B$ is a scalar multiple of the other, so $B$ is a basis for the plane. Since

⟨ [- 1 1 0 1]^{T}, [6 1 7 1]^{T} ⟩ = - 6 + 2 + 0 + 4 = 0,

the set $B$ is an orthogonal basis for the plane. The projection of the vector $v = [2 0 - 1 3]^{T}$ onto the plane spanned by $w_{1} = [- 1 1 0 1]^{T}$ and $w_{2} = [6 1 7 1]^{T}$ is given by

\begin{aligned} \frac{⟨ v, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} w_{1} + \frac{⟨ v, w_{2} ⟩}{⟨ w_{2}, w_{2} ⟩} w_{2} & = \frac{10}{7} [- 1 1 0 1]^{T} + \frac{3}{189} [6 1 7 1]^{T} \\ = \frac{1}{189} [- 252 273 21 273] T \\ = [- \frac{4}{3} \frac{13}{9} \frac{1}{9} \frac{13}{9}] . \end{aligned}

To measure how close close $[- \frac{4}{3} \frac{13}{9} \frac{1}{9} \frac{13}{9}]$ is to $[2 0 - 1 3]^{T},$ we calculate

\begin{aligned} | | [- \frac{4}{3} \frac{13}{9} \frac{1}{9} \frac{13}{9}] - [2 0 - 1 3]^{T} | | & = | | [- \frac{10}{3} \frac{13}{9} \frac{10}{9} - \frac{14}{9}] | | \\ = \sqrt{\frac{100}{9} + \frac{338}{81} + \frac{300}{81} + \frac{784}{81}} \\ = \frac{1}{9} \sqrt{2322} \\ \approx 5.35 . \end{aligned}

🔗

(b)

Express the vector $[2 0 - 1 3]^{T}$ as the sum of a vector in $W$ and a vector orthogonal to $W .$

If $v = [2 0 - 1 3]^{T},$ then ${proj}_{W} v$ is in $W$ and

{proj}_{W^{⊥}} v = v - {proj}_{W} v = [- \frac{10}{3} \frac{13}{9} \frac{10}{9} - \frac{14}{9}]

is in $W^{⊥},$ and $v = {proj}_{W} v + {proj}_{W^{⊥}} v .$

🔗

Subsection Summary

An inner product $⟨, ⟩$ on a vector space $V$ is a mapping from $V \times V \to R$ satisfying
1. $⟨ u, v ⟩ = ⟨ v, u ⟩$ for all $u$ and $v$ in $V,$
2. $⟨ u + v, w ⟩ = ⟨ u, w ⟩ + ⟨ v, w ⟩$ for all $u,$ $v,$ and $w$ in $V,$
3. $⟨ c u, v ⟩ = c ⟨ u, v ⟩$ for all $u,$ $v$ in $V$ and $c \in R,$
4. $⟨ u, u ⟩ \geq 0$ for all $u$ in $V$ and $⟨ u, u ⟩ = 0$ if and only if $u = 0 .$
An inner product space is a pair $V,$ $⟨, ⟩$ where $V$ is a vector space and $⟨, ⟩$ is an inner product on $V .$
The length of a vector $v$ in an inner product space $V$ is defined to be the real number $| | v | | = \sqrt{⟨ v, v ⟩} .$
The distance between two vectors $u$ and $v$ in an inner product space $V$ is the scalar $| | u - v | | .$
The angle $θ$ between two vectors $u$ and $v$ is the angle which satisfies $0 \leq θ \leq π$ and

$\cos (θ) = \frac{⟨ u, v ⟩}{| | u | | | | v | |} .$
Two vectors $u$ and $v$ in an inner product space $V$ are orthogonal if $⟨ u, v ⟩ = 0 .$
A subset $S$ of an inner product space is an orthogonal set if $⟨ u, v ⟩ = 0$ for all $u \neq v$ in $S .$
A basis for a subspace of an inner product space is an orthogonal basis if the basis is also an orthogonal set.
Let $B = {v_{1}, v_{2}, \dots, v_{m}}$ be an orthogonal basis for a subspace of an inner product space $V .$ Let $x$ be a vector in $W .$ Then

$x = \sum_{i = 1}^{m} c_{i} v_{i},$

where

$c_{i} = \frac{⟨ x, v_{i} ⟩}{⟨ v_{i}, v_{i} ⟩}$

for each $i .$
An orthogonal basis $B = {v_{1}, v_{2}, \dots, v_{m}}$ for a subspace $W$ of an inner product space $V$ is an orthonormal basis if $| | v_{k} | | = 1$ for each $k$ from 1 to $m .$
If $B = {w_{1}, w_{2}, \dots, w_{m}}$ is an orthogonal basis for $V$ and $x \in V,$ then

$[x]_{B} = [\begin{matrix} \frac{⟨ x, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} \\ \frac{⟨ x, w_{2} ⟩}{⟨ w_{2}, w_{2} ⟩} \\ ⋮ \\ \frac{⟨ x, w_{m} ⟩}{⟨ w_{m}, w_{m} ⟩} \end{matrix}] .$
The projection of the vector $v$ in an inner product space $V$ onto a subspace $W$ of $V$ is the vector

${proj}_{W} v = \frac{⟨ v, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} w_{1} + \frac{⟨ v, w_{2} ⟩}{⟨ w_{2}, w_{2} ⟩} w_{2} + \dots + \frac{⟨ v, w_{m} ⟩}{⟨ w_{m}, w_{m} ⟩} w_{m},$

where ${w_{1}, w_{2}, \dots, w_{m}}$ is an orthogonal basis of $W .$ Projections are important in that ${proj}_{W} v$ is the best approximation of the vector $v$ by a vector in $W$ in the least squares sense.
With $W$ as in (a), the projection of $v$ orthogonal to $W$ is the vector

${proj}_{W^{⊥}} v = v - {proj}_{W} v .$

The norm of ${proj}_{W^{⊥}} v$ provides a measure of how well ${proj}_{W} v$ approximates the vector $v .$
The orthogonal complement of a subset $S$ of an inner product space $V$ is the set

$S^{⊥} = {v \in V : ⟨ v, u ⟩ = 0 for all u \in S} .$

🔗

Exercises Exercises

🔗

1.

Let $C [a, b]$ be the set of all continuous real valued functions on the interval $[a, b] .$ If $f$ is in $C [a, b],$ we can extend $f$ to a continuous function from $R$ to $R$ by letting $F$ be the function defined by

F (x) = {\begin{cases} f (a) & if x < a \\ f (x) & if a \leq x \leq b \\ f (b) & if b < x \end{cases} .

In this way we can view $C [a, b]$ as a subset of $F,$ the vector space of all functions from $R$ to $R .$ Verify that $C [a, b]$ is a vector space.

Hint.

Use properties of continuous functions.

🔗

2.

Use the definition of an inner product to determine which of the following defines an inner product on the indicated space. Verify your answers.

🔗

(a)

$⟨ u, v ⟩ = u_{1} v_{1} - u_{2} v_{1} - u_{1} v_{2} + 3 u_{2} v_{2}$ for $u = [u_{1} u_{2}]^{T}$ and $v = [v_{1} v_{2}]^{T}$ in $R^{2}$

🔗

(b)

$⟨ f, g ⟩ = \int_{a}^{b} f (x) g (x) d x$ for $f, g \in C [a, b]$ (where $C [a, b]$ is the vector space of all continuous functions on the interval $[a, b]$ )

🔗

(c)

$⟨ f, g ⟩ = f^{'} (0) g^{'} (0)$ for $f, g \in D (- 1, 1)$ (where $D (a, b)$ is the vector space of all differentiable functions on the interval $(a, b)$ )

🔗

(d)

$⟨ u, v ⟩ = (A u) \cdot (A v)$ for $u, v \in R^{n}$ and $A$ an invertible $n \times n$ matrix

🔗

3.

We can sometimes visualize an inner product in $R^{2}$ or $R^{3}$ (or other spaces) by describing the unit circle $S^{1},$ where

S^{1} = {v \in V : | | v | | = 1}

in that inner product space. For example, in the inner product space $R^{2}$ with the dot product as inner product, the unit circle is just our standard unit circle. Inner products, however, can distort this familiar picture of the unit circle. Describe the points on the unit circle $S^{1}$ in the inner product space $R^{2}$ with inner product $⟨ [u_{1} u_{2}], [v_{1} v_{2}] ⟩ = 2 u_{1} v_{1} + 3 u_{2} v_{2}$ using the following steps.

🔗

(a)

Let $x = [x y] \in R^{2} .$ Set up an equation in $x$ and $y$ that is equivalent to the vector equation $| | x | | = 1 .$

🔗

(b)

Describe the graph of the equation you found in $R^{2} .$ It should have a familiar form. Draw a picture to illustrate. What do you think of calling this graph a “circle”?

🔗

4.

Define $⟨, ⟩$ on $R^{2}$ by $⟨ [u_{1} u_{2}]^{T}, [v_{1} v_{2}]^{T} ⟩ = 4 u_{1} v_{1} + 2 u_{2} v_{2} .$

🔗

(a)

Show that $⟨, ⟩$ is an inner product.

🔗

(b)

The inner product $⟨, ⟩$ can be represented as a matrix transformation $⟨ u, v ⟩ = u^{T} A v,$ where $u$ and $v$ are written as column vectors. Find a matrix $A$ that represents this inner product.

🔗

5.

This exercise is a generalization of Exercise 4. Define $⟨, ⟩$ on $R^{n}$ by

⟨ [u_{1} u_{2} \dots u_{n}]^{T}, [v_{1} v_{2} \dots v_{n}]^{T} ⟩ = a_{1} u_{1} v_{1} + a_{2} u_{2} v_{2} + \dots + a_{n} u_{n} v_{n}

for some positive scalars $a_{1},$ $a_{2},$ $\dots,$ $a_{n} .$

🔗

(a)

Show that $⟨, ⟩$ is an inner product.

🔗

(b)

🔗

6.

Is the sum of two inner products on an inner product space $V$ an inner product on $V ?$ If yes, prove it. If no, provide a counterexample. (By the sum of inner products we mean a function $⟨, ⟩$ satisfying

⟨ u, v ⟩ = ⟨ u, v ⟩_{1} + ⟨ u, v ⟩_{2}

for all $u$ and $v$ in $V,$ where $⟨, ⟩_{1}$ and $⟨, ⟩_{2}$ are inner products on $V .$ )

🔗

7.

🔗

(a)

Does $⟨ u, v ⟩ = u^{T} A v$ define an inner product on $R^{n}$ for every $n \times n$ matrix $A ?$ Verify your answer.

🔗

(b)

If your answer to part (a) is no, are there any types of matrices for which $⟨ u, v ⟩ = u^{T} A v$ defines an inner product?

Hint.

See Exercise 4 and Exercise 5.

🔗

8.

The trace of an $n \times n$ matrix $A = [a_{i j}]$ has some useful properties.

🔗

(a)

Show that $trace (A + B) = trace (A) + trace (B)$ for any $n \times n$ matrices $A$ and $B .$

🔗

(b)

Show that $trace (c A) = c trace (A)$ for any $n \times n$ matrix $A$ and any scalar $c .$

🔗

(c)

Show that $trace (A^{T}) = trace (A)$ for any $n \times n$ matrix.

🔗

9.

Let $V$ be an inner product space and $u, v$ be two vectors in $V .$

🔗

(a)

Check that if $v = 0,$ the Cauchy-Schwarz inequality

| ⟨ u, v ⟩ | \leq | | u | | | | v | |

holds.

Hint.

Evaluate each side of the inequality.

🔗

(b)

Assume $v \neq 0 .$ Let $λ = ⟨ u, v ⟩ / | | v | |^{2}$ and $w = u - λ v .$ Use the fact that $| | w | |^{2} \geq 0$ to conclude the Cauchy-Schwarz inequality in this case.

Hint.

Write $| | w | |^{2}$ as an inner product and expand.

🔗

10.

The Frobenius inner product is defined as

⟨ A, B ⟩ = trace (A B^{T}) .

for $n \times n$ matrices $A$ and $B .$ Verify that $⟨ A, B ⟩$ defines an inner product on $M_{n \times n} .$

🔗

11.

Let $A = [a_{i j}]$ and $B = [b_{i j}]$ be two $n \times n$ matrices.

🔗

(a)

Show that if $n = 2,$ then the Frobenius inner product (see Exercise 10) of $A$ and $B$ is

⟨ A, B ⟩ = a_{11} b_{11} + a_{12} b_{12} + a_{21} b_{21} + a_{22} b_{22} .

Hint.

Expand the inner product.

🔗

(b)

Extend part (a) to the general case. That is, show that for an arbitrary $n,$

⟨ A, B ⟩ = \sum_{i = 1}^{n} \sum_{j = 1}^{n} a_{i j} b_{i j} .

Hint.

Expand the inner product.

🔗

(c)

Compare the Frobenius inner product to the scalar product of two vectors.

Hint.

Convert $A$ and $B$ to vectors in $R^{n^{2}}$ whose entries are the entries in the first row followed by the entries in the second row and so on.

🔗

12.

Let $B = {[1 1 1]^{T}, [1 - 1 0]^{T}}$ and let $W = Span B$ in $R^{3} .$

🔗

(a)

Show that $B$ is an orthogonal basis for $W,$ using the dot product as inner product.

🔗

(b)

Explain why the vector $v = [0 2 2]^{T}$ is not in $W .$

🔗

(c)

Find the vector in $W$ that is closest to $v .$ How close is this vector to $v ?$

🔗

13.

Let $R^{3}$ be the inner product space with inner product

⟨ [u_{1} u_{2} u_{3}]^{T}, [v_{1} v_{2} v_{3}]^{T} ⟩ = u_{1} v_{1} + 2 u_{2} v_{2} + u_{3} v_{3} .

Let $B = {[1 1 1]^{T}, [1 - 1 1]^{T}}$ and let $W = Span B$ in $R^{3} .$

🔗

(a)

Show that $B$ is an orthogonal basis for $W,$ using the given inner product.

🔗

(b)

Explain why the vector $v = [0 2 2]^{T}$ is not in $W .$

Hint.

Try to write $v$ in terms of the basis vectors for $W .$

🔗

(c)

Find the vector in $W$ that is closest to $v .$ How close is this vector to $v ?$

🔗

14.

Let $P_{2}$ be the inner product space with inner product

⟨ p (t), q (t) ⟩ = \int_{0}^{1} p (t) q (t) d t .

Let $B = {1, 1 - 2 t}$ and let $W = Span B$ in $P_{2} .$

🔗

(a)

Show that $B$ is an orthogonal basis for $W,$ using the given inner product.

🔗

(b)

Explain why the polynomial $q (t) = t^{2}$ is not in $W .$

🔗

(c)

Find the vector in $W$ that is closest to $q (t) .$ How close is this vector to $q (t) ?$

🔗

15.

Prove the remaining properties of Theorem 35.2. That is, if $⟨, ⟩$ is an inner product on a vector space $V$ and $u, v,$ and $w$ are vectors in $V$ and $c$ is any scalar, then

🔗

(a)

$⟨ 0, v ⟩ = ⟨ v, 0 ⟩ = 0$

Hint.

Use the fact that $0 = 0 + 0 .$

🔗

(b)

$⟨ u, c v ⟩ = c ⟨ u, v ⟩$

Hint.

Use the fact that $⟨ u, v ⟩ = ⟨ v, u ⟩ .$

🔗

(c)

$⟨ v + w, u ⟩ = ⟨ v, u ⟩ + ⟨ w, u ⟩$

Hint.

Same hint as part (b).

🔗

(d)

$⟨ u - v, w ⟩ = ⟨ w, u - v ⟩ = ⟨ u, w ⟩ - ⟨ v, w ⟩ = ⟨ w, u ⟩ - ⟨ w, v ⟩$

Hint.

Use the fact that $u - v = u + (- v) .$

🔗

16.

Prove the following theorem referenced in Activity 35.6.

🔗

Theorem 35.20.

Hint.

Mimic Theorem 35.20.

🔗

17.

Prove Theorem 35.8.

Hint.

Mimic Theorem 24.3.

🔗

18.

Let $V$ be a vector space with basis ${v_{1}, v_{2}, \dots, v_{n}} .$ Define $⟨, ⟩$ as follows:

⟨ u, w ⟩ = \sum_{i = 1}^{n} u_{i} w_{i}

if $u = \sum_{i = 1}^{n} u_{i} v_{i}$ and $w = \sum_{i = 1}^{n} w_{i} v_{i}$ in $V .$ (Since the representation of a vector as a linear combination of basis elements is unique, this mapping is well-defined.) Show that $⟨, ⟩$ is an inner product on $V$ and conclude that any finite dimensional vector space can be made into an inner product space.

🔗

19.

Label each of the following statements as True or False. Provide justification for your response.

🔗

(a) True/False.

An inner product on a vector space $V$ is a function from $V$ to the real numbers.

🔗

(b) True/False.

If $⟨, ⟩$ is an inner product on a vector space $V,$ and if $v$ is a vector in $V,$ then the set $W = {x \in V : ⟨ x, v ⟩ = 0}$ is a subspace of $V .$

🔗

(c) True/False.

There is exactly one inner product on each inner product space.

🔗

(d) True/False.

If $x,$ $y,$ and $z$ are vectors in an inner product space with $⟨ x, z ⟩ = ⟨ y, z ⟩,$ then $x = y .$

🔗

(e) True/False.

If $⟨ x, y ⟩ = 0$ for all vectors $y$ in an inner product space $V,$ then $x = 0 .$

🔗

(f) True/False.

If $u$ and $v$ are vectors in an inner product space and the distance from $u$ to $v$ is the same as the distance from $u$ to $- v,$ then $u$ and $v$ are orthogonal.

🔗

(g) True/False.

If $W$ is a subspace of an inner product space and a vector $v$ is orthogonal to every vector in a basis of $W,$ then $v$ is in $W^{⊥} .$

🔗

(h) True/False.

If ${v_{1}, v_{2}, v_{3}}$ is an orthogonal basis for an inner product space $V,$ then so is ${c v_{1}, v_{2}, v_{3}}$ for any nonzero scalar $c .$

🔗

(i) True/False.

An inner product $⟨ u, v ⟩$ in an inner product space $V$ results in another vector in $V .$

🔗

(j) True/False.

An inner product in an inner product space $V$ is a function that maps pairs of vectors in $V$ to the set of non-negative real numbers.

🔗

(k) True/False.

The vector space of all $n \times n$ matrices can be made into an inner product space.

🔗

(l) True/False.

Any non-zero multiple of an inner product on space $V$ is also an inner product on $V .$

🔗

(m) True/False.

Every set of $k$ non-zero orthogonal vectors in a vector space $V$ of dimension $k$ is a basis for $V .$

🔗

(n) True/False.

For any finite-dimensional inner product space $V$ and a subspace $W$ of $V,$ $W$ is a subspace of $(W^{⊥})^{⊥} .$

🔗

(o) True/False.

If $W$ is a subspace of an inner product space, then $W \cap W^{⊥} = {0} .$

🔗

Subsection Project: Fourier Series and Musical Tones

🔗

Joseph Fourier first studied trigonometric polynomials to understand the flow of heat in metallic plates and rods. The resulting series, called Fourier series, now have applications in a variety of areas including electrical engineering, vibration analysis, acoustics, optics, signal processing, image processing, geology, quantum mechanics, and many more. For our purposes, we will focus on synthesized music.

🔗

Pure musical tones are periodic sine waves. Simple electronic circuits can be designed to generate alternating current. Alternating current is current that is periodic, and hence is described by a combination of

\sin (k x)

and

\cos (k x)

for integer values of

k .

To synthesize an instrument like a violin, we can project the instrument's tones onto trigonometric polynomials — and then we can produce them electronically. As we will see, these projections are least squares approximations onto certain vector spaces. The website falstad.com/fourier/ provides a tool for hearing sounds digitally created by certain functions. For example, you can listen to the sound generated by a sawtooth function

f

of the form

f (x) = {\begin{cases} x & if - π < x \leq π, \\ f (x - 2 π), & if π < x, \\ f (x + 2 π), & if x \leq - π . \end{cases}

🔗

Try out some of the tones on this website (click on the Sound button to hear the tones). You can also alter the tones by clicking on any one of the white dots and moving it up or down. and play with the buttons. We will learn much about what this website does in this project.

🔗

Pure tones are periodic and so are modeled by trigonometric functions. In general, trigonometric polynomials can be used to produce good approximations to periodic phenomena. A trigonometric polynomial is an object of the form

\begin{aligned} c_{0} + c_{1} \cos (x) + d_{1} \sin (x) & + c_{2} \cos (2 x) + d_{2} \sin (2 x) + \dots \\ + c_{n} \cos (n x) + d_{n} \sin (n x) + \dots, \end{aligned}

🔗

where the

c_{i}

and

d_{j}

are real constants. With judicious choices of these constants, we can approximate periodic and other behavior with trigonometric polynomials. The first step for us will be to understand the relationships between the summands of these trigonometric polynomials in the inner product space

C [- π, π]

⁶¹ of continuous functions from

[π, π]

R

with the inner product

\begin{matrix} (35.3) & ⟨ f, g ⟩ = \frac{1}{π} \int_{- π}^{π} f (x) g (x) d x . \end{matrix}

🔗

Our first order of business is to verify that (35.3) is, in fact, an inner product.

🔗

Project Activity 35.10.

Let $C [a, b]$ be the set of continuous real-valued functions on the interval $[a, b] .$ In Exercise 1 in Section 35 we are asked to show that $C [a, b]$ is a vector space, while Exercise 2 in Section 35 asks us to show that $⟨ f, g ⟩ = \int_{a}^{b} f (x) g (x) d x$ defines an inner product on $C [a, b] .$ However, (36.2) is slightly different than this inner product. Show that any positive scalar multiple of an inner product is an inner product, and conclude that (35.3) defines an inner product on $C [- π, π] .$ (We will see why we introduce the factor of $\frac{1}{π}$ later.)

🔗

Now we return to our inner product space

C [- π, π]

with inner product (35.3). Given a function

g

C [- π, π],

we approximate

g

using only a finite number of the terms in a trigonometric polynomial. Let

W_{n}

be the subspace of

C [- π, π]

spanned by the functions

1, \cos (x), \sin (x), \cos (2 x), \sin (2 x), \dots, \cos (n x), \sin (n x) .

🔗

One thing we need to know is the dimension of

W_{n} .

🔗

Project Activity 35.11.

We start with the initial case of $W_{1} .$

🔗

(a)

Show directly that the functions $1,$ $\cos (x),$ and $\sin (x)$ are orthogonal.

🔗

(b)

What is the dimension of $W_{1} ?$ Explain.

🔗

Now we need to see if what happened in Project Activity 35.11 happens in general. A few tables of integrals and some basic facts from trigonometry can help.

🔗

Project Activity 35.12.

A table of integrals shows the following for $k \neq m$ (up to a constant):

\begin{aligned} (35.4) & \int \cos (m x) \cos (k x) d x & = \frac{1}{2} (\frac{\sin ((k - m) x)}{k - m} + \frac{\sin ((k + m) x)}{k + m}) \\ (35.5) & \int \sin (m x) \sin (k x) d x & = \frac{1}{2} (\frac{\sin ((k - m) x)}{k - m} - \frac{\sin ((k + m) x)}{k + m}) \\ (35.6) & \int \cos (m x) \sin (k x) d x & = \frac{1}{2} (\frac{\cos ((m - k) x)}{m - k} - \frac{\sin ((m + k) x)}{m + k}) \\ (35.7) & \int \cos (m x) \sin (m x) d x & = - \frac{1}{2 m} \cos^{2} (m x) \end{aligned}

🔗

(a)

Use (35.4) to show that $\cos (m x)$ and $\cos (k x)$ are orthogonal in $C [- π, π]$ if $k \neq m .$

🔗

(b)

Use (35.5) to show that $\sin (m x)$ and $\sin (k x)$ are orthogonal in $C [- π, π]$ if $k \neq m .$

🔗

(c)

Use (35.6) to show that $\cos (m x)$ and $\sin (k x)$ are orthogonal in $C [- π, π]$ if $k \neq m .$

🔗

(d)

Use (35.7) to show that $\cos (m x)$ and $\sin (m x)$ are orthogonal in $C [- π, π] .$

🔗

(e)

What is $\dim (W_{n}) ?$ Explain.

🔗

Once we have an orthogonal basis for

W_{n},

we might want to create an orthonormal basis for

W_{n} .

Throughout the remainder of this project, unless otherwise specified, you should use a table of integrals or any appropriate technological tool to find integrals for any functions you need.

🔗

Project Activity 35.13.

Show that the set

B_{n} = {\frac{1}{\sqrt{2}}, \cos (x), \cos (2 x), \dots, \cos (n x), \sin (x), \sin (2 x), \dots, \sin (n x)}

is an orthonormal basis for $W_{n} .$ Use the fact that the norm of a vector $v$ in an inner product space with inner product $⟨, ⟩$ is defined to be $\sqrt{⟨ v, v ⟩} .$ (This is where the factor of $\frac{1}{π}$ will be helpful.)

🔗

Now we need to recall how to find the best approximation to a vector by a vector in a subspace, and apply that idea to approximate an arbitrary function

g

with a trigonometric polynomial in

W_{n} .

Recall that the best approximation of a function

g

C [- π, π]

is the projection of

g

onto

W_{n} .

If we have an orthonormal basis

{h_{0}, h_{1}, h_{2}, \dots, h_{2 n}}

W_{n},

then the projection of

g

onto

W_{n}

{proj}_{W_{n}} g = ⟨ g, h_{0} ⟩ h_{0} + ⟨ g, h_{1} ⟩ h_{1} + ⟨ g, h_{2} ⟩ h_{2} + \dots + ⟨ g, h_{2 n} ⟩ h_{2 n} .

🔗

With this idea, we can find formulas for the coefficients when we project an arbitrary function onto

W_{n} .

🔗

Project Activity 35.14.

If $g$ is an arbitrary function in $C [- π, π],$ we will write the projection of $g$ onto $W_{n}$ as

\begin{aligned} a_{0} (\frac{1}{\sqrt{2}}) + a_{1} \cos (x) + b_{1} \sin (x) + a_{2} \cos (2 x) & + b_{2} \sin (2 x) + \dots \\ + a_{n} \cos (n x) + b_{n} \sin (n x) . \end{aligned}

The $a_{i}$ and $b_{j}$ are the Fourier coefficients for $f .$ The expression $a_{n} \cos (n x) + b_{n} \sin (n x)$ is called the $n$ th harmonic of $g .$ The first harmonic is called the fundamental frequency. The human ear cannot hear tones whose frequencies exceed 20000 Hz, so we only hear finitely many harmonics (the projections onto $W_{n}$ for some $n$ ).

🔗

(a)

Show that

\begin{matrix} (35.8) & a_{0} = \frac{1}{\sqrt{2} π} \int_{- π}^{π} g (x) d x . \end{matrix}

Explain why $\frac{a_{0}}{\sqrt{2}}$ gives the average value of $g$ on $[- π, π] .$ You may want to go back and review average value from calculus. This is saying that the best constant approximation of $g$ on $[- π, π]$ is its average value, which makes sense.

🔗

(b)

Show that for $m \geq 1,$

\begin{matrix} (35.9) & a_{m} = \frac{1}{π} \int_{- π}^{π} g (x) \cos (m x) d x . \end{matrix}

🔗

(c)

Show that for $m \geq 1,$

\begin{matrix} (35.10) & b_{m} = \frac{1}{π} \int_{- π}^{π} g (x) \sin (m x) d x . \end{matrix}

🔗

Let us return to the sawtooth function defined earlier and find its Fourier coefficients.

🔗

Project Activity 35.15.

Let $f$ be defined by $f (x) = x$ on $[- π, π]$ and repeated periodically afterwards with period $2 π .$ Let $p_{n}$ be the projection of $f$ onto $W_{n} .$

🔗

(a)

Evaluate the integrals to find the projection $p_{1} .$

🔗

(b)

Use appropriate technology to find the projections $p_{10},$ $p_{20},$ and $p_{30}$ for the sawtooth function $f .$ Draw pictures of these approximations against $f$ and explain what you see.

🔗

(c)

Now we find formulas for all the Fourier coefficients. Use the fact that $x \cos (m x)$ is an odd function to explain why $a_{m} = 0$ for each $m .$ Then show that $b_{m} = (- 1)^{m + 1} \frac{2}{m}$ for each $m .$

🔗

(d)

Go back to the website falstad.com/fourier/ and replay the sawtooth tone. Explain what the white buttons represent.

🔗

Project Activity 35.16.

This activity is not connected to the idea of musical tones, so can be safely ignored if so desired. We conclude with a derivation of a very fascinating formula that you may have seen for $\sum_{n = 1}^{\infty} \frac{1}{n^{2}} .$ To do so, we need to analyze the error in approximating a function $g$ with a function in $W_{n} .$

Let $p_{n}$ be the projection of $g$ onto $W_{n} .$ Notice that $p_{n}$ is also in $W_{n + 1} .$ It is beyond the scope of this project, but in “nice” situations we have $| | g - p_{n} | | \to 0$ as $n \to \infty .$ Now $g - p_{n}$ is orthogonal to $p_{n},$ so the Pythagorean theorem shows that

| | g - p_{n} | |^{2} + | | p_{n} | |^{2} = | | g | |^{2} .

Since $| | g - p_{n} | |^{2} \to 0$ as $n \to \infty,$ we can conclude that

\begin{matrix} (35.11) & lim_{n \to \infty} | | p_{n} | |^{2} = | | g | |^{2} . \end{matrix}

We use these ideas to derive a formula for $\sum_{n = 1}^{\infty} \frac{1}{n^{2}} .$

🔗

(a)

Use the fact that $B_{n}$ is an orthonormal basis to show that

| | p_{n} | |^{2} = a_{0}^{2} + a_{1}^{2} + b_{1}^{2} + \dots + a_{n}^{2} + b_{n}^{2} .

Conclude that

\begin{matrix} (35.12) & | | g | |^{2} = a_{0}^{2} + a_{1}^{2} + b_{1}^{2} + \dots + a_{n}^{2} + b_{n}^{2} + \dots . \end{matrix}

🔗

(b)

For the remainder of this activity, let $f$ be the sawtooth function defined by $f (x) = x$ on $[- π, π]$ and repeated periodically afterwards. We determined the Fourier coefficients $a_{i}$ and $b_{j}$ of this function in Project Activity 35.15.

🔗

(i)

Show that

a_{0}^{2} + a_{1}^{2} + b_{1}^{2} + \dots + a_{n}^{2} + b_{n}^{2} + \dots = 4 \sum_{n = 1}^{\infty} \frac{1}{n^{2}} .

🔗

(ii)

Calculate $| | f | |^{2}$ using the inner product and compare to (35.12) to find a surprising formula for $\sum_{n = 1}^{\infty} \frac{1}{n^{2}} .$

With suitable adjustments, we can work over any interval that is convenient, but for the sake of simplicity in this project, we will restrict ourselves to the interval

[- π, π] .