The Gram-Schmidt Process in Inner Product Spaces

Section 36 The Gram-Schmidt Process in Inner Product Spaces

Focus Questions

By the end of this section, you should be able to give precise and thorough answers to the questions listed below. You may want to keep these questions in mind to focus your thoughts as you complete the section.

What is the Gram-Schmidt process and why is it useful?

🔗

Subsection Application: Gaussian Quadrature

🔗

Since integration of functions is difficult, approximation techniques for definite integrals are very important. In calculus we are introduced to various methods, e.g., the midpoint, trapezoid, and Simpson's rule, for approximating definite integrals. These methods divide the interval of integration into subintervals and then use values of the integrand at the endpoints to approximate the integral. These are useful methods when approximating integrals from tabulated data, but there are better methods for other types of integrands. If we make judicious choices in the points we use to evaluate the integrand, we can obtain more accurate results with less work. One such method is Gaussian quadrature (which, for example, is widely used in solving problems of radiation heat transfer in direct integration of the equation of transfer of radiation over space), which we explore later in this section. This method utilizes the Gram-Schmidt process to produce orthogonal polynomials.

🔗

Subsection Introduction

🔗

We have seen that orthogonal bases make computations very convenient, and the Gram-Schmidt process allowed us to create orthogonal bases in

R^{n}

using the dot product as inner product. In this section we will see how the Gram-Schmidt process works in any inner product space.

🔗

Subsection The Gram-Schmidt Process using Inner Products

🔗

Our goal is to understand how the Gram-Schmidt process works in any inner product space.

🔗

Preview Activity 36.1.

Let $p_{1} (t) = t,$ $p_{2} (t) = 1 + t + t^{2},$ and $p_{3} (t) = t^{3}$ in $P_{3} .$ Let $B = {p_{1} (t), p_{2} (t), p_{3} (t)}$ and let $W = Span B .$ Throughout this activity, use the inner product on $P_{3}$ defined by

⟨ p (t), q (t) ⟩ = \int_{- 1}^{1} p (t) q (t) d t .

We want to construct an orthogonal basis $C = {q_{1} (t), q_{2} (t), q_{3} (t)}$ from $B$ so that $Span C = Span B .$ To begin, we start by letting $q_{1} (t) = p_{1} (t) .$

🔗

(a)

Now we want to find a polynomial in $W$ that is orthogonal to $q_{1} (t) .$ Let $W_{1} = Span {q_{1} (t)} .$ Explain why $q_{2} (t) = {proj}_{⊥ W_{1}} p_{2} (t)$ is in $W$ and is orthogonal to $q_{1} (t) .$ Then calculate the polynomial $q_{2} (t) .$

🔗

(b)

Next we need to find a third polynomial $q_{3} (t)$ that is in $W$ and is orthogonal to both $q_{1} (t)$ and $q_{2} (t) .$ Let $W_{2} = Span {q_{1} (t), q_{2} (t)} .$ Explain why $q_{3} (t) = {proj}_{⊥ W_{2}} p_{3} (t)$ is in $W$ and is orthogonal to both $q_{1} (t)$ and $q_{2} (t) .$ Then calculate the polynomial $q_{3} (t) .$

🔗

(c)

Explain why the set ${q_{1} (t), q_{2} (t), q_{3} (t)}$ is an orthogonal basis for $W .$

🔗

Preview Activity 36.1 shows the first steps of the Gram-Schmidt process to construct an orthogonal basis from any basis of an inner product space. To understand how the process works in general, let

{v_{1}, v_{2}, \dots, v_{m}}

be a basis for a subspace

W

of an inner product space

V .

Let

w_{1} = v_{1}

and let

W_{1} = Span {w_{1}} .

Since

w_{1} = v_{1}

we have that

W_{1} = Span {w_{1}} = Span {v_{1}} .

Now consider the subspace

W_{2} = Span {v_{1}, v_{2}}

🔗

W .

The vectors

v_{1} = w_{1}

and

v_{2}

are possibly not orthogonal, but we know the orthogonal projection of

v_{2}

onto

W_{1}^{⊥}

is orthogonal to

w_{1} .

Let

w_{2} = {proj}_{⊥ W_{1}} v_{2} = v_{2} - \frac{⟨ v_{2}, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} w_{1} .

🔗

Then

{w_{1}, w_{2}}

is an orthogonal set. Note that

w_{1} = v_{1} \neq 0,

and the fact that

v_{2} \notin W_{1}

implies that

w_{2} \neq 0 .

So the set

{w_{1}, w_{2}}

is linearly independent, being a set of non-zero orthogonal vectors. Now the question is whether

Span {w_{1}, w_{2}} = Span {v_{1}, v_{2}} .

Note that

w_{2}

is a linear combination of

v_{1}

and

v_{2},

w_{2}

is in

Span {v_{1}, v_{2}} .

Since

Span {w_{1}, w_{2}}

is a 2-dimensional subspace of the 2-dimensional space

W_{2},

it must be true that

Span {w_{1}, w_{2}} = W_{2} = Span {v_{1}, v_{2}} .

🔗

Now we take the next step, adding

v_{3}

into the mix. Let

W_{3} = Span {v_{1}, v_{2}, v_{3}} = Span {w_{1}, w_{2}, v_{3}} .

🔗

The vector

w_{3} = {proj}_{⊥ W_{2}} v_{3} = v_{3} - \frac{⟨ v_{3}, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} w_{1} - \frac{⟨ v_{3}, w_{2} ⟩}{⟨ w_{2}, w_{2} ⟩} w_{2}

🔗

is orthogonal to both

w_{1}

and

w_{2}

and, by construction,

w_{3}

is a linear combination of

v_{1},

v_{2},

and

v_{3} .

w_{3}

is in

W_{3} .

The fact that

v_{3} \notin W_{2}

implies that

w_{3} \neq 0

and

{w_{1}, w_{2}, w_{3}}

is a linearly independent set. Since

Span {w_{1}, w_{2}, w_{3}}

is a 3-dimensional subspace of the 3-dimensional space

W_{3},

we conclude that

Span {w_{1}, w_{2}, w_{3}}

equals

Span {v_{1}, v_{2}, v_{3}} .

🔗

We continue inductively in this same manner. If we have constructed a set

{w_{1},

w_{2},

w_{3},

\dots,

w_{k - 1}}

k - 1

orthogonal vectors such that

Span {w_{1}, w_{2}, w_{3}, \dots, w_{k - 1}} = Span {v_{1}, v_{2}, v_{3}, \dots, v_{k - 1}},

🔗

then we let

\begin{aligned} w_{k} & = {proj}_{⊥ W_{k - 1}} v_{k} \\ = v_{k} - \frac{⟨ v_{k}, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} w_{1} - \frac{⟨ v_{k}, w_{2} ⟩}{⟨ w_{2}, w_{2} ⟩} w_{2} - \dots - \frac{⟨ v_{k}, w_{k - 1} ⟩}{⟨ w_{k - 1}, w_{k - 1} ⟩} w_{k - 1}, \end{aligned}

🔗

where

W_{k - 1} = Span {w_{1}, w_{2}, w_{3}, \dots, w_{k - 1}} .

🔗

We know that

w_{k}

is orthogonal to

w_{1},

w_{2},

\dots,

w_{k - 1} .

Since

w_{1},

w_{2},

\dots,

w_{k - 1},

and

v_{k}

are all in

W_{k} = Span {v_{1}, v_{2}, \dots, v_{k}}

we see that

w_{k}

is also in

W_{k} .

Since

v_{k} \notin W_{k - 1}

implies that

w_{k} \neq 0

and

{w_{1}, w_{2}, \dots, w_{k}}

is a linearly independent set. Then

Span {w_{1}, w_{2}, w_{3}, \dots, w_{k}}

is a

k

-dimensional subspace of the

k

-dimensional space

W_{k},

so it follows that

Span {w_{1}, w_{2}, w_{3}, \dots, w_{k}} = W_{k} = Span {v_{1}, v_{2}, v_{3}, \dots, v_{k}} .

🔗

This process will end when we have an orthogonal set

{w_{1},

w_{2},

w_{3},

\dots,

w_{m}}

with

Span {w_{1},

w_{2},

w_{3},

\dots,

w_{m}}

W .

🔗

We summarize the process in the following theorem.

🔗

Theorem 36.1. The Gram-Schmidt Process.

Let ${v_{1}, v_{2}, \dots, v_{m}}$ be a basis for a subspace $W$ of an inner product space $V .$ The set ${w_{1}, w_{2}, w_{3}, \dots, w_{m}}$ defined by

$w_{1} = v_{1},$
$w_{2} = v_{2} - \frac{⟨ v_{2}, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} w_{1},$
$w_{3} = v_{3} - \frac{⟨ v_{3}, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} w_{1} - \frac{⟨ v_{3}, w_{2} ⟩}{⟨ w_{2}, w_{2} ⟩} w_{2},$ $⋮$
$w_{m} = v_{m} - \frac{⟨ v_{m}, w_{1} ⟩}{⟨ w_{1}, w_{1} ⟩} w_{1} - \frac{⟨ v_{m}, w_{2} ⟩}{⟨ w_{2}, w_{2} ⟩} w_{2} - \dots - \frac{⟨ v_{m}, w_{m - 1} ⟩}{⟨ w_{m - 1}, w_{m - 1} ⟩} w_{m - 1} .$

is an orthogonal basis for $W .$ Moreover,

Span {w_{1}, w_{2}, w_{3}, \dots, w_{k}} = Span {v_{1}, v_{2}, v_{3}, \dots, v_{k}}

for $1 \leq k \leq m .$

🔗

The Gram-Schmidt process builds an orthogonal basis

{w_{1}, w_{2}, w_{3}, \dots, w_{m}}

for us from a given basis. To make an orthonormal basis

{u_{1}, u_{2}, u_{3}, \dots, u_{m}},

all we need do is normalize each basis vector: that is, for each

i,

we let

u_{i} = \frac{w_{i}}{| | w_{i} | |} .

🔗

Activity 36.2.

Use the Gram-Schmidt process to find an orthogonal basis for

W = Span {[\begin{matrix} 1 \\ 0 \\ 0 \\ 1 \end{matrix}], [\begin{array}{r} 0 \\ 1 \\ 0 \\ - 2 \end{array}], [\begin{array}{r} 4 \\ 0 \\ - 1 \\ 0 \end{array}]}

using the inner product

⟨ [u_{1} u_{2} u_{3} u_{4}]^{T}, [v_{1} v_{2} v_{3} v_{4}]^{T} ⟩ = u_{1} v_{1} + 2 u_{2} v_{2} + 2 u_{3} v_{3} + u_{4} v_{4} .

🔗

Subsection Examples

🔗

What follows are worked examples that use the concepts from this section.

🔗

Example 36.2.

Let $W = Span {M_{1}, M_{2}, M_{3}, M_{4}},$ where

M_{1} = [\begin{array}{ccc} 1 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{array}], M_{2} = [\begin{array}{ccc} 0 & 0 & 2 \\ 1 & 0 & 1 \\ 0 & 1 & 0 \end{array}], M_{3} = [\begin{array}{ccc} 0 & 0 & 0 \\ 4 & 1 & 3 \\ 0 & 0 & 0 \end{array}], and M_{4} = [\begin{array}{ccc} 12 & 0 & 0 \\ 0 & 0 & 0 \\ 1 & 0 & 0 \end{array}] .

Find an orthonormal basis for $W$ using the Frobenius inner product $⟨ A, B ⟩ = trace (A B^{T}) .$

Solution.

Recall that the Frobenius inner product is just like a dot product for matrices. First note that $M_{1},$ $M_{2},$ $M_{3},$ and $M_{4}$ are linearly independent. We let $N_{1} = M_{1}$ and the Gram-Schmidt process gives us

\begin{aligned} N_{2} & = M_{2} - \frac{⟨ M_{2}, N_{1} ⟩}{⟨ N_{1}, N_{1} ⟩} N_{1} \\ = [\begin{array}{ccc} 0 & 0 & 2 \\ 1 & 0 & 1 \\ 0 & 1 & 0 \end{array}] - \frac{3}{3} [\begin{array}{ccc} 1 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{array}] \\ = [\begin{array}{rcc} - 1 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{array}] \end{aligned}

\begin{aligned} N_{3} & = M_{3} - \frac{⟨ M_{3}, N_{1} ⟩}{⟨ N_{1}, N_{1} ⟩} N_{1} - \frac{⟨ M_{3}, N_{2} ⟩}{⟨ N_{2}, N_{2} ⟩} N_{2} \\ = [\begin{array}{ccc} 0 & 0 & 0 \\ 4 & 1 & 3 \\ 0 & 0 & 0 \end{array}] - \frac{3}{3} [\begin{array}{ccc} 1 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{array}] - \frac{4}{4} [\begin{array}{rcc} - 1 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{array}] \\ = [\begin{array}{crr} 0 & 0 & - 2 \\ 3 & 1 & 2 \\ 0 & - 1 & 0 \end{array}] \end{aligned}

and

\begin{aligned} N_{4} & = M_{4} - \frac{⟨ M_{4}, N_{1} ⟩}{⟨ N_{1}, N_{1} ⟩} N_{1} - \frac{⟨ M_{4}, N_{2} ⟩}{⟨ N_{2}, N_{2} ⟩} N_{2} - \frac{⟨ M_{4}, N_{3} ⟩}{⟨ N_{3}, N_{3} ⟩} N_{3} \\ = [\begin{array}{ccc} 12 & 0 & 0 \\ 0 & 0 & 0 \\ 1 & 0 & 0 \end{array}] - \frac{12}{3} [\begin{array}{ccc} 1 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{array}] - \frac{- 12}{4} [\begin{array}{rcc} - 1 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{array}] - \frac{0}{19} [\begin{array}{crr} 0 & 0 & - 2 \\ 3 & 1 & 2 \\ 0 & - 1 & 0 \end{array}] \\ = [\begin{array}{ccr} 5 & 0 & - 1 \\ 3 & 0 & - 4 \\ 1 & 3 & 0 \end{array}] . \end{aligned}

Then ${N_{1}, N_{2}, N_{3}, N_{4}}$ is an orthogonal basis for $W .$ An orthonormal basis is found by dividing each vector by its magnitude, so

{\frac{1}{\sqrt{3}} [\begin{array}{ccc} 1 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{array}], \frac{1}{2} [\begin{array}{rcc} - 1 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{array}], \frac{1}{\sqrt{19}} [\begin{array}{crr} 0 & 0 & - 2 \\ 3 & 1 & 2 \\ 0 & - 1 & 0 \end{array}], \frac{1}{\sqrt{61}} [\begin{array}{ccr} 5 & 0 & - 1 \\ 3 & 0 & - 4 \\ 1 & 3 & 0 \end{array}]}

is an orthonormal basis for $W .$

🔗

Example 36.3.

Consider the space $V = C [0, 1]$ with inner product $⟨ f, g ⟩ = \int_{0}^{1} f (x) g (x) d x .$

🔗

(a)

Find the polynomial in $P_{2}$ (considered as a subspace of $V$ ) that is closest to $h (x) = \sqrt{x} .$ Use technology to calculate any required integrals. Draw a graph of your approximation against the graph of $h .$

Solution.

Our job is to find ${proj}_{P_{2}} h (x) .$ To do this, we need an orthogonal basis of $P_{2} .$ We apply the Gram-Schmidt process to the standard basis ${1, t, t^{2}}$ of $P_{2}$ to obtain an orthogonal basis ${p_{1} (t), p_{2} (t), p_{3} (t)}$ of $P_{2} .$ We start with $p_{1} (t) = 1,$ then

\begin{aligned} p_{2} (t) & = t - \frac{⟨ t, 1 ⟩}{⟨ 1, 1 ⟩} (1) \\ = t - \frac{1}{2} \end{aligned}

and

\begin{aligned} p_{3} (t) & = t^{2} - \frac{⟨ t^{2}, 1 ⟩}{⟨ 1, 1 ⟩} (1) - \frac{⟨ t^{2}, t - \frac{1}{2} ⟩}{⟨ t - \frac{1}{2}, t - \frac{1}{2} ⟩} (t - \frac{1}{2}) \\ = t^{2} - \frac{1}{3} - \frac{\frac{1}{12}}{\frac{1}{12}} (t - \frac{1}{2}) \\ = t^{2} - t + \frac{1}{6} . \end{aligned}

Then

\begin{aligned} {proj}_{P_{2}} \sqrt{x} & = \frac{⟨ \sqrt{x}, 1 ⟩}{⟨ 1, 1 ⟩} (1) + \frac{⟨ \sqrt{x}, x - \frac{1}{2} ⟩}{⟨ x - \frac{1}{2}, x - \frac{1}{2} ⟩} (x - \frac{1}{2}) \\ + \frac{⟨ \sqrt{x}, x^{2} - x + \frac{1}{12} ⟩}{⟨ x^{2} - x + \frac{1}{6}, x^{2} - x + \frac{1}{6} ⟩} (x^{2} - x + \frac{1}{6}) \\ = \frac{\frac{2}{3}}{1} (1) + \frac{\frac{1}{15}}{\frac{1}{12}} (x - \frac{1}{2}) - \frac{\frac{1}{315}}{\frac{1}{180}} (x^{2} - x + \frac{1}{6}) \\ = \frac{2}{3} + (\frac{4}{5}) (x - \frac{1}{2}) - \frac{4}{7} (x^{2} - x + \frac{1}{6}) \\ = - \frac{4}{7} x^{2} + \frac{48}{35} x + \frac{6}{35} . \end{aligned}

A graph of the approximation and $h$ are shown in Figure 36.4

Figure 36.4. The graphs of $\sqrt{x}$ and $- \frac{4}{7} x^{2} + \frac{48}{35} x + \frac{6}{35} .$

🔗

(b)

Provide a numeric measure of the error in approximating $\sqrt{x}$ by the polynomial you found in part (a).

Solution.

The norm of ${proj}_{P_{2}^{⊥}} \sqrt{x} = \sqrt{x} - (- \frac{4}{7} x^{2} + \frac{48}{35} x + \frac{6}{35})$ tells us how well our projection $- \frac{4}{7} x^{2} + \frac{48}{35} x + \frac{6}{35}$ approximates $\sqrt{x} .$ Technology shows that

| | {proj}_{P_{2}^{⊥}} \sqrt{x} | | = \sqrt{⟨ {proj}_{P_{2}^{⊥}} \sqrt{x}, {proj}_{P_{2}^{⊥}} \sqrt{x} ⟩} = \frac{\sqrt{2}}{70} \approx 0.02 .

🔗

Subsection Summary

The Gram-Schmidt process produces an orthogonal basis from any basis. It works as follows. Let ${v_{1}, v_{2}, \dots, v_{m}}$ be a basis for a subspace $W$ of $R^{n} .$ The set ${w_{1},$ $w_{2},$ $w_{3},$ $\dots,$ $w_{m}}$ defined by
- $w_{1} = v_{1},$
- $w_{2} = v_{2} - \frac{v_{2} \cdot w_{1}}{w_{1} \cdot w_{1}} w_{1},$
- $w_{3} = v_{3} - \frac{v_{3} \cdot w_{1}}{w_{1} \cdot w_{1}} w_{1} - \frac{v_{3} \cdot w_{2}}{w_{2} \cdot w_{2}} w_{2},$ $⋮$
- $w_{m} = v_{m} - \frac{v_{m} \cdot w_{1}}{w_{1} \cdot w_{1}} w_{1} - \frac{v_{m} \cdot w_{2}}{w_{2} \cdot w_{2}} w_{2} - \dots - \frac{v_{m} \cdot w_{m - 1}}{w_{m - 1} \cdot w_{m - 1}} w_{m - 1} .$
is an orthogonal basis for $W .$ Moreover,

$Span {w_{1}, w_{2}, w_{3}, \dots, w_{k}} = Span {v_{1}, v_{2}, v_{3}, \dots, v_{k}}$

for each $k$ between 1 and $m .$

🔗

Exercises Exercises

🔗

1.

Let $V = C [0, 1]$ with the inner product $⟨ f (t), g (t) ⟩ = \int_{0}^{1} f (t) g (t) d t .$ Let $W = P_{2} .$ Note that $W$ is a subspace of $V .$

🔗

(a)

Find the polynomial $q (t)$ in $W$ that is closest to the function $h$ defined by $h (t) = \frac{2}{1 + t^{2}}$ in the least squares sense. That is, find the projection of $h (t)$ onto $W .$

Hint.

Recall the work done in Activity 25.5.

🔗

(b)

Find the second order Taylor polynomial $P_{2} (t)$ for $h (t)$ centered at 0.

🔗

(c)

Plot $h (t),$ $q (t),$ and $P_{2} (t)$ on the same set of axes. Which approximation provides a better fit to $h$ on this interval. Why?

🔗

2.

Each set $S$ is linearly independent. Use the Gram-Schmidt process to create an orthogonal set of vectors with the same span as $S .$ Then find an orthonormal basis for the same span.

🔗

(a)

$S = {1 + t, 1 - t, t - t^{2}}$ in $P_{2}$ using the inner product

⟨ p (t), q (t) ⟩ = \int_{- 1}^{1} p (t) q (t) d t

🔗

(b)

$S = {[\begin{matrix} 1 \\ 0 \\ 2 \end{matrix}], [\begin{array}{r} - 1 \\ 1 \\ 1 \end{array}], [\begin{matrix} 1 \\ 1 \\ 0 \end{matrix}]}$ in $R^{3}$ using the inner product $⟨ u, v ⟩ = (A u) \cdot (A v),$ where $A = [\begin{array}{ccc} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 1 & 1 & 0 \end{array}]$

🔗

(c)

$S = {[\begin{matrix} 1 \\ 0 \\ 1 \\ 0 \\ 1 \\ 0 \\ 1 \end{matrix}], [\begin{array}{r} - 1 \\ 2 \\ 3 \\ 0 \\ 1 \\ 0 \\ 1 \end{array}], [\begin{array}{r} 1 \\ 0 \\ 4 \\ - 1 \\ 2 \\ 0 \\ 1 \end{array}], [\begin{matrix} 1 \\ 1 \\ 1 \\ 1 \\ 1 \\ 1 \\ 1 \end{matrix}]}$ in $R^{7}$ with the weighted inner product

\begin{aligned} ⟨ [u_{1} & u_{2} u_{3} u_{4} u_{5} u_{6} u_{7}]^{T}, [v_{1} v_{2} v_{3} v_{4} v_{5} v_{6} v_{7}]^{T} ⟩ \\ = u_{1} v_{1} + u_{2} v_{2} + u_{3} v_{3} + 2 u_{4} v_{4} + 2 u_{5} v_{5} + u_{6} v_{6} + u_{7} v_{7} \end{aligned}

🔗

3.

Let $S = {1, \cos (t), \sin (t)} .$

🔗

(a)

Show that $S$ is a linearly independent set in $C [0, π] .$

Hint.

Evaluate an appropriate linear combination at several well chosen values to set up a linear system.

🔗

(b)

Use the Gram-Schmidt process to find an orthogonal basis from $S$ using the inner product

⟨ f (t), g (t) ⟩ = \int_{0}^{π} f (t) g (t) d t

🔗

4.

Find an orthonormal basis for the subspace $W$ of $P_{2}$ that consists of all polynomials of the form $a + (a + b) t + b t^{2}$ using the inner product $⟨ p (t), q (t) ⟩ = \int_{0}^{1} p (t) q (t) d t .$

🔗

5.

The Legendre polynomials form an orthonormal basis for the infinite dimensional inner product space $P$ of all polynomials using the inner product

⟨ p (t), q (t) ⟩ = \int_{- 1}^{1} p (t) q (t) d t .

The Legendre polynomials have applications to differential equations, statistics, numerical analysis, and physics (e.g., they appear when solving Schrödinger equation in three dimensions for a central force). The Legendre polynomials are found by using the Gram-Schmidt process to find an orthogonal basis from the standard basis ${1, t, t^{2}, \dots}$ for $P .$ Find the first four Legendre polynomials by creating a orthonormal basis from the set ${1, t, t^{2}, t^{3}} .$

🔗

6.

The sine integral function $Si$ has applications in physics and engineering. We define $Si$ as

Si (x) = \int_{0}^{x} \frac{\sin (t)}{t} d t .

Since we cannot find an elementary formula for $Si (x),$ we use approximations. Find the best approximation to $Si (x)$ in $P_{3}$ with inner product $⟨ p (t), q (t) ⟩ = \int_{0}^{1} p (t) q (t) d t .$ Use appropriate technology for computations and round output six places to the right of the decimal.

🔗

7.

Recall from Exercise 18 in Section 35 that any finite dimensional vector space $V$ can be made into an inner product space by setting a basis $B$ for $V$ and defining

\begin{matrix} (36.1) & ⟨ u, w ⟩_{B} = \sum_{i = 1}^{n} u_{i} w_{i} \end{matrix}

if $u = \sum_{i = 1}^{n} u_{i} v_{i}$ and $w = \sum_{i = 1}^{n} w_{i} v_{i}$ in $V .$ Let $V = P_{2}$ and $B = {1 + t, 1 + t^{2}, t + t^{2}} .$ (You may assume that $B$ is a basis for $V .$ )

🔗

(a)

Find an orthogonal basis for $V$ containing the polynomial $p_{1} (t) = 1$ using the inner product (36.1).

🔗

(b)

Find the best approximation possible as measured by the inner product (36.1) to the polynomial $1 + 2 t + 3 t^{2}$ by a polynomial in $P_{1} .$

🔗

8.

Let $a,$ $b,$ and $c$ be three distinct fixed real numbers and define $⟨, ⟩ : P_{2} \to R$ by

\begin{matrix} (36.2) & ⟨ p (t), q (t) ⟩ = p (a) q (a) + p (b) q (b) + p (c) q (c) . \end{matrix}

🔗

(a)

Show that $⟨, ⟩$ as defined in (36.2) is an inner product on $P_{2} .$

🔗

(b)

Is the mapping

\begin{matrix} (36.3) & ⟨ p (t), q (t) ⟩ = p (a) q (a) + p (b) q (b) \end{matrix}

an inner product on $P_{2} ?$ Justify your answer.

🔗

(c)

Let $a = - 1,$ $b = 1,$ and $c = 3 .$ Find an orthogonal basis for $P_{2}$ using the inner product (36.2).

🔗

(d)

Continue with $a = - 1,$ $b = 1,$ and $c = 3 .$ Find the best approximation possible as measured by the inner product (36.2) to the polynomial $1 + 2 t + 3 t^{2}$ by a polynomial in $P_{1} .$

🔗

(e)

If $W = Span {1 + t},$ find $W^{⊥}$ in $P_{2}$ with respect to the inner product (36.2).

🔗

9.

Label each of the following statements as True or False. Provide justification for your response. Throughout, let $V$ be a vector space.

🔗

(a) True/False.

If ${w_{1}, w_{2}}$ is a basis for a subspace $W$ of an inner product space $V,$ then the vector $\frac{v \cdot w_{1}}{w_{1} \cdot w_{1}} w_{1} + \frac{v \cdot w_{2}}{w_{2} \cdot w_{2}} w_{2}$ is the vector in $W$ closest to $v .$

🔗

(b) True/False.

If $W$ is a subspace of an inner product space $V,$ then the vector $v - {proj}_{W} v$ is orthogonal to every vector in $W .$

🔗

(c) True/False.

If $u_{1},$ $u_{2},$ $u_{3}$ are vectors in an inner product space $V,$ then the Gram-Schmidt process constructs an orthogonal set of vectors ${v_{1}, v_{2}, v_{3}}$ with the same span as ${u_{1}, u_{2}, u_{3}} .$

🔗

(d) True/False.

Any set ${u_{1}, u_{2}, \dots, u_{k}}$ of orthogonal vectors in an inner product space $V$ can be extended to an orthogonal basis of $V .$

🔗

(e) True/False.

Every nontrivial finite dimensional subspace of an inner product space has an orthogonal basis.

🔗

(f) True/False.

In any inner product space $V,$ if $W$ is a subspace satisfying $W^{⊥} = {0},$ then $W = V .$

🔗

Subsection Project: Gaussian Quadrature and Legendre Polynomials

🔗

Simpson's rule is a reasonably accurate method for approximating definite integrals since it models the integrand on subintervals with quadratics. For that reason, Simpson's rule provides exact values for integrals of all polynomials of degree less than or equal to 2. In Gaussian quadrature, we will use a family of polynomials to determine points at which to evaluate an integral of the form

\int_{- 1}^{1} f (t) d t .

By allowing ourselves to select evaluation points that are not uniformly distributed across the interval of integration, we will be able to approximate our integrals much more efficiently. The method is constructed so as to obtain exact values for as large of degree polynomial integrands as possible. As a result, if we can approximate our integrand well with polynomials, we can obtain very good approximations with Gaussian quadrature with minimal effort.

🔗

The Gaussian quadrature approximation has the form

\begin{matrix} (36.4) & \int_{- 1}^{1} f (t) d t \approx w_{1} f (t_{1}) + w_{2} f (t_{2}) + \dots + w_{n} f (t_{n}) = \sum_{i = 1}^{n} w_{i} f (t_{i}), \end{matrix}

🔗

where the

w_{i}

(weights) and the

t_{i}

(nodes) are points in the interval

[- 1, 1]

⁶². Gaussian quadrature describes how to find the weights and the points in (36.4) to obtain suitable approximations. We begin to explore Gaussian quadrature with the simplest cases.

🔗

Project Activity 36.3.

In this activity we find through direct calculation the node and weight with $n = 1$ so that

\begin{matrix} (36.5) & w_{1} f (t_{1}) \approx \int_{- 1}^{1} f (t) d t . \end{matrix}

There are two unknowns in this situation ( $w_{1}$ and $t_{1}$ ) and so we will need 2 equations to find these unknowns. Keep in mind that we want to have the approximation (36.5) be exact for as large of degree polynomials as possible.

🔗

(a)

Assume equality in (36.5) if we choose $f (t) = 1 .$ Use the resulting equation to find $w_{1} .$

🔗

(b)

Assume equality in (36.5) if we choose $f (t) = t .$ Use the resulting equation to find $t_{1} .$

🔗

(c)

Verify that (36.5) is in fact an equality for any linear polynomial of the form $f (t) = a_{0} + a_{1} t,$ using the values of $w_{1}$ and $t_{1}$ you found

🔗

We do one more specific case before considering the general case.

🔗

Project Activity 36.4.

In this problem we find through direct calculation the nodes and weights with $n = 2$ so that

\begin{matrix} (36.6) & w_{1} f (t_{1}) + w_{2} f (t_{2}) \approx \int_{- 1}^{1} f (t) d t . \end{matrix}

There are four unknowns in this situation ( $w_{1}, w_{2}$ and $t_{1}, t_{2}$ ) and so we will need 4 equations to find these unknowns. Keep in mind that we want to have the approximation (36.6) be exact for as large of degree polynomials as possible. In this case we will use $f (t) = 1,$ $f (t) = t,$ $f (t) = t^{2},$ and $f (t) = t^{3} .$

🔗

(a)

Assume equality in (36.6) if we choose $f (t) = 1 .$ This gives us an equation in $w_{1}$ and $w_{2} .$ Find this equation.

🔗

(b)

Assume equality in (36.6) if we choose $f (t) = t .$ This gives us an equation in $w_{1}, w_{2}$ and $t_{1}, t_{2} .$ Find this equation.

🔗

(c)

Assume equality in (36.6) if we choose $f (t) = t^{2} .$ This gives us an equation in $w_{1}, w_{2}$ and $t_{1}, t_{2} .$ Find this equation.

🔗

(d)

Assume equality in (36.6) if we choose $f (t) = t^{3} .$ This gives us an equation in $w_{1}, w_{2}$ and $t_{1}, t_{2} .$ Find this equation.

🔗

(e)

Solve this system of 4 equations in 4 unknowns. You can do this by hand or with any other appropriate tool. Show that $t_{1}$ and $t_{2}$ are the roots of the polynomial $t^{2} - \frac{1}{3} .$

🔗

(f)

Verify that (36.6) is in fact an equality with the values of $w_{1}, w_{2}$ and $t_{1}, t_{2}$ you found for any polynomial of the form $f (t) = a_{0} + a_{1} t + a_{2} t^{2} + a_{3} t^{3} .$

🔗

Other than solving a system of linear equations as in Project Activity 36.4, it might be reasonable to ask what the connection is between Gaussian quadrature and linear algebra. We explore that connection now.

🔗

In the general case, we want to find the weights and nodes to make the approximation exact for as large degree polynomials as possible. We have

2 n

unknowns

w_{1},

w_{2},

\dots,

w_{n}

and

t_{1},

t_{2},

\dots,

t_{n},

so we need to impose

2 n

conditions to determine the unknowns. We will require equality for the

2 n

functions

t^{i}

for

i

from 0 to

2 n - 1 .

This yields the equations

\begin{aligned} w_{1} \cdot 1 + w_{2} \cdot 1 + \dots + w_{n} \cdot 1 & = \int_{- 1}^{1} 1 d t = t |_{- 1}^{1} = 2 \\ w_{1} t_{1} + w_{2} t_{2} + \dots + w_{n} t_{n} & = \int_{- 1}^{1} t d t = \frac{t^{2}}{2} |_{- 1}^{1} = 0 \\ w_{1} t_{1}^{2} + w_{2} t_{2}^{2} + \dots + w_{n} t_{n}^{2} & = \int_{- 1}^{1} t^{2} d t = \frac{t^{3}}{3} |_{- 1}^{1} = \frac{2}{3} \\ ⋮ & ⋮ \\ w_{1} t_{1}^{2 n - 1} + w_{2} t_{2}^{2 n - 1} + \dots + w_{n} t_{n}^{2 n - 1} & = \int_{- 1}^{1} t^{2 n - 1} d t = \frac{t^{2 n}}{2 n} |_{- 1}^{1} = 0 . \end{aligned}

🔗

In the

i

th equation the right hand side is

\int_{- 1}^{1} t^{i} d t = \frac{t^{i + 1}}{i + 1} |_{- 1}^{1} = {\begin{cases} \frac{2}{i + 1} & if i is even, \\ 0 & if i is odd . \end{cases}

🔗

Project Activity 36.5.

It is inefficient to always solve these systems of equations to find the nodes and weights, especially since there is a more elegant way to find the nodes.

🔗

(a)

Use appropriate technology to find the equations satisfied by the $t_{i}$ for $n = 3,$ $n = 4,$ and $n = 5 .$

Now we will see the more elegant way to find the nodes. As we will show for some cases, the nodes can be found as roots of a set of orthogonal polynomials in $P_{n}$ with the inner product $⟨ f (t), g (t) ⟩ = \int_{- 1}^{1} f (t) g (t) d t .$ Begin with the basis $S_{n} = {1, t, t^{2}, \dots, t^{n}}$ of $P_{n} .$ Use appropriate technology to find an orthogonal basis $B_{5}$ for $P_{5}$ obtained by applying the Gram-Schmidt process to $S_{n} .$ The polynomials in this basis are called Legendre polynomials. Check that the nodes are roots of the Legendre polynomials by finding roots of these polynomials using any method. Explain why the $t_{i}$ appear to be roots of the Legendre polynomials.

🔗

Although it would take us beyond the scope of this project to verify this fact, the nodes in the

n

th Gaussian quadrature approximation (36.4) are in fact the roots of the

n

th order Legendre polynomial. In other words, if

p_{n} (t)

is the

n

th order Legendre polynomial, then

t_{1},

t_{2},

\dots,

t_{n}

are the roots of

p_{n} (t)

[- 1, 1] .

Gaussian quadrature as described in (36.4) using the polynomial

p_{n} (t)

is exact if the integrand

f (t)

is a polynomial of degree less than

2 n .

🔗

We can find the corresponding weights,

w_{i},

using the formula⁶³

\begin{matrix} (36.7) & w_{i} = \frac{2}{(1 - t_{i}^{2}) (q_{n}^{'} (t_{i}))^{2}}, \end{matrix}

🔗

where

q_{i} (t)

is the

i

th order Legendre polynomial scaled so that

q_{i} (1) = 1 .

🔗

Project Activity 36.6.

Let us see now how good the integral estimates are with Gaussian quadrature method using an example. Use Gaussian quadrature with the indicated value of $n$ to approximate $\int_{- 1}^{1} e^{t} \cos (t) d t .$ Be sure to explain how you found your nodes and weights (approximate the nodes and weights to 8 decimal places). Compare the approximations with the actual value of the integral. Use technology as appropriate to help with calculations.

🔗

(a)

$n = 3$

🔗

(b)

$n = 4$

🔗

(c)

$n = 5$

🔗

Our Gaussian quadrature formula was derived for integrals on the interval

[- 1, 1] .

We conclude by seeing how a definite integral on an interval can be converted to one on the interval

[- 1, 1] .

🔗

Project Activity 36.7.

Consider the problem of approximating an integral of the form $I = \int_{a}^{b} g (x) d x .$ Show that the change of variables $x = \frac{(b - a) t}{2} + \frac{a + b}{2},$ $f (t) = \frac{(b - a) g (x)}{2}$ reduces the integral $I$ to the form $I = \int_{- 1}^{1} f (t) d t .$ (This change of variables can be derived by finding a linear function that maps the interval $[a, b]$ to the interval $[- 1, 1] .$ )

As we will see later, integrations over

[a, b]

can be converted to an integral over

[- 1, 1]

with a change of variable.

Abramowitz, Milton; Stegun, Irene A., eds. (1972), sec. 25.4, Integration, Handbook of Mathematical Functions (with Formulas, Graphs, and Mathematical Tables), Dover,