The Characteristic Equation

Section 18 The Characteristic Equation

Focus Questions

By the end of this section, you should be able to give precise and thorough answers to the questions listed below. You may want to keep these questions in mind to focus your thoughts as you complete the section.

What is the characteristic polynomial of a matrix?
What is the characteristic equation of a matrix?
How and why is the characteristic equation of a matrix useful?
How many different eigenvalues can an $n \times n$ matrix have?
How large can the dimension of the eigenspace corresponding to an eigenvalue be?

🔗

Subsection Application: Modeling the Second Law of Thermodynamics

🔗

Pour cream into your cup of coffee and the cream spreads out; straighten up your room and it soon becomes messy again; when gasoline is mixed with air in a car's cylinders, it explodes if a spark is introduced. In each of these cases a transition from a low energy state (your room is straightened up) to a higher energy state (a messy, disorganized room) occurs. This can be described by entropy — a measure of the energy in a system. Low energy is organized (like ice cubes) and higher energy is not (like water vapor). It is a fundamental property of energy (as described by the second law of thermodynamics) that the entropy of a system cannot decrease. In other words, in the absence of any external intervention, things never become more organized.

🔗

The Ehrenfest model³⁴ is a Markov process proposed to explain the statistical interpretation of the second law of thermodynamics using the diffusion of gas molecules. This process can be modeled as a problem of balls and bins, as we will do later in this section. The characteristic polynomial of the transition matrix will help us find the eigenvalues and allow us to analyze our model.

🔗

Subsection Introduction

🔗

We have seen that the eigenvalues of an

n \times n

matrix

A

are the scalars

λ

so that

A - λ I_{n}

has a nontrivial null space. Since a matrix has a nontrivial null space if and only if the matrix is not invertible, we can also say that

λ

is an eigenvalue of

A

\begin{matrix} (18.1) & det (A - λ I_{n}) = 0 . \end{matrix}

🔗

This equation is called the characteristic equation of

A .

It provides us an algebraic way to find eigenvalues, which can then be used in finding eigenvectors corresponding to each eigenvalue. Suppose we want to find the eigenvalues of

A = [\begin{array}{cc} 1 & 1 \\ 1 & 3 \end{array}] .

Note that

A - λ I_{2} = [\begin{array}{cc} 1 - λ & 1 \\ 1 & 3 - λ \end{array}],

🔗

with determinant

(1 - λ) (3 - λ) - 1 = λ^{2} - 4 λ + 2 .

Hence, the eigenvalues

λ_{1}, λ_{2}

are the solutions of the characteristic equation

λ^{2} - 4 λ + 2 = 0 .

Using quadratic formula, we find that

λ_{1} = 2 + \sqrt{2}

and

λ_{2} = 2 - \sqrt{2}

are the eigenvalues.

🔗

In this activity, our goal will be to use the characteristic equation to obtain information about eigenvalues and eigenvectors of a matrix with real entries.

🔗

Preview Activity 18.1.

🔗

(a)

For each of the following parts, use the characteristic equation to determine the eigenvalues of $A .$ Then, for each eigenvalue $λ,$ find a basis of the corresponding eigenspace, i.e., $Nul (A - λ I) .$ You might want to recall how to find a basis for the null space of a matrix from Section 13. Also, make sure that your eigenvalue candidate $λ$ yields nonzero eigenvectors in $Nul (A - λ I)$ for otherwise $λ$ will not be an eigenvalue.

🔗

(i)

$A = [\begin{array}{cr} 2 & 0 \\ 0 & - 3 \end{array}]$

🔗

(ii)

$A = [\begin{array}{cc} 1 & 2 \\ 0 & 1 \end{array}]$

🔗

(iii)

$A = [\begin{array}{cc} 1 & 4 \\ 2 & 3 \end{array}]$

🔗

(b)

Use your eigenvalue and eigenvector calculations of the above problem as a guidance to answer the following questions about a matrix with real entries.

🔗

(i)

At most how many eigenvalues can a $2 \times 2$ matrix have? Is it possible to have no eigenvalues? Is it possible to have only one eigenvalue? Explain.

🔗

(ii)

If a matrix is an upper-triangular matrix (i.e., all entries below the diagonal are 0's, as in the first two matrices of the previous problem), what can you say about its eigenvalues? Explain.

🔗

(iii)

How many linearly independent eigenvectors can be found for a $2 \times 2$ matrix? Is it possible to have a matrix without 2 linearly independent eigenvectors? Explain.

🔗

(c)

Using the characteristic equation, determine which matrices have 0 as an eigenvalue.

🔗

Subsection The Characteristic Equation

🔗

Until now, we have been given eigenvalues or eigenvectors of a matrix and determined eigenvectors and eigenvalues from the known information. In this section we use determinants to find (or approximate) the eigenvalues of a matrix. From there we can find (or approximate) the corresponding eigenvectors. The tool we will use is a polynomial equation, the characteristic equation, of a square matrix whose roots are the eigenvalues of the matrix. The characteristic equation will then provide us with an algebraic way of finding the eigenvalues of a square matrix.

🔗

We have seen that the eigenvalues of a square matrix

A

are the scalars

λ

so that

A - λ I

has a nontrivial null space. Since a matrix has a nontrivial null space if and only if the matrix is not invertible, we can also say that

λ

is an eigenvalue of

A

\begin{matrix} (18.2) & det (A - λ I) = 0 . \end{matrix}

🔗

Note that if

A

is an

n \times n

matrix, then

det (A - λ I)

is a polynomial of degree

n .

Furthermore, if

A

has real entries, the polynomial has real coefficients. This polynomial, and the equation (18.2) are given special names.

🔗

Definition 18.1.

Let $A$ be an $n \times n$ matrix. The characteristic polynomial of $A$ is the polynomial

det (A - λ I_{n}),

where $I_{n}$ is the $n \times n$ identity matrix. The characteristic equation of $A$ is the equation

det (A - λ I_{n}) = 0 .

🔗

So the characteristic equation of

A

gives us an algebraic way of finding the eigenvalues of

A .

🔗

Activity 18.2.

🔗

(a)

Find the characteristic polynomial of the matrix $A = [\begin{array}{crc} 3 & - 2 & 5 \\ 1 & 0 & 7 \\ 0 & 0 & 1 \end{array}],$ and use the characteristic polynomial to find all of the eigenvalues of $A .$

🔗

(b)

Verify that 1 and 2 are the only eigenvalues of the matrix $[\begin{array}{cccc} 1 & 0 & 0 & 1 \\ 1 & 2 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{array}] .$

🔗

As we argued in Preview Activity 18.1, a

2 \times 2

matrix can have at most 2 eigenvalues. For an

n \times n

matrix, the characteristic polynomial will be a degree

n

polynomial, and we know from algebra that a degree

n

polynomial can have at most

n

roots. Since an eigenvalue of a matrix is a root of the characteristic polynomial of that matrix, we can conclude that an

n \times n

matrix can have at most

n

distinct eigenvalues. Activity 18.2 (b) shows that a

4 \times 4

matrix may have fewer than

4

eigenvalues, however. Note that one of these eigenvalues, the eigenvalue 1, appears three times as a root of the characteristic polynomial of the matrix. The number of times an eigenvalue appears as a root of the characteristic polynomial is called the (algebraic) multiplicity of the eigenvalue. More formally:

🔗

Definition 18.2.

The (algebraic) multiplicity of an eigenvalue $λ$ of a matrix $A$ is the largest integer $m$ so that $(x - λ)^{m}$ divides the characteristic polynomial of $A .$

🔗

Thus, in Activity 18.2 (b) the eigenvalue 1 has multiplicity 3 and the eigenvalue 2 has multiplicity 1. Notice that if we count the eigenvalues of an

n \times n

matrix with their multiplicities, the total will always be

n .

🔗

A

is a matrix with real entries, then the characteristic polynomial will have real coefficients. It is possible that the characteristic polynomial can have complex roots, and that the matrix

A

has complex eigenvalues. The Fundamental Theorem of Algebra shows us that if a real matrix has complex eigenvalues, then those eigenvalues will appear in conjugate pairs, i.e., if

λ_{1} = a + i b

is an eigenvalue of

A,

then

λ_{2} = a - i b

is another eigenvalue of

A .

Furthermore, for an odd degree polynomial, since the complex eigenvalues will come in conjugate pairs, we will be able to find at least one real eigenvalue.

🔗

We now summarize the information we have so far about eigenvalues of an

n \times n

real matrix:

🔗

Theorem 18.3.

Let $A$ be an $n \times n$ matrix with real entries. Then

There are at most $n$ eigenvalues of $A .$ If each eigenvalue (including complex eigenvalues) is counted with its multiplicity, there are exactly $n$ eigenvalues.
If $A$ has a complex eigenvalue $λ,$ the complex conjugate of $λ$ is also an eigenvalue of $A .$
If $n$ is odd, $A$ has at least one real eigenvalue.
If $A$ is upper or lower-triangular, the eigenvalues are the entries on the diagonal.

🔗

Subsection Eigenspaces, A Geometric Example

🔗

Recall that for each eigenvalue

λ

of an

n \times n

matrix

A,

the eigenspace of

A

corresponding to the eigenvalue

λ

Nul (A - λ I_{n}) .

These eigenspaces can tell us important information about the matrix transformation defined by

A .

For example, consider the matrix transformation

T

from

R^{3}

R^{3}

defined by

T (x) = A x,

where

A = [\begin{array}{ccc} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 2 \end{array}] .

🔗

We are interested in understanding what this matrix transformation does to vectors in

R^{3} .

First we note that

A

has eigenvalues

λ_{1} = 1

and

λ_{2} = 2,

with

λ_{1}

having multiplicity

2 .

There is a pair

v_{1} = [\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}]

and

v_{2} = [\begin{matrix} 0 \\ 1 \\ 0 \end{matrix}]

of linearly independent eigenvectors for

A

corresponding to the eigenvalue

λ_{1}

and an eigenvector

v_{3} = [\begin{matrix} 1 \\ 1 \\ 1 \end{matrix}]

for

A

corresponding to the eigenvalue

λ_{2} .

Note that the vectors

v_{1},

v_{2},

and

v_{3}

are linearly independent (recall from Theorem that eigenvectors corresponding to different eigenvalues are always linearly independent). So any vector

b

R^{3}

can be written uniquely as a linear combination of

v_{1},

v_{2},

and

v_{3} .

Let's now consider the action of the matrix transformation

T

on a linear combination of

v_{1},

v_{2},

and

v_{2} .

Note that

\begin{aligned} T (c_{1} v_{1} + c_{2} v_{2} + c_{3} v_{3}) & = c_{1} T (v_{1}) + c_{2} T (v_{2}) + c_{3} T (v_{3}) \\ = c_{1} λ_{1} v_{1} + c_{2} λ_{1} v_{2} + c_{3} λ_{2} v_{3} \\ (18.3) & = (1) (c_{1} v_{1} + c_{2} v_{2}) + (2) c_{3} v_{3} . \end{aligned}

🔗

Equation (18.3) illustrates that it is most convenient to view the action of

T

in the coordinate system where

Span {v_{1}}

serves as the

x

-axis,

Span {v_{2}}

serves as the

y

-axis, and

Span {v_{3}}

as the

z

-axis. In this case, we can visualize that when we apply the transformation

T

to a vector

b = c_{1} v_{1} + c_{2} v_{2} + c_{3} v_{3}

R^{3}

the result is an output vector that is unchanged in the

v_{1}

v_{2}

plane and scaled by a factor of

2

in the

v_{3}

direction. For example, consider the box whose sides are determined by the vectors

v_{1},

v_{2},

and

v_{3}

as shown in Figure 18.4. The transformation

T

stretches this box by a factor of

2

in the

v_{3}

direction and leaves everything else alone, as illustrated in Figure 18.4. So the entire

Span {v_{1}, v_{2}})

is unchanged by

T,

but

Span {v_{3}})

is scaled by

2 .

In this situation, the eigenvalues and eigenvectors provide the most convenient perspective through which to visualize the action of the transformation

T .

🔗
Figure 18.4. A box and a transformed box.

🔗

This geometric perspective illustrates how each eigenvalue and the corresponding eigenspace of

A

tells us something important about

A .

So it behooves us to learn a little more about eigenspaces.

🔗

Subsection Dimensions of Eigenspaces

🔗

There is a connection between the dimension of the eigenspace of a matrix corresponding to an eigenvalue and the multiplicity of that eigenvalue as a root of the characteristic polynomial. Recall that the dimension of a subspace of

R^{n}

is the number of vectors in a basis for the eigenspace. We investigate the connection between dimension and multiplicity in the next activity.

🔗

Activity 18.3.

🔗

(a)

Find the dimension of the eigenspace for each eigenvalue of matrix $A = [\begin{array}{crc} 3 & - 2 & 5 \\ 1 & 0 & 7 \\ 0 & 0 & 1 \end{array}]$ from Activity 18.2 (a).

🔗

(b)

Find the dimension of the eigenspace for each eigenvalue of matrix $A = [\begin{array}{cccc} 1 & 0 & 0 & 1 \\ 1 & 2 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{array}]$ from Activity 18.2 (b).

🔗

(c)

Consider now a $3 \times 3$ matrix with 3 distinct eigenvalues $λ_{1}, λ_{2}, λ_{3} .$

🔗

(i)

Recall that a polynomial of degree can have at most three distinct roots. What does that say about the multiplicities of $λ_{1}, λ_{2}, λ_{3} ?$

🔗

(ii)

Use the fact that eigenvectors corresponding to distinct eigenvalues are linearly independent to find the dimensions of the eigenspaces for $λ_{1}, λ_{2}, λ_{3} .$

🔗

The examples in Activity 18.3 all provide instances of the principle that the dimension of an eigenspace corresponding to an eigenvalue

λ

cannot exceed the multiplicity of

λ .

Specifically:

🔗

Theorem 18.5.

If $λ$ is an eigenvalue of $A,$ the dimension of the eigenspace corresponding to $λ$ is less than or equal to the multiplicity of $λ .$

🔗

The examples we have seen raise another important point. The matrix

A = [\begin{array}{ccc} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 2 \end{array}]

from our geometric example has two eigenvalues

1

and

2,

with the eigenvalue 1 having multiplicity 2. If we let

E_{λ}

represent the eigenspace of

A

corresponding to the eigenvalue

λ,

then

\dim (E_{1}) = 2

and

\dim (E_{2}) = 1 .

If we change this matrix slightly to the matrix

B = [\begin{array}{crc} 2 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{array}]

we see that

B

has two eigenvalues

1

and

2,

with the eigenvalue 1 having multiplicity 2. However, in this case we have

\dim (E_{1}) = 1

(like the example in from Activity 18.2 (a) and Activity 18.3 (a)). In this case the vector

v_{1} = [1 0 0]^{T}

forms a basis for

E_{2}

and the vector

v_{2} = [0 1 0]^{T}

forms a basis for

E_{1} .

We can visualize the action of

B

on the square formed by

v_{1}

and

v_{2}

in the

x y

-plane as a scaling by 2 in the

v_{1}

direction as shown in Figure 18.6, but since we do not have a third linearly independent eigenvector, the action of

B

in the direction of

[0 0 1]^{T}

is not so clear.

🔗
Figure 18.6. A box and a transformed box.

🔗

So the action of a matrix transformation can be more easily visualized if the dimension of each eigenspace is equal to the multiplicity of the corresponding eigenvalue. This geometric perspective leads us to define the geometric multiplicity of an eigenvalue.

🔗

Definition 18.7.

The geometric multiplicity of an eigenvalue of an $n \times n$ matrix $A$ is the dimension of the corresponding eigenspace $Nul (A - λ I_{n}) .$

🔗

Subsection Examples

🔗

What follows are worked examples that use the concepts from this section.

🔗

Example 18.8.

Let $A = [\begin{array}{rcr} - 1 & 0 & - 2 \\ 2 & 1 & 2 \\ 0 & 0 & 1 \end{array}] .$

🔗

(a)

Find the characteristic polynomial of $A .$

Solution.

The characteristic polynomial of $A$ is

\begin{aligned} p (λ) & = det (A - λ I_{3}) \\ = det ([\begin{array}{ccc} - 1 - λ & 0 & - 2 \\ 2 & 1 - λ & 2 \\ 0 & 0 & 1 - λ \end{array}]) \\ = (- 1 - λ) (1 - λ) (1 - λ) . \end{aligned}

🔗

(b)

Factor the characteristic polynomial and find the eigenvalues of $A .$

Solution.

The eigenvalues of $A$ are the solutions to the characteristic equation. Since

p (λ) = (- 1 - λ) (1 - λ) (1 - λ) = 0

implies $λ = - 1$ or $λ = 1,$ the eigenvalues of $A$ are $1$ and $- 1 .$

🔗

(c)

Find a basis for each eigenspace of $A .$

Solution.

To find a basis for the eigenspace of $A$ corresponding to the eigenvalue $1,$ we find a basis for $Nul (A - I_{3}) .$ The reduced row echelon form of $A - I_{3} = [\begin{array}{rcr} - 2 & 0 & - 2 \\ 2 & 0 & 2 \\ 0 & 0 & 0 \end{array}]$ is $[\begin{array}{ccc} 1 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{array}] .$ If $x = [\begin{matrix} x_{1} \\ x_{2} \\ x_{3} \end{matrix}],$ then $(A - I_{3}) x = 0$ has general solution

x = [\begin{matrix} x_{1} \\ x_{2} \\ x_{3} \end{matrix}] = [\begin{array}{r} - x_{3} \\ x_{2} \\ x_{3} \end{array}] = x_{2} [\begin{matrix} 0 \\ 1 \\ 0 \end{matrix}] + x_{3} [\begin{array}{r} - 1 \\ 0 \\ 1 \end{array}] .

Therefore, ${[0 1 0]^{T}, [- 1 0 1]^{T}}$ is a basis for the eigenspace of $A$ corresponding to the eigenvalue $1 .$ To find a basis for the eigenspace of $A$ corresponding to the eigenvalue $- 1,$ we find a basis for $Nul (A + I_{3}) .$ The reduced row echelon form of $A + I_{3} = [\begin{array}{ccr} 0 & 0 & - 2 \\ 2 & 2 & 2 \\ 0 & 0 & 2 \end{array}]$ is $[\begin{array}{ccc} 1 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{array}] .$ If $x = [\begin{matrix} x_{1} \\ x_{2} \\ x_{3} \end{matrix}],$ then $(A + I_{3}) x = 0$ has general solution

x = [\begin{matrix} x_{1} \\ x_{2} \\ x_{3} \end{matrix}] = [\begin{array}{r} - x_{2} \\ x_{2} \\ 0 \end{array}] = x_{2} [\begin{array}{r} - 1 \\ 1 \\ 0 \end{array}] .

Therefore, a basis for the eigenspace of $A$ corresponding to the eigenvalue $- 1$ is ${[- 1 1 0]^{T}} .$

🔗

(d)

Is it possible to find a basis for $R^{3}$ consisting of eigenvectors of $A ?$ Explain.

Solution.

Let $v_{1} = [0 1 0]^{T}, [- 1 0 1]^{T},$ $v_{2} = [- 1 0 1]^{T},$ and $v_{3} = [- 1 1 0]^{T} .$ Since eigenvectors corresponding to different eigenvalues are linearly independent, and since neither $v_{1}$ nor $v_{2}$ is a scalar multiple of the other, we can conclude that the set ${v_{1}, v_{2}, v_{3}}$ is a linearly independent set with $3 = \dim (R^{3})$ vectors. Therefore, ${v_{1}, v_{2}, v_{3}}$ is a basis for $R^{3}$ consisting of eigenvectors of $A .$

🔗

Example 18.9.

Find a $3 \times 3$ matrix $A$ that has an eigenvector $v_{1} = [1 0 1]^{T}$ with corresponding eigenvalue $λ_{1} = 2,$ an eigenvector $v_{2} = [0 2 - 3]^{T}$ with corresponding eigenvalue $λ_{2} = - 3,$ and an eigenvector $v_{3} = [- 4 0 5]^{T}$ with corresponding eigenvalue $λ_{3} = 5 .$ Explain your process.

Solution.

We are looking for a $3 \times 3$ matrix $A$ such that $A v_{1} = 2 v_{1},$ $A v_{2} = - 3 v_{2}$ and $A v_{3} = 5 v_{3} .$ Since $v_{1},$ $v_{2},$ and $v_{3}$ are eigenvectors corresponding to different eigenvalues, $v_{1},$ $v_{2},$ and $v_{3}$ are linearly independent. So the matrix $[v_{1} v_{2} v_{3}]$ is invertible. It follows that

\begin{aligned} A [v_{1} v_{2} v_{3}] & = [A v_{1} A v_{2} A v_{3}] \\ A [\begin{array}{rcr} 1 & 0 & - 4 \\ 0 & 2 & 0 \\ 1 & - 3 & 5 \end{array}] & = [2 v_{1} - 3 v_{2} 5 v_{3}] \\ A [\begin{array}{crr} 1 & 0 & - 4 \\ 0 & 2 & 0 \\ 1 & - 3 & 5 \end{array}] & = [\begin{array}{crr} 2 & 0 & - 20 \\ 0 & - 6 & 0 \\ 2 & 9 & 25 \end{array}] \\ A & = [\begin{array}{crr} 2 & 0 & - 20 \\ 0 & - 6 & 0 \\ 2 & 9 & 25 \end{array}] {[\begin{array}{crr} 1 & 0 & - 4 \\ 0 & 2 & 0 \\ 1 & - 3 & 5 \end{array}]}^{- 1} \\ A & = [\begin{array}{crr} 2 & 0 & - 20 \\ 0 & - 6 & 0 \\ 2 & 9 & 25 \end{array}] [\begin{array}{rcc} \frac{5}{9} & \frac{2}{3} & \frac{4}{9} \\ 0 & \frac{1}{2} & 0 \\ - \frac{1}{9} & \frac{1}{6} & \frac{1}{9} \end{array}] \\ A & = [\begin{array}{rrr} \frac{10}{3} & - 2 & - \frac{4}{3} \\ 0 & - 3 & 0 \\ - \frac{5}{3} & 10 & \frac{11}{3} \end{array}] . \end{aligned}

🔗

Subsection Summary

🔗

In this section we studied the characteristic polynomial of a matrix and similar matrices.

If $A$ is an $n \times n$ matrix, the characteristic polynomial of $A$ is the polynomial

$det (A - λ I_{n}),$

where $I_{n}$ is the $n \times n$ identity matrix.
If $A$ is an $n \times n$ matrix, the characteristic equation of $A$ is the equation

$det (A - λ I_{n}) = 0 .$
The characteristic equation of a square matrix provides us an algebraic method to find the eigenvalues of the matrix.
The eigenvalues of an upper or lower-triangular matrix are the entries on the diagonal.
There are at most $n$ eigenvalues of an $n \times n$ matrix.
For a real matrix $A,$ if an eigenvalue $λ$ of $A$ is complex, then the complex conjugate of $λ$ is also an eigenvalue.
The algebraic multiplicity of an eigenvalue $λ$ is the multiplicity of $λ$ as a root of the characteristic equation.
The dimension of the eigenspace corresponding to an eigenvalue $λ$ is less than or equal to the algebraic multiplicity of $λ .$

🔗

Exercises Exercises

🔗

1.

There is a useful relationship between the determinant and eigenvalues of a matrix $A$ that we explore in this exercise.

🔗

(a)

Let $B = [\begin{array}{cc} 2 & 3 \\ 8 & 4 \end{array}] .$ Find the determinant of $B$ and the eigenvalues of $B,$ and compare $det (B)$ to the eigenvalues of $B .$

🔗

(b)

Let $A$ be an $n \times n$ matrix. In this part of the exercise we argue the general case illustrated in the previous part — that $det (A)$ is the product of the eigenvalues of $A .$ Let $p (λ) = det (A - λ I_{n})$ be the characteristic polynomial of $A .$

🔗

(i)

Let $λ_{1},$ $λ_{2},$ $\dots,$ $λ_{n}$ be the eigenvalues of $A$ (note that these eigenvalues may not all be distinct). Recall that if $r$ is a root of a polynomial $q (x),$ then $(x - r)$ is a factor of $q (x) .$ Use this idea to explain why

p (λ) = (- 1)^{n} (λ - λ_{1}) (λ - λ_{2}) \dots (λ - λ_{n}) .

🔗

(ii)

Explain why $p (0) = λ_{1} λ_{2} \dots λ_{n} .$

🔗

(iii)

Why is $p (0)$ also equal to $det (A) .$ Explain how we have shown that $det (A)$ is the product of the eigenvalues of $A .$

🔗

2.

Find the eigenvalues of the following matrices. For each eigenvalue, determine its algebraic and geometric multiplicity.

🔗

(a)

$A = [\begin{array}{ccc} 1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{array}]$

🔗

(b)

$A = [\begin{array}{ccc} 2 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 1 & 2 \end{array}]$

🔗

3.

Let $A$ be an $n \times n$ matrix. Use the characteristic equation to explain why $A$ and $A^{T}$ have the same eigenvalues.

🔗

4.

Find three $3 \times 3$ matrices whose eigenvalues are 2 and 3, and for which the dimensions of the eigenspaces for $λ = 2$ and $λ = 3$ are different.

🔗

5.

Suppose $A$ is an $n \times n$ matrix and $B$ is an invertible $n \times n$ matrix. Explain why the characteristic polynomial of $A$ is the same as the characteristic polynomial of $B A B^{- 1},$ and hence, as a result, the eigenvalues of $A$ and $B A B^{- 1}$ are the same.

🔗

6.

Label each of the following statements as True or False. Provide justification for your response.

🔗

(a) True/False.

If the determinant of a $2 \times 2$ matrix $A$ is positive, then $A$ has two distinct real eigenvalues.

🔗

(b) True/False.

If two $2 \times 2$ matrices have the same eigenvalues, then the have the same eigenvectors.

🔗

(c) True/False.

The characteristic polynomial of an $n \times n$ matrix has degree $n .$

🔗

(d) True/False.

If $R$ is the reduced row echelon form of an $n \times n$ matrix $A,$ then $A$ and $R$ have the same eigenvalues.

🔗

(e) True/False.

If $R$ is the reduced row echelon form of an $n \times n$ matrix $A,$ and $v$ is an eigenvector of $A,$ then $v$ is an eigenvector of $R .$

🔗

(f) True/False.

Let $A$ and $B$ be $n \times n$ matrices with characteristic polynomials $p_{A} (λ)$ and $p_{B} (λ),$ respectively. If $A \neq B,$ then $p_{A} (λ) \neq p_{B} (λ) .$

🔗

(g) True/False.

Every matrix has at least one eigenvalue.

🔗

(h) True/False.

Suppose $A$ is a $3 \times 3$ matrix with three distinct eigenvalues. Then any three eigenvectors, one for each eigenvalue, will form a basis of $R^{3} .$

🔗

(i) True/False.

If an eigenvalue $λ$ is repeated 3 times among the eigenvalues of a matrix, then there are at most 3 linearly independent eigenvectors corresponding to $λ .$

🔗

Subsection Project: The Ehrenfest Model

🔗

To realistically model the diffusion of gas molecules we would need to consider a system with a large number of balls as substitutes for the gas molecules. However, the main idea can be seen in a model with a much smaller number of balls, as we will do now. Suppose we have two bins that contain a total of

4

balls between them. Label the bins as Bin 1 and Bin 2. In this case we can think of entropy as the number of different possible ways the balls can be arranged in the system. For example, there is only

1

way for all of the balls to be in Bin 1 (low entropy), but there are

4

ways that we can have one ball in Bin 1 (choose any one of the four different balls, which can be distinguished from each other) and

3

balls in Bin 2 (higher entropy). The highest entropy state has the balls equally distributed between the bins (with

6

different ways to do this).

🔗

We assume that there is a way for balls to move from one bin to the other (like having gas molecules pass through a permeable membrane). A way to think about this is that we select a ball (from ball 1 to ball 4, which are different balls) and move that ball from its current bin to the other bin. Consider a “move” to be any instance when a ball changes bins. A state is any configuration of balls in the bins at a given time, and the state changes when a ball is chosen at random and moved to the other bin. The possible states are to have 0 balls in Bin 1 and 4 balls in Bin 2 (State 0, entropy 1), 1 ball in Bin 1 and 3 in Bin 2 (State 1, entropy 4), 2 balls in each Bin (State 2, entropy 6), 3 balls in Bin 1 and 1 ball in Bin 2 (State 3, entropy 4), and 4 balls in Bin 1 and 0 balls in Bin 2 (State 4, entropy 1). These states are shown in Figure 18.10.

🔗

Project Activity 18.4.

To model the system of balls in bins we need to understand how the system can transform from one state to another. It suffices to count the number of balls in Bin 1 (since the remaining balls will be in Bin 2). Even though the balls are labeled, our count only cares about how many balls are in each bin. Let $x_{0} = [x_{0}, x_{1}, x_{2}, x_{3}, x_{4}]^{T},$ where $x_{i}$ is the probability that Bin 1 contains $i$ balls, and let $x_{1} = {[x_{0}^{1}, x_{1}^{1}, x_{2}^{1}, x_{3}^{1}, x_{4}^{1}]}^{T},$ where $x_{i}^{1}$ is the probability that Bin 1 contains $i$ balls after the first move. We will call the vectors $x_{0}$ and $x_{1}$ probability distributions of balls in bins. Note that since all four balls have to be placed in some bin, the sum of the entries in our probability distribution vectors must be $1 .$ Recall that a move is an instance when a ball changes bins. We want to understand how $x_{1}$ is obtained from $x_{0} .$ In other words, we want to figure out what the probability that Bin 1 contains 0, 1, 2, 3, or 4 balls after one ball changes bins if our initial probability distribution of balls in bins is $x_{0} .$

We begin by analyzing the ways that a state can change. For example,

Suppose there are $0$ balls in Bin 1. (In our probability distribution $x_{0},$ this happens with probability $x_{0} .$ ) Then there are four balls in Bin 2. The only way for a ball to change bins is if one of the four balls moves from Bin 2 to Bin 1, putting us in State 1. Regardless of which ball moves, we will always be put in State 1, so this happens with a probability of $1 .$ In other words, if the probability that Bin 1 contains $0$ balls is $x_{0},$ then there is a probability of $(1) x_{0}$ that Bin 1 will contain 1 ball after the move.
Suppose we have 1 ball in Bin 1. There are four ways this can happen (since there are four balls, and the one in Bin 1 is selected at random from the four balls), so the probability of a given ball being in Bin 1 is $\frac{1}{4} .$
- If the ball in Bin 1 moves, that move puts us in State $0 .$ In other words, if the probability that Bin 1 contains 1 ball is $x_{1},$ then there is a probability of $\frac{1}{4} x_{1}$ that Bin 1 will contain $0$ balls after a move.
- If any of the $3$ balls in Bin 2 moves (each moves with probability $\frac{3}{4}$ ), that move puts us in State 2. In other words, if the probability that Bin 1 contains 1 ball is $x_{1},$ then there is a probability of $\frac{3}{4} x_{1}$ that Bin 1 will contain $2$ balls after a move.

🔗

(a)

Complete this analysis to explain the probabilities if there are $2,$ $3,$ or $4$ balls in Bin 1.

🔗

(b)

Explain how the results of part (a) show that

\begin{aligned} x_{0}^{1} & = & 0 x_{0} & + & \frac{1}{4} x_{1} & + & 0 x_{2} & + & 0 x_{3} & + & 0 x_{4} \\ x_{1}^{1} & = & 1 x_{0} & + & 0 x_{1} & + & \frac{1}{2} x_{2} & + & 0 x_{3} & + & 0 x_{4} \\ x_{2}^{1} & = & 0 x_{0} & + & \frac{3}{4} x_{1} & + & 0 x_{2} & + & \frac{3}{4} x_{3} & + & 0 x_{4} \\ x_{3}^{1} & = & 0 x_{0} & + & 0 x_{1} & + & \frac{1}{2} x_{2} & + & 0 x_{3} & + & 1 x_{4} \\ x_{4}^{1} & = & 0 x_{0} & + & 0 x_{1} & + & 0 x_{2} & + & \frac{1}{4} x_{3} & + & 0 x_{4} \end{aligned}

🔗

The system we developed in Project Activity 18.4 has matrix form

x_{1} = T x_{0},

🔗

where

T

is the transition matrix

T = [\begin{array}{ccccc} 0 & \frac{1}{4} & 0 & 0 & 0 \\ 1 & 0 & \frac{1}{2} & 0 & 0 \\ 0 & \frac{3}{4} & 0 & \frac{3}{4} & 0 \\ 0 & 0 & \frac{1}{2} & 0 & 1 \\ 0 & 0 & 0 & \frac{1}{4} & 0 \end{array}] .

🔗

Subsequent moves give probability distribution vectors

\begin{aligned} x_{2} & = T x_{1} \\ x_{3} & = T x_{2} \\ ⋮ & ⋮ \\ x_{k} & = T x_{k - 1} . \end{aligned}

🔗

This example is an example of a Markov process (see Definition 9.4). There are several questions we can ask about this model. For example, what is the long-term behavior of this system, and how does this model relate to entropy? That is, given an initial probability distribution vector

x_{0},

the system will have probability distribution vectors

x_{1},

x_{2},

\dots

after subsequent moves. What happens to the vectors

x_{k}

k

goes to infinity, and what does this tell us about entropy? To answer these questions, we will first explore the sequence

{x_{k}}

numerically, and then use the eigenvalues and eigenvectors of

T

to analyze the sequence

{x_{k}} .

🔗

Project Activity 18.5.

Use appropriate technology to do the following.

🔗

(a)

Suppose we begin with a probability distribution vector $x_{0} = [1 0 0 0 0]^{T} .$ Calculate vectors $x_{k}$ for enough values of $k$ so that you can identify the long term behavior of the sequence. Describe this behavior.

🔗

(b)

Repeat part (a) with

🔗

(i)

$x_{0} = {[0 \frac{1}{2} \frac{1}{2} 0 0]}^{T}$

🔗

(ii)

$x_{0} = {[0 \frac{1}{3} \frac{1}{3} 0 \frac{1}{3}]}^{T}$

🔗

(iii)

$x_{0} = {[\frac{1}{5} \frac{1}{5} \frac{1}{5} \frac{1}{5} \frac{1}{5}]}^{T}$

🔗

In what follows, we investigate the behavior of the sequence

{x_{k}}

that we uncovered in Project Activity 18.5.

🔗

Project Activity 18.6.

We use the characteristic polynomial to find the eigenvalues of $T .$

🔗

(a)

Find the characteristic polynomial of $T .$ Factor the characteristic polynomial into a product of linear polynomials to show that the eigenvalues of $T$ are $0,$ $1,$ $- 1,$ $\frac{1}{2}$ and $- \frac{1}{2} .$

🔗

(b)

As we will see a bit later, certain eigenvectors for $T$ will describe the end behavior of the sequence ${x_{k}} .$ Find eigenvectors for $T$ corresponding to the eigenvalues $1$ and $- 1 .$ Explain how the eigenvector for $T$ corresponding to the eigenvalue $1$ explains the behavior of one of the sequences was saw in Project Activity 18.5. (Any eigenvector of $T$ with eigenvalue $1$ is called an equilibrium or steady state vector.)

🔗

Now we can analyze the behavior of the sequence

{x_{k}} .

🔗

Project Activity 18.7.

To make the notation easier, we will let $v_{1}$ be an eigenvector of $T$ corresponding to the eigenvalue $0,$ $v_{2}$ an eigenvector of $T$ corresponding to the eigenvalue $1,$ $v_{3}$ an eigenvector of $T$ corresponding to the eigenvalue $- 1,$ $v_{4}$ an eigenvector of $T$ corresponding to the eigenvalue $\frac{1}{2},$ and $v_{5}$ an eigenvector of $T$ corresponding to the eigenvalue $- \frac{1}{2} .$

🔗

(a)

Explain why ${v_{1}, v_{2}, v_{3}, v_{4}, v_{5}}$ is a basis of $R^{5} .$

🔗

(b)

Let $x_{0}$ be any initial probability distribution vector. Explain why we can write $x_{0}$ as

x_{0} = a_{1} v_{1} + a_{2} v_{2} + a_{3} v_{3} + a_{4} v_{4} + a_{5} v_{5} = \sum_{i = 1}^{5} a_{i} v_{i}

for some scalars $a_{1},$ $a_{2},$ $a_{3},$ $a_{4},$ and $a_{5} .$

🔗

We can now use the eigenvalues and eigenvectors of

T

to write the vectors

x_{k}

in a convenient form. Let

λ_{1} = 0,

λ_{2} = 1,

λ_{3} = - 1,

λ_{4} = \frac{1}{2},

and

λ_{5} = - \frac{1}{2} .

Notice that

\begin{aligned} x_{1} & = T x_{0} \\ = T (a_{1} v_{1} + a_{2} v_{2} + a_{3} v_{3} + a_{4} v_{4} + a_{5} v_{5}) \\ = a_{1} T v_{1} + a_{2} T v_{2} + a_{3} T v_{3} + a_{4} T v_{4} + a_{5} T v_{5} \\ = a_{1} λ_{1} v_{1} + a_{2} λ_{2} v_{2} + a_{3} λ_{3} v_{3} + a_{4} λ_{4} v_{4} + a_{5} λ_{5} v_{5} \\ = \sum_{i = 1}^{5} a_{i} λ_{i} v_{i} . \end{aligned}

🔗

Similarly

x_{2} = T x_{1} = T (\sum_{i = 1}^{5} a_{i} λ_{i} v_{i}) = \sum_{i = 1}^{5} a_{i} λ_{i} T v_{i} = \sum_{i = 1}^{5} a_{i} λ_{i}^{2} v_{i} .

🔗

We can continue in this manner to ultimately show that for each positive integer

k

we have

\begin{matrix} (18.4) & x_{k} = \sum_{i = 1}^{5} a_{i} λ_{i}^{k} v_{i} \end{matrix}

🔗

when

x_{0} = \sum_{i = 1}^{5} a_{i} v_{i} .

🔗

Project Activity 18.8.

Recall that we are interested in understanding the behavior of the sequence ${x_{k}}$ as $k$ goes to infinity.

🔗

(a)

Equation (18.4) shows that we need to know $lim_{k \to \infty} λ_{i}^{k}$ for each $i$ in order to analyze $lim_{k \to \infty} x_{k} .$ Calculate or describe these limits.

🔗

(b)

Use the result of part (a), Equation (18.4), and Project Activity 18.6 (b) to explain why the sequence ${x_{k}}$ is either eventually fixed or oscillates between two states. Compare to the results from Project Activity 18.5. How are these results related to entropy? You may use the facts that

$v_{1} = [1 0 - 2 0 1]^{T}$ is an eigenvector for $T$ corresponding to the eigenvalue $0,$
$v_{2} = [1 4 6 4 1]^{T}$ is an eigenvector for $T$ corresponding to the eigenvalue $1,$
$v_{3} = [1 - 4 6 - 4 1]^{T}$ is an eigenvector for $T$ corresponding to the eigenvalue $- 1,$
$v_{4} = [- 1 - 2 0 2 1]^{T}$ is an eigenvector for $T$ corresponding to the eigenvalue $\frac{1}{2},$
$v_{5} = [- 1 2 0 - 2 1]^{T}$ is an eigenvector for $T$ corresponding to the eigenvalue $- \frac{1}{2} .$

named after Paul and Tatiana Ehrenfest who introduced it in “Über zwei bekannte Einwände gegen das Boltzmannsche H-Theorem,” Physikalishce Zeitschrift, vol. 8 (1907), pp. 311-314)