Matrix Operations

Section 8 Matrix Operations

Focus Questions

By the end of this section, you should be able to give precise and thorough answers to the questions listed below. You may want to keep these questions in mind to focus your thoughts as you complete the section.

Under what conditions can we add two matrices and how is the matrix sum defined?
Under what conditions can we multiply a matrix by a scalar and how is a scalar multiple of a matrix defined?
Under what conditions can we multiply two matrices and how is the matrix product defined?
What properties do matrix addition, scalar multiplication of matrices and matrix multiplication satisfy? Are these properties similar to properties that are satisfied by vector operations?
What are two properties that make matrix multiplication fundamentally different than our standard product of real numbers?
What is the interpretation of matrix multiplication from the perspective of linear transformations?
How is the transpose of a matrix defined?

🔗

Subsection Application: Algorithms for Matrix Multiplication

🔗

Matrix multiplication is widely used in applications ranging from scientific computing and pattern recognition to counting paths in graphs. As a consequence, much work is being done in developing efficient algorithms for matrix multiplication.

🔗

We will see that a matrix product can be calculated through the row-column method. Recall that the product of two

2 \times 2

matrices

A = [\begin{array}{cc} a_{11} & a_{12} \\ a_{21} & a_{22} \end{array}]

and

B = [\begin{array}{cc} b_{11} & b_{12} \\ b_{21} & b_{22} \end{array}]

is given by

A B = [\begin{array}{cc} a_{11} b_{11} + a_{12} b_{21} & a_{11} b_{12} + a_{12} b_{22} \\ a_{21} b_{11} + a_{22} b_{21} & a_{21} b_{12} + a_{22} b_{22} \end{array}],

🔗

This product involves eight scalar multiplications and some scalar additions. As we will see, multiplication is more computationally expensive than addition, so we will focus on multiplication. In 1969, a German mathematician named Volker Strassen showed¹³ that the product of two

2 \times 2

matrices can be calculated using only seven multiplications. While this is not much of an improvement, the Strassen algorithm can be applied to larger matrices, using matrix partitions (which allow for parallel computation), and its publication led to additional research on faster algorithms for matrix multiplication. More details are provided later in this section.

🔗

Subsection Introduction

🔗

A vector is a list of numbers in a specified order and a matrix is an ordered array of objects. In fact, a vector can be thought of as a matrix of size

n \times 1 .

Vectors and matrices are so alike in this way that it would seem natural that we can define operations on matrices just as we did with vectors.

🔗

Recall that a matrix is made of rows and columns — the entries reading from left to right form the rows of the matrix and the entries reading from top to bottom form the columns. The number of rows and columns of a matrix is called the size of the matrix, so an

m \times n

matrix has

m

rows and

n

columns. If we label the entry in the

i

th row and

j

th column of a matrix

A

a_{i j},

then we write

A = [a_{i j}] .

🔗

We can generalize the operations of addition and scalar multiplication on vectors to matrices similarly. Given two matrices

A = [a_{i j}]

and

B = [b_{i j}]

of the same size, we define the sum

A + B

A + B = [a_{i j} + b_{i j}]

🔗

when the sizes of the matrices

A

and

B

match. In other words, for matrices of the same size the matrix addition is defined by adding corresponding entries in the matrices. For example,

[\begin{array}{rc} 1 & 2 \\ - 2 & 3 \end{array}] + [\begin{array}{cc} 1 & 3 \\ 2 & 4 \end{array}] = [\begin{array}{cc} 2 & 5 \\ 0 & 7 \end{array}] .

🔗

We define the scalar multiple of a matrix

A = [a_{i j}]

by scalar

c

to be the matrix

c A

defined by

c A = [c a_{i j}],

🔗

This means that we multiply each entry of the matrix

A

by the scalar

c .

As an example,

3 [\begin{array}{rc} 1 & 2 \\ - 2 & 3 \end{array}] = [\begin{array}{rc} 3 & 6 \\ - 6 & 9 \end{array}] .

🔗

Even though we did not have a multiplication operation on vectors, we had a matrix-vector product, which is a special case of a matrix-matrix product since a vector is a matrix with one column. However, generalizing the matrix-vector product to a matrix-matrix product is not immediate as it is not immediately clear what we can do with the other columns. We will consider this question in this section.

🔗

Note that all of the matrix operations can be performed on a calculator. After entering each matrix in the calculator, just use

+,

-

and × operations to find the result of the matrix operation. (Just for fun, try using

\div

with matrices to see if it will work.)

🔗

Preview Activity 8.1.

🔗

(a)

Pick three different varying sizes of pairs of $A, B$ matrices which can be added. For each pair:

🔗

(i)

Find the matrices $A + B$ and $B + A .$

🔗

(ii)

How are the two matrices $A + B$ and $B + A$ related? What does this tell us about matrix addition?

🔗

(b)

Let $A = [\begin{array}{rc} 1 & 0 \\ - 2 & 8 \end{array}],$ $B = [\begin{array}{cc} 1 & 1 \\ 3 & 4 \end{array}],$ and $C = [\begin{array}{cr} 0 & - 5 \\ 1 & 6 \end{array}] .$ Determine the entries of the matrix $A + 2 B - 7 C .$

🔗

(c)

Now we turn to multiplication of matrices. Our first goal is to find out what conditions we need on the sizes of matrices $A$ and $B$ if the matrix-matrix product $A B$ is defined and what the size of the resulting product matrix is. We know the condition and the size of the result in the special case of $B$ being a vector, i.e., a matrix with one column. So our conjectures for the general case should match what we know in the special case. In each part of this problem, use any appropriate tool (e.g., your calculator, Maple, Mathematica, Wolfram $|$ Alpha) to determine the matrix product $A B,$ if it exists. If you obtain a product, write it down and explain how its size is related to the sizes of $A$ and $B .$ If you receive an error, write down the error and guess why the error occurred and/or what it means.

🔗

(i)

$A = [\begin{array}{ccc} 1 & 2 & 0 \\ 0 & 1 & 1 \end{array}] and B = [\begin{array}{crc} 3 & 5 & 0 \\ 0 & - 2 & 1 \end{array}]$

🔗

(ii)

$A = [\begin{array}{ccc} 1 & 2 & 0 \\ 0 & 1 & 1 \end{array}] and B = [\begin{array}{crc} 3 & 0 \\ 5 & - 2 \\ 0 & 1 \end{array}]$

🔗

(iii)

$A = [\begin{array}{cc} 1 & 2 \\ 3 & 4 \end{array}] and B = [\begin{array}{ccc} 1 & 1 & 1 \\ 1 & 0 & 1 \\ 0 & 2 & 0 \end{array}]$

🔗

(iv)

$A = [\begin{array}{cc} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ 7 & 8 \end{array}] and B = [\begin{array}{rcc} 1 & 2 & 3 \\ - 1 & 1 & 1 \end{array}]$

🔗

(v)

Make a guess for the condition on the sizes of two matrices $A, B$ for which the product $A B$ is defined. How is the size of the product matrix related to the sizes of $A$ and $B ?$

🔗

(d)

The final matrix products, when defined, in problem 8.1.c might seem unrelated to the individual matrices at first. In this problem, we will uncover this relationship using our knowledge of the matrix-vector product. Let $A = [\begin{array}{rr} 3 & - 1 \\ - 2 & 3 \end{array}]$ and $B = [\begin{array}{ccc} 0 & 2 & 1 \\ 1 & 3 & 2 \end{array}] .$

🔗

(i)

Calculate $A B$ using any tool.

🔗

(ii)

Using the matrix-vector product, calculate $A x$ where $x$ is the first column (i.e., calculate $A [\begin{matrix} 0 \\ 1 \end{matrix}]$ ), and then the second column of $B$ (i.e., calculate $A [\begin{matrix} 2 \\ 3 \end{matrix}]$ ), and then the third column of $B$ (i.e., calculate $A [\begin{matrix} 1 \\ 2 \end{matrix}]$ ). Do you notice these output vectors within $A B ?$

🔗

(iii)

Describe as best you can a definition of $A B$ using the matrix-vector product.

🔗

Subsection Properties of Matrix Addition and Multiplication by Scalars

🔗

Just as we were able to define an algebra of vectors with addition and multiplication by scalars, we can define an algebra of matrices. We will see that the properties of these operations on matrices are immediate generalizations of the properties of the operations on vectors. We will then see how the matrix product arises through the connection of matrices to linear transformations. Finally, we define the transpose of a matrix. The transpose of a matrix will be useful in applications such as graph theory and least-squares fitting of curves, as well as in advanced topics such as inner product spaces and the dual space of a vector space.

🔗

We learned in Preview Activity 8.1 that we can add two matrices of the same size together by adding corresponding entries and we can multiply any matrix by a scalar by multiplying each entry of the matrix by that scalar. More generally, if

A = [a_{i j}]

and

B = [b_{i j}]

are

m \times n

matrices and

c

is any scalar, then

A + B = [a_{i j} + b_{i j}] and c A = [c a_{i j}] .

🔗

As we have done each time we have introduced a new operation, we ask what properties the operation has. For example, you determined in Preview Activity 8.1 that addition of matrices is a commutative operation. More specifically, for every two

m \times n

matrices

A

and

B,

A + B = B + A .

We can use similar arguments to verify the following properties of matrix addition and multiplication by scalars. Notice that these properties are very similar to the properties of addition and scalar multiplication of vectors we discussed earlier. This should come as no surprise since the

n

-dimensional vectors are

n \times 1

matrices. In a strange twist, we will see that matrices themselves can be considered as vectors when we discuss vector spaces in a later section.

🔗

Theorem 8.1.

Let $A,$ $B,$ and $C$ be $m \times n$ matrices and let $a$ and $b$ be scalars. Then

$A + B = B + A$ (this property tells us that matrix addition is commutative)
$(A + B) + C = A + (B + C)$ (this property tells us that matrix addition is associative)
The $m \times n$ matrix $0$ whose entries are all 0 has the property that $A + 0 = A .$ The matrix $0$ is called the zero matrix (It is generally clear from the context what the size of the 0 matrix is.).
The scalar multiple $(- 1) A$ of the matrix $A$ has the property that $(- 1) A + A = 0 .$ The matrix $(- 1) A = - A$ is called the additive inverse of the matrix $A .$
$(a + b) A = a A + b A$ (this property tells us that scalar multiplication of matrices distributes over scalar addition)
$a (A + B) = a A + a B$ (this property tells us that scalar multiplication of matrices distributes over matrix addition)
$(a b) A = a (b A)$
$1 A = A .$

🔗

Later on, we will see that these properties define the set of all

m \times n

matrices as a vector space. These properties just say that, regarding addition and multiplication by scalars, we can manipulate matrices just as we do real numbers. Note, however, we have not yet defined an operation of multiplication on matrices. That is the topic for the next section.

🔗

Subsection A Matrix Product

🔗

As we saw in Preview Activity 8.1, a matrix-matrix product can be found in a way which makes use of and also generalizes the matrix-vector product.

🔗

Definition 8.2.

The matrix product of a $k \times m$ matrix $A$ and an $m \times n$ matrix $B = [b_{1} b_{2} \dots b_{n}]$ with columns $b_{1},$ $b_{2},$ $\dots,$ $b_{n}$ is the $k \times n$ matrix

[A b_{1} A b_{2} \dots A b_{n}] .

🔗

We now consider the motivation behind this definition by thinking about the matrix transformations corresponding to each of the matrices

A, B

and

A B .

Recall that left multiplication by an

m \times n

matrix

B

defines a transformation

T

from

R^{n}

R^{m}

T (x) = B x .

The domain of

T

R^{n}

because the number of components of

x

have to match the number of entries in each of row of

B

in order for the matrix-vector product

B x

to be defined. Similarly, a

k \times m

matrix

A

defines a transformation

A

from

R^{m}

R^{k} .

Since transformations are functions, we can compose them as long as the output vectors of the inside transformation lie in the domain of the outside transformation. Therefore if

T

is the inside transformation and

S

is the outside transformation, the composition

S \circ T

is defined. So a natural question to ask is if we are given

a transformation $T$ from $R^{n}$ to $R^{m}$ where $T (x) = B x$ for an $m \times n$ matrix $B$ and
a transformation $S$ from $R^{m}$ to $R^{k}$ with $S (y) = A y$ for some $k \times m$ matrix $A,$

🔗

is there a matrix that represents the transformation

S \circ T

defined by

(S \circ T) (x) = S (T (x)) ?

We investigate this question in the next activity in the special case of a

2 \times 3

matrix

A

and a

3 \times 2

matrix

B .

🔗

Activity 8.2.

In this activity, we look for the meaning of the matrix product from a transformation perspective. Let $S$ and $T$ be matrix transformations defined by

S (y) = A y and T (x) = B x,

where

A = [\begin{array}{ccc} 1 & 2 & 0 \\ 0 & 1 & 1 \end{array}] and B = [\begin{array}{cr} 3 & 0 \\ 5 & - 2 \\ 0 & 1 \end{array}] .

🔗

(a)

What are the domains and codomains of $S$ and $T ?$ Why is the composite transformation $S \circ T$ defined? What is the domain of $S \circ T ?$ What is the codomain of $S \circ T ?$ (Recall that $S \circ T$ is defined by $(S \circ T) (x) = S (T (x)),$ i.e., we substitute the output $T (x)$ as the input into the transformation $S .$ )

🔗

(b)

Let $x = [\begin{matrix} x \\ y \end{matrix}] .$ Determine the components of $T (x) .$

🔗

(c)

Find the components of $S \circ T (x) = S (T (x)) .$

🔗

(d)

Find a matrix $C$ so that $S (T (x)) = C x .$

🔗

(e)

Use the definition of composition of transformations and the definitions of the $S$ and $T$ transformations to explain why it is reasonable to define $A B$ to be the matrix $C .$ Does the matrix $C$ agree with the

A B = [\begin{array}{cr} 13 & - 4 \\ 5 & - 1 \end{array}]

you found in Preview Activity 8.1 using technology?

🔗

We now consider this result in the general case of a

k \times m

matrix

A

and an

m \times n

matrix

B,

where

A

and

B

define matrix transformations

S

and

T,

respectively. In other words,

S

and

T

are matrix transformations defined by

S (x) = A x

and

T (x) = B x .

The domain of

S

R^{m}

and the codomain is

R^{k} .

The domain of

T

R^{n}

and the codomain is

R^{m} .

The composition

S \circ T

is defined because the output vectors of

T

are in

R^{m}

and they lie in the domain of

S .

The domain of

S \circ T

is the same as the domain of

T

since the input vectors first go through the

T

transformation. The codomain of

S \circ T

is the same as the codomain of

S

since the final output vectors are produced by applying the

S

transformation.

🔗

Let us see how we can obtain the matrix corresponding to the transformation

S \circ T .

Let

B = [b_{1} b_{2} \dots b_{n}],

where

b_{j}

is the

j

th column of

B,

and let

x = [\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{matrix}] .

Recall that the matrix vector product

B x

is the linear combination of the columns of

B

with the corresponding weights from

x .

T (x) = B x = x_{1} b_{1} + x_{2} b_{2} + \dots + x_{n} b_{n} .

🔗

Note that each of the

b_{j}

vectors are in

R^{m}

since

B

is an

m \times n

matrix. Therefore, each of these vectors can be multiplied by matrix

A

and we can evaluate

S (B x) .

Therefore,

S \circ T

is defined and

\begin{matrix} (8.1) & (S \circ T) (x) = S (T (x)) = A (B x) = A (x_{1} b_{1} + x_{2} b_{2} + \dots + x_{n} b_{n}) . \end{matrix}

🔗

The properties of matrix-vector products show that

\begin{matrix} (8.2) & A (x_{1} b_{1} + x_{2} b_{2} + \dots + x_{n} b_{n}) = x_{1} A b_{1} + x_{2} A b_{2} + \dots + x_{n} A b_{n} . \end{matrix}

🔗

This expression is a linear combination of

A b_{i}

's with

x_{i}

's being the weights. Therefore, if we let

C

be the matrix with columns

A b_{1},

A b_{2},

\dots,

A b_{n},

that is

C = [A b_{1} A b_{2} \dots A b_{n}],

🔗

then

\begin{matrix} (8.3) & x_{1} A b_{1} + x_{2} A b_{2} + \dots + x_{n} A b_{n} = C x \end{matrix}

🔗

by definition of the matrix-vector product. Combining equations (8.1), (8.2), and (8.3) shows that

(S \circ T) (x) = C x

🔗

where

C = [A b_{1} A b_{2} \dots A b_{n}] .

🔗

Also note that since

T (x) = B x

and

S (y) = A y,

we find

\begin{matrix} (8.4) & (S \circ T) (x) = S (T (x)) = S (B x) = A (B (x)) . \end{matrix}

🔗

Since the matrix representing the transformation

S \circ T

is the matrix

[A b_{1} A b_{2} \dots A b_{n}]

🔗

where

b_{1},

b_{2},

\dots,

b_{n}

are the columns of the matrix

B,

it is natural to define

A B

to be this matrix in light of equation (8.4).

🔗

Matrix multiplication has some properties that are unfamiliar to us as the next activity illustrates.

🔗

Activity 8.3.

Let $A = [\begin{array}{rr} 3 & - 1 \\ - 2 & 6 \end{array}],$ $B = [\begin{array}{cc} 0 & 2 \\ 1 & 3 \end{array}],$ $C = [\begin{array}{cc} 1 & 1 \\ 1 & 1 \end{array}],$ $D = [\begin{array}{rr} 3 & - 3 \\ - 3 & 3 \end{array}]$ and $E = [\begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array}] .$

🔗

(a)

Find the indicated products (by hand or using a calculator).

A B B A D C A C B C A E E B

🔗

(b)

Is matrix multiplication commutative? Explain.

🔗

(c)

Is there an identity element for matrix multiplication? In other words, is there a matrix $I$ for which $A I = I A = A$ for any matrix $A ?$ Explain.

🔗

(d)

If $a$ and $b$ are real numbers with $a b = 0,$ then we know that either $a = 0$ or $b = 0 .$ Is this same property true with matrix multiplication? Explain.

🔗

(e)

If $a,$ $b,$ and $c$ are real numbers with $c \neq 0$ and $a c = b c,$ we know that $a = b .$ Is this same property true with matrix multiplication? Explain.

🔗

As we saw in Activity 8.3, there are matrices

A, B

for which

A B \neq B A .

On the other hand, there are matrices for which

A B = B A .

For example, this equality will always hold for a square matrix

A

and if

B

is the identity matrix of the same size. It also holds if

A = B .

If the equality

A B = B A

holds, we say that matrices

A

and

B

commute. So the identity matrix commutes with all square matrices of the same size and every matrix

A

commutes with

A^{k}

for any power

k .

🔗

There is an alternative method of calculating a matrix product that we will often use that we illustrate in the next activity. This alternate version depends on the product of a row matrix with a vector. Suppose

A = [a_{1} a_{2} \dots a_{n}]

is a

1 \times n

matrix and

x = [\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{matrix}]

is an

n \times 1

vector. Then the product

A x

is the

1 \times 1

vector

[a_{1} a_{2} \dots a_{n}] [\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{matrix}] = [a_{1} x_{1} + a_{2} x_{2} + \dots + a_{n} x_{n}] .

🔗

In this situation, we usually identify the

1 \times 1

matrix with its scalar entry and write

\begin{matrix} (8.5) & [a_{1} a_{2} \dots a_{n}] \cdot [\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{matrix}] = a_{1} x_{1} + a_{2} x_{2} + \dots + a_{n} x_{n} . \end{matrix}

🔗

The product

\cdot

in (8.5) is called the scalar or dot product of

[a_{1} a_{2} \dots a_{n}]

with

[\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{n} \end{matrix}] .

🔗

Activity 8.4.

Let $A = [\begin{array}{crr} 1 & - 1 & 2 \\ 3 & 0 & - 4 \\ 2 & - 5 & 1 \end{array}]$ and $B = [\begin{array}{cr} 4 & - 2 \\ 6 & 0 \\ 1 & 3 \end{array}] .$

Let $a_{i}$ be the $i$ th row of $A$ and $b_{j}$ the $j$ th column of $B .$ For example, $a_{1} = [1 - 1 2]$ and $b_{2} = [\begin{array}{r} - 2 \\ 0 \\ 3 \end{array}] .$

Calculate the entries of the matrix $C,$ where

C = [\begin{array}{cc} a_{1} \cdot b_{1} & a_{1} \cdot b_{2} \\ a_{2} \cdot b_{1} & a_{2} \cdot b_{2} \\ a_{3} \cdot b_{1} & a_{3} \cdot b_{2} \end{array}],

where $a_{i} \cdot b_{j}$ refers to the scalar product of row $i$ of $A$ with column $j$ of $B .$ ¹⁴ Compare your result with the result of $A B$ calculated via the product of $A$ with the columns of $B .$

🔗

Activity 8.4 shows that these is an alternate way to calculate a matrix product. To see how this works in general, let

A = [a_{i j}]

be a

k \times m

matrix and

B = [b_{1} b_{2} \dots b_{n}]

m \times n

matrix. We know that

A B = [A b_{1} A b_{2} \dots A b_{n}] .

🔗

Now let

r_{1},

r_{2},

\dots,

r_{k}

be the rows of

A

so that

A = [\begin{matrix} r_{1} \\ r_{2} \\ ⋮ \\ r_{k} \end{matrix}] .

First we argue that if

x = [\begin{matrix} x_{1} \\ x_{2} \\ ⋮ \\ x_{m} \end{matrix}],

then

A x = [\begin{matrix} r_{1} \cdot x \\ r_{2} \cdot x \\ ⋮ \\ r_{k} \cdot x \end{matrix}] .

🔗

This is the scalar product (or dot product) definition of the matrix-vector product.

🔗

To show that this definition gives the same result as the linear combination definition of matrix-vector product, we first let

A = [c_{1} c_{2} \dots c_{m}],

where

c_{1},

c_{2},

\dots,

c_{m}

are the columns of

A .

By our linear combination definition of the matrix-vector product, we obtain

\begin{aligned} A x & = x_{1} c_{1} + x_{2} c_{2} + \dots + x_{m} c_{m} \\ = x_{1} [\begin{array}{c} a_{11} \\ a_{21} \\ ⋮ \\ a_{k 1} \end{array}] + x_{2} [\begin{array}{c} a_{12} \\ a_{22} \\ ⋮ \\ a_{k 2} \end{array}] + \dots + x_{m} [\begin{array}{c} a_{1 m} \\ a_{2 m} \\ ⋮ \\ a_{k m} \end{array}] \\ = [\begin{array}{c} a_{11} x_{1} + a_{12} x_{2} + \dots + a_{1 m} x_{m} \\ a_{21} x_{1} + a_{22} x_{2} + \dots + a_{2 m} x_{m} \\ ⋮ \\ a_{k 1} x_{1} + a_{k 2} x_{2} + \dots + a_{k m} x_{m} \end{array}] \\ = [\begin{array}{c} r_{1} \cdot x \\ r_{2} \cdot x \\ ⋮ \\ r_{k} \cdot x \end{array}] . \end{aligned}

🔗

Therefore, the above work shows that both linear combination and scalar product definitions give the same matrix-vector product.

🔗

Applying this to the matrix product

A B

defined in terms of the matrix-vector product, we see that

A b_{j} = [\begin{matrix} r_{1} \cdot b_{j} \\ r_{2} \cdot b_{j} \\ ⋮ \\ r_{k} \cdot b_{j} \end{matrix}] .

🔗

So the

i, j

th entry of the matrix product

A B

is found by taking the scalar product of the

i

th row of

A

with the

j

th column of

B .

In other words,

(A B)_{i j} = r_{i} \cdot b_{j}

🔗

where

r_{i}

is the

i

th row of

A

and

b_{j}

is the

j

th column of

B .

🔗

Subsection Properties of Matrix Multiplication

🔗

Activity 8.3 shows that we must be very careful not to assume that matrix multiplication behaves like multiplication of real numbers. However, matrix multiplication does satisfy some familiar properties. For example, we now have an addition and multiplication of matrices under certain conditions, so we might ask if matrix multiplication distributes over matrix addition. To answer this question we take two arbitrary

k \times m

matrices

A

and

B

and an arbitrary

m \times n

matrix

C = [c_{1} c_{2} \dots c_{n}] .

Then

\begin{aligned} (A + B) C & = [(A + B) c_{1} (A + B) c_{2} \dots (A + B) c_{n}] \\ = [A c_{1} + B c_{1} A c_{2} + B c_{2} \dots A c_{n} + B c_{n}] \\ = [A c_{1} A c_{2} \dots A c_{n}] + [B c_{1} B c_{2} \dots B c_{n}] \\ = A C + B C . \end{aligned}

🔗

Similar arguments can be used to show the following properties of matrix multiplication.

🔗

Theorem 8.3.

Let $A,$ $B,$ and $C$ be matrices of the appropriate sizes for all sums and products to be defined and let $a$ be a scalar. Then

$(A B) C = A (B C)$ (this property tells us that matrix multiplication is associative)
$(A + B) C = A C + B C$ (this property tells us that matrix multiplication on the right distributes over matrix addition)
$A (B + C) = A B + A C$ (this property tells us that matrix multiplication on the left distributes over matrix addition)
There is a square matrix $I_{n}$ with the property that $A I_{n} = A$ or $I_{n} A = A$ for whichever product is defined.
$a (A B) = (a A) B = A (a B)$

🔗

We verified the second part of this theorem and will assume that all of the properties of this theorem hold. The matrix

I_{n}

introduced in Theorem 8.3 is called the (multiplicative) identity matrix. We usually omit the word multiplicative and refer to the

I_{n}

simply as the identity matrix. This does not cause any confusion since we refer to the additive identity matrix as simply the zero matrix.

🔗

Definition 8.4.

Let $n$ be a positive integer. The $n \times n$ identity matrix $I_{n}$ is the matrix $I_{n} = [a_{i j}],$ where $a_{i i} = 1$ for each $i$ and $a_{i j} = 0$ if $i \neq j .$

🔗

We also write the matrix

I_{n}

I_{n} = [\begin{array}{ccccccc} 1 & 0 & 0 & 0 & \dots & 0 & 0 \\ 0 & 1 & 0 & 0 & \dots & 0 & 0 \\ 0 & 0 & 1 & 0 & \dots & 0 & 0 \\ ⋮ & ⋱ & ⋮ \\ 0 & 0 & 0 & 0 & \dots & 1 & 0 \\ 0 & 0 & 0 & 0 & \dots & 0 & 1 \end{array}] .

🔗

The matrix

I_{n}

has the property that for any

n \times n

matrix

A,

A I_{n} = I_{n} A = A .

🔗

I_{n}

is a multiplicative identity in the set of all

n \times n

matrices. More generally, for an

m \times n

matrix

A,

A I_{n} = I_{m} A = A .

🔗

Subsection The Transpose of a Matrix

🔗

One additional operation on matrices is the transpose. The transpose of a matrix occurs in many useful formulas in linear algebra and in applications of linear algebra.

🔗

Definition 8.5.

The transpose of an $m \times n$ matrix $A = [a_{i j}]$ is the $n \times m$ matrix $A^{T}$ whose $i, j$ th entry is $a_{j i} .$

🔗

Written out, the transpose of the

m \times n

matrix

A = [\begin{array}{ccccc} a_{11} & a_{12} & \dots & a_{1 n - 1} & a_{1 n} \\ a_{21} & a_{22} & \dots & a_{2 n - 1} & a_{2 n} \\ ⋮ & ⋱ & ⋮ \\ a_{m 1} & a_{m 2} & \dots & a_{m n - 1} & a_{m n} \end{array}]

🔗

is the

n \times m

matrix

A^{T} = [\begin{array}{ccccc} a_{11} & a_{21} & \dots & a_{m - 11} & a_{m 1} \\ a_{12} & a_{22} & \dots & a_{m - 12} & a_{m 2} \\ ⋮ & ⋱ & ⋮ \\ a_{1 n} & a_{2 n} & \dots & a_{m - 1 n} & a_{m n} \end{array}] .

🔗

In other words, the transpose of a matrix

A

is the matrix

A^{T}

whose rows are the columns of

A .

Alternatively, the transpose of

A

is the matrix

A^{T}

whose columns are the rows of

A .

We can also view the transpose of

A

as the reflection of

A

across its main diagonal, where the diagonal of a matrix

A = [a_{i j}]

consists of the entries of the form

[a_{i i}] .

🔗

Activity 8.5.

🔗

(a)

Find the transpose of each of the indicated matrices.

[\begin{array}{cccc} 1 & 2 & 3 & 4 \\ 5 & 6 & 7 & 8 \end{array}] [\begin{array}{r} 1 \\ - 1 \\ 0 \end{array}] [\begin{array}{cr} 1 & 2 \\ 4 & - 3 \\ 0 & - 1 \end{array}]

🔗

(b)

Find the transpose of the new matrix for each part above. What can you conjecture based on your results? There are certain special types of matrices that are given names.

🔗

(c)

There are certain special types of matrices that are given names.

🔗

Definition 8.6.

Let $A$ be a square matrix whose $i j$ th entry is $a_{i j} .$

The matrix $A$ is a diagonal matrix if $a_{i j} = 0$ whenever $i \neq j .$
The matrix $A$ is a symmetric matrix if $A^{T} = A .$
The matrix $A$ is an upper triangular if $a_{i j} = 0$ whenever $i > j .$
The matrix $A$ is a lower triangular if $a_{i j} = 0$ whenever $i < j .$

🔗

(i)

Find an example of a diagonal matrix $A .$ What can you say about $A^{T} ?$

🔗

(ii)

Find an example of a non-diagonal symmetric matrix $B .$ If $B^{T} = B,$ must $B$ be a square matrix?

🔗

(iii)

Find an example of an upper triangular matrix $C .$ What kind of a matrix is $C^{T} ?$

🔗

We will see later that diagonal matrices are important in that their powers are easy to calculate. Symmetric matrices arise frequently in applications such as in graph theory as adjacency matrices and in quantum mechanics as observables, and have many useful properties including being diagonalizable and having real eigenvalues, as we will also see later.

🔗

Subsection Properties of the Matrix Transpose

🔗

As with every other operation, we want to understand what properties the matrix transpose has. Properties of transposes are shown in the following theorem.

🔗

Theorem 8.7.

Let $A$ and $B$ be matrices of the appropriate sizes and let $a$ be a scalar. Then

${(A^{T})}^{T} = A$
$(A + B)^{T} = A^{T} + B^{T}$
$(A B)^{T} = B^{T} A^{T}$
$(a A)^{T} = a A^{T}$

🔗

The one property that might seem strange is the third one. To understand this property, suppose

A

is an

m \times n

matrix and

B

n \times k

matrix so that the product

A B

is defined. We will argue that

(A B)^{T} = B^{T} A^{T}

by comparing the

i, j

th entry of each side.

First notice that the $i, j$ th entry of $(A B)^{T}$ is the $j, i$ th entry of $A B .$ The $j, i$ th entry of $A B$ is found by taking the scalar product of the $j$ th row of $A$ with the $i$ th column of $B .$ Thus, the $i, j$ th entry of $(A B)^{T}$ is the scalar product of the $j$ th row of $A$ with the $i$ th column of $B .$
The $i, j$ th entry of $B^{T} A^{T}$ is the scalar product of the $i$ th row of $B^{T}$ with the $j$ th column of $A^{T} .$ But the $i$ th row of $B^{T}$ is the $i$ th column of $B$ and the $j$ th column of $A^{T}$ is the $j$ th row of $A .$ So the $i, j$ th entry of $B^{T} A^{T}$ is the scalar product of the $j$ th row of $A$ with the $i$ th column of $B .$

🔗

Since the two matrices

(A B)^{T}

and

B^{T} A^{T}

have the same size and same corresponding entries, they are the same matrix.

🔗

Subsection Examples

🔗

What follows are worked examples that use the concepts from this section.

🔗

Example 8.8.

Let

\begin{array}{ccc} A = [\begin{array}{ccrc} 1 & 2 & 0 & 1 \\ 3 & 0 & - 4 & 5 \\ 7 & 6 & - 1 & 0 \end{array}] & B = [\begin{array}{rcr} - 2 & 4 & - 3 \\ 5 & 1 & 9 \\ 1 & 1 & - 2 \end{array}] \\ C = [\begin{array}{crc} 0 & - 1 & 6 \\ 3 & - 2 & 5 \\ 1 & 0 & 4 \end{array}] & D = [\begin{array}{cr} 10 & - 4 \\ 5 & 2 \\ 8 & - 1 \end{array}] \\ E = [\begin{array}{cr} 1 & 0 \\ 4 & - 3 \\ 5 & - 1 \end{array}] & and & F = [\begin{array}{rcr} - 2 & 1 & 5 \\ 6 & 3 & - 8 \\ 1 & 0 & - 1 \\ 7 & 0 & - 5 \end{array}] . \end{array} .

Determine the results of the following operations, if defined. If not defined, explain why.

🔗

(a)

$A F$

Solution.

Since $A$ is a $3 \times 4$ matrix and $F$ is a $4 \times 3$ matrix, the number of columns of $A$ equals the number of rows of $F$ and the matrix produce $A F$ is defined. Recall that if $F = [f_{1} f_{2} f_{3}],$ where $f_{1},$ $f_{2},$ $f_{3}$ are the columns of $F,$ then $A F = [A f_{1} A f_{2} A f_{3}] .$ Recall also that $A f_{1}$ is the linear combination of the columns of $A$ with weights from $f_{1},$ so

\begin{aligned} A f_{1} & = [\begin{array}{ccrc} 1 & 2 & 0 & 1 \\ 3 & 0 & - 4 & 5 \\ 7 & 6 & - 1 & 0 \end{array}] [\begin{array}{r} - 2 \\ 6 \\ 1 \\ 7 \end{array}] \\ = (- 2) [\begin{array}{c} 1 \\ 3 \\ 7 \end{array}] + (6) [\begin{array}{c} 2 \\ 0 \\ 6 \end{array}] + (1) [\begin{array}{r} 0 \\ - 4 \\ - 1 \end{array}] + (7) [\begin{array}{c} 1 \\ 5 \\ 0 \end{array}] \\ = [\begin{array}{c} - 2 + 12 + 0 + 7 \\ - 6 + 0 - 4 + 35 \\ - 14 + 36 - 1 + 0 \end{array}] \\ = [\begin{array}{c} 17 \\ 25 \\ 21 \end{array}], \end{aligned}

\begin{aligned} A f_{2} & = [\begin{array}{ccrc} 1 & 2 & 0 & 1 \\ 3 & 0 & - 4 & 5 \\ 7 & 6 & - 1 & 0 \end{array}] [\begin{array}{c} 1 \\ 3 \\ 0 \\ 0 \end{array}] \\ = (1) [\begin{array}{c} 1 \\ 3 \\ 7 \end{array}] + (3) [\begin{array}{c} 2 \\ 0 \\ 6 \end{array}] + (0) [\begin{array}{r} 0 \\ - 4 \\ - 1 \end{array}] + (0) [\begin{array}{c} 1 \\ 5 \\ 0 \end{array}] \\ = [\begin{array}{c} 1 + 6 + 0 + 0 \\ 3 + 0 + 0 + 0 \\ 7 + 18 + 0 + 0 \end{array}] \\ = [\begin{array}{c} 7 \\ 3 \\ 25 \end{array}], \end{aligned}

and

\begin{aligned} A f_{3} & = [\begin{array}{ccrc} 1 & 2 & 0 & 1 \\ 3 & 0 & - 4 & 5 \\ 7 & 6 & - 1 & 0 \end{array}] [\begin{array}{r} 5 \\ - 8 \\ - 1 \\ - 5 \end{array}] \\ = (5) [\begin{array}{c} 1 \\ 3 \\ 7 \end{array}] - (8) [\begin{array}{c} 2 \\ 0 \\ 6 \end{array}] - (1) [\begin{array}{r} 0 \\ - 4 \\ - 1 \end{array}] - (5) [\begin{array}{c} 1 \\ 5 \\ 0 \end{array}] \\ = [\begin{array}{c} 5 - 16 - 0 - 5 \\ 15 - 0 + 4 - 25 \\ 35 - 48 + 1 - 0 \end{array}] \\ = [\begin{array}{r} - 16 \\ - 6 \\ - 12 \end{array}] . \end{aligned}

So $A F = [\begin{array}{ccr} 17 & 7 & - 16 \\ 25 & 3 & - 6 \\ 21 & 25 & - 12 \end{array}] .$ Alternatively, if $A = [\begin{matrix} a_{1} \\ a_{2} \\ a_{3} \\ a_{4} \end{matrix}],$ then the matrix product $A F$ is the matrix whose $i j$ entry is $a_{i} \cdot f_{j} .$ Using this method we have

A F = [\begin{array}{ccc} a_{1} \cdot f_{1} & a_{1} \cdot f_{2} & a_{1} \cdot f_{3} \\ a_{2} \cdot f_{1} & a_{2} \cdot f_{2} & a_{2} \cdot f_{3} \\ a_{3} \cdot f_{1} & a_{3} \cdot f_{2} & a_{3} \cdot f_{3} \end{array}] .

Now

\begin{aligned} a_{1} \cdot f_{1} & = (1) (- 2) + (2) (6) + (0) (1) + (1) (7) = 17 \\ a_{1} \cdot f_{2} & = (1) (1) + (2) (3) + (0) (0) + (1) (0) = 7 \\ a_{1} \cdot f_{3} & = (1) (5) + (2) (- 8) + (0) (- 1) + (1) (- 5) = - 16 \\ a_{2} \cdot f_{1} & = (3) (- 2) + (0) (6) + (- 4) (1) + (5) (7) = 25 \\ a_{2} \cdot f_{2} & = (3) (1) + (0) (3) + (- 4) (0) + (5) (0) = 3 \\ a_{2} \cdot f_{3} & = (3) (5) + (0) (- 8) + (- 4) (- 1) + (5) (- 5) = - 6 \\ a_{3} \cdot f_{1} & = (7) (- 2) + (6) (6) + (- 1) (1) + (0) (7) = 21 \\ a_{3} \cdot f_{2} & = (7) (1) + (6) (3) + (- 1) (0) + (0) (0) = 25 \\ a_{3} \cdot f_{3} & = (7) (5) + (6) (- 8) + (- 1) (- 1) + (0) (- 5) = - 12, \end{aligned}

so $A F = [\begin{array}{ccr} 17 & 7 & - 16 \\ 25 & 3 & - 6 \\ 21 & 25 & - 12 \end{array}] .$

🔗

(b)

$A (B C)$

Solution.

Since $B C$ is a $3 \times 3$ matrix but $A$ is $3 \times 4,$ the number of columns of $A$ is not equal to the number of rows of $B C .$ We conclude that $A (B C)$ is not defined.

🔗

(c)

$(B C) A$

Solution.

Since $B C$ is a $3 \times 3$ matrix and $A$ is $3 \times 4,$ the number of columns of $B C$ is equal to the number of rows of $A .$ Thus, the quantity $(B C) A$ is defined. First we calculate $B C$ using the dot product of the rows of $B$ with the columns of $C .$ Letting $B = [\begin{matrix} b_{1} \\ b_{2} \\ b_{3} \end{matrix}]$ and $C = [c_{1} c_{2} c_{3}],$ where $b_{1},$ $b_{2},$ and $b_{3}$ are the rows of $B$ and $c_{1},$ $c_{2},$ and $c_{3}$ are the columns of $C,$ we have

B C = [\begin{array}{ccc} b_{1} \cdot c_{1} & b_{1} \cdot c_{2} & b_{1} \cdot c_{3} \\ b_{2} \cdot c_{1} & b_{2} \cdot c_{2} & b_{2} \cdot c_{3} \\ b_{3} \cdot c_{1} & b_{3} \cdot c_{2} & b_{3} \cdot c_{3} \end{array}] .

Now

\begin{aligned} b_{1} \cdot c_{1} & = (- 2) (0) + (4) (3) + (- 3) (1) = 9 \\ b_{1} \cdot c_{1} & = (- 2) (- 1) + (4) (- 2) + (- 3) (0) = - 6 \\ b_{1} \cdot c_{1} & = (- 2) (6) + (4) (5) + (- 3) (4) = - 4 \\ b_{1} \cdot c_{1} & = (5) (0) + (1) (3) + (9) (1) = 12 \\ b_{1} \cdot c_{1} & = (5) (- 1) + (1) (- 2) + (9) (0) = - 7 \\ b_{1} \cdot c_{1} & = (5) (6) + (1) (5) + (9) (4) = 71 \\ b_{1} \cdot c_{1} & = (1) (0) + (1) (3) + (- 2) (1) = 1 \\ b_{1} \cdot c_{1} & = (1) (- 1) + (1) (- 2) + (- 2) (0) = - 3 \\ b_{1} \cdot c_{1} & = (1) (6) + (1) (5) + (- 2) (4) = 3, \end{aligned}

so $B C = [\begin{array}{crr} 9 & - 6 & - 4 \\ 12 & - 7 & 71 \\ 1 & - 3 & 3 \end{array}] .$ If $B C = [\begin{matrix} r_{1} \\ r_{2} \\ r_{3} \end{matrix}]$ and $A = [s_{1} s_{2} s_{3} s_{4}],$ where $r_{1},$ $r_{2},$ and $r_{3}$ are the rows of $B C$ and $s_{1},$ $s_{2},$ $s_{3},$ and $s_{4}$ are the columns of $A,$ then

(B C) A = [\begin{array}{cccc} r_{1} \cdot s_{1} & r_{1} \cdot s_{2} & r_{1} \cdot r_{3} & r_{1} \cdot s_{4} \\ r_{2} \cdot s_{1} & r_{2} \cdot s_{2} & r_{2} \cdot s_{3} & r_{2} \cdot s_{4} \\ r_{3} \cdot s_{1} & r_{3} \cdot s_{2} & r_{3} \cdot s_{3} & r_{3} \cdot s_{4} \end{array}] .

Now

\begin{aligned} r_{1} \cdot s_{1} & = (9) (1) + (- 6) (3) + (- 4) (7) = - 37 \\ r_{1} \cdot s_{2} & = (9) (2) + (- 6) (0) + (- 4) (6) = - 6 \\ r_{1} \cdot s_{3} & = (9) (0) + (- 6) (- 4) + (- 4) (- 1) = 28 \\ r_{1} \cdot s_{4} & = (9) (1) + (- 6) (5) + (- 4) (0) = - 21 \\ r_{2} \cdot s_{1} & = (12) (1) + (- 7) (3) + (71) (7) = 488 \\ r_{2} \cdot s_{2} & = (12) (2) + (- 7) (0) + (71) (6) = 450 \\ r_{2} \cdot s_{3} & = (12) (0) + (- 7) (- 4) + (71) (- 1) = - 43 \\ r_{2} \cdot s_{4} & = (12) (1) + (- 7) (5) + (71) (0) = - 23 \\ r_{3} \cdot s_{1} & = (1) (1) + (- 3) (3) + (3) (7) = 13 \\ r_{3} \cdot s_{2} & = (1) (2) + (- 3) (0) + (3) (6) = 20 \\ r_{3} \cdot s_{3} & = (1) (0) + (- 3) (- 4) + (3) (- 1) = 9 \\ r_{3} \cdot s_{4} & = (1) (1) + (- 3) (5) + (3) (0) = - 14, \end{aligned}

so $(B C) A = [\begin{array}{rrrr} - 37 & - 6 & 28 & - 21 \\ 488 & 450 & - 43 & - 23 \\ 13 & 20 & 9 & - 14 \end{array}] .$

🔗

(d)

$(B + C) D$

Solution.

Since $B$ and $C$ are both $3 \times 3$ matrices, their sum is defined and is a $3 \times 3$ matrix. Because $D$ is $3 \times 2$ matrix, the number of columns of $B + C$ is equal to the number of rows of $D .$ Thus, the quantity $(B + C) D$ is defined and, using the row-column method of matrix multiplication as earlier,

\begin{aligned} (B + C) D & = ([\begin{array}{rcr} - 2 & 4 & - 3 \\ 5 & 1 & 9 \\ 1 & 1 & - 2 \end{array}] + [\begin{array}{crc} 0 & - 1 & 6 \\ 3 & - 2 & 5 \\ 1 & 0 & 4 \end{array}]) [\begin{array}{cr} 10 & - 4 \\ 5 & 2 \\ 8 & - 1 \end{array}] \\ = [\begin{array}{ccc} - 2 + 0 & 4 - 1 & - 3 + 6 \\ 5 + 3 & 1 - 2 & 9 + 5 \\ 1 + 1 & 1 + 0 & - 2 + 4 \end{array}] [\begin{array}{cr} 10 & - 4 \\ 5 & 2 \\ 8 & - 1 \end{array}] \\ = [\begin{array}{rrc} - 2 & 3 & 3 \\ 8 & - 1 & 14 \\ 2 & 1 & 2 \end{array}] [\begin{array}{cr} 10 & - 4 \\ 5 & 2 \\ 8 & - 1 \end{array}] \\ = [\begin{array}{cr} 19 & 11 \\ 187 & - 48 \\ 41 & - 8 \end{array}] . \end{aligned}

🔗

(e)

$D^{T} E$

Solution.

Since $D^{T}$ is a $2 \times 3$ matrix and $E$ is $3 \times 2,$ the number of columns of $D^{T}$ is equal to the number of rows of $E .$ Thus, $D^{T} E$ is defined and

\begin{aligned} D^{T} E & = {[\begin{array}{cr} 10 & - 4 \\ 5 & 2 \\ 8 & - 1 \end{array}]}^{T} [\begin{array}{cr} 1 & 0 \\ 4 & - 3 \\ 5 & - 1 \end{array}] \\ = [\begin{array}{rcr} 10 & 5 & 8 \\ - 4 & 2 & - 1 \end{array}] [\begin{array}{cr} 1 & 0 \\ 4 & - 3 \\ 5 & - 1 \end{array}] \\ = [\begin{array}{rr} 70 & - 23 \\ - 1 & - 5 \end{array}] . \end{aligned}

🔗

(f)

${(A^{T} + F)}^{T}$

Solution.

The fact that $A$ is a $3 \times 4$ matrix means that $A^{T}$ is a $4 \times 3$ matrix. Since $F$ is also a $4 \times 3$ matrix, the sum $A^{T} + F$ is defined. The transpose of any matrix is also defined, so ${(A^{T} + F)}^{T}$ is defined and

\begin{aligned} {(A^{T} + F)}^{T} & = {({[\begin{array}{ccrc} 1 & 2 & 0 & 1 \\ 3 & 0 & - 4 & 5 \\ 7 & 6 & - 1 & 0 \end{array}]}^{T} + [\begin{array}{rcr} - 2 & 1 & 5 \\ 6 & 3 & - 8 \\ 1 & 0 & - 1 \\ 7 & 0 & - 5 \end{array}])}^{T} \\ = {([\begin{array}{crr} 1 & 3 & 7 \\ 2 & 0 & 6 \\ 0 & - 4 & - 1 \\ 1 & 5 & 0 \end{array}] + [\begin{array}{rcr} - 2 & 1 & 5 \\ 6 & 3 & - 8 \\ 1 & 0 & - 1 \\ 7 & 0 & - 5 \end{array}])}^{T} \\ = {([\begin{array}{ccc} 1 - 2 & 3 + 1 & 7 + 5 \\ 2 + 6 & 0 + 3 & 6 - 8 \\ 0 + 1 & - 4 + 0 & - 1 - 1 \\ 1 + 7 & 5 + 0 & 0 - 5 \end{array}])}^{T} \\ = {([\begin{array}{rrr} - 1 & 4 & 12 \\ 8 & 3 & - 2 \\ 1 & - 4 & - 2 \\ 8 & 5 & - 5 \end{array}])}^{T} \\ = [\begin{array}{rrrr} - 1 & 8 & 1 & 8 \\ 4 & 3 & - 4 & 5 \\ 12 & - 2 & - 2 & - 5 \end{array}] . \end{aligned}

🔗

Example 8.9.

Let $A = [\begin{array}{cr} 2 & - 1 \\ 7 & - 2 \end{array}]$ and $B = [\begin{array}{rc} 4 & 6 \\ - 3 & 5 \end{array}] .$

🔗

(a)

Determine the matrix sum $A + B .$ Then use this sum to calculate $(A + B)^{2} .$

Solution.

Adding corresponding terms shows that $A + B = [\begin{array}{cc} 6 & 5 \\ 4 & 3 \end{array}] .$ Squaring this sum yields the result $(A + B)^{2} = [\begin{array}{cc} 56 & 45 \\ 36 & 29 \end{array}] .$

🔗

(b)

Now calculate $(A + B)^{2}$ in a different way. Use the fact that matrix multiplication distributes over matrix addition to expand (like foiling) $(A + B)^{2}$ into a sum of matrix products. The calculate each summand and add to find $(A + B)^{2} .$ You should obtain the same result as part (a). If not, what could be wrong?

Solution.

Expanding $(A + B)^{2}$ (remember that matrix multiplication is not commutative) gives us

\begin{aligned} (A + B)^{2} & = (A + B) (A + B) \\ = A^{2} + A B + B A + B^{2} \\ = [\begin{array}{rr} - 3 & 0 \\ 0 & - 3 \end{array}] + [\begin{array}{cc} 11 & 7 \\ 34 & 32 \end{array}] + [\begin{array}{cr} 50 & - 16 \\ 29 & - 7 \end{array}] + [\begin{array}{rc} - 2 & 54 \\ - 27 & 7 \end{array}] \\ = [\begin{array}{cc} 56 & 45 \\ 36 & 29 \end{array}] \end{aligned}

just as in part (a). If instead you obtained the matrix $[\begin{array}{cc} 17 & 68 \\ 41 & 68 \end{array}]$ you likely made the mistake of equating $(A + B)^{2}$ with $A^{2} + 2 A B + B^{2} .$ These two matrices are not equal in general, because we cannot say that $A B$ is equal to $B A .$

🔗

Subsection Summary

🔗

In this section we defined a matrix sum, scalar multiples of matrices, the matrix product, and the transpose of a matrix.

The sum of two $m \times n$ matrices $A = [a_{i j}]$ and $B = [b_{i j}]$ is the $m \times n$ matrix $A + B$ whose $i, j$ th entry is $a_{i j} + b_{i j} .$
If $A = [a_{i j}]$ is an $m \times n$ matrix, the scalar multiple $k A$ of $A$ by the scalar $k$ is the $m \times n$ matrix whose $i, j$ th entry is $k a_{i j} .$
If $A$ is a $k \times m$ matrix and $B = [b_{1} b_{2} \dots b_{n}]$ is an $m \times n$ matrix, then the matrix product $A B$ of the matrices $A$ and $B$ is the $k \times n$ matrix

$[A b_{1} A b_{2} \dots A b_{n}] .$

The matrix product is defined in this way so that the matrix of a composite $S \circ T$ of linear transformations is the product of matrices of $S$ and $T .$
An alternate way of calculating the product of an $k \times m$ matrix $A$ with rows $r_{1},$ $r_{2},$ $\dots,$ $r_{k}$ and an $m \times n$ matrix $B$ with columns $b_{1},$ $b_{2},$ $\dots,$ $b_{n}$ is that the product $A B$ is the $k \times n$ matrix whose $i, j$ th entry is $r_{i} \cdot b_{j} .$
Matrix multiplication does not behave as the standard multiplication on real numbers. For example, we can have a product of two non-zero matrices equal to the zero matrix and there is no cancellation law for matrix multiplication.
The transpose of an $m \times n$ matrix $A = [a_{i j}]$ is the $n \times m$ matrix $A^{T}$ whose $i, j$ th entry is $a_{j i} .$

🔗

Exercises Exercises

🔗

1.

Calculate $A B$ for each of the following matrix pairs by hand in two ways.

🔗

(a)

$A = [\begin{array}{cc} 1 & 0 \\ 0 & 1 \\ 0 & 0 \end{array}],$ $B = [\begin{array}{cc} a & b \\ c & d \end{array}]$

🔗

(b)

$A = [\begin{array}{ccr} 1 & 0 & - 1 \end{array}],$ $B = [\begin{array}{cc} 1 & 2 \\ 2 & 3 \\ 3 & 4 \end{array}]$

🔗

2.

For each of the following $A$ matrices, find all $2 \times 2$ matrices $B = [\begin{array}{cc} a & b \\ c & d \end{array}]$ which commute with the given $A .$ (Two matrices $A$ and $B$ commute with each other if $A B = B A .$ )

🔗

(a)

$A = [\begin{array}{cc} 2 & 0 \\ 0 & 2 \end{array}]$

🔗

(b)

$A = [\begin{array}{cc} 2 & 0 \\ 0 & 3 \end{array}]$

🔗

(c)

$A = [\begin{array}{cc} 0 & 1 \\ 0 & 0 \end{array}]$

🔗

3.

Find all possible, if any, $X$ matrices satisfying each of the following matrix equations.

🔗

(a)

$[\begin{array}{cc} 1 & 2 \\ 0 & 2 \end{array}] X = [\begin{array}{cc} 0 & 1 \\ 0 & 0 \end{array}]$

🔗

(b)

$[\begin{array}{rr} 1 & - 2 \\ - 2 & 4 \end{array}] X = [\begin{array}{cc} 0 & 1 \\ 0 & 0 \end{array}]$

🔗

(c)

$[\begin{array}{rr} 1 & - 2 \\ - 2 & 4 \end{array}] X = [\begin{array}{cc} 0 & 1 \\ 0 & - 2 \end{array}]$

🔗

4.

For each of the following $A$ matrices, compute $A^{2} = A A, A^{3} = A A A, A^{4} .$ Use your results to conjecture a formula for $A^{m} .$ Interpret your answer geometrically using the transformation interpretation.

🔗

(a)

$A = [\begin{array}{cc} 2 & 0 \\ 0 & 3 \end{array}]$

🔗

(b)

$A = [\begin{array}{cc} 1 & 1 \\ 0 & 1 \end{array}]$

🔗

(c)

$A = [\begin{array}{cr} 0 & - 1 \\ 1 & 0 \end{array}]$

🔗

5.

If $A v = 2 v$ for unknown $A$ matrix and $v$ vector, determine an expression for $A^{2} v,$ $A^{3} v,$ …, $A^{m} v .$

🔗

6.

If $A v = 2 v$ and $A u = 3 u,$ find an expression for $A^{m} (a v + b u)$ in terms of $v$ and $u .$

🔗

7.

A matrix $A$ is a nilpotent matrix if $A^{m} = 0,$ i.e., $A^{m}$ is the zero matrix, for some positive integer $m .$ Explain why the matrices

A = [\begin{array}{cc} 0 & a \\ 0 & 0 \end{array}], B = [\begin{array}{ccc} 0 & a & b \\ 0 & 0 & c \\ 0 & 0 & 0 \end{array}]

are nilpotent matrices.

🔗

8.

Suppose $A$ is an $n \times n$ matrix for which $A^{2} = 0 .$ Show that there is a matrix $B$ for which $(I_{n} + A) B = I_{n}$ where $I_{n}$ is the identity matrix of size $n .$

🔗

9.

Let $A,$ $B,$ and $C$ be $m \times n$ matrices and let $a$ and $b$ be scalars. Verify Theorem 8.1. That is, show that

🔗

(a)

$A + B = B + A$

🔗

(b)

$(A + B) + C = A + (B + C)$

🔗

(c)

The $m \times n$ matrix $0$ whose entries are all 0 has the property that $A + 0 = A .$

🔗

(d)

The scalar multiple $(- 1) A$ of the matrix $A$ has the property that $(- 1) A + A = 0 .$

🔗

(e)

$(a + b) A = a A + b A$

🔗

(f)

$a (A + B) = a A + a B$

🔗

(g)

$(a b) A = a (b A)$

🔗

(h)

$1 A = A .$

🔗

10.

Let $A,$ $B,$ and $C$ be matrices of the appropriate sizes for all sums and products to be defined and let $a$ be a scalar. Verify the remaining parts of Theorem 8.3. That is, show that

🔗

(a)

$(A B) C = A (B C)$

🔗

(b)

$A (B + C) = A B + A C$

🔗

(c)

There is a square matrix $I_{n}$ with the property that $A I_{n} = A$ or $I_{n} A = A$ for whichever product is defined.

🔗

(d)

$a (A B) = (a A) B = A (a B)$

🔗

11.

Let $A = [a_{i j}]$ and $B = [b_{i j}]$ be matrices of the appropriate sizes, and let $a$ be a scalar. Verify the remaining parts of Theorem 8.7. That is, show that

🔗

(a)

${(A^{T})}^{T} = A$

🔗

(b)

$(A + B)^{T} = A^{T} + B^{T}$

🔗

(c)

$(a A)^{T} = a A^{T}$

🔗

12.

The matrix exponential is an important tool in solving differential equations. Recall from calculus that the Taylor series expansion for $e^{x}$ centered at $x = 0$ is

e^{x} = \sum_{n = 0}^{\infty} \frac{x^{n}}{n!} = 1 + x + \frac{x^{2}}{2!} + \frac{x^{3}}{3!} + \dots,

and that this Taylor series converges to $e^{x}$ for every real number $x .$ We extend this idea to define the matrix exponential $e^{A}$ for any square matrix $A$ with real entries as

e^{A} = \sum_{n = 0}^{\infty} \frac{1}{n!} A^{n} = I_{n} + A + \frac{1}{2!} A^{2} + \frac{1}{3!} A^{3} + \dots

We explore this idea with an example. Let $B = [\begin{array}{cr} 2 & 0 \\ 0 & - 1 \end{array}] .$

🔗

(a)

Calculate $B^{2},$ $B^{3},$ $B^{4} .$ Explain why $B^{n} = [\begin{array}{cc} 2^{n} & 0 \\ 0 & (- 1)^{n} \end{array}]$ for any positive integer $n .$

🔗

(b)

Show that $I_{2} + B + B^{2} + B^{3} + B^{4}$ is equal to

[\begin{array}{cc} 1 + 2 + \frac{2^{2}}{2} + \frac{2^{3}}{3!} + \frac{2^{4}}{4!} & 0 \\ 0 & 1 + (- 1) + \frac{(- 1)^{2}}{2} + \frac{(- 1)^{3}}{3!} + \frac{(- 1)^{4}}{4!} \end{array}] .

🔗

(c)

Explain why $e^{B} = [\begin{array}{cc} e^{2} & 0 \\ 0 & e^{- 1} \end{array}] .$

🔗

13.

Show that if $A$ and $B$ are $2 \times 2$ rotation matrices, then $A B$ is also a $2 \times 2$ rotation matrix.

🔗

14.

Label each of the following statements as True or False. Provide justification for your response. Throughout, assume that matrices are of the appropriate sizes so that any matrix sums or products are defined.

🔗

(a) True/False.

For any three matrices $A, B, C$ with $A \neq 0,$ $A B = A C$ implies $B = C .$

🔗

(b) True/False.

For any three matrices $A, B, C$ with $A \neq 0,$ $A B = C A$ implies $B = C .$

🔗

(c) True/False.

If $A^{2}$ is the zero matrix, then $A$ itself is the zero matrix.

🔗

(d) True/False.

If $A B = B A$ for every $n \times n$ matrix $B,$ then $A$ is the identity matrix $I_{n} .$

🔗

(e) True/False.

If matrix products $A B$ and $B A$ are both defined, then $A$ and $B$ are both square matrices of the same size.

🔗

(f) True/False.

If $x_{1}$ is a solution for $A x = b_{1}$ (i.e., that $A x_{1} = b_{1}$ ) and $x_{2}$ is a solution for $B x = b_{2},$ then $x_{1} + x_{2}$ is a solution for $(A + B) x = b_{1} + b_{2} .$

🔗

(g) True/False.

If $B$ is an $m \times n$ matrix with two equal columns, then the matrix $A B$ has two equal columns for every $k \times m$ matrix.

🔗

(h) True/False.

If $A^{2} = I_{2},$ then $A = - I_{2}$ or $A = I_{2} .$

🔗

Subsection Project: Strassen's Algorithm and Partitioned Matrices

🔗

Strassen's algorithm is an algorithm for matrix multiplication that can be more efficient than the standard row-column method. To understand this method, we begin with the

2 \times 2

case which will highlight the essential ideas.

🔗

Project Activity 8.6.

We first work with the $2 \times 2$ case.

🔗

(a)

Let $A = [a_{i j}] = [\begin{array}{cc} 1 & 2 \\ 3 & 4 \end{array}]$ and $B = [b_{i j}] = [\begin{array}{cc} 5 & 6 \\ 7 & 8 \end{array}] .$

🔗

(i)

Calculate the matrix product $A B .$

🔗

(ii)

Rather than using eight multiplications to calculate $A B,$ Strassen came up with the idea of using the following seven products:

\begin{aligned} h_{1} & = (a_{11} + a_{22}) (b_{11} + b_{22}) \\ h_{2} & = (a_{21} + a_{22}) b_{11} \\ h_{3} & = a_{11} (b_{12} - b_{22}) \\ h_{4} & = a_{22} (b_{21} - b_{11}) \\ h_{5} & = (a_{11} + a_{12}) b_{22} \\ h_{6} & = (a_{21} - a_{11}) (b_{11} + b_{12}) \\ h_{7} & = (a_{12} - a_{22}) (b_{21} + b_{22}) . \end{aligned}

Calculate $h_{1}$ through $h_{7}$ for the given matrices $A$ and $B .$ Then calculate the quantities

h_{1} + h_{4} - h_{5} + h_{7}, h_{3} + h_{5}, h_{2} + h_{4}, and h_{1} + h_{3} - h_{2} + h_{6} .

What do you notice?

🔗

(b)

Now we repeat part (a) in general. Suppose we want to calculate the matrix product $A B$ for arbitrary $2 \times 2$ matrices $A = [\begin{array}{cc} a_{11} & a_{12} \\ a_{21} & a_{22} \end{array}]$ and $B = [\begin{array}{cc} b_{11} & b_{12} \\ b_{21} & b_{22} \end{array}] .$ Let

\begin{aligned} h_{1} & = (a_{11} + a_{22}) (b_{11} + b_{22}) \\ h_{2} & = (a_{21} + a_{22}) b_{11} \\ h_{3} & = a_{11} (b_{12} - b_{22}) \\ h_{4} & = a_{22} (b_{21} - b_{11}) \\ h_{5} & = (a_{11} + a_{12}) b_{22} \\ h_{6} & = (a_{21} - a_{11}) (b_{11} + b_{12}) \\ h_{7} & = (a_{12} - a_{22}) (b_{21} + b_{22}) . \end{aligned}

Show that

A B = [\begin{array}{cc} h_{1} + h_{4} - h_{5} + h_{7} & h_{3} + h_{5} \\ h_{2} + h_{4} & h_{1} + h_{3} - h_{2} + h_{6} \end{array}] .

🔗

The next step is to understand how Strassen's algorithm can be applied to larger matrices. This involves the idea of partitioned (or block) matrices. Recall that the matrix-matrix product of the

k \times m

matrix

A

and the

m \times n

matrix

B = [b_{1} b_{2} \dots b_{n}]

is defined as

A B = [A b_{1} A b_{2} \dots A b_{n}] .

🔗

In this process, we think of

B

as being partitioned into

n

columns. We can expand on this idea to partition both

A

and

B

when calculating a matrix-matrix product.

🔗

Project Activity 8.7.

We illustrate the idea of partitioned matrices with an example. Let $A = [\begin{array}{crcrc} 1 & - 2 & 3 & - 6 & 4 \\ 7 & 5 & 2 & - 1 & 0 \\ 3 & - 8 & 1 & 0 & 9 \end{array}] .$ We can partition $A$ into smaller matrices

A = [\begin{array}{crcrc} 1 & - 2 & 3 & - 6 & 4 \\ 7 & 5 & 2 & - 1 & 0 \\ 3 & - 8 & 1 & 0 & 9 \end{array}],

which are indicated by the vertical and horizontal lines. As a shorthand, we can describe this partition of $A$ as

A = [\begin{array}{cc} A_{11} & A_{12} \\ A_{21} & A_{22} \end{array}],

where $A_{11} = [\begin{array}{crc} 1 & - 2 & 3 \\ 7 & 5 & 2 \end{array}],$ $A_{12} = [\begin{array}{rc} - 6 & 4 \\ - 1 & 0 \end{array}],$ $A_{21} = [\begin{array}{crc} 3 & - 8 & 1 \end{array}],$ and $A_{22} = [0 9] .$ The submatrices $A_{i j}$ are called blocks. If $B$ is a matrix such that $A B$ is defined, then $B$ must have five rows. As an example, $A B$ is defined if $B = [\begin{array}{cc} 1 & 3 \\ 2 & 0 \\ 4 & 1 \\ 6 & 5 \\ 4 & 2 \end{array}] .$ The partition of $A$ breaks $A$ up into blocks with three and two columns, respectively. So if we partition $B$ into blocks with three and two rows, then we can use the blocks to calculate the matrix product $A B .$ For example, partition $B$ as

B = [\begin{array}{cc} 1 & 3 \\ 2 & 0 \\ 4 & 1 \\ 6 & 5 \\ 4 & 2 \end{array}] = [\begin{matrix} B_{11} \\ B_{21} \end{matrix}] .

Show that

A B = [\begin{array}{cc} A_{11} & A_{12} \\ A_{21} & A_{22} \end{array}] [\begin{matrix} B_{11} \\ B_{21} \end{matrix}] = [\begin{array}{cc} A_{11} B_{11} + A_{12} B_{21} \\ A_{21} B_{11} + A_{22} B_{21} \end{array}] .

🔗

An advantage to using partitioned matrices is that computations with them can be done in parallel, which lessens the time it takes to do the work. In general, we can multiply partitioned matrices as though the submatrices are scalars. That is,

[\begin{array}{cccc} A_{11} & A_{12} & \dots & A_{1 m} \\ A_{21} & A_{22} & \dots & A_{2 m} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ A_{i 1} & A_{i 2} & \dots & A_{i m} \\ ⋮ & ⋮ & ⋱ & ⋮ \\ A_{k 1} & A_{k 2} & \dots & A_{k m} \end{array}] [\begin{array}{cccccc} B_{11} & B_{12} & \dots & B_{1 j} & \dots & B_{1 n} \\ B_{21} & B_{22} & \dots & B_{2 j} & \dots & B_{2 n} \\ ⋮ & ⋮ & ⋱ & ⋮ & ⋱ & ⋮ \\ B_{m 1} & B_{m 2} & \dots & B_{m j} & \dots & B_{m n} \end{array}] = [P_{i j}],

🔗

where

P_{i j} = A_{i 1} B_{1 j} + A_{i 2} B_{2 j} + \dots + A_{i m} B_{m j} = \sum_{t = 1}^{m} A_{i t} B_{t j},

🔗

provided that all the submatrix products are defined.

🔗

Now we can apply Strassen's algorithm to larger matrices using partitions. This method is sometimes referred to as divide and conquer.

🔗

Project Activity 8.8.

Let $A$ and $B$ be two $r \times r$ matrices. If $r$ is not a power of $2,$ then pad the rows and columns of $A$ and $B$ with zeros to make them of size $2^{m} \times 2^{m}$ for some integer $m .$ (From a practical perspective, we might instead just use unequal block sizes.) Let $n = 2^{m} .$ Partition $A$ and $B$ as

A = [\begin{array}{cc} A_{11} & A_{12} \\ A_{21} & A_{22} \end{array}] and B = [\begin{array}{cc} B_{11} & B_{12} \\ B_{21} & B_{22} \end{array}],

where each submatrix is of size $\frac{n}{2} \times \frac{n}{2} .$ Now we use the Strassen algorithm just as in the $2 \times 2$ case, treating the submatrices as if they were scalars (with the additional constraints of making sure that the dimensions match up so that products are defined, and ensuring we multiply in the correct order). Letting

\begin{aligned} M_{1} & = (A_{11} + A_{22}) (B_{11} + B_{22}) \\ M_{2} & = (A_{21} + A_{22}) B_{11} \\ M_{3} & = A_{11} (B_{12} - B_{22}) \\ M_{4} & = A_{22} (B_{21} - B_{11}) \\ M_{5} & = (A_{11} + A_{12}) B_{22} \\ M_{6} & = (A_{21} - A_{11}) (B_{11} + B_{12}) \\ M_{7} & = (A_{12} - A_{22}) (B_{21} + B_{22}), \end{aligned}

then the same algebra as in Project Activity 8.6 shows that

A B = [\begin{array}{cc} M_{1} + M_{4} - M_{5} + M_{7} & M_{3} + M_{5} \\ M_{2} + M_{4} & M_{1} + M_{3} - M_{2} + M_{6} \end{array}] .

Apply Strassen's algorithm to calculate the matrix product $A B,$ where

A = [\begin{array}{crr} 1 & 3 & - 1 \\ 2 & 4 & 6 \\ 7 & - 2 & 5 \end{array}] and B = [\begin{array}{crc} 2 & 5 & 3 \\ 2 & - 4 & 1 \\ 1 & 6 & 4 \end{array}] .

🔗

While Strassen's algorithm can be more efficient, it does not always speed up the process. We investigate this in the next activity.

🔗

Project Activity 8.9.

We introduce a little notation to help us describe the efficiency of our calculations. We won't be formal with this notation, rather work with it in an informal way. Big O (the letter “O” ) notation is used to describe the complexity of an algorithm. Generally speaking, in computer science big O notation can be used to describe the run time of an algorithm, the space used by the algorithm, or the number of computations required. The letter “O” is used because the behavior described is also called the order. Big O measures the asymptotic time of an algorithm, not its exact time. For example, if it takes $6 n^{2} - n + 8$ steps to complete an algorithm, then we say that the algorithm grows at the order of $n^{2}$ (we ignore the constants and the smaller power terms, since they become insignificant as $n$ increases) and we describe its growth as $O (n^{2}) .$ To measure the efficiency of an algorithm to determine a matrix product, we will measure the number of operations it takes to calculate the product.

🔗

(a)

Suppose $A$ and $B$ are $n \times n$ matrices. Explain why the operation of addition (that is, calculating $A + B$ ) is $O (n^{2}) .$

🔗

(b)

Suppose $A$ and $B$ are $n \times n$ matrices. How many multiplications are required to calculate the matrix product $A B ?$ Explain.

🔗

(c)

The standard algorithm for calculating a matrix product of two $n \times n$ matrices requires $n^{3}$ multiplications and a number of additions. Since additions are much less costly in terms of operations, the standard matrix product is $O (n^{3}) .$ We won't show it here, but using Strassen's algorithm on a product of $2^{m} \times 2^{m}$ matrices is $O (n^{\log_{2} (7)}),$ where $n = 2^{m} .$ That means that Strassen's algorithm applied to an $n \times n$ matrix (where $n$ is a power of $2$ ) requires approximately $n^{\log_{2} (7)}$ multiplications. We use this to analyze situations to determine when Strassen's algorithm is computationally more efficient than the standard algorithm.

🔗

(i)

Suppose $A$ and $B$ are $5 \times 5$ matrices. Determine the number of multiplications required to calculate the matrix product $A B$ using the standard matrix product. Then determine the approximate number of multiplications required to calculate the matrix product $A B$ using Strassen's algorithm. Which is more efficient? (Remember, we can only apply Strassen's algorithm to square matrices whose sizes are powers of $2 .$ )

🔗

(ii)

Repeat part i. with $125 \times 125$ matrices. Which method is more efficient?

🔗

As a final note, Strassen's algorithm is approximately

O (n^{2.81}) .

As of 2018, the best algorithm for matrix multiplication, developed by Virginia Williams at Stanford University, is approximately

O (n^{2.373}) .

¹⁵

Strassen, Volker, Gaussian Elimination is not Optimal, Number. Math. 13, p. 354-356, 1969

Recall from Exercise 5 of Section 5 that the scalar product

u \cdot v

of a

1 \times n

matrix

u = [u_{1} u_{2} \dots u_{n}]

and an

n \times 1

vector

v = [\begin{matrix} v_{1} \\ v_{2} \\ ⋮ \\ v_{n} \end{matrix}]

u \cdot v = u_{1} v_{1} + u_{2} v_{2} + u_{3} v_{3} + \dots + u_{n} v_{n} .

V. V. Williams, Multiplying matrices in

O (n^{2.373})

time, Stanford University, (2014).