An Inquiry-Based Introduction to Linear Algebra and Applications

Subsection Application: The Knight's Tour

Chess is a game played on an \(8 \times 8\) grid which utilizes a variety of different pieces. One piece, the knight, is different from the other pieces in that it can jump over other pieces. However, the knight is limited in how far it can move in a given turn. For these reasons, the knight is a powerful, but often under-utilized, piece.

A knight can move two units either horizontally or vertically, and one unit perpendicular to that. Four knight moves are as illustrated in Figure 4.1, and the other four moves are the opposites of these.

The knight's tour problem is the mathematical problem of finding a knight's tour, that is a sequence of knight moves so the the knight visits each square exactly once. While we won't consider a knight's tour in this text, we will see using linear combinations of vectors that a knight can move from its initial position to any other position on the board, and that it is possible to determine a sequence of moves to make that happen.

Subsection Introduction

So far we learned of a convenient method to represent a linear system using matrices. We now consider another representation of a linear system using vectors. Vectors can represent concepts in the physical world like velocity, acceleration, and force — but we will be interested in vectors as algebraic objects in this class. Vectors will form the foundation for everything we will do in linear algebra. For now, the following definition will suffice.

Any set that satisfies the properties listed in Theorem 4.3 is called a field. Our vectors are made from elements of a field, we call those elements of the field scalars.

We will algebraically represent a vector as a matrix with one column. For example, \(\vv = \left[ \begin{array}{c} 1\\ 2 \end{array} \right]\) is a vector with 2 entries, and we say that \(\vv\) is a vector in 2-space. By 2-space we mean \(\R^2\text{,}\) which can be geometrically modeled as the plane. Here the symbol \(\R\) indicates that the entries of \(\vv\) are real numbers and the superscript 2 tells us that \(\vv\) has two entries. Similarly, vectors in \(\R^3\) have three entries, e.g., \(\left[ \begin{array}{r} 1\\ 3\\ -1 \end{array} \right]\text{.}\) The collection of column vectors with three entries can be geometrically modeled as three-dimensional space. If a vector \(\vv\) has \(n\) entries we say that \(\vv\) is a vector in \(\R^n\) (or \(n\)-space). Vectors are also often indicated with arrows, so we might also see a vector \(\vv\) written as \(\overrightarrow{v}\text{.}\) It is important when writing to differentiate between a vector \(\vv\) and a scalar \(v\text{.}\) These are quite different objects and it is up to us to make sure we are clear what a symbol represents. We will use boldface letters to represent vectors.

A vector like \(\left[ \begin{array}{c} 1\\ 2 \end{array} \right]\) is called a column vector of size \(2 \times 1\) (two rows, one column). We can define an addition operation on two vectors of the same size by adding corresponding components, such as

Similarly, we can define scalar multiplication of a vector by multiplying each component of the vector by the scalar. For example,

Since we can add vectors and multiply vectors by scalars, we can then add together scalar multiples of vectors. For completeness, we define vector subtraction as adding a scalar multiple:

This definition is equivalent to defining subtraction of \(\vu\) from \(\vv\) by subtracting components of \(\vu\) from the corresponding components of \(\vv\text{.}\)

Subsection Vectors and Vector Operations

As discussed in Preview Activity 4.1, a vector is simply a list of numbers. We can add vectors of like size and multiply vectors by scalars. These operations define a structure on the set of all vectors with the same number of components that will be our major object of study in linear algebra. Ultimately we will expand our idea of vectors to a more general context and study what we will call vector spaces.

In Preview Activity 4.1 we saw how to add vectors and multiply vectors by scalars in \(\R^2\text{,}\) and this idea extends to \(\R^n\) for any \(n\text{.}\) Before we do so, one thing we didn't address in Preview Activity 4.1 is what it means for two vectors to be equal. It should seem reasonable that two vectors are equal if and only if they have the same corresponding components. More formally, if we let

be vectors in \(\R^n\text{,}\) then \(\vu = \vv\) if \(u_i = v_i\) for every \(i\) between 1 and \(n\text{.}\) Note that this statement implies that a vector in \(\R^2\) cannot equal a vector in \(\R^3\) because they don't have the same number of components. With this in mind we can now define the sum \(\vu + \vv\) of the vectors \(\vu\) and \(\vv\) to be the vector in \(\R^n\) defined by

In other words, to add two vectors of the same size, we add corresponding components.

Similarly, we can define scalar multiplication of a vector. If \(c\) is a scalar, then the scalar multiple \(c \vv\) of the vector \(\vv\) is the vector in \(\R^n\) defined by

In other words, the scalar multiple \(c\vv\) of the vector \(\vv\) is the vector obtained by multiplying each component of the vector \(\vv\) by the scalar \(c\text{.}\) Since we can add vectors and multiply vectors by scalars, we can then add together scalar multiples of vectors. For completeness, we define vector subtraction as adding a scalar multiple:

This definition is equivalent to defining subtraction of \(\vu\) from \(\vv\) by subtracting components of \(\vu\) from the corresponding components of \(\vv\text{.}\)

After defining operations on objects, we should wonder what kinds of properties these operations have. For example, with the operation of addition of real numbers we know that \(1+2\) is equal to \(2+1\text{.}\) This is called the commutative property of scalar addition and says that order does not matter when we add real numbers. It is natural for us to ask if similar properties hold for the vector operations, addition and scalar multiplication, we defined. You showed in Preview Activity 4.1 that the addition operation is also commutative on vectors in \(\R^2\text{.}\)

In the activity below we consider how the two operations, addition and scalar multiplication, interact with each other. In real numbers, we know that multiplication is distributive over addition. Is that true with vectors as well?

Similar arguments can be used to show the following properties of vector addition and multiplication by scalars.

We will later see that the above properties make the set \(\R^n\) a vector space. These properties just say that, for the most part, we can manipulate vectors just as we do real numbers. Please note, though, that there is no multiplication or division of vectors.

Subsection Geometric Representation of Vectors and Vector Operations

We can geometrically represent a vector \(\vv = \left[ \begin{array}{c} v_1\\ v_2 \end{array} \right]\) in \(\R^2\) as the point \((v_1, v_2)\) in the plane as we did in Preview Activity 4.1. We can similarly represent a vector \(\vv = \left[ \begin{array}{c} v_1\\ v_2\\ v_3 \end{array} \right]\) in \(\R^3\) as the point \((v_1, v_2, v_3)\) in the three-dimensional space. This geometric representation will be handy when we consider collections of infinitely many vectors, as we will do when we consider the span of a collection of vectors later in this section.

We can also represent the vector \(\vv = \left[ \begin{array}{c} v_1\\ v_2 \end{array} \right]\) in \(\R^2\) as the directed line segment (or arrow) from the origin to the point \((v_1, v_2)\) as shown in Figure 4.5 to aid in the visualization.

The fact that the vector in Figure 4.5 is represented by the directed line segment from the origin to the point (4,6) means that this vector is the vector \(\vv = \left[ \begin{array}{c} 4\\ 6 \end{array} \right]\text{.}\) If \(O\) is the origin and \(P\) is the point \((4,6)\text{,}\) we will also denote this vector as \(\overrightarrow{OP}\) — so

In this way we can think of vectors as having direction and length. With the Pythagorean Theorem, we can see that the length of a vector \(\vv = \left[ \begin{array}{c} v_1\\v_2 \end{array} \right]\) is \(\sqrt{v_1^2+v_2^2}\text{.}\) This idea can be applied to vectors in any space. If \(\vv = \left[ \begin{array}{c} v_1 \\ v_2 \\ v_3 \\ \vdots \\ \ v_n \end{array} \right]\) is a vector in \(\R^n\text{,}\) then the length of \(\vv\text{,}\) denoted \(|\vv|\) is the scalar

Thinking of vectors having direction and length is especially useful in visualizing the addition of vectors. The geometric interpretation of the sum of two vectors can be seen in Figure 4.6 and Figure 4.7.

Let \(\vu = \left[ \begin{array}{c} 4\\ 6 \end{array} \right]\) and \(\vv = \left[ \begin{array}{r} 3\\ -2 \end{array} \right]\text{.}\) Then \(\vu + \vv = \left[ \begin{array}{c} 7\\ 4 \end{array} \right]\) as shown in Figure 4.6. Figure 4.7 provides a context to interpret this vector sum geometrically. Using the parallelogram imposed on the three vectors, we see that if vectors \(\vu\) and \(\vv\) are both placed to start at the origin, then the vector sum \(\vu + \vv\) can be visualized geometrically as the directed line segment from the origin to the fourth corner of the parallelogram.

In Preview Activity 4.1 we considered scalar multiples of a vector in \(\R^2\text{.}\) The arrow representation helps in visualizing scalar multiples as well. Geometrically, a scalar multiple \(c\vv\) of a nonzero vector \(\vv\) is a vector in the same direction as \(\vv\) if \(c>0\) and in the opposite direction as \(\vv\) if \(c\lt 0\text{.}\) If \(c>1\text{,}\) scalar multiplication stretches the vector, while \(0\lt c\lt 1\) shrinks the vector. We also saw that the collection of all scalar multiples of a vector \(\vv\) in \(\R^2\) gives us a line through the origin and \(\vv\text{,}\) except when \(\vv=\vzero\) in which case we only obtain \(\vzero\text{.}\) In other words, for a nonzero vector \(\vv\text{,}\) the set \(S = \{c\vv : c \text{ is a scalar } \}\) is the line through the origin and \(\vv\) in \(\R^2\text{.}\)

All of these properties generalize to vectors in \(\R^3\text{.}\) Specifically, the scalar multiple \(c\vv\) is a vector in the same or opposite direction as \(\vv\) based on the sign of \(c\text{,}\) and is a stretched or shrunken version of \(\vv\) based on whether \(|c|>1\) or \(|c|\lt 1\text{.}\) Also, the collection of all multiples of a non-zero vector \(\vv\) in \(\R^3\) form a line through the origin.

Subsection Linear Combinations of Vectors

The concept of linear combinations is one of the fundamental ideas in linear algebra. We will use linear combinations to describe almost every important concept in linear algebra — the span of a set of vectors, the range of a linear transformation, bases, the dimension of a vector space — to name just a few.

In Preview Activity 4.1, we considered the sets of all scalar multiples of a single nonzero vector in \(\R^2\) and in \(\R^3\text{.}\) We also considered the set of all sums of scalar multiples of two nonzero vectors. These results so far gives us an idea of geometrical descriptions of sets of vectors generated by one or two vectors. Oftentimes we are interested in what vectors can be made from a given collection of vectors. For example, suppose we have two different water-benzene-acetic acid chemical solutions, one with 40% water, 50% benzene and 10% acetic acid, the other with 52% water, 42% benzene and 6% acid. An experiment we want to conduct requires a chemical solution with 43% water, 48% benzene and 9% acid. We would like to know if we make this new chemical solution by mixing the first two chemical solutions, or do we have to run to the chemical solutions market to get the chemical solution we want.

We can set up a system of equations for each ingredient and find the answer. But we can also consider each chemical solution as a vector, where the components represent the water, benzene and acid percentages. So the two chemical solutions we have are represented by the vectors \(\vv_1 = \left[ \begin{array}{c} 40\\50\\10 \end{array} \right]\) and \(\vv_2 = \left[ \begin{array}{c} 52\\42\\6 \end{array} \right]\text{.}\) If we mix the two chemical solutions with varying amounts of each ingredient, then the question of whether we can make the desired chemical solution becomes the question of whether the equation

has a solution. (You will determine if this equation has a solution in Exercise 5.)

We might also be interested in what other chemical solutions we can make from the two given solutions. This amounts to determining which vectors can be written in the form \(c_1\left[ \begin{array}{c} 40\\50\\10 \end{array} \right]+ c_2\left[ \begin{array}{c} 52\\42\\6 \end{array} \right]\) for scalars \(c_1\) and \(c_2\text{.}\) Vectors that are created from sums of scalar multiples of given vectors are called linear combinations of those vectors. More formally,

In the chemical solutions example, the vector \(c_1\left[ \begin{array}{c} 40\\50\\10 \end{array} \right]+ c_2\left[ \begin{array}{c} 52\\42\\6 \end{array} \right]\) for scalars \(c_1\) and \(c_2\) is a linear combination of the vectors \(\left[ \begin{array}{c} 40\\50\\10 \end{array} \right]\) and \(\left[ \begin{array}{c} 52\\42\\6 \end{array} \right]\) with weights \(c_1\) and \(c_2\text{,}\) and the set of linear combinations of the given chemical solution vectors tells us exactly which chemical solutions we can make from the given ones. This is one example of how linear combinations can arise in applications.

The set of all linear combinations of a fixed collection of vectors has a very nice algebraic structure and, in small dimensions, allows us to use a geometrical description to aid our understanding. In the above example, this collection gives us the type of chemical solutions we can make by combining the first two solutions in varying amounts.

Note that to find the weights that make \(\vw\) a linear combination of the vectors \(\vv_1\) and \(\vv_2\text{,}\) we simply solved the linear system corresponding to the augmented matrix

where the vectors \(\vv_1\text{,}\) \(\vv_2\text{,}\) and \(\vw\) form the columns of an augmented matrix, and the solution of the system gave us the weights of the linear combination. In general, if we want to find weights \(c_1\text{,}\) \(c_2\text{,}\) \(\ldots\text{,}\) \(c_m\) so that a vector \(\vw\) in \(\R^n\) is a linear combination of the vectors \(\vv_1\text{,}\) \(\vv_2\text{,}\) \(\ldots\text{,}\) \(\vv_m\) in \(\R^n\text{,}\) we solve the system corresponding to the augmented matrix

Any solution to this system will gives us the weights. If this system has no solutions, then \(\vw\) cannot be written as a linear combination of the vectors \(\vv_1\text{,}\) \(\vv_2\text{,}\) \(\ldots\text{,}\) \(\vv_m\text{.}\) This shows us the equivalence of the linear system and its vector equation representation. Specifically, we have the following result.

Subsection The Span of a Set of Vectors

As we saw in the previous section, the question of whether a system of linear equations has a solution is equivalent to the question of whether the vector obtained by the non-coefficient constants in the system is a linear combination of the vectors obtained from the columns of the coefficient matrix of the system. So if we were interested in finding for which constants the system has a solution, we would look for the collection of all linear combinations of the columns. We call this collection the span of these vectors. In this section we investigate the concept of span both algebraically and geometrically.

Our work in Preview Activity 4.1 seems to indicate that the span of a set of vectors, i.e., the collection of all linear combinations of this set of vectors, has a nice structure. As we mentioned above, the span of a set of vectors represents the collection of all constant vectors for which a linear system has a solution, but we will also see that other important objects in linear algebra can be represented as the span of a set of vectors.

We now investigate what the span of a set of one or two vectors is, both from an algebraic and geometric perspective, and consider what happens for more general spanning sets.

Subsection Examples

What follows are worked examples that use the concepts from this section.

Subsection Summary

• A vector is a list of numbers in a specified order.

• We add two vectors of the same size by adding corresponding components. In other words, if \(\vu\) and \(\vv\) are vectors of the same size and \(u_i\) and \(v_i\) are the \(i\) components of \(\vu\) and \(\vv\text{,}\) respectively, then \(\vu +\vv\) is the vector whose \(i\)th component is \(u_i+v_i\) for each \(i\text{.}\) Geometrically, we represent the sum of two vectors using the Parallelogram Rule: The vector \(\vu+\vv\) is the directed line segment from the origin to the 4th point of the parallelogram formed by the origin and the vectors \(\vu, \vv\text{.}\)

• A scalar multiple of a vector is found by multiplying each component of the vector by that scalar. In other words, if \(v_i\) is the \(i\) component of the vector \(\vv\) and \(c\) is any scalar, then \(c\vv\) is the vector whose \(i\) component is \(cv_i\) for each \(i\text{.}\) Geometrically, a scalar multiple of a nonzero vector \(\vv\) is a vector in the same direction as \(\vv\) if \(c>0\) and in the opposite direction if \(c\lt 0\text{.}\) If \(|c|>1\text{,}\) the vector is stretched, and if \(|c|\lt 1\text{,}\) the vector is shrunk.

• An important concept is that of a linear combination of vectors. In words, a linear combination of a collection of vectors is a sum of scalar multiples of the vectors. More formally, we defined a linear combination of vectors \(\vv_1\text{,}\) \(\vv_2\text{,}\) \(\ldots\text{,}\) \(\vv_m\) in \(\R^n\) is any vector of the form \(c_1 \vv_1 + c_2 \vv_2 + \cdots + c_m \vv_m\text{,}\) where \(c_1\text{,}\) \(c_2\text{,}\) \(\ldots\text{,}\) \(c_m\) are scalars.

• To find weights \(c_1\text{,}\) \(c_2\text{,}\) \(\ldots\text{,}\) \(c_m\) so that a vector \(\vw\) in \(\R^n\) is a linear combination of the vectors \(\vv_1\text{,}\) \(\vv_2\text{,}\) \(\ldots\text{,}\) \(\vv_m\) in \(\R^n\text{,}\) we simply solve the system corresponding to the augmented matrix

  \begin{equation*} [\vv_1 \ \vv_2 \ \vv_3 \ \cdots \ \vv_m | \ \vw]\text{.} \end{equation*}

• The collection of all linear combinations of a set of vectors is called the span of the set of vectors. More formally, the span of the vectors \(\vv_1\text{,}\) \(\vv_2\text{,}\) \(\ldots\text{,}\) \(\vv_m\) in \(\R^n\) is the set

  \begin{equation*} \{c_1\vv_1 + c_2\vv_2 + \cdots + c_m\vv_m : c_1, c_2, \ldots, c_m \text{ are scalars } \}\text{,} \end{equation*}

  which we denote as \(\Span \{\vv_1, \vv_2, \ldots, \vv_m\}\text{.}\) Geometrically, the span of a single nonzero vector \(\vv\) in any dimension is the line through the origin and the vector \(\vv\text{.}\) The span of two vectors \(\vv_1, \vv_2\) in any dimension neither of which is a multiple of the other is a plane through the origin containing both vectors.

Subsection Project: Analyzing Knight Moves

To understand where a knight can move in a chess game, we need to know the initial setup. A chess board is an \(8 \times 8\) grid. To be able to refer to the individual positions on the board, we will place the board so that its lower left corner is at the origin, make each square in the grid have side length \(1\text{,}\) and label each square with the point at the lower left corner. This is illustrated at left in Figure 4.13.

Each player has two knights to start the game, for one player the knights would begin in positions \((1,0)\) and \((6,0)\text{.}\) Because of the symmetry of the knight's moves, we will only analyze the moves of the knight that begins at position \((1,0)\text{.}\) This knight has only three allowable moves from its starting point (assuming that the board is empty), as shown at right in Figure 4.13. The questions we will ask are: given any position on the board, can the knight move from its start position to that position using only knight moves and, what sequence of moves will make that happen. To answer these questions we will use linear combinations of knight moves described as vectors.

Each knight move can be described by a vector. A move one position to the right and two up can be represented as \(\vn_1 = \left[ \begin{array}{c} 1\\2 \end{array} \right]\text{.}\) Three other moves are \(\vn_2 = \left[ \begin{array}{r} -1\\2 \end{array} \right]\text{,}\) \(\vn_3 = \left[ \begin{array}{c} 2\\1 \end{array} \right]\text{,}\) and \(\vn_4 = \left[ \begin{array}{r} -2\\1 \end{array} \right]\text{.}\) The other four knight moves are the additive inverses of these four. Any sequence of moves by the knight is given by the linear combination

A word of caution: the knight can only make complete moves, so we are restricted to integer (either positive or negative) values for \(x_1\text{,}\) \(x_2\text{,}\) \(x_3\text{,}\) and \(x_4\text{.}\) You can use the GeoGebra app at geogebra.org/m/dfwtskrj to see the effects the weights have on the knight moves. We should note here that since addition of vectors is commutative, the order in which we apply our moves does not matter. However, we may need to be careful with the order so that our knight does not leave the chess board.

Project Activity 4.6 shows that it is possible for our knight to move to position \((5,2)\) on the board. We would like to know if it is possible to move to any position on the board. That is, we would like to know if the integer span of the four moves \(\vn_1\text{,}\) \(\vn_2\text{,}\) \(\vn_3\text{,}\) and \(\vn_4\) will allow our knight to cover the entire board. This takes a bit more work.

To answer our question if our knight can reach any position, we now need to determine if we can always find integer values of \(x_3\) and \(x_4\) to make equations (4.3) and (4.4) have integer solutions. In other words, we need to find values of \(x_3\) and \(x_4\) so that \(-5x_3+3x_4+b+2(a-1)\) and \(3x_3-5x_4+b-2(a-1)\) are multiples of \(4\text{.}\) How we do this could depend on the parity (even or odd) of \(a\) and \(b\text{.}\) For example, if \(a\) is odd and \(b\) is even, say \(a = 2r+1\) and \(b = 2s\) for some integers \(r\) and \(s\text{,}\) then

With a little trial and error we can see that if we let \(x_3 = x_4 = s\text{,}\) then \(x_1 = r\) and \(x_2 = -r\) is a solution with integer weights. For example, when \(a=5\) and \(b=2\) we have \(r=2\) and \(s = 1\text{.}\) This makes \(x_1 = 2\text{,}\) \(x_2 = -2\text{,}\) \(x_3 = 1 = x_4\text{.}\) Compare this to the solution(s) you found in Project Activity 4.6. This analysis shows us how to move our knight to any position \((a,b)\) where \(a\) is odd and \(b\) is even.

Project Activity 4.8 shows that for any position on the chess board, using linear combinations of move vectors, we can find a sequence of moves that takes our knight to that position. (We actually haven't shown that these moves can be made so that our knight always stays on the board — we leave that question to you.)