An Inquiry-Based Introduction to Linear Algebra and Applications

Subsection Application: Bézier Curves

Bézier curves are simple curves that were first developed in 1959 by French mathematician Paul de Casteljau, who was working at the French automaker Citroën. The curves were made public in 1962 by Pierre Bézier who used them in his work designing automobiles at the French car maker Renault. In addition to automobile design, Bézier curves have many other uses. Two of the most common applications of Bézier curves are font design and drawing tools. As an example, the letter “S” in Palatino font is shown using Bézier curves in Figure 6.1. If you've used Adobe Illustrator, Photoshop, Macromedia Freehand, Fontographer, or any other of a number of drawing programs, then you've used Bézier curves. At the end of this section we will see how Bézier curves can be defined using linearly independent vectors and linear combinations of vectors.

Subsection Introduction

In Section 4 we saw how to represent water-benzene-acetic acid chemical solutions with vectors, where the components represent the water, benzene and acid percentages. We then considered a problem of determining if a given chemical solution could be made by mixing other chemical solutions. Suppose we now have three different water-benzene-acetic acid chemical solutions, one with 40% water, 50% benzene and 10% acetic acid, the second with 52% water, 42% benzene and 6% acid, and a third with 46% water, 46% benzene and 8% acid. We represent the first chemical solution with the vector \(\vv_1 = \left[ \begin{array}{c} 40\\50\\10 \end{array} \right]\text{,}\) the second with the vector \(\vv_2 = \left[ \begin{array}{c} 52\\42\\6 \end{array} \right]\text{,}\) and the third with the vector \(\vv_3 = \left[ \begin{array}{c} 46\\46\\8 \end{array} \right]\text{.}\) By combining these three chemical solutions we can make a chemical solution with 43% water, 48% benzene and 9% acid as follows

However, if we had noticed that the third chemical solution can actually be made from the first two, that is,

we might have realized that we don't need the third chemical solution to make the 43% water, 48% benzene and 9% acid chemical solution. In fact,

(See Exercise 5 of Section 4.) Using the third chemical solution (represented by \(\vv_3\)) uses more information than we actually need to make the desired 43% water, 48% benzene and 9% acid chemical solution because the vector \(\vv_3\) is redundant — all of the material we need to make \(\vv_3\) is contained in \(\vv_1\) and \(\vv_2\text{.}\) This is the basic idea behind linear independence — representing information in the most efficient way.

Information is often contained in and conveyed through vectors — especially linear combinations of vectors. In this section we will investigate the concepts of linear dependence and independence of a set of vectors. Our goal is to be able to efficiently determine when a given set of vectors forms a minimal spanning set. A minimal spanning set is a spanning set that contains the smallest number of vectors to obtain all of the vectors in the span. An important aspect of a minimal spanning set is that every vector in the span can be written in one and only one way as a linear combination of the vectors in the minimal spanning set. This will allow us to define the important notion of the dimension of a vector space.

Subsection Linear Independence

In this section we will investigate the concepts of linear independence of a set of vectors. Our goal is to be able to efficiently determine when a given set of vectors forms a minimal spanning set. This will involve the concepts of span and linear independence. Minimal spanning sets are important in that they provide the most efficient way to represent vectors in a space, and will later allow us to define the dimension of a vector space.

In Preview Activity 6.1 we considered the case where we had a set \(\{ \vv_1, \vv_2, \vv_3 \}\) of three vectors, and the vector \(\vv_3\) was in the span of \(\{ \vv_1, \vv_2 \}\text{.}\) So the vector \(\vv_3\) did not add anything to the span of \(\{ \vv_1, \vv_2 \}\text{.}\) In other words, the set \(\{ \vv_1, \vv_2, \vv_3 \}\) was larger than it needed to be in order to generate the vectors in its span — that is, \(\Span \{ \vv_1, \vv_2, \vv_3 \} = \Span \{ \vv_1, \vv_2 \}\text{.}\) However, neither of the vectors in the set \(\{ \vv_1, \vv_2 \}\) could be removed without changing its span. In this case, the set \(\{ \vv_1, \vv_2 \}\) is what we will call a minimal spanning set or basis for \(\Span S\text{.}\) There are two important properties that make \(\{ \vv_1, \vv_2 \}\) a basis for \(\Span S\text{.}\) The first is that every vector in \(\Span S\) can be written as linear combinations of \(\vv_1\) and \(\vv_2\) (we also use the terminology that the vectors \(\vv_1\) and \(\vv_2\) span \(\Span S\)), and the second is that every vector in \(\Span S\) can be written in exactly one way as a linear combination of \(\vv_1\) and \(\vv_2\text{.}\) This second property is the property of linear independence, and it is the property that makes the spanning set minimal.

To make a spanning set minimal, we want to be able to write every vector in the span in a unique way in terms of the spanning vectors. Notice that the zero vector can always be written as a linear combination of any set of vectors using 0 for all of the weights. So to have a minimal or linearly independent spanning set, that is, to have a unique representation for each vector in the span, it will need to be the case that the only way we can write the zero vector as a linear combination of a set of vectors is if all of the weights are 0. This leads us to the definition of a linearly independent set of vectors.

Alternatively, we say that the vectors \(\vv_1, \vv_2, \cdots, \vv_k\) are linearly independent (or dependent) if the set \(\{ \vv_1, \vv_2, \cdots, \vv_k \}\) is linearly independent (or dependent).

Note that the definition tells us that a set \(\{ \vv_1, \vv_2, \cdots, \vv_k \}\) of vectors in \(\R^n\) is linearly dependent if there are scalars \(x_1, x_2, \cdots, x_n\text{,}\) not all of which are 0 so that

Activity 6.2.

Which of the following sets in \(\R^2\) or \(\R^3\) is linearly independent and which is linearly dependent? Why? For the linearly dependent sets, write one of the vectors as a linear combination of the others, if possible.

(a)

\(S_1 = \left\{\left[\begin{array}{c} 2 \\ 0 \\ 1 \end{array}\right], \left[\begin{array}{r} -2 \\ 8 \\ 1 \end{array}\right], \left[\begin{array}{r} -4 \\ 8 \\ 0 \end{array}\right]\right\}\)

(b)

\(S_2 = \left\{\left[\begin{array}{c} 1 \\ 2 \\ 1 \end{array}\right], \left[\begin{array}{c} 0 \\ 2 \\ 3 \end{array}\right]\right\}\)

Hint.

What relationship must exist between two vectors if they are linearly dependent?

(c)

The vectors \(\vu\text{,}\) \(\vv\text{,}\) and \(\vw\) as shown in Figure 6.3.

Task 6.2.a and Task 6.2.c illustrate how we can write one of the vectors in a linearly dependent set as a linear combination of the others. This would allow us to write at least one of the vectors in the span of the set in more than one way as a linear combination of vectors in this set. We prove this result in general in the following theorem.

The next activity is intended to help set the stage for the proof of Theorem 6.4.

Now we provide a formal prof of Theorem 6.4, using the ideas from Activity 6.3.

With a linearly dependent set, at least one of the vectors in the set is a linear combination of the others. With a linearly independent set, this cannot happen — no vector in the set can be written as a linear combination of the others. This result is given in the next theorem. You may be able to see how Theorem 6.4 and Theorem 6.5 are logically equivalent.

Activity 6.4 contains the general ideas to show that any vector in the span of a linearly independent set can be written in one and only one way as a linear combination of the vectors in the set. The weights of such a linear combination provide us a coordinate system for the vectors in terms of the basis. Two familiar concepts of coordinate systems are the Cartesian coordinates and \(xy\)-plane, and \(xyz\)-space. We will revisit the coordinate system idea in a later chapter.

In the next theorem we state and prove the general case of any number of linearly independent vectors productin unique representations as linear combinations.

Subsection Determining Linear Independence

The definition and our previous work give us a straightforward method for determining when a set of vectors in \(\R^n\) is linearly independent or dependent.

Subsection Minimal Spanning Sets

It is important to note the differences and connections between linear independence, span, and minimal spanning set.

• The set \(S = \left\{ \left[ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right], \left[ \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right] \right\}\) is not a minimal spanning set for \(\R^3\) even though \(S\) is a linearly independent set. Note that \(S\) does not span \(\R^3\) since the vector \(\left[ \begin{array}{c} 0 \\ 0 \\ 1 \end{array} \right]\) is not in \(\Span \ S\text{.}\)

• The set \(T = \left\{ \left[ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right], \left[ \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right] , \left[ \begin{array}{c} 0 \\ 0 \\ 1 \end{array} \right] , \left[ \begin{array}{c} 1 \\ 1 \\ 1 \end{array} \right] \right\}\) is not a minimal spanning set for \(\R^3\) even though \(\Span \ T = \R^3\text{.}\) Note that

  \begin{equation*} \left[ \begin{array}{c} 1 \\ 1 \\ 1 \end{array} \right] = \left[ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right] + \left[ \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right] + \left[ \begin{array}{c} 0 \\ 0 \\ 1 \end{array} \right]\text{,} \end{equation*}

  so \(T\) is not a linearly independent set.

• The set \(U = \left\{ \left[ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right], \left[ \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right] , \left[ \begin{array}{c} 0 \\ 0 \\ 1 \end{array} \right] \right\}\) is a minimal spanning set for \(\R^3\) since it satisfies both characteristics of a minimal spanning set: \(\Span \ U = \R^3\) AND \(U\) is linearly independent.

The three concepts — linear independence, span, and minimal spanning set — are different. The important point to note is that minimal spanning set must be both linearly independent and span the space.

To find a minimal spanning set we will often need to find a smallest subset of a given set of vectors that has the same span as the original set of vectors. In this section we determine a method for doing so.

Activity 6.6 illustrates how we can use a matrix to determine a minimal spanning set for a given set of vectors \(\{\vv_1, \vv_2, \ldots, \vv_k\}\) in \(\R^n\text{.}\)

• Form the matrix \(A = [\vv_1 \ \vv_2 \ \cdots \ \vv_k]\text{.}\)

• Find the reduced row echelon form \([B \ | \ \vzero]\) of \([A \ | \ \vzero]\text{.}\) If \(B\) contains non-pivot columns, say for example that the \(i\)th column is a non-pivot column, then we can choose the weight \(x_i\) corresponding to the \(i\)th column to be 1 and all weights corresponding to the other non-pivot columns to be 0 to make a linear combination of the columns of \(A\) that is equal to \(\vzero\text{.}\) This allows us to write \(\vv_i\) as a linear combination of the vectors corresponding to the pivot columns of \(A\) as we did in the proof of Theorem 6.5. So every vector corresponding to a non-pivot column is in the span of the set of vectors corresponding to the pivot columns. The vectors corresponding to the pivot columns are linearly independent, since the matrix with those columns has every column as a pivot column. Thus, the set of vectors corresponding to the pivot columns of \(A\) forms a minimal spanning set for \(\{\vv_1, \vv_2, \ldots, \vv_k\}\text{.}\)

Activity 6.6 also illustrates a general process by which we can find a minimal spanning set — that is the smallest subset of vectors that has the same span. This process will be useful later when we consider vectors in arbitrary vector spaces. The idea is that if we can write one of the vectors in a set \(S\) as a linear combination of the remaining vectors, then we can remove that vector from the set and maintain the same span. In other words, begin with the span of a set \(S\) and follow these steps:

Step 1

If \(S\) is a linearly independent set, we already have a minimal spanning set.

Step 2

If \(S\) is not a linearly independent set, then one of the vectors in \(S\) is a linear combination of the others. Remove that vector from \(S\) to obtain a new set \(T\text{.}\) It will be the case that \(\Span \ T = \Span \ S\text{.}\)

Step 3

If \(T\) is a linearly independent set, then \(T\) is a minimal spanning set. If not, repeat steps 2 and 3 for the set \(T\) until you arrive at a linearly independent set.

This process is guaranteed to stop as long as the set contains at least one nonzero vector. A verification of the statement in Step 2 that \(\Span \ T = \Span \ S\) is given in the next theorem.

The result of Theorem 6.7 is that if we have a finite set \(S\) of vectors in \(\R^n\text{,}\) we can eliminate those vectors that are linear combinations of others until we obtain a smallest set of vectors that still has the same span. As mentioned earlier, we call such a minimal spanning set a basis.

We have worked with a familiar basis in \(\R^2\) throughout our mathematical careers. A vector \(\left[ \begin{array}{c} a \\ b \end{array} \right]\) in \(\R^2\) can be written as

So the set \(\{\ve_1, \ve_2\}\text{,}\) where \(\ve_1 = \left[ \begin{array}{c} 1 \\ 0 \end{array} \right]\) and \(\ve_2 = \left[ \begin{array}{c} 0 \\ 1 \end{array} \right]\) spans \(\R^2\text{.}\) Since the columns of \([\ve_1 \ \ve_2]\) are linearly independent, so is the set \(\{\ve_1, \ve_2\}\text{.}\) Therefore, the set \(\{\ve_1, \ve_2\}\) is a basis for \(\R^2\text{.}\) The vector \(\ve_1\) is in the direction of the positive \(x\)-axis and the vector \(\ve_2\) is in the direction of the positive \(y\)-axis, so decomposing a vector \(\left[ \begin{array}{c} a \\ b \end{array} \right]\) as a linear combination of \(\ve_1\) and \(\ve_2\) is akin to identifying the vector with the point \((a,b)\) as we discussed earlier. The set \(\{\ve_1, \ve_2\}\) is called the standard basis for \(\R^2\text{.}\)

This idea is not restricted to \(\R^2\text{.}\) Consider the vectors

in \(\R^n\text{.}\) That is, the vector \(\ve_i\) is the vector with a 1 in the \(i\)th position and 0s everywhere else. Since the matrix \([\ve_1 \ \ve_2 \ \cdots \ \ve_n]\) has a pivot in each row and column, the set \(\{\ve_1, \ve_2, \ldots, \ve_n\}\) is a basis for \(\R^n\text{.}\) The set \(\{\ve_1, \ve_2, \ldots, \ve_n\}\) is called the standard basis for \(\R^n\text{.}\)

As we will see later, bases¹² are of fundamental importance in linear algebra in that bases will allow us to define the dimension of a vector space and will provide us with coordinate systems.

We conclude this section with an important theorem that is similar to Theorem 5.3.

Subsection Examples

What follows are worked examples that use the concepts from this section.

Subsection Summary

• A set \(\{\vv_1, \vv_2, \ldots, \vv_k\}\) of vectors in \(\R^n\) is linearly independent if the vector equation

  \begin{equation*} x_1 \vv_1 + x_2 \vv_2 + \cdots + x_k \vv_k = \vzero \end{equation*}

  for scalars \(x_1, x_2, \ldots, x_k\) has only the trivial solution

  \begin{equation*} x_1 = x_2 = x_3 = \cdots = x_k = 0\text{.} \end{equation*}

  Another way to think about this is that a set of vectors is linearly independent if no vector in the set can be written as a linear combination of the other vectors in the set.

• A set \(\{\vv_1, \vv_2, \ldots, \vv_k\}\) of vectors in \(\R^n\) is linearly dependent if the vector equation

  \begin{equation*} x_1 \vv_1 + x_2 \vv_2 + \cdots + x_k \vv_k = \vzero \end{equation*}

  has a nontrivial solution. That is, we can find scalars \(x_1, x_2, \ldots, x_k\) that are not all 0 so that

  \begin{equation*} x_1 \vv_1 + x_2 \vv_2 + \cdots + x_k \vv_k = \vzero\text{.} \end{equation*}

  Another way to think about this is that a set of vectors is linearly dependent if at least one vector in the set can be written as a linear combination of the other vectors in the set.

• If \(S\) is a set of vectors, a subset \(B\) of \(S\) is a basis for \(\Span \ S\) if \(B\) is a linearly independent set and \(\Span \ B = \Span \ S\text{.}\)

• Given a nonzero set \(S\) of vectors, we can remove vectors from \(S\) that are linear combinations of remaining vectors in \(S\) to obtain a linearly independent subset of \(S\) that has the same span as \(S\text{.}\)

• The columns of a matrix \(A\) are linearly independent if the equation \(A \vx = \vzero\) has only the trivial solution \(\vx = \vzero\text{.}\)

• The set \(\{\vv_1, \vv_2, \ldots, \vv_k\}\) is linearly independent if and only if every column of the matrix \(A = [\vv_1 \ \vv_2 \ \vv_3 \ \cdots \ \vv_k]\text{,}\) is a pivot column.

• If \(A = [\vv_1 \ \vv_2 \ \vv_3 \ \cdots \ \vv_k]\text{,}\) then the vectors in the pivot columns of \(A\) form a minimal spanning set for \(\Span \{\vv_1, \vv_2, \ldots, \vv_k\}\text{.}\)

Subsection Project: Generating Bézier Curves

Bézier curves can be created as linear combinations of vectors. In this section we will investigate how cubic Bézier curves (the ones used for fonts) can be realized through linear and quadratic Bézier curves. We begin with linear Bézier curves.

For each value of \(t\text{,}\) the vector \(\vq = (1-t)\vp_0 + t\vp_1\) is a linear combination of the vectors \(\vp_0\) and \(\vp_1\text{.}\) Note that when \(t=0\text{,}\) we have \(\vq = \vp_0\) and when \(t=1\) we have \(\vq = \vp_1\text{,}\) and for \(0 \leq t \leq 1\) Project Activity 6.8 shows that the vectors \(\vq\) trace out the line segment from \(\vp_0\) to \(\vp_1\text{.}\) The span \(\{(1-t)\vp_0 + t\vp_1\}\) of the vectors \(\vp_0\) and \(\vp_1\) for \(0 \leq t \leq 1\) is a linear Bézier curve. Once we have a construction like this, it is natural in mathematics to extend it and see what happens. We do that in the next activity to construct quadratic Bézier curves.

The span \(\{(1-t)\vq_0 + t \vq_1\}\) of the vectors \(\vq_0\) and \(\vq_1\text{,}\) or the set of points traced out by the vectors \(\vr\) for \(0 \leq t \leq 1\text{,}\) is a quadratic Bézier curve. To understand why this curve is called quadratic, we examine the situation in a general context in the following activity.

Notice that if any one of the \(\vp_i\) lies on the line determined by the other two vectors, then the quadratic Bézier curve is just a line segment. So to obtain something non-linear we need to choose our vectors so that that doesn't happen.

Quadratic Bézier curves are limited, because their graphs are parabolas. For applications we need higher order Bézier curves. In the next activity we consider cubic Bézier curves.

The span \(\{(1-t)\vr_0 + t \vr_1\}\) of the vectors \(\vr_0\) and \(\vr_1\text{,}\) or the set of points traced out by the vectors \(\vs\) for \(0 \leq t \leq 1\text{,}\) is a cubic Bézier curve. To understand why this curve is called cubic, we examine the situation in a general context in the following activity.

Just as with the quadratic case, we need certain subsets of the set of control vectors to be linearly independent so that the cubic Bézier curve does not degenerate to a quadratic or linear Bézier curve.

More complicated and realistic shapes can be represented by piecing together two or more Bézier curves as illustrated with the letter “S” in Figure 6.1. Suppose we have two cubic Bézier curves, the first with control points \(\vp_0\text{,}\) \(\vp_1\text{,}\) \(\vp_2\text{,}\) and \(\vp_3\) and the second with control points \(\vp_0'\text{,}\) \(\vp_1'\text{,}\) \(\vp_2'\text{,}\) and \(\vp_3'\text{.}\) You may have noticed that \(\vp_1\) lies on the tangent line to the first Bézier curve at \(\vp_0\) and that \(\vp_2\) lies on the tangent line to the first Bézier curve at \(\vp_3\text{.}\) (Play around with the program Cubic Bezier to convince yourself of these statements. This can be proved in a straightforward manner using vector calculus.) So if we want to make a smooth curve from these two Bézier curves, the curves will need to join together smoothly at \(\vp_3\) and \(\vp_0'\text{.}\) This will force \(\vp_3 = \vp_0'\) and the tangents at \(\vp_3 = \vp_0'\) will have to match. This implies that \(\vp_2\text{,}\) \(\vp_3\text{,}\) and \(\vp_1'\) all have to lie on this common tangent line. Keeping this idea in mind, use the GeoGebra file Cubic Bezier Pair at geogebra.org/m/UwxQ6RPk to find control points for the pair of Bézier curves that create your own letter S.