An Inquiry-Based Introduction to Linear Algebra and Applications

Subsection Application: Approximating Area Under a Curve

We know from basic geometry how to find areas of circles and triangles. However, it is much more difficult to find areas of other geometric objects. In fact, it is generally an impossible problem to determine the exact area bounded by a complicated curve. For this reason, approximation methods are used. One such method involves approximating curves using quadratic functions.

Unless you have learned some calculus, you have probably never calculated the area under a parabola. In the ancient work Quadrature of the Parabola (3rd century BC), Archimedes determined a method for finding the area of a region bounded by a parabola by using mechanics and then by geometric methods. Once we know how to calculate the area of a region bounded by a parabola, Simpson's Rule uses parabolas to approximate a function, and then approximates the area under the graph of the graph of the function by using the areas under the parabolas. In order to use Simpsons Rule, we need to know how to exactly fit a quadratic function to three points. More details about this process can be found at the end of this section. This idea of fitting a polynomial to a set of data points has uses in other areas as well. For example, two common applications of Bézier curves are font design and drawing tools. When fitting a polynomial to a large set of data points, our systems of equations can become quite large, and can be difficult to solve by hand. In this section we will see how to use matrices to more conveniently represent systems of equations of any size. We also consider how the elimination process works on the matrix representation of a linear system and how we can determine the existence of solutions and the form of solutions of a linear system.

Subsection Introduction

When working with a linear system, the labels for the variables are irrelevant to the solution — the only thing that matters is the coefficients of the variables in the equations and the constants on the other side of the equations. For example, given a linear system of the form

the important information in the system can be represented as

where we interpret the first three numbers in each horizontal row to represent the coefficients of the variables \(a\text{,}\) \(b\) and \(c\text{,}\) respectively, and the last number to be the constant on the right hand side of the equation. This tells us that we can record all the necessary information about our system in a rectangular array of numbers. Such an array is called a matrix.

We usually delineate a matrix by enclosing its entries in square brackets \([ * ]\text{.}\) For the system in (2.1), there are two corresponding matrices:

The matrix on the left is the matrix of the coefficients of the system, and is called the coefficient matrix of the system. The matrix on the right is the matrix of coefficients and the constants, and is called the augmented matrix of the system (where we say we augment the coefficient matrix with the additional column of constants). We will separate the augmented column from the coefficient matrix with a vertical line to keep it clear that the last column is an augmented column of constants and not a column of coefficients.⁷

Subsection Simplifying Linear Systems Represented in Matrix Form

Once we have stored the information about a linear system in an augmented matrix, we can perform the elementary operations directly on the augmented matrix.

Recall that the allowable operations on a system of equations are the following:

1. Replacing one equation by the sum of that equation and a scalar multiple of another equation.

2. Interchanging the positions of two equations.

3. Replacing an equation by a nonzero scalar multiple of itself.

Recall that we use these elementary operations to transform a system, with the ultimate goal of finding a simpler, equivalent system that we can solve. Since each row of an augmented matrix corresponds to an equation, we can translate these operations on equations to corresponding operations on rows (called row operations or elementary row operations):

1. Replacing one row by the sum of that row and a scalar multiple of another row.

2. Interchanging two rows.

3. Replacing a row by a nonzero scalar multiple of itself.

Activity 2.2 illustrates how we can perform all of the operations on equations with operations on the rows of augmented matrices to reduce a system to a solvable form. Each time we perform an operation on the system of equations (or on the rows of an augmented matrix) we obtain an equivalent system (or an augmented matrix corresponding to an equivalent system). For completeness, we list the operations on equations and the corresponding row operations below that can be used to solve our polynomial fitting system. Throughout the process we will let \(E_1\text{,}\) \(E_2\text{,}\) and \(E_3\) be the first, second, and third equations in the system and \(R_1\text{,}\) \(R_2\text{,}\) and \(R_3\) the first, second, and third rows of the augmented matrices. The notation \(E_1+E_2\) placed next to equation \(E_2\) means means that we replace the second equation in the system with the sum of the first two equations. We start with the system

On the left we demonstrate the operations on equations and on the right the corresponding operations on rows of the augmented matrix.

Now we can solve the last equation for \(a_0\) to find that \(a_0=5\text{.}\) The second equation gives us \(a_1 = 2\text{.}\)⁸ Finally, using the first equation with the already determined values of \(a_0\) and \(a_1\) gives us \(a_2=-1\text{.}\) Thus we have found the solution to the polynomial fitting system to be \(a_2=-1\text{,}\) \(a_1=2\text{,}\) and \(a_0=5\text{.}\)

We summarize the steps of the (partial) elimination on matrices we used above to solve a general linear system in the variables \(x_1\text{,}\) \(x_2\text{,}\) \(\ldots\text{,}\) \(x_n\text{.}\)

1. Interchange equations if needed to ensure that the coefficient of \(x_1\) (or, more generally, the first non-zero variable) in the first equation is non-zero.

2. Use the first equation to eliminate \(x_1\) (or, the first non-zero variable) from other equations by adding a multiple of the first equation to the others.

3. After \(x_1\) is eliminated from all equations but the first equation, focus on the rest of the equations. Repeat the process of elimination on these equations to eliminate \(x_2\) (or, the next non-zero variable) all but the second equation.

4. Once the process of eliminating variables recursively is finished, solve for the variables in a backwards fashion starting with the last equation and substituting known values in the equations above as they become known.

This elimination method where the variables are eliminated from lower equations is called the forward elimination phase as it eliminates variables in the forward direction. Solving for variables using substitution into upper equations is called back substitution. The matrix representation of a linear system after the forward elimination process is said to be in row echelon form. We will define this form and the elimination process on the matrices more precisely in the next section.

Subsection Linear Systems with Infinitely Many Solutions

Each of the systems that we solved so far have had a unique (exactly one) solution. The geometric representation of linear systems with two equations in two variables shows that this does not always have to be the case. We also have linear systems with no solution and systems with infinitely many solutions. We now consider the problem of how to represent the set of solutions of a linear system that has infinitely many solutions. (Systems with infinitely many solutions will also be of special interest to us a bit later when we study eigenspaces of a matrix.)

After using the elimination method, the first non-zero coefficient (from the left) of each equation in the linear system is in a different position. We call each such coefficient a pivot and a variable corresponding to a pivot a basic variable. In the system

the basic variables are \(a_2, a_1, a_0\) for the first, second, and third equations, respectively. In the system,

the basic variables are \(x_1\) and \(x_2\) for the first and second equations, respectively, while the third equation does not have a basic variable. Through back-substitution, we can solve for each variable in a unique way if each appears as the basic variable in an equation. If, however, a variable is free, meaning that it is not the basic variable of an equation, we cannot solve for that variable explicitly. We instead assign a distinct parameter to each such free variable and solve for the basic variables in terms of these parameters.

Subsection Linear Systems with No Solutions

We saw in the previous section that geometrically two parallel and distinct lines represent a linear system with two equations in two unknowns which has no solution. Similarly, two parallel and distinct planes in three dimensions represent a linear system with two equations in three unknowns which has no solution. We can have at least four different geometric configurations of three planes in three dimensions representing a system with no solution. But how do these geometrical configurations manifest themselves algebraically?

We summarize our observations about when a system has a solution, and which of those cases has a unique solution.

Subsection Examples

What follows are worked examples that use the concepts from this section.

Subsection Summary

• A matrix is just a rectangular array of numbers or objects.

• Given a system of linear equations, with the variables listed in the same order in each equation, we represent the system by writing the coefficients of the first equation as the first row of a matrix, the coefficients of the second equation as the second row, and so on. This creates the coefficient matrix of the system. We then augment the coefficient matrix with a column of the constants that appear in the equations. This gives us the augmented matrix of the system.

• The operations that we can perform on equations translate exactly to row operations that we can perform on an augmented matrix:

  1. Replacing one row by the sum of that row and a scalar multiple of another row.

  2. Interchanging two rows.

  3. Replacing a row by a nonzero scalar multiple of itself.

• The forward elimination phase of the elimination method recursively eliminates the variables in a linear system to reach an equivalent but simplified system.

• The first non-zero entry in an equation in a linear system after elimination is called a pivot.

• A basic variable in a linear system corresponds to a pivot of the system. A free variable is a variable that is not basic.

• A linear system can be inconsistent (no solutions), have a unique solution (if consistent and every variable is a basic variable), or have infinitely many solutions (if consistent and there is a free variable).

• A linear system has no solutions if, after elimination, there is an equation of the form \(0=b\) where \(b\) is a nonzero number.

• A linear system after the elimination method can be solved using back-substitution. The free variables can be chosen arbitrarily and the basic variables can be solved in terms of the free variables through the back-substitution process.

Subsection Project: Polynomial Interpolation to Approximate the Area Under a Curve

Suppose we want to approximate the area of the region shown in Figure 2.8. As discussed in the introduction, we can approximate the area under a curve by approximating the curve by quadratics. First, we will see how Archimedes approached the problem of finding the area of a quadratic region, then we will determine how to determine a quadratic function that passes through three points, then we put it all together to approximate the area under a curve as in Figure 2.8.

Archimedes approached the problem of calculating the area of a quadratic region in the following way. Given a quadratic \(q(x)\) on an interval \([a,b]\text{,}\) Archimedes drew in a base given by the secant line connecting the points \((a,q(a))\) and \((b,q(b))\) as illustrated at left in Figure 2.9. Then he found the point in the interval \([a,b]\) at which the tangent line to the curve is parallel to the secant line. Archimedes gave an argument using mechanics (based on balance points), and then another using geometry (through a method of exhaustion) to show that the area of the parabolic region is \(\frac{4}{3}\) times the area of the triangle determined by the endpoints and the point of tangency as shown at right in Figure 2.9.

Although we won't go through the details, a conclusion we can draw from Archimedes argument, using the formula for the area of a rectangle and the area of a triangle, is that the area between the graph of a quadratic with equation \(q(x) = ax^2+bx+c\) and the \(x\)-axis on an interval \([x_1, x_2]\) as illustrated in Figure 2.10 is

To approximate the area under the graph of a function, we will approximate the function itself with a collection of quadratics, and then use equation (2.3) repeatedly. To do this, we need to know how to fit a quadratic curve to a three points. We consider that question now.

Suppose we are given a collection of three points in the plane: \((x_1, y_1), (x_2, y_2)\) and \((x_3, y_3)\text{.}\) There is exactly one quadratic polynomial \(p(x)\) which goes through these points, i.e. there is exactly one quadratic \(p(x)\) such that for each \(x_i\text{,}\) \(p(x_i)=y_i\text{.}\) This is an example of polynomial curve fitting.

As an example, we use the points \((-1, 2)\text{,}\) \((1,6)\text{,}\) \((2,5)\text{.}\) To fit a quadratic to these points, consider a general quadratic of the form \(p(x)=a_2x^2+a_1x+a_0\text{.}\) By substituting the \(x\) value of each of the given points and setting that equal to the \(y\) value of that point, we find three equations

that give us a system of three equations in the three unknowns \(a_2\text{,}\) \(a_1\text{,}\) and \(a_0\text{:}\)

This system is the example we considered in Preview Activity 2.1, whose solution is \(a_2 = -1\text{,}\) \(a_1=2\text{,}\) and \(a_0 = 5\text{.}\) A graph of \(q(x) = -x^2+2x+5\) along with the three points \((-1, 2)\text{,}\) \((1,6)\text{,}\) \((2,5)\) is shown in Figure 2.11.

Now that we know how to fit a quadratic to three points, we next approximate a curve with a collection of quadratics. The method we use is to break the interval on which our curve is defined into several subintervals and create quadratics on each subinterval. The basic idea is contained in our first project activity.

Project Activity 2.6 illustrates how we can model a function on an interval using a sequence of quadratic functions. Now we apply this polynomial curve fitting technique to derive the general formula for approximating the area between a graph of a function \(f\) and the \(x\)-axis. We use parabolic arcs to approximate the graph of \(f\) on each subinterval.

We start by dividing the interval \([a,b]\) over which our function is defined into some number of subintervals. We need an even number of subintervals, since we have to use three points to define each parabola. Let \(n = 2m\) be the number of subintervals we use. In order to make the calculations a bit easier, let the subintervals all have the same length, which we denote by \(\Delta x\) (the symbol \(\Delta\) is often used in mathematics to indicate a change in a quantity). Since we have \(n\) subintervals, the length of each subinterval will be \(\Delta x = \frac{b-a}{n}\text{.}\) For each \(k\) we let \(x_k = a+k \Delta x\) and \(y_k = f(x_k)\text{.}\) Note that \(x_0 = a\) and \(x_n=b\text{.}\) This labeling scheme is illustrated in Figure 2.12

We approximate \(f\) on each subinterval using a quadratic. So we need to find the quadratic \(Q(x) = c_2x^2+c_1x+c_0\) that passes through two consecutive end points as well as the midpoint of a subinterval. That is, we need to find the coefficients of \(Q\) so that \(Q\) passes through the points \((x_k,y_k)\text{,}\) \((x_{k+2}, y_{k+2})\text{,}\) and the midpoint \((x_{k+1},y_{k+1})\) on the interval \([x_k, x_{k+2}]\) (so that we have three points to which to fit a parabola) as shown at left in Figure 2.13. Note that the length of the interval \([x_k, x_{k+2}]\) is \(2\Delta x\text{.}\) To make the calculations easier, we will translate our function so that our leftmost point is \((-r, y_k)\text{.}\) Then the middle point is \((0, y_{k+1})\) and the rightmost point is \((r, y_{k+2})\) as illustrated at right in Figure 2.13, where \(r = \Delta x\text{.}\)

We conclude with an example.