The Structure of \R^n

Section 12 The Structure of $R^{n}$

Focus Questions

By the end of this section, you should be able to give precise and thorough answers to the questions listed below. You may want to keep these questions in mind to focus your thoughts as you complete the section.

What properties make $R^{n}$ a vector space?
What is a subspace of $R^{n} ?$
What properties do we need to verify to show that a set of vectors is a subspace of $R^{n} ?$ Why?
What important structure does the span of a set of vectors in $R^{n}$ have?

🔗

Subsection Application: Connecting GDP and Consumption in Romania

🔗

It is common practice in the sciences to run experiments and collect data. Once data is collected it is necessary to find some way to analyze the data and predict future behavior from the data. One method is to find a curve that best “fits” the data, and one widely used method for curve fitting is called the least squares method.

🔗

For example, economists are often interested in consumption, which is the purchase of goods and services for use by households. In “A Statistical Analysis of GDP and Final Consumption Using Simple Linear Regression, the Case of Romania 1990-2010”,²⁷ the authors collect data and then use simple linear regression to compare GDP (gross domestic product) to consumption in Romania. The data they used is seen in Table 12.1, with a corresponding scatterplot of the data (with consumption as independent variable and GDP as dependent variable). The units for GDP and consumption are milliions of leu (the currency of Romania is the leu — on December 21, 2018, one leu was worth approximately $0.25 U.S.) The authors conclude their paper with the following statement:

However, we can appreciate that linear regression model describes the correlation between the value of gross domestic product and the value of final consumption and may be transcribed following form: PIB = -3127.51+ 1.22 CF. Analysis of correlation between GDP and final consumption (private consumption and public consumption) will result in an increase of 1.22 units of monetary value of gross domestic product. We can conclude that the Gross Domestic Product of our country is strongly influenced by the private and public consumption.

Table 12.1. GDP and consumption in Romania

Year	GDP	Consumption
$1990$	$85.8$	$68.0$
$1991$	$220.4$	$167.3$
$1992$	$602.9$	$464.3$
$1993$	$2003.9$	$1523.6$
$1994$	$4977.3$	$3845.2$
$1995$	$7648.9$	$6257.7$
$1996$	$11384.2$	$9713.8$
$1997$	$25529.8$	$21972.2$
$1998$	$37055.1$	$33311.2$
$1999$	$55191.4$	$49311.9$
$2000$	$80984.6$	$69587.4$
$2001$	$117945.8$	$100731.7$
$2002$	$152017.0$	$127118.8$
$2003$	$197427.6$	$168818.7$
$2004$	$247368.0$	$211054.6$
$2005$	$288954.6$	$251038.1$
$2006$	$344650.6$	$294867.6$
$2007$	$416006.8$	$344937.0$
$2008$	$514700.0$	$420917.5$
$2009$	$498007.5$	$402246.0$
$2010$	$513640.8$	$405422.4$

Figure 12.2. GDP and consumption data plot.

🔗

As we can see from the scatterplot, the relationship between the GDP and consumption is not exactly linear, but looks to be very close. To make correlations between GDP and consumption as the authors did, we need to understand how they determined their approximate linear relationship between the variables. With a good approximation function we can then compare the variables, extrapolate from the data, and make predictions or interpolate and estimate between data points. For example, we could use our approximation function to predict, as the authors did, how changes in consumption affect GDP (or vice versa). Later in this section we will see how to find the least squares line to fit this data — the best linear approximation to the data. This involves finding a vector in a certain subspace of

R^{2}

that is closest to a given vector. Linear least squares approximation is a special case of a more general process that we will encounter in later sections where we learn how to project sets onto subspaces.

🔗

Subsection Introduction

🔗

The set

R^{n}

with vector addition and scalar multiplication has a nice algebraic structure. These operations satisfy a number of properties, such as associativity and commutativity of vector addition, the existence of an additive identity and additive inverse, distribution of scalar multiplication over vector addition, and others. These properties make it easier to work with the whole space since we can express the vectors as linear combinations of basis vectors in a unique way. This algebraic structure makes

R^{n}

a vector space.

🔗

There are many subsets of

R^{n}

that have this same structure. These subsets are called subspaces of

R^{n} .

These are the sets of vectors for which the addition of any two vectors is defined within the set, the scalar multiple of any vector by any scalar is defined within the set and the set contains the zero vector. One type of subset with this structure is the span of a set of vectors.

🔗

Recall that the span of a set of vectors

{v_{1}, v_{2}, \dots, v_{k}}

R^{n}

is the set of all linear combinations of the vectors. For example, if

v_{1} = [\begin{matrix} 1 \\ 1 \\ 0 \end{matrix}]

and

v_{2} = [\begin{matrix} 1 \\ 0 \\ 1 \end{matrix}],

then a linear combination of these two vectors is of the form

c_{1} v_{1} + c_{2} v_{2} = c_{1} [\begin{matrix} 1 \\ 1 \\ 0 \end{matrix}] + c_{2} [\begin{matrix} 1 \\ 0 \\ 1 \end{matrix}] = [\begin{matrix} c_{1} + c_{2} \\ c_{1} \\ c_{2} \end{matrix}] .

🔗

One linear combination can be obtained by letting

c_{1} = 2, c_{2} = - 3,

which gives the vector

2 v_{1} - 3 v_{2} = [\begin{matrix} - 1 \\ - 3 \\ 2 \end{matrix}] .

All such linear combinations form the span of the vectors

v_{!}

and

v_{2} .

In this case, these vectors will form a plane through the origin in

R^{3} .

🔗

Now we will investigate if the span of two vectors form a subspace, i.e. if it has the same structure as a vector space.

🔗

Preview Activity 12.1.

Let $w_{1}$ and $w_{2}$ be two vectors in $R^{n} .$ Let $W = Span {w_{1}, w_{2}} .$

🔗

(a)

For $W$ to be a subspace of $R^{n},$ the sum of any two vectors in $W$ must also be in $W .$

🔗

(i)

Pick two specific examples of vectors $u, y$ in $W$ (keeping $w_{1}, w_{2}$ unknown/general vectors). For example, one specific $u$ would be $2 w_{1} - 3 w_{2}$ as we used in the above example. Find the sum of $u, y .$ Is the sum also in $W ?$ Explain.

Hint.

What does it mean for a vector to be in $W ?$

🔗

(ii)

Now let $u$ and $y$ be arbitrary vectors in $W .$ Explain why $u + y$ is in $W .$

🔗

(b)

For $W$ to be a subspace of $R^{n},$ any scalar multiple of any vector in $W$ must also be in $W .$

🔗

(i)

Pick a specific example $u$ in $W .$ Explain why $2 u, - 3 u, π u$ are all also in $W .$

🔗

(ii)

Now let $a$ be an arbitrary scalar and let $u$ be an arbitrary vector in $W .$ Explain why the vector $a u$ is in $W .$

🔗

(c)

For $W$ to be a subspace of $R^{n},$ the zero vector must also be in $W .$ Explain why the zero vector is in $W .$

🔗

(d)

Does vector addition being commutative for vectors in $R^{n}$ imply that vector addition is also commutative for vectors in $W ?$ Explain your reasoning.

🔗

(e)

Suppose we have an arbitrary $u$ in $W .$ There is an additive inverse of $u$ in $R^{n} .$ In other words, there is a $u^{'}$ such that $u + u^{'} = 0 .$ Should this $u^{'}$ be also in $W ?$ If so, explain why. If not, give a counterexample.

🔗

(f)

Look at the other properties of vector addition and scalar multiplication of vectors in $R^{n}$ listed in Theorem 4.4 in Section 4. Which of these properties should also hold for vectors in $W ?$

🔗

Subsection Vector Spaces

🔗

The set of

n

-dimensional vectors with the vector addition and scalar multiplication satisfy many properties, such as addition being commutative and associative, existence of an additive identity, and others. The set

R^{n}

with these properties is an example of a vector space, a general structure examples of which include many other algebraic structures as we will see later.

🔗

Definition 12.3.

A set $V$ on which an operation of addition and a multiplication by scalars is defined is a vector space if for all $u,$ $v,$ and $w$ in $V$ and all scalars $a$ and $b :$

$u + v$ is an element of $V$ (we say that $V$ is closed under the addition in $V$ ),
$u + v = v + u$ (we say that the addition in $V$ is commutative),
$(u + v) + w = u + (v + w)$ (we say that the addition in $V$ is associative),
there is a vector $0$ in $V$ so that $u + 0 = u$ (we say that $V$ contains an additive identity or zero vector $0$ ),
for each $x$ in $V$ there is an element $y$ in $V$ so that $x + y = 0$ (we say that $V$ contains an additive inverse $y$ for each element $x$ in $V$ ),
$a u$ is an element of $V$ (we say that $V$ is closed under multiplication by scalars),
$(a + b) u = a u + b u$ (we say that multiplication by scalars distributes over scalar addition),
$a (u + v) = a u + a v$ (we say that multiplication by scalars distributes over addition in $V$ ),
$(a b) u = a (b u),$
$1 u = u .$

🔗

Theorem 4.4 in Section 4 shows that

R^{n}

is itself a vector space. As we will see, there are many other sets that have the same algebraic structure. By focusing on this structure and the properties of these operations, we can extend the theory of vectors we developed so far to a broad range of objects, making it easier to work with them. For example, we can consider linear combinations of functions or matrices, or define a basis for different types of sets of objects. Such algebraic tools provide us with new ways of looking at these sets of objects, including a geometric intuition when working with these sets. In this section, we will analyze subsets of

R^{n}

which behave similar to

R^{n}

algebraically. We will call such sets subspaces. In a later chapter we will encounter different kinds of sets that are also vector spaces.

🔗

Definition 12.4.

A subset $W$ of $R^{n}$ is a subspace of $R^{n}$ if $W$ itself is a vector space using the same operations as in $R^{n} .$

🔗

The following example illustrates the process for demonstrating that a subset of

R^{n}

is a subspace of

R^{n} .

🔗

Example 12.5.

There are many subsets of $R^{n}$ that are themselves vector spaces. Consider as an example the set $W$ of vectors in $R^{2}$ defined by

W = {[\begin{matrix} x \\ 0 \end{matrix}] | x is a real number} .

In other words, $W$ is the set of vectors in $R^{2}$ whose second component is 0. To see that $W$ is itself a vector space, we need to demonstrate that $W$ satisfies all of the properties listed in Definition 12.3.

To prove the first property, we need to show that the sum of any two vectors in $W$ is again in $W .$ So we need to choose two arbitrary vectors in $W .$ Let $u = [\begin{matrix} x \\ 0 \end{matrix}]$ and $v = [\begin{matrix} y \\ 0 \end{matrix}]$ be vectors in $W .$ Note that

u + v = [\begin{matrix} x \\ 0 \end{matrix}] + [\begin{matrix} y \\ 0 \end{matrix}] = [\begin{matrix} x + y \\ 0 \end{matrix}] .

Since the second component of $u + v$ is 0, it follows that $u + v$ is in $W .$ Thus, the set $W$ is closed under addition.

For the second property, that addition is commutative in $W,$ we can just use the fact that if $u$ and $v$ are in $W,$ they are also vectors in $R^{2}$ and $u + v = v + u$ is satisfied in $R^{2} .$ So the property also holds in $W .$

A similar argument can be made for property (3).

Property (4) states the existence of the additive identity in $W .$ Note that $0$ is an additive identity in $R^{2}$ and if it is also an element in $W,$ then it will automatically be the additive identity of $W .$ Since the zero vector can be written as $0 = [\begin{matrix} x \\ 0 \end{matrix}]$ with $x = 0,$ $0$ is in $W .$ Thus, $W$ satisfies property 4.

We will postpone property (5) for a bit since we can show that other properties imply property (5).

Property (6) is a closure property, just like property (1). We need to verify that any scalar multiple of any vector in $W$ is again in $W .$ Consider an arbitrary vector $u$ and an arbitrary scalar $a .$ Now

a u = a [\begin{matrix} x \\ 0 \end{matrix}] = [\begin{matrix} a x \\ 0 \end{matrix}] .

Since the vector $a u$ has a 0 as its second component, we see that $a u$ is in $W .$ Thus, $W$ is closed under scalar multiplication.

Properties (7), (8), (9) and (10) only depend on the operations of addition and multiplication by scalars in $R^{2} .$ Since these properties depend on the operations and not the vectors, these properties will transfer to $W .$

We still have to justify property (5) though. Note that since $1 - 1 = 0$ in real numbers, by applying property (7) with $a = 1,$ $b = - 1,$ we find that

0 = 0 u = (a + b) u = a u + b u = u + (- 1) u .

Therefore, $(- 1) u$ is an additive inverse for $u .$ Therefore, to show that the additive inverse of any $u$ in $W$ is also in $W,$ we simply note that any multiple of $u$ is also in $W$ and hence $(- 1) u$ must also be in $W .$

Since $W$ satisfies all of the properties of a vector space, $W$ is a vector space. Any subset of $R^{n}$ that is itself a vector space using the same operations as in $R^{n}$ is called a subspace of $R^{n} .$

🔗

Example 12.5 and our work Preview Activity 12.1 bring out some important ideas. When checking that a subset

W

of a vector space

R^{n}

is also a vector space, we can use the fact that all of the properties of the operations in

R^{n}

are transferred to any closed subset

W .

This implies that properties (2), (3), (7)-(10) are all automatically satisfied for

W

as well. Property (5) follows from the others. So we only need to check properties (1), (4) and (6). In fact, as we argued in the above example, property (4) also needs to be checked by simply checking that

0

R^{n}

is in

W .

We summarize this result in the following theorem.

🔗

Theorem 12.6.

A subset $W$ of $R^{n}$ is a subspace of $R^{n}$ if

whenever $u$ and $v$ are in $W$ it is also true that $u + v$ is in $W$ (that is, $W$ is closed under addition),
whenever $u$ is in $W$ and $a$ is a scalar it is also true that $a u$ is in $W$ (that is, $W$ is closed under scalar multiplication),
$0$ is in $W .$

🔗

The next activity provides some practice using Theorem 12.6.

🔗

Activity 12.2.

Use Theorem 12.6 to answer the following questions. Justify your responses. For sets which lie inside $R^{2},$ sketch a pictorial representation of the set and explain why your picture confirms your answer.

🔗

(a)

Is the set $W = {[\begin{matrix} x \\ y \end{matrix}] | y = 2 x}$ a subspace of $R^{2} ?$

🔗

(b)

Is the set $W = {[\begin{matrix} x \\ 0 \\ 1 \end{matrix}] | x is a scalar}$ a subspace of $R^{3} ?$

🔗

(c)

Is the set $W = {[\begin{matrix} x \\ x + y \end{matrix}] | x, y are scalars}$ a subspace of $R^{2} ?$

🔗

(d)

Is the set $W = {[\begin{matrix} x \\ y \end{matrix}] | y = 2 x + 1}$ a subspace of $R^{2} ?$

🔗

(e)

Is the set $W = {[\begin{matrix} x \\ y \end{matrix}] | y = x^{2}}$ a subspace of $R^{2} ?$

🔗

(f)

Is the set $W = {[\begin{matrix} 0 \\ 0 \\ 0 \\ 0 \end{matrix}]}$ a subspace of $R^{4} ?$

🔗

(g)

Is the set $W = {[\begin{matrix} x \\ y \\ z \end{matrix}] | x^{2} + y^{2} + z^{2} \leq 1}$ a subspace of $R^{3} ?$ Note that $W$ is the unit sphere (a.k.a. unit ball) in $R^{3} .$

🔗

(h)

Is the set $W = R^{2}$ a subspace of $R^{3} ?$

🔗

There are several important points that we can glean from Activity 12.2.

A subspace is a vector space within a larger vector space, similar to a subset being a set within a larger set.
The set containing the zero vector in $R^{n}$ is a subspace of $R^{n},$ and it is the only finite subspace of $R^{n} .$
Every subspace of $R^{n}$ must contain the zero vector.
No nonzero subspace is bounded — since a subspace must include all scalar multiples of its vectors, a subspace cannot be contained in a finite sphere or box.
Since vectors in $R^{k}$ have $k$ components, vectors in $R^{k}$ are not contained in $R^{n}$ when $n \neq k .$ However, if $n > k,$ then we can think of $R^{n}$ as containing a copy (what we call an isomorphic image) of $R^{k}$ as the set of vectors with zeros as the last $n - k$ components.

🔗

Subsection The Subspace Spanned by a Set of Vectors

🔗

One of the most convenient ways to represent a subspace of

R^{n}

is as the span of a set of vectors. In Preview Activity 12.1 we saw that the span of two vectors is a subspace of

R^{n} .

In the next theorem we verify this result for the span of an arbitrary number of vectors, extending the ideas you used in Preview Activity 12.1. Expressing a set of vectors as the span of some number of vectors is a quick way of justifying that this set is a subspace and it also provides us a geometric intuition for the set of vectors.

🔗

Theorem 12.7.

Let $v_{1},$ $v_{2},$ $\dots,$ $v_{k}$ be vectors in $R^{n} .$ Then $Span {v_{1}, v_{2}, \dots, v_{k}}$ is a subspace of $R^{n} .$

🔗

Proof.

Let $v_{1},$ $v_{2},$ $\dots,$ $v_{k}$ be vectors in $R^{n} .$ Let $W = Span {v_{1}, v_{2}, \dots, v_{k}} .$ To show that $W$ is a subspace of $R^{n}$ we need to show that $W$ is closed under addition and multiplication by scalars and that $0$ is in $W .$

First we show that $W$ is closed under addition. Let $u$ and $w$ be vectors in $W .$ This means that $u$ and $w$ are linear combinations of $v_{1},$ $v_{2},$ $\dots,$ $v_{k} .$ So there are scalars $a_{1},$ $a_{2},$ $\dots,$ $a_{k}$ and $b_{1},$ $b_{2},$ $\dots,$ $b_{k}$ so that

u = a_{1} v_{1} + a_{2} v_{2} + \dots + a_{k} v_{k} and w = b_{1} v_{1} + b_{2} v_{2} + \dots + b_{k} v_{k} .

To demonstrate that $u + w$ is in $W,$ we need to show that $u + w$ is a linear combination of $v_{1},$ $v_{2},$ $\dots,$ $v_{k} .$ Using the properties of vector addition and scalar multiplication, we find

\begin{aligned} u + w & = (a_{1} v_{1} + a_{2} v_{2} + \dots + a_{k} v_{k}) + (b_{1} v_{1} + b_{2} v_{2} + \dots + b_{k} v_{k}) \\ = (a_{1} + b_{1}) v_{1} + (a_{2} + b_{2}) v_{2} + \dots + (a_{k} + b_{k}) v_{k} . \end{aligned}

Thus $u + w$ is a linear combination of $v_{1},$ $v_{2},$ $\dots,$ $v_{k}$ and $W$ is closed under vector addition.

Next we show that $W$ is closed under scalar multiplication. Let $u$ be in $W$ and $c$ be a scalar. Then

c u = c (a_{1} v_{1} + a_{2} v_{2} + \dots + a_{k} v_{k}) = (c a_{1}) v_{1} + (c a_{2}) v_{2} + \dots + (c a_{k}) v_{k}

and $c u$ is a linear combination of $v_{1},$ $v_{2},$ $\dots,$ $v_{k}$ and $W$ is closed under multiplication by scalars.

Finally, we show that $0$ is in $W .$ Since

0 = 0 v_{1} + 0 v_{2} + \dots + 0 v_{k},

$0$ is in $W .$

Since $W$ satisfies all of the properties of a subspace as given in definition of a subspace, we conclude that $W$ is a subspace of $R^{n} .$

🔗

The subspace

W = Span {v_{1}, v_{2}, \dots, v_{k}}

is called the subspace of

R^{n}

spanned by

v_{1}, v_{2}, \dots, v_{k}

. We also use the phrase “subspace generated by

v_{1}, v_{2}, \dots, v_{k}

” since the vectors

v_{1}, v_{2}, \dots, v_{k}

are the building blocks of all vectors in

W .

🔗

Activity 12.3.

🔗

(a)

Describe geometrically as best as you can the subspaces of $R^{3}$ spanned by the following sets of vectors. ${[\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}]}$ ${[\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}], [\begin{array}{r} 0 \\ 1 \\ 0 \end{array}]}$

🔗

(b)

Express the following set of vectors as the span of some vectors to show that this set is a subspace. Can you give a geometric description of the set?

W = {[\begin{matrix} 2 x + y - z \\ y \\ z \\ - x + 3 z \end{matrix}] : x, y, z real numbers}

🔗

One additional conclusion we can draw from Activity 12.2 and Activity 12.3 is that subspaces of

R^{n}

are made up of “flat” subsets. The span of a single nonzero vector is a line (which is flat), and the span of a set of two distinct nonzero vectors is a plane (which is also flat). So subspaces of

R^{n}

are linear (or “flat”) subsets of

R^{n} .

That is why we can recognize that the non-flat parabola in Activity 12.2 is not a subspace of

R^{2} .

🔗

Subsection Examples

🔗

What follows are worked examples that use the concepts from this section.

🔗

Example 12.8.

Let $W = {[\begin{matrix} 2 r + s + t \\ r + t \\ r + s \end{matrix}] : r, s, t \in R} .$

🔗

(a)

Show that $W$ is a subspace of $R^{3} .$

Solution.

Every vector in $W$ has the form

[\begin{matrix} 2 r + s + t \\ r + t \\ r + s \end{matrix}] = r [\begin{matrix} 2 \\ 1 \\ 1 \end{matrix}] + s [\begin{matrix} 1 \\ 0 \\ 1 \end{matrix}] + t [\begin{matrix} 1 \\ 1 \\ 0 \end{matrix}]

for some real numbers $r,$ $s,$ and $t .$ Thus,

W = Span {[\begin{matrix} 2 \\ 1 \\ 1 \end{matrix}], [\begin{matrix} 1 \\ 0 \\ 1 \end{matrix}], [\begin{matrix} 1 \\ 1 \\ 0 \end{matrix}]} .

As a span of a set of vectors, we know that $W$ is a subspace of $R^{3} .$

🔗

(b)

Describe in detail the geometry of the subspace $W$ (e.g., is it a line, a union of lines, a plane, a union of planes, etc.)

Solution.

Let $v_{1} = [\begin{matrix} 2 \\ 1 \\ 1 \end{matrix}],$ $v_{2} = [\begin{matrix} 1 \\ 0 \\ 1 \end{matrix}],$ and $v_{3} = [\begin{matrix} 1 \\ 1 \\ 0 \end{matrix}] .$ The reduced row echelon form of $[v_{1} v_{2} v_{3}]$ is $[\begin{array}{ccr} 1 & 0 & 1 \\ 0 & 1 & - 1 \\ 0 & 0 & 0 \end{array}] .$ The pivot columns of $[v_{1} v_{2} v_{3}]$ form a linearly independent set with the same span as ${v_{1}, v_{2}, v_{3}},$ So $W = Span {v_{1}, v_{2}}$ and $W$ forms the plane in $R^{3}$ through the origin and the points $(2, 1, 1)$ and $(1, 0, 1) .$

🔗

Example 12.9.

🔗

(a)

Let $X = Span {[\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}]}$ and let $Y = Span {[\begin{matrix} 0 \\ 1 \\ 0 \end{matrix}]} .$ That is, $X$ is the $x$ -axis and $Y$ the $y$ -axis in three-space. Let

X + Y = {x + y : x \in X and y \in Y} .

🔗

(i)

Is $[\begin{matrix} 2 \\ 3 \\ 0 \end{matrix}]$ in $X + Y ?$ Justify your answer.

Solution.

We let $X = Span {[\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}]}$ and $Y = Span {[\begin{matrix} 0 \\ 1 \\ 0 \end{matrix}]} .$ T

Let $w = [\begin{matrix} 2 \\ 3 \\ 0 \end{matrix}],$ $x = 2 [\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}],$ and $y = 3 [\begin{matrix} 0 \\ 1 \\ 0 \end{matrix}] .$ Since $w = x + y$ with $x \in X$ and $y \in Y$ we conclude that $w \in X + Y .$

🔗

(ii)

Is $[\begin{matrix} 1 \\ 1 \\ 1 \end{matrix}]$ in $X + Y ?$ Justify your answer.

Solution.

We let $X = Span {[\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}]}$ and $Y = Span {[\begin{matrix} 0 \\ 1 \\ 0 \end{matrix}]} .$ T

Every vector in $X$ has the form $a e_{1}$ for some scalar $a$ (where $e_{1} = [\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}],$ and every vector in $Y$ has the form $b e_{2}$ for some scalar $b$ (where $e_{2} = [\begin{matrix} 0 \\ 1 \\ 0 \end{matrix}]$ ). So every vector in $X + Y$ is of the form $a e_{1} + b e_{2} = [\begin{matrix} a \\ b \\ 0 \end{matrix}] .$ Since the vector $[\begin{matrix} 1 \\ 1 \\ 1 \end{matrix}]$ does not have a $0$ in the third component, we conclude that in $[\begin{matrix} 1 \\ 1 \\ 1 \end{matrix}]$ is not in $X + Y .$

🔗

(iii)

Assume that $X + Y$ is a subspace of $R^{3} .$ Describe in detail the geometry of this subspace.

Solution.

We let $X = Span {[\begin{matrix} 1 \\ 0 \\ 0 \end{matrix}]}$ and $Y = Span {[\begin{matrix} 0 \\ 1 \\ 0 \end{matrix}]} .$ T

As we just argued, every vector in $X + Y$ has the form $a e_{1} + b e_{2} .$ So $X + Y = Span {e_{1}, e_{2}},$ which is the $x y$ -plane in $R^{3} .$

🔗

(b)

Now let $W_{1}$ and $W_{2}$ be arbitrary subspaces of $R^{n}$ for some positive integer $n .$ Let

W_{1} + W_{2} = {w_{1} + w_{2} : w_{1} \in W_{1} and w_{2} \in W_{2}} .

Show that $W_{1} + W_{2}$ is a subspace of $R^{n} .$ The set $W_{1} + W_{2}$ is called the sum of the subspaces $W_{1}$ and $W_{2} .$

Solution.

To see why the set $W_{1} + W_{2}$ is a subspace of $R^{3},$ suppose that $x$ and $y$ are in $W_{1} + W_{2} .$ Then $x = u_{1} + u_{2}$ and $y = z_{1} + z_{2}$ for some $u_{1}, z_{1}$ in $W_{1}$ and some $u_{2}, z_{2}$ in $W_{2} .$ Then

x + y = (u_{1} + u_{2}) + (z_{1} + z_{2}) = (u_{1} + z_{1}) + (u_{2} + z_{2}) .

Since $W_{1}$ is a subspace of $R^{3}$ it follows that $u_{1} + z_{1} \in W_{1} .$ Similarly, $u_{2} + z_{2} \in W_{2} .$ This makes $x + y$ an element of $W_{1} + W_{2} .$ Also, suppose that $a$ is a scalar. Then

a x = a (u_{1} + u_{2}) = a u_{1} + a u_{2} .

Since $W_{1}$ is a subspace of $R^{3}$ it follows that $a u_{1} \in W_{1} .$ Similarly, $a u_{2} \in W_{2} .$ This makes $a x$ an element of $W_{1} + W_{2} .$ Finally, since $0$ is in both $W_{1}$ and $W_{2},$ and $0 = 0 + 0,$ it follows that $0$ is an element of $W_{1} + W_{2} .$ We conclude that $W_{1} + W_{2}$ is a subspace of $R^{3} .$

🔗

Subsection Summary

A vector space is a set $V$ with operations of addition and scalar multiplication defined on $V$ such that for all $u,$ $v,$ and $w$ in $V$ and all scalars $a$ and $b :$
1. $u + v$ is an element of $V$ (we say that $V$ is closed under the addition in $V$ ),
2. $u + v = v + u$ (we say that the addition in $V$ is commutative),
3. $(u + v) + w = u + (v + w)$ (we say that the addition in $V$ is associative),
4. there is a vector $0$ in $V$ so that $u + 0 = u$ (we say that $V$ contains an additive identity or zero vector $0$ ),
5. for each $x$ in $V$ there is an element $y$ in $V$ so that $x + y = 0$ (we say that $V$ contains an additive inverse $y$ for each element $x$ in $V$ ),
6. $a u$ is an element of $V$ (we say that $V$ is closed under multiplication by scalars),
7. $(a + b) u = a u + b u$ (we say that multiplication by scalars distributes over scalar addition),
8. $a (u + v) = a u + a v$ (we say that multiplication by scalars distributes over addition in $V$ ),
9. $(a b) u = a (b u),$
10. $1 u = u .$
For every $n,$ $R^{n}$ is a vector space.
A subset $W$ of $R^{n}$ is a subspace of $R^{n}$ if $W$ is a vector space using the same operations as in $R^{n} .$
To show that a subset $W$ of $R^{n}$ is a subspace of $R^{n},$ we need to prove the following:
1. $u + v$ is in $W$ whenever $u$ and $v$ are in $W$ (when this property is satisfied we say that $W$ is closed under addition),
2. $a u$ is in $W$ whenever $a$ is a scalar and $u$ is in $W$ (when this property is satisfied we say that $W$ is closed under multiplication by scalars),
3. $0$ is in $W .$
The remaining properties of a vector space are properties of the operation, and as long as we use the same operations as in $R^{n},$ the operation properties follow the operations.
The span of any set of vectors in $R^{n}$ is a subspace of $R^{n} .$

🔗

Exercises Exercises

🔗

1.

Each of the following regions or graphs determines a ≈subset $W$ of $R^{2} .$ For each region, discuss each of the subspace properties of Theorem 12.4 and explain with justification if the set $W$ satisfies each property or not.

🔗

2.

Determine which of the following sets $W$ is a subspace of $R^{n}$ for the indicated value of $n .$ Justify your answer.

🔗

(a)

$W = {[x 0]^{T} : x is a scalar}$

🔗

(b)

$W = {[2 x + y x - y x + y]^{T} : x, y are scalars}$

🔗

(c)

$W = {[x + 1 x - 1]^{T} : x is a scalar}$

🔗

(d)

$W = {[x y x z y z]^{T} : x, y, z are scalars}$

🔗

3.

Find a subset of $R^{2}$ that is closed under addition and scalar multiplication, but that does not contain the zero vector, or explain why no such subset exists.

🔗

4.

Let $v$ be a vector in $R^{2} .$ What is the smallest subspace of $R^{2}$ that contains $v ?$ Explain. Describe this space geometrically.

🔗

5.

What is the smallest subspace of $R^{2}$ containing the first quadrant? Justify your answer.

🔗

6.

Let $u,$ $v,$ and $w$ be vectors in $R^{3}$ with $w = u + v .$ Let $W_{1} = Span {u, v}$ and $W_{2} = Span {u, v, w} .$

🔗

(a)

If $x$ is in $W_{1},$ must $x$ be in $W_{2} ?$ Explain.

🔗

(b)

If $y$ is in $W_{2},$ must $y$ be in $W_{1} ?$ Explain.

🔗

(c)

What is the relationship between $Span {u, v}$ and $Span {u, v, w} ?$ Be specific.

🔗

7.

Let $m$ and $n$ be positive integers, and let $v$ be in $R^{n} .$ Let $W = {A v : A \in M_{m \times n}} .$

🔗

(a)

As an example, let $v = [2 1]^{T}$ in $R^{2}$ with $W = {A v : A \in M_{2 \times 2}} .$

🔗

(i)

Show that the vector $[2 1]^{T}$ is in $W$ by finding a matrix $A$ that places $[2 1]^{T}$ in $W .$

🔗

(ii)

Show that the the vector $[4 2]^{T}$ is in $W$ by finding a matrix $A$ that places $[4 2]^{T}$ in $W .$

🔗

(iii)

Show that the vector $[6 - 1]^{T}$ is in $W$ by finding a matrix $A$ that places $[6 - 1]^{T}$ in $W .$

🔗

(iv)

Show that $W = R^{2} .$

🔗

(b)

Show that, regardless of the vector $v$ selected, $W$ is a subspace of $R^{m} .$

🔗

(c)

Characterize all of the possibilities for what the subspace $W$ can be.

Hint.

There is more than one possibility.

🔗

8.

Let $S_{1}$ and $S_{2}$ be subsets of $R^{3}$ such that $Span S_{1} = Span S_{2} .$ Must it be the case that $S_{1}$ and $S_{2}$ contain at least one vector in common? Justify your answer.

🔗

9.

Assume $W_{1}$ and $W_{2}$ are two subspaces of $R^{n} .$ Is $W_{1} \cap W_{2}$ also a subspace of $R^{n} ?$ Is $W_{1} \cup W_{2}$ also a subspace of $R^{n} ?$ Justify your answer. (Note: The notation $W_{1} \cap W_{2}$ refers to the vectors common to both $W_{1}, W_{2},$ while the notation $W_{1} \cup W_{2}$ refers to the vectors that are in at least one of $W_{1}, W_{2} .$ )

🔗

10.

Determine whether the plane defined by the equation $5 x + 3 y - 2 z = 0$ is a subspace in $R^{3} .$

🔗

11.

If $W$ is a subspace of $R^{n}$ and $u$ is a vector in $R^{n}$ not in $W,$ determine whether

u + W = {u + v : v is a vector in W}

is a subspace of $R^{n} .$

🔗

12.

Two students are talking about examples of subspaces.

Student 1: The $x$ -axis in $R^{2}$ is a subspace. It is generated by the vector $[\begin{matrix} 1 \\ 0 \end{matrix}] .$

Student 2: Similarly $R^{2}$ is a subspace of $R^{3} .$

Student 1: I'm not sure if that will work. Can we fit $R^{2}$ inside $R^{3} ?$ Don't we need $W$ to be a subset of $R^{3}$ if it is a subspace of $R^{3} ?$

Student 2: Of course we can fit $R^{2}$ inside $R^{3} .$ We can think of $R^{2}$ as vectors $[\begin{matrix} a \\ b \\ 0 \end{matrix}] .$ That's the $x y$ -plane.

Student 1: I don't know. The vector $[\begin{matrix} a \\ b \\ 0 \end{matrix}]$ is not exactly same as $[\begin{matrix} a \\ b \end{matrix}] .$

Student 2: Well, $R^{2}$ is a plane and so is the $x y$ -plane. So they must be equal, shouldn't they?

Student 1: But there are infinitely many planes in $R^{3} .$ They can't all be equal to $R^{2} .$ They all “look like” $R^{2}$ but I don't think we can say they are equal.

Which student is correct? Is

R^{2}

a subspace of

R^{3},

or not? Justify your answer.

🔗

13.

Given two subspaces $H_{1}, H_{2}$ of $R^{n},$ define

H_{1} + H_{2} = {w ∣ w = u + v where u in H_{1}, v in H_{2}} .

Show that $H_{1} + H_{2}$ is a subspace of $R^{n}$ containing both $H_{1}, H_{2}$ as subspaces. The space $H_{1} + H_{2}$ is the sum of the subspaces $H_{1}$ and $H_{2} .$

🔗

14.

Label each of the following statements as True or False. Provide justification for your response.

🔗

(a) True/False.

Any line in $R^{n}$ is a subspace in $R^{n} .$

🔗

(b) True/False.

Any line through the origin in $R^{n}$ is a subspace in $R^{n} .$

🔗

(c) True/False.

Any plane through the origin in $R^{n}$ is a subspace in $R^{n} .$

🔗

(d) True/False.

In $R^{4},$ the points satisfying $x y = 2 t + z$ form a subspace.

🔗

(e) True/False.

In $R^{4},$ the points satisfying $x + 3 y = 2 z$ form a subspace.

🔗

(f) True/False.

Any two nonzero vectors generate a plane subspace in $R^{3} .$

🔗

(g) True/False.

The space $R^{2}$ is a subspace of $R^{3} .$

🔗

(h) True/False.

If $W$ is a subspace of $R^{n}$ and $u$ is in $W,$ then the line through the origin and $u$ is in $W .$

🔗

(i) True/False.

There are four types of subspaces in $R^{3} :$ ${0},$ line through origin, plane through origin and the whole space $R^{3} .$

🔗

(j) True/False.

There are four types of subspaces in $R^{4} :$ ${0},$ line through origin, plane through origin and the whole space $R^{4} .$

🔗

(k) True/False.

The vectors $[\begin{matrix} 1 \\ 1 \\ 1 \end{matrix}],$ $[\begin{matrix} 1 \\ 2 \\ 1 \end{matrix}]$ and $[\begin{matrix} 2 \\ 3 \\ 2 \end{matrix}]$ form a subspace in $R^{3} .$

🔗

(l) True/False.

The vectors $[\begin{matrix} 1 \\ 1 \\ 1 \end{matrix}]$ and $[\begin{matrix} 1 \\ 2 \\ 1 \end{matrix}]$ form a basis of a subspace in $R^{3} .$

🔗

Subsection Project: Least Squares Linear Approximation

🔗

We return to the problem of finding the least squares line to fit the GDP-consumption data. We will start our work in a more general setting, determining the method for fitting a linear function to fit any data set, like the GDP-consumption data, in the least squares sense. Then we will apply our result to the GDP-consumption data.

🔗

Project Activity 12.4.

Suppose we want to fit a linear function $p (x) = m x + b$ to our data. For the sake of our argument, let us assume the general case where we have $n$ data points labeled as $(x_{1}, y_{1}),$ $(x_{2}, y_{2}),$ $(x_{3}, y_{3}),$ $\dots,$ $(x_{n}, y_{n}) .$ (In the GDP-consumption data $n = 21 .$ ) In the unlikely event that the graph of $p (x)$ actually passes through these data points, then we would have the system of equations

\begin{aligned} y_{1} & = b + m x_{1} \\ y_{2} & = b + m x_{2} \\ (12.1) & y_{3} & = b + m x_{3} \\ ⋮ & ⋮ \\ y_{n} & = b + m x_{n} \end{aligned}

in the unknowns $b$ and $m .$

🔗

(a)

As a small example to illustrate, write the system (\ref{eq:LS_system}) using the threepoints $(x_{1}, y_{1}) = (1, 2),$ $(x_{2}, y_{2}) = (3, 4),$ and $(x_{3}, y_{3}) = (5, 6) .$ Identify the unknowns and then write this system in the form $M a = y .$ Explicitly identify thematrix $M$ and the the vectors $a$ and $y .$

🔗

(b)

Identify the specific matrix $M$ and the specific vectors $a$ and $y$ using the data inTable \ref{T:GDP_consumption}. Explain why the system $M a = y$ is inconsistent.(Remember, we are treating consumption as the independent variable and GDP as the dependentvariable.) What does the result tell us about the data?

🔗

Project Activity 12.4 shows that the GDP-consumption data does not lie on a line. So instead of attempting to find coefficients

b

and

m

that give a solution to this system, which may be impossible, we instead look for a vector

a^{*}

that provides us with something that is “close” to a solution.

🔗

If we could find

b

and

m

that give a solution to the system

M a = y,

then

M a - y

would be zero. If we can't make

M a - y

exactly equal to the vector

0,

we could instead try to minimize

M a - y

in some way. One way is to minimize the length

| | M a - y | |

of the vector

M a - y .

🔗

If we minimize the quantity

| | M a - y | |,

then we will have minimized a function given by a sum of squares. That is,

| | M a - y | |

is calculated to be

\begin{matrix} (12.2) & \sqrt{(b + m x_{1} - y_{1})^{2} + (b + m x_{2} - y_{2})^{2} + \dots + (b + m x_{n} - y_{n})^{2}} . \end{matrix}

🔗

This is why the method we will derive is called the method of least squares. This method provides us with a vector “solution” in a subspace that is related to

M .

We can visualize

| | M a - y | |

as in Figure 12.10. In this figure the data points are shown along with a linear approximation (not the best for illustrative purposes). The lengths of the vertical line segments are the summands

(b + m x_{i} - y_{i})

in (12.2). So we are trying to minimize the sum of the squares of these line segments.

🔗
Figure 12.10. Error in the linear approximation.

🔗

Suppose that

a^{*}

minimizes

| | M a - y | | .

Then the vector

M a^{*}

is the vector that is closest to

y

of all of the vectors of the form

M x .

The fact that the vectors of the form

M x

make a subspace will be useful in what follows. We verify that fact in the next project activity.

🔗

Project Activity 12.5.

Let $A$ be an arbitrary $m \times k$ matrix. Explain why the set $C = {A x : x \in R^{k}}$ is a subspace of $R^{m} .$

🔗

Project Activity 12.5 shows us that even though the GDP-consumption system

M a = y

does not have a solution, we can find a vector that is close to a solution in the subspace

{M x : x \in R^{2}} .

That is, find a vector

a^{*}

R^{2}

such that

M a^{*}

is as close (in the least squares sense) to

y

as we can get. In other words, the error

| | M a^{*} - y | |

is as small as possible. In the following activity we see how to find

a^{*} .

🔗

Project Activity 12.6.

Let

S = \sqrt{(b + m x_{1} - y_{1})^{2} + (b + m x_{2} - y_{2})^{2} + \dots + (b + m x_{n} - y_{n})^{2}},

the quantity we want to minimize. The variables in $S$ are $m$ and $b,$ so we can think of $S$ as a function of the two independent variables $m$ and $b .$ The square root makes calculations more complicated, so it is helpful to notice that $S$ will be a minimum when $S^{2}$ is a minimum. Since $S^{2}$ is also function of the two variables $b$ and $m,$ the minimum value of $S^{2}$ will occur when the partial derivatives of $S^{2}$ with respect to $b$ and $m$ are both $0$ (if you haven't yet taken a multivariable calculus course, you can just assume that this is correct). This yields the equations

\begin{aligned} (12.3) & 0 & = \sum_{i = 1}^{n} (m x_{i} + b - y_{i}) x_{i} \\ (12.4) & 0 & = \sum_{i = 1}^{n} (m x_{i} + b - y_{i}) . \end{aligned}

In this activity we solve equations (12.3) and (12.4) for the unknowns $b$ and $m .$ (Do this in a general setting without using specific values for the $x_{i}$ and $y_{i} .$ )

🔗

(a)

Let $r = \sum_{i = 1}^{n} x_{i}^{2},$ $s = \sum_{i = 1}^{n} x_{i},$ $t = \sum_{i = 1}^{n} y_{i},$ and $u = \sum_{i = 1}^{n} x_{i} y_{i} .$ Show that the equations (12.3) and (12.4) can be written in the form

\begin{aligned} 0 & = b s + m r - u \\ 0 & = b n + m s - t . \end{aligned}

Note that this is a system of two linear equations in the unknowns $b$ and $m .$

🔗

(b)

Write the system from part (a) in matrix form $A x = b .$ Then use techniques from linear algebra to solve the linear system to show that

\begin{matrix} (12.5) & b = \frac{t r - u s}{n r - s^{2}} = \frac{(\sum_{i = 1}^{n} y_{i}) (\sum_{i = 1}^{n} x_{i}^{2}) - (\sum_{i = 1}^{n} x_{i}) (\sum_{i = 1}^{n} x_{i} y_{i})}{n (\sum_{i = 1}^{n} x_{i}^{2}) - {(\sum_{i = 1}^{n} x_{i})}^{2}} \end{matrix}

and

\begin{matrix} (12.6) & m = \frac{n u - t s}{n r - s^{2}} = \frac{n (\sum_{i = 1}^{n} x_{i} y_{i}) - (\sum_{i = 1}^{n} x_{i}) (\sum_{i = 1}^{n} y_{i})}{n (\sum_{i = 1}^{n} x_{i}^{2}) - {(\sum_{i = 1}^{n} x_{i})}^{2}} . \end{matrix}

🔗

Project Activity 12.7.

Use the formulas (12.5) and (12.6) to find the values of

b

and

m

for the regression line to fit the GDP-consumption data in Table 12.1. You may use the fact that the sum of the GDP data is

3.5164030 \times 10^{6},

the sum of the consumption data is

2.9233750 \times 10^{6},

the sum of the squares of the consumption data is

8.806564894 \times 10^{11},

and the sum of the products of the GDP and consumption data is

1.069946378 \times 10^{12} .

Compare to the results the authors obtained in the paper “A Statistical Analysis of GDP and Final Consumption Using Simple Linear Regression, the Case of Romania 1990-2010”.

Bălăcescu, Aniela & Zaharia, Marian. (2012). A STATISTICAL ANALYSIS OF GDP AND FINAL CONSUMPTION USING SIMPLE LINEAR REGRESSION. THE CASE OF ROMANIA 1990?2010. Annals - Economy Series. 4. 26-31. Available from: Research Gate ²⁸.

researchgate.net/publication/227382939_A_STATISTICAL_ANALYSIS_OF_GDP_AND_FINAL_CONSUMPTION_USING_SIMPLE_LINEAR_REGRESSION_THE_CASE_OF_ROMANIA_1990-2010

Section 12 The Structure of Rn

Focus Questions

Subsection Application: Connecting GDP and Consumption in Romania

Subsection Introduction

Preview Activity 12.1.

(a)

(i)

(ii)

(b)

(i)

(ii)

(c)

(d)

(e)

(f)

Subsection Vector Spaces

Definition 12.3.

Definition 12.4.

Example 12.5.

Theorem 12.6.

Activity 12.2.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Subsection The Subspace Spanned by a Set of Vectors

Theorem 12.7.

Proof.

Activity 12.3.

(a)

(b)

Subsection Examples

Example 12.8.

(a)

Solution.

(b)

Solution.

Example 12.9.

(a)

(i)

Solution.

(ii)

Solution.

(iii)

Solution.

(b)

Solution.

Subsection Summary

Exercises Exercises

1.

(a)

(b)

(c)

(d)

2.

(a)

(b)

(c)

(d)

3.

4.

5.

6.

(a)

(b)

(c)

7.

(a)

(i)

(ii)

(iii)

(iv)

(b)

(c)

8.

9.

Section 12 The Structure of $R^{n}$