Bases and Dimension

Matrices are important in the study of vector spaces. They provide a wealth of examples of
spaces (the spaces whose vectors are the matrices themselves as well as null spaces, column
spaces, and row spaces as we have seen) and they also provide basic mappings between vector
spaces as we shall see in a later chapter. Fundamental notions for all vector spaces are the
concepts that we now consider: linear independence and spanning.

Definition

Suppose that v₁, v₂, ..., v_n are vectors in some vector space V. We say that they are linearly
independent if c₁ v₁ + c₂ v₂ + ... + c_n v_n = 0 implies that c₁ = c₂ = ... = c_n = 0, i.e., the only linear
combination of the vectors that gives the zero vector is the one in which the coefficient of each
vector is zero. The sequence of vectors is linearly dependent if there exist scalars c₁, c₂, ..., c_n
that are not all zero with c₁ v₁ + c₂ v₂ + ... + c_n v_n = 0.

Examples

1. The vectors (8, -1, 9), (4, 3, -7), (22, 6, -4) in R^3 are linearly dependent since
3(8, -1, 9) + 5 (4, 3, -7) + (-2) (22, 6, -4) = (0, 0, 0) = 0.

2. The vectors in M_2×2 are clearly linearly independent

since if then from the 1, 1 position

we must have a = 0, from the 1, 2 position b = 0, and from the 2, 1 position c = 0.

3. The vectors x^2 + 5x -2 and 3x^2 + 10 in P₃, the set of polynomials of degree 3 or less with
real coefficients , are linearly independent since if a(x^2 + 5x -2) + b(3x^2 + 10) = 0 = 0,
then the coefficient of x in the sum must be 0, so a = 0, and then, e.g., the coefficient of
x^2 , that is, a + 3b, must be 0, so b = 0.

4. Suppose that the sequence of vectors v₁, v₂, ..., v_n is linearly dependent and that w is any
vector. Then v₁, v₂, ..., v_n, w is a linearly dependent set of vectors. Why?

5. Any set of vectors which includes the zero vector is linearly dependent.

Theorem

Suppose that v₁, v₂, ..., v_n is a linearly dependent set of at least two vectors (n >1). Then there is
some vector in the set which is a linear combination of the others.

Proof

Since the set is linearly dependent, there are scalars c₁, c₂, ..., c_n, not all 0 such that c₁ v₁ + c₂ v₂ +
... + c_n v_n = 0. Choose one of these scalars which is non-zero. By renumbering the vectors and
scalars, if necessary, we may assume that Solving for v_n we get v_n = -c₁/c_n v₁ - c₂/c_n v₂ - ...
- c_n-1/c_n v_n-1. Thus, v_n is a linear combination of the remaining vectors in the set.

It should be clear that the theorem above can be stated as an if and only if result; if some vector
in a set can be written as a linear combination of the others, then the set is linearly dependent.
Thus linear dependence of a set of vectors can be characterized in terms of writing some vector
in the set as a linear combination of the others.

Example

We have defined <v₁, v₂, ..., v_n> to be the subspace generated by the vectors v₁, v₂, ..., v_n,
Suppose that the v_i are linearly dependent. By the theorem above one of the vectors can be
written as a linear combination of the others, say v_n is a linear combination of v₁, v₂, ..., v_n-1. It is
then easy to see that <v₁, v₂, ..., v_n> = <v₁, v₂, ..., v_n-1>, i.e. this subspace is also generated by a
smaller subset of the v_i.

How does one decide whether or not a sequence of vectors is linear independent? Write the
definition - simply set up a linear combination of the vectors using variables for the scalars and
equate this to 0. Then solve this equation to see if there is a solution other than the trivial
solution in which each scalar is 0.

Example

Determine whether the fol lowing set of vectors is linearly independent.
(4, -3, 9, 5), (0, 7, 1, -2), (-5, 2, 0, 6), (1, 6, -8, 0)
Using the definition the question is exactly can we ex press the zero vector, 0, as a nontrivial (not
all the scalars being zero) linear combination of these vectors? Thus we proceed to solve the
vector equation below for the scalars a, b, c, and d.
a(4, -3, 9, 5) + b (0, 7, 1, -2) + c (-5, 2, 0, 6) + d (1, 6, -8, 0) = (0, 0, 0, 0)
This becomes
(4a - 5c + d, -3a + 7b + 2c + 6d, 9a + b - 8d, 5a - 2b + 6c) = (0, 0, 0, 0)
which, equating the components of the vectors on each side of the equation, amounts to the
homogeneous system

Therefore we row reduce the coefficient matrix

to the equivalent matrix

This tells us that the only solution to the system (and so for the scalars in the vector equation) is
the trivial solution. Hence, the vectors are linearly independent. If we replace the last vector in
the set by (9, 2, 10, -3), the coefficient matrix becomes

Since this matrix reduces tothe corresponding

system of linear equations has many solutions (take d to be any value and adjust a, b, and c
accordingly) and the new set of vectors is linearly dependent.

Definition

Suppose that v₁, v₂, ..., v_n are vectors in the vector space V and that S is a subset of V. The
vectors span S provided each element of S can be written as a linear combination of the vectors
v₁, v₂, ..., v_n .

Examples

1. Decide whether (676, -677, 363) and (95, 71, -68) are in the span of (41, -25, 16),
(-38, 61, -29), and (53, 119, -36).

Combining the augmented matrices resulting from the vector equations
a (41, -25, 16) + b (-38, 61, -29) + c (53, 119, -36) = (676, -677, 363) and also this left
hand side equal to (95, 71, -68) we get the matrix

The equivalent reduced row echelon form of this matrix is

Therefore, (676, -677, 363) is in the span of these vectors but (95, 71, - 68) is not.

2. Prove that the vectorsspan the space

M_2×2. We need to show that for an arbitrary element in M_2×2, there are scalars

Performing the scalar multiplications, adding the matrices and equating corresponding
elements results in the system

The coefficient matrix of the system row reduces to the identity

matrix so M_2×2 is spanned by these vectors.

3. Notice that any set that is spanned by the vectors v₁, v₂, ..., v_n is contained in <v₁, v₂, ...,
v_n> and is also spanned by v₁, v₂, ..., v_n+1 for any vector v_n+1.

A subspace is defined by its generators. As a vector space, the number of generators in a
minimal set of generators for any subspace completely determines the space. This is actually
true of any vector space which motivates the following definition.

Definition
The vectors v₁, v₂, ..., v_n form a basis for the vector space V if
1. They are linearly independent and
2. They span V.

Example

1. The standard basis for Rⁿ is the set e₁, e₂, ..., e_n where each e_i has all zero components
except for a 1 in its i^th component. In R^3 we have the standard basis e₁ = (1, 0, 0),
e₂ = (0, 1, 0), and (0, 0, 1).

2. For the vector space of n by m matrices, M_m×n, the standard basis consists of the matrices
e_ij for i = 1, 2, ..., m and j = 1, 2, ..., n where e_ij = (a_st) and a_st = 0 if and a_ij = 1.
Thus, e.g., M_2×3 has as its standard basis the vectors

3. The vector space consisting of all polynomials of degree n or less, P_n, has
as its standard basis.

Every spanning set of a vector space contains a basis for the space

Theorem

Suppose that S = {v₁, v₂, ..., v_n} spans the vector space V. Then there is a basis of V consisting
of a subset of S.

Proof

S is a linearly independent set, then S is a basis for V. So suppose that S is a linearly dependent
set. Then there is some vector (say v_n) in S which is a linear combination of the others. By a
previous example we know <v₁, v₂, ..., v_n> = <v₁, v₂, ..., v_n-1>. Thus the set {v₁, v₂, ..., v_n-1} spans
V. Repeating his argument we arrive at a subset of S which is linearly independent and also
spans V, i.e. is a basis for V.

Any two bases for a fixed vector space have the same number of elements.

Theorem

Suppose that S = {v₁, v₂, ..., v_n} and T = {w₁, w₂,..., w_m} are both bases for a vector space V.
Then n = m.

Definition

The dimension of a vector space V, denoted dim(V), is the number os elements in any basis of
V.

Examples
From the standard basis given earlier we observe that dim(Rⁿ) = n, dim(M_m×n) = nm, and dim(P_n)
= n + 1.

Theorem (Unique Representation)
Suppose that {v₁, v₂, ..., v_n} is a basis for V and w is a vector in V. Then there exist unique
scalars c₁, c₂, ..., c_n such that w = c₁ v₁ + c₂ v₂ + ... + c_n v_n.

Proof
Since S is a basis, it spans V so w is a linear combination of the v_i. This gives the existence. For
the uniqueness, suppose that w = c₁ v₁ + c₂ v₂ + ... + c_n v_n and also w = d₁ v₁ + d₂ v₂ + ... + d_n v_n.
Then 0 = w - w = (c₁ - d₁) v₁ + (c₂ - d₂) v₂ + ... + (c_n - d_n) v_n. But since S is a linearly independent
set, c_i - d_i = 0 for each i. Hence, c_i = d_i for all i = 1, 2, ..., n which proves uniqueness.