# Linear Algebra: Linear Combinations

## Linear Independence

Let $v1_, \ldots, v_n$ be vectors in a vector space $V$ over a field $F$. Let $v \in \text{span}(v_1, \ldots, v_n)$. By the definition of span, there exist some set of coefficients $a_1, \ldots, a_n \in F$ such that

$v = \sum\limits_{i=1}^{n} a_i v_i$

You might be curious as to whether the set of $a_i$ are unique. Is there only one combination of coefficients that produce $v$? How can we tell? The answer is to suppose the contrary. Suppose the coefficients aren't unique. Suppose there is another set of coefficients, $b_1, \ldots, b_n \in F$ such that

$v = \sum\limits_{i=1}^{n} b_i v_i$

Then we can see that

$v = \sum\limits_{i=1}^{n} a_i v_i \\ \sum\limits_{i=1}^{n} b_i v_i = \sum\limits_{i=1}^{n} a_i v_i \\ 0 = \sum\limits_{i=1}^{n} a_i v_i - \sum\limits_{i=1}^{n} b_i v_i \\ 0 = \sum\limits_{i=1}^{n} (a_i - b_i) v_i \\$

If the only way this sum works out is for for each $a_i$ to equal $b_i$, then the coefficients are in fact unique. It turns out that this situation of uniqueness is fairly important in linear algebra, so it has its own name.

A set of vectors $v_1, \ldots, v_n$ in a vector space $V$ over a field $F$ is linearly independent if the only values of $c_1, \ldots, c_n \in F$ that make $c_1v_1 + \ldots + c_nv_n$ equal $0$ are $c_1=\ldots=c_n=0$. The empty set is defined to be linearly independent.

A set of vectors are linearly dependent if they are not linearly independent.

## Problems

1. Explain why a set of vectors containing $0$ is never linearly independent.

Linear independence requires that there be only one combination of coefficients that makes the set of vectors sum to $0$. However, the $0$ vector can have any coefficient from the field.

If you thought you could be clever and use a field that had only one element, then you're not clever enough, as a field must by definition have at least two elements - the $0$ element and the $1$ element.

2. What are the conditions for a set of a single vector to be linearly independent?

Let $v$ be the vector in question.  If it is the $0$ vector, the set is linearly dependent. If not, then the only coefficient $c$ that satisfies $cv = 0$ is $0$. Therefore $v$ can be any vector but the $0$ vector.

3. Under what conditions is a set of two vectors linearly independent?

Let $v_1, v_2$ be vectors in a vector space $V$ over a field $F$. The question is what coefficients $c_1, c_2 \in F$ ensure that $c_1v_1 + c_2v_2 \neq 0$. For $c_1 \neq 0$, rewrite the equation as

$v_1 + (c_1)^{-1}c_2v \neq 0 \\ v_1 \neq -(c_1)^{-1}c_2v \\$

We see that $v_1$ cannot be a scalar multiple of $v_2$. If $c_1=0$, then repeat the process with $c_2 \neq 0$ to arrive at the same conclusion. Thus in order two be linearly independent, two vectors cannot be scalar multiples of each other.

4. Determine whether the vectors $\langle 1, 0, 0 \rangle, \langle 0, 1, 0 \rangle, \langle 0, 0, 1 \rangle \in \mathbb{R}^3$ are linearly independent.

The set of vectors is linearly independent if the only set of coefficients satisfying the equation

$0 = a_1 \langle 1, 0, 0 \rangle + a_2 \langle 0, 1, 0 \rangle + a_3 \langle 0, 0, 1 \rangle$

are the coefficients $c_1=c_2=c_3=0$. Suppose there is another set of coefficients also satisfying the equation:

$0 = b_1 \langle 1, 0, 0 \rangle + b_2 \langle 0, 1, 0 \rangle + b_3 \langle 0, 0, 1 \rangle$

Let's see if they're unique.

$0 = (a_1 - b_1) \langle 1, 0, 0 \rangle + (a_2 - b_2) \langle 0, 1, 0 \rangle + (a_3 - b_3) \langle 0, 0, 1 \rangle \\ 0 = \langle (a_1 - b_1), 0, 0 \rangle + \langle 0, (a_2 - b_2), 0 \rangle + \langle 0, 0, (a_3 - b_3) \rangle \\ \langle 0, 0, 0 \rangle = \langle (a_1 - b_1), 0, 0 \rangle + \langle 0, (a_2 - b_2), 0 \rangle + \langle 0, 0, (a_3 - b_3) \rangle \\ \langle 0, 0, 0 \rangle = \langle (a_1 - b_1), (a_2 - b_2), (a_3 - b_3) \rangle \\$

The only way for each vector component $a_i - b_i$ to equal $0$ is for $b_i$ to equal $a_i$. Therefore the coefficients are uniquely equal to $0$, so the three vectors in question are linearly independent.

5. Recall the vectors $e_1, \ldots, e_n \in F^n$, where $F$ is a field, such that $e_1 = \langle 1, 0, \ldots, 0 \rangle, e_2 = \langle 0, 1, \ldots, 0 \rangle, \ldots, e_n = \langle 0, 0, \ldots, 1 \rangle$. Show that these vectors are linearly independent.

To show a set of vectors is linearly independent, we must show that the only coefficients that cause the weighted sum to go to $0$ are $c_1=c_2=\ldots=c_n=0$. To show this uniqueness, we imagine there is another set of coefficients, $d_1, d_2, \ldots, d_n$, that also satsifies the equation, and prove that the two sets of coefficients are equal.

$c_1e_1 + c_2e_2 + \ldots + c_ne_n = 0 \\ c_1e_1 + c_2e_2 + \ldots + c_ne_n = d_1e_1 + d_2e_2 + \ldots + d_ne_n \\ 0 = (d_1e_1 - c_1e_1) + (d_2e_2 - c_2e_2) + \ldots + (d_ne_n - c_ne_n) \\ 0 = \langle d_1-c_1, 0, \ldots, 0 \rangle + \langle 0, d_2-c_2, \ldots, 0 \rangle + \langle 0, 0, \ldots, d_n - c_n \rangle \\ \langle 0, 0, \ldots, 0 \rangle = \langle d_1-c_1, 0, \ldots, 0 \rangle + \langle 0, d_2-c_2, \ldots, 0 \rangle + \langle 0, 0, \ldots, d_n - c_n \rangle \\$

Clearly each $d_i=c_i$, therefore the coefficients are all uniquely $0$, and the set of vectors $e_1, e_2, \ldots, e_n$ is linearly independent.

6. Show that a set of vectors $v_1, \ldots, v_n$ in a vector space $V$ over $F$ is linearly independent if and only if every for each $v \in \text{span}(v_1, \ldots, v_n)$ there is a unique set of coefficients $c_1, \ldots, c_n \in F$ such that $v = c_1v_1 + \ldots c_nv_n$.

If there are unique coefficients $c_1, \ldots, c_n \in F$ for each $v \in V$ such that $v = c_1v_1 + \ldots + c_nv_n$, then the equation $0 = c_1v_1 + \ldots + c_nv_n$ has unique coefficients that all equal $0$.

Conversely, suppose $v_1, \ldots, v_n$ are linearly independent, and let $v \in \text{span}(v_1, \ldots, v_n)$. Then

$v = c_1v_1 + \ldots + c_nv_n$

for some set of coefficients $c_1, \ldots, c_n \in F$. Suppose further that there is another set of coefficients, $d_1, \ldots, d_n \in F$ that also form a linear combination:

$v = d_1v_1 + \ldots + d_nv_n$

Then

$c_1v_1 + \ldots + c_nv_n = d_1v_1 + \ldots + d_nv_n \\ 0 = (d_1 - c_1)v_1 + \ldots + (d_n - c_n)v_n \\$

Because the set of vectors $v_1, \ldots, v_n$ is linearly independent, all the above coefficients $(d_i-c_i)$ equal $0$. This means that $c_i=d_i$, and therefore that the coefficients $c_1, \ldots, c_n$ are unique.