Real Analysis: The Real Numbers

Euclidean Space

Definition

The cartesian product of $\mathbb{R}$ with itself $n$ times, for some positive $n \in \mathbb{N}$, is denoted as $\mathbb{R}^n$, and consists of all the $n$-tuples of real numbers. These tuples, called vectors, are enclosed by square brackets rather than parentheses both to distinguish them from algebraic parentheses and to convey the additional algebraic and geometric interpretation that comes with them, which we introduce below. For example, $\mathbb{R}^3$ consists of all triplets of real numbers, such as $[1, 2, 3]$ and $[-9, 0, \sqrt{2}]$. Vectors in $\mathbb{R}^n$ are abbreviated as bold letters in order to contrast them with elements of $\mathbb{R}$, which we in turn call scalars, and their components are indexed sequentially: If $\mathbf{x} \in \mathbb{R}^4$, then $\mathbf{x} = [x_1, x_2, x_3, x_4]$.

$\mathbb{R}$ can be visualized as a number line, $\mathbb{R}^2$ as a plane, and $\mathbb{R}^3$ as a 3-dimensional space.

Algebra on $\mathbb{R}^n$

The algebraic operations on $\mathbb{R}^n$ are not obviously inferred from those on $\mathbb{R}$. In fact, much of the algebra on $\mathbb{R}^n$ is treated as its own subject called linear algebra. However, we define three arithmetic operations here, as they are necessary for later topics.

Addition is a binary operation $+ : \mathbb{R}^n \times \mathbb{R}^n \rightarrow \mathbb{R}^n$ where $[x_1, \ldots, x_n] + [y_1, \ldots, y_n] = [x_1 + y_1, \ldots, x_n + y_n]$. Addition is associative and commutative, every element has an inverse, and the $\textbf{0}$ element defined as $\textbf{0} = [0, \ldots, 0]$ acts as an additive identity. Unlike for $\mathbb{R}$, however, there is no need to define these facts to be true, as they can be proven directly from the definition.

Scalar Multiplication is the operation $\cdot : \mathbb{R} \times \mathbb{R}^n \rightarrow \mathbb{R}^n$ where $c[x_1, \ldots, x_n] = [cx_1, \ldots, cx_n]$. Scalar multiplication is not associative or commutative, but $1 \in \mathbb{R}$ still acts as an identity operator.

The dot product of two vectors is the binary operation $\cdot : \mathbb{R}^n \times \mathbb{R}^n \rightarrow \mathbb{R}$ where $\textbf{x} \cdot \textbf{y} = \sum\limits_{i=1}^n x_iy_i$.

$p$-Norms

We would like to be able to measure the distance an element of $\mathbb{R}^n$ is from the origin. The calculation for $\mathbb{R}$ is simple: if $x > 0$, then the distance is $x$, and if $x < 0$, then the distance is $-x$. In other words, the distance is identically the absolute value of $x$. This distance is also called the magnitude of $x$.

When we consider the distance of a vector $\textbf{x} \in \mathbb{R}^n$ from $0$, we have more freedom in how we measure it. One possible method is to sum the absolute values of each of the components. Another is to use the Pythagorean theorem to calculate the straight line distance. The method we will describe here is a generalization of both of these approaches called the $p$-norm.

Define the $p$-norm of a vector $\mathbf{x} \in \mathbb{R}^n$ as the function $\|\cdot\|_p : \mathbb{R}^n \rightarrow \mathbb{R}$ where

$$\|\textbf{x}\|_p = \left(\sum\limits_{i=1}^n |x_i|^p \right)^{\frac{1}{p}}$$

Observe that the $1$-norm is the sum of the absolute value of the components mentioned above. The $1$-norm is also called the Manhattan norm or the taxicab norm, named after the borough of New York City that is laid out (mostly) as a grid, where the actual distance traveled between two locations is not the straight line between them, since nobody can walk through or jump over buildings, but the number of axis-aligned blocks between them.

Next, note that the $2$-norm is the Pythagorean distance of a point from the origin. The $2$-norm is also called the Euclidean norm, since it measures the distance between two points along a straight line in Euclidean space. (Alas, it doesn't seem that Pythagoras retained naming credit on this one.) Note that $\|\textbf{x}\|_2^2 = \textbf{x} \cdot \textbf{x}$.

There is a special norm, called the maximum norm, which is defined as $\|\textbf{x}\|_{\infty} = \text{max}(x_1, \ldots, x_n)$. In other words, the maximum norm returns the absolute value of the vector component with the greatest magnitude. Depending on the context, the $p$-norm of $\textbf{x}$ can also be called its magnitude.

Problems

1. Associativity of Vector Addition: Show that $(\textbf{x} + \textbf{y}) + \textbf{z} = \textbf{x} + (\textbf{y} + \textbf{z})$ for all $\textbf{x}, \textbf{y}, \textbf{z} \in \mathbb{R}^n$.

$(\textbf{x} + \textbf{y}) + \textbf{z} = ([x_1, \ldots, x_n] + [y_1, \ldots, y_n]) + [z_1, \ldots, z_n] \\ (\textbf{x} + \textbf{y}) + \textbf{z} = [x_1 + y_1, \ldots, x_n + y_n] + [z_1, \ldots, z_n] \\ (\textbf{x} + \textbf{y}) + \textbf{z} = [(x_1 + y_1) + z_1, \ldots, (x_n + y_n) + z_n] \\ (\textbf{x} + \textbf{y}) + \textbf{z} = [x_1 + (y_1 + z_1), \ldots, x_n + (y_n + z_n)] \\ (\textbf{x} + \textbf{y}) + \textbf{z} = [x_1, \ldots, x_n] + [y_1 + z_1, \ldots, y_n + z_n] \\ (\textbf{x} + \textbf{y}) + \textbf{z} = [x_1, \ldots, x_n] + ([y_1, \ldots, y_n] + [z_1, \ldots, z_n]) \\ (\textbf{x} + \textbf{y}) + \textbf{z} = \textbf{x} + (\textbf{y} + \textbf{z}) \\$

2. Commutativity of Vector Addition: Show that $\textbf{x} + \textbf{y} = \textbf{y} + \textbf{x}$ for all $\textbf{x}, \textbf{y} \in \mathbb{R}^n$.

$\textbf{x} + \textbf{y} = [x_1, \ldots, x_n] + [y_1, \ldots, y_n] \\ \textbf{x} + \textbf{y} = [x_1 + y_1, \ldots, x_n + y_n] \\ \textbf{x} + \textbf{y} = [y_1 + x_1, \ldots, y_n + x_n] \\ \textbf{x} + \textbf{y} = [y_1, \ldots, y_n] + [x_1, \ldots, x_n] \\ \textbf{x} + \textbf{y} = \textbf{y} + \textbf{x} \\$

3. Additive Identity for Vector Addition: Show that $\textbf{x} + \textbf{0} = \textbf{x}$ for all $\textbf{x} \in \mathbb{R}^n$.

$\textbf{x} + \textbf{0} = [x_1, \ldots, x_n] + [0, \ldots, 0] \\ \textbf{x} + \textbf{0} = [x_1 + 0, \ldots, x_n + 0] \\ \textbf{x} + \textbf{0} = [x_1, \ldots, x_n] \\ \textbf{x} + \textbf{0} = \textbf{x}$

4. Existence of Additive Inverses for Vector Addition: For every $\textbf{x} \in \mathbb{R}^n$, show that there exists a vector, call it $-\textbf{x}$, such that $\textbf{x} + (-\textbf{x}) = \textbf{0}$.

Let $-\textbf{x} = [-x_1, \ldots, -x_n]$. Clearly $-\textbf{x} \in \mathbb{R}^n$. Likewise, $\textbf{x} + (-\textbf{x}) = [x_1 - x_1, \ldots, x_n - x_n] = [0, \ldots, 0] \textbf{0}$.

5. Scalar Multiplicative Identity: Show that $1\textbf{x} = \textbf{x}$ for all $\textbf{x} \in \mathbb{R}^n$.

$1\textbf{x} = 1[x_1, \ldots, x_n] \\ 1\textbf{x} = [1x_1, \ldots, 1x_n] \\ 1\textbf{x} = [x_1, \ldots, x_n] \\ 1\textbf{x} = \textbf{x}$

6. Distributive Law (Part 1): Show that $c(\textbf{x} + \textbf{y}) = c\textbf{x} + c\textbf{y}$.

$c(\textbf{x} + \textbf{y}) = c([x_1, \ldots, x_n] + [y_1, \ldots, y_n]) \\ c(\textbf{x} + \textbf{y}) = c[x_1 + y_1, \ldots, x_n + y_n] \\ c(\textbf{x} + \textbf{y}) = [c(x_1 + y_1), \ldots, c(x_n + y_n)] \\ c(\textbf{x} + \textbf{y}) = [cx_1 + cy_1, \ldots, cx_n + cy_n] \\ c(\textbf{x} + \textbf{y}) = [cx_1, \ldots, cx_n] + [cy_1, \ldots, cy_n] \\ c(\textbf{x} + \textbf{y}) = c[x_1, \ldots, x_n] + c[y_1, \ldots, y_n] \\ c(\textbf{x} + \textbf{y}) = c\textbf{x} + c\textbf{y}$

7. Distributive Law (Part 2): Show that $(a + b)\textbf{x} = a\textbf{x} + b\textbf{x}$.

$(a + b)\textbf{x} = (a + b)[x_1, \ldots, x_n] \\ (a + b)\textbf{x} = [(a + b)x_1, \ldots, (a + b)x_n] \\ (a + b)\textbf{x} = [ax_1 + bx_1, \ldots, ax_n + bx_n] \\ (a + b)\textbf{x} = [ax_1, \ldots, ax_n] + [bx_1, \ldots, bx_n] \\ (a + b)\textbf{x} = a[x_1, \ldots, x_n] + b[x_1, \ldots, x_n] \\ (a + b)\textbf{x} = a\textbf{x} + b\textbf{x} \\$

8. Show that $\|\mathbf{x}\|_p \geq 0$ for all $x \in \mathbb{R}^n$ and all positive $p \in \mathbb{N}$.

The absolute value of each component $|x_i|$ is non-negative by definition. According to the ordering properties on $\mathbb{R}$, the product of non-negative real numbers is non-negative, so $|x_i|^p$ is non-negative. The ordering properties also dictate that sums of non-negative numbers are also non-negative, so $\sum\limits_{i=1}^p |x_i|^p$ is non-negative. Finally, by the definition of rational exponents, we conclude that $\left( \sum\limits_{i=1}^p |x_i|^p \right)^{\frac{1}{p}}$ is non-negative.

9. Show that $\|\textbf{x}\|_p = 0$ if and only if $\textbf{x} = \textbf{0}$.

Assume $\textbf{x} = \textbf{0}$. Then

$\|\textbf{x}\|_p = \|\textbf{0}\|_p \\ \|\textbf{x}\|_p = \left( \sum\limits_{i=1}^{n} |0|^p \right)^{\frac{1}{p}} \\ \|\textbf{x}\|_p = \left( \sum\limits_{i=1}^{n} 0 \right)^{\frac{1}{p}} \\ \|\textbf{x}\|_p = 0^{\frac{1}{p}} \\ \|\textbf{x}\|_p = 0 \\$

Conversely, assume $\|\textbf{x}\|_p = 0$. Then

$0 = \left( \sum\limits_{i=1}^{n} |x_i|^p \right)^{\frac{1}{p}} \\ 0^p = \sum\limits_{i=1}^{n} |x_i|^p \\ 0 = \sum\limits_{i=1}^{n} |x_i|^p \\$

Proof by contradiction. Assume that $\textbf{x} \neq \textbf{0}$. Then there is some $x_i \neq 0$. Without loss of generality, let $i=1$. Then $|x_1| > 0$, and $-|x_1|^p = \sum\limits_{i=2}^n |x_i|^p$. But the absolute value of a real number is non-negative, and a product of $p$ non-negative numbers is non-negative, as is a sum of $n-1$ non-negative numbers. Therefore $x_i$ could not have been nonzero, thus $\textbf{x} = \textbf{0}$ after all.

10. Show that $\|a\textbf{x}\|_p = |a|\|\textbf{x}\|_p$.

$\|a\textbf{x}\|_p = \left( \sum\limits_{i=1}^{n} |ax_i|^p \right)^{\frac{1}{p}} \\ \|a\textbf{x}\|_p = \left( \sum\limits_{i=1}^{n} |a|^p|x_i|^p \right)^{\frac{1}{p}} \\ \|a\textbf{x}\|_p = \left( |a|^p\sum\limits_{i=1}^{n} |x_i|^p \right)^{\frac{1}{p}} \\ \|a\textbf{x}\|_p = \left(|a|^p\right)^{\frac{1}{p}}\left( \sum\limits_{i=1}^{n} |x_i|^p \right)^{\frac{1}{p}} \\ \|a\textbf{x}\|_p = |a|\left( \sum\limits_{i=1}^{n} |x_i|^p \right)^{\frac{1}{p}} \\ \|a\textbf{x}\|_p = |a|\|\textbf{x}\|_p$

11. Consider $\textbf{x}, \textbf{y} \in \mathbb{R}^n$ where $n \geq 2$, and also consider $p, q \in \mathbb{N}^+$ where $p \neq q$. If $\|\textbf{x}\|_p < \|\textbf{y}\|_p$, does it follow that$\|\textbf{x}\|_q < \|\textbf{y}\|_q$?

No.

Let $\textbf{x} = [0, 2]$ and $\textbf{y} = [\sqrt{2}, \sqrt{2}]$. Then $\|\textbf{x}\|_1 = 2$ and $\|\textbf{y}\|_1 = 2\sqrt{2}$, but $\|\textbf{x}\|_2 = 2$ and $\|\textbf{y}\|_2 = 2$.

12. Commutativity of Dot Product: Show that $\textbf{x} \cdot \textbf{y} = \textbf{y} \cdot \textbf{x}$ for all $\textbf{x}, \textbf{y} \in \mathbb{R}^n$.

$\textbf{x} \cdot \textbf{y} = \sum\limits_{i=1}^n x_i y_i \\ \textbf{x} \cdot \textbf{y} = \sum\limits_{i=1}^n y_i x_i \\ \textbf{x} \cdot \textbf{y} = \textbf{y} \cdot \textbf{x}$

13. Distributivity of Dot Product over Addition: Show that

1. $\textbf{x} \cdot (\textbf{y} + \textbf{z}) = (\textbf{x} \cdot \textbf{y}) + (\textbf{x} \cdot \textbf{z})$

2. $(\textbf{x} + \textbf{y}) \cdot \textbf{z} = (\textbf{x} \cdot \textbf{y}) + (\textbf{x} \cdot \textbf{z})$

1. $\textbf{x} \cdot (\textbf{y} + \textbf{z}) = \sum\limits_{i=1}^n x_i(y_i + z_i) \\ \textbf{x} \cdot (\textbf{y} + \textbf{z}) = \sum\limits_{i=1}^n x_iy_i + x_iz_i \\ \textbf{x} \cdot (\textbf{y} + \textbf{z}) = \sum\limits_{i=1}^n x_iy_i + \sum\limits_{i=1}^n x_iz_i \\ \textbf{x} \cdot (\textbf{y} + \textbf{z}) = (\textbf{x} \cdot \textbf{y}) + (\textbf{x} \cdot \textbf{z})$

2. $(\textbf{x} + \textbf{y}) \cdot \textbf{z} = \sum\limits_{i=1}^n (x_i + y_i)z_i \\ (\textbf{x} + \textbf{y}) \cdot \textbf{z} = \sum\limits_{i=1}^n x_iz_i + y_iz_i \\ (\textbf{x} + \textbf{y}) \cdot \textbf{z} = \sum\limits_{i=1}^n x_iz_i + \sum\limits_{i=1}^n y_iz_i \\ (\textbf{x} + \textbf{y}) \cdot \textbf{z} = (\textbf{x} \cdot \textbf{z}) + (\textbf{y} \cdot \textbf{z}) \\$

14. Show that $\textbf{x} \cdot \textbf{x} \geq 0$ for all $x \in \mathbb{R}^n$.

$\textbf{x} \cdot \textbf{x} = \sum\limits_{i=1}^n x_i x_i = \sum\limits_{i=1}^n x_i^2$. The square of any number is non-negative, and the sum of non-negative numbers is non-negative, so the result follows.

15. Cauchy-Schwarz Inequality: Show that $|\textbf{x} \cdot \textbf{y}| \leq \|\textbf{x}\|_2\|\textbf{y}\|_2$.

Hint: Consider $\|\textbf{y}\|_2\textbf{x}$ and $\|\textbf{x}\|_2\textbf{y}^2$.

Consider each vector as scaled by the squared length of the other: $\|\textbf{y}\|_2\textbf{x}$ and $\|\textbf{x}\|_2\textbf{y}$.

$0 \leq \left(\|\textbf{y}\|_2\textbf{x} - \|\textbf{x}\|_2\textbf{y}\right) \cdot \left(\|\textbf{y}\|_2\textbf{x} - \|\textbf{x}\|_2\textbf{y}\right) \\ 0 \leq \|\textbf{y}\|_2^2(\textbf{x} \cdot \textbf{x}) - 2\|\textbf{x}\|_2\|\textbf{y}\|_2(\textbf{x} \cdot \textbf{y}) + \|\textbf{x}\|_2^2(\textbf{y} \cdot \textbf{y}) \\ 0 \leq 2\|\textbf{x}\|_2^2\|\textbf{y}\|_2^2 - 2\|\textbf{x}\|_2\|\textbf{y}\|_2(\textbf{x} \cdot \textbf{y}) \\ 2\|\textbf{x}\|_2\|\textbf{y}\|_2(\textbf{x} \cdot \textbf{y}) \leq 2\|\textbf{x}\|_2^2\|\textbf{y}\|_2^2 \\ \textbf{x} \cdot \textbf{y} \leq \|\textbf{x}\|_2\|\textbf{y}\|_2$

Likewise,

$0 \leq \left(\|\textbf{y}\|_2\textbf{x} + \|\textbf{x}\|_2\textbf{y}\right) \cdot \left(\|\textbf{y}\|_2\textbf{x} + \|\textbf{x}\|_2\textbf{y}\right) \\ 0 \leq \|\textbf{y}\|_2^2(\textbf{x} \cdot \textbf{x}) + 2\|\textbf{x}\|_2\|\textbf{y}\|_2(\textbf{x} \cdot \textbf{y}) + \|\textbf{x}\|_2^2(\textbf{y} \cdot \textbf{y}) \\ 0 \leq 2\|\textbf{x}\|_2^2\|\textbf{y}\|_2^2 + 2\|\textbf{x}\|_2\|\textbf{y}\|_2(\textbf{x} \cdot \textbf{y}) \\ -2\|\textbf{x}\|_2\|\textbf{y}\|_2(\textbf{x} \cdot \textbf{y}) \leq 2\|\textbf{x}\|_2^2\|\textbf{y}\|_2^2 \\ -\textbf{x} \cdot \textbf{y} \leq \|\textbf{x}\|_2\|\textbf{y}\|_2$

Therefore $|\textbf{x} \cdot \textbf{y}| \leq \|\textbf{x}\|_2\|\textbf{y}\|_2$