Real Analysis: Derivatives
Derivatives
Motivation: Quantifying "Steepness"
How quickly is a function increasing or decreasing at a given point? This is the question the notion of a derivative seeks to answer.
The steepness of a real function $f$ at a point $x$ may be approximated by constructing the secant line between $x$ and another nearby point $x_0.$ The formula for the slope of a line is $\frac{\text{rise}}{\text{run}},$ and thus the slope of the secant line between $x$ and $x_0$ is
$$\dfrac{f(x_0) - f(x)}{x_0 - x}.$$
The closer $x_0$ gets to $x,$ the more accurate the approximation. The limit of this expression as $x_0$ approaches $x$ is called the derivative of $f.$
The Derivative of a Real Function
Let $f$ be a real function. The derivative of $f$ at a point $x$ is given by the following limit, if it exists:
$$f'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0) - f(x)}{x_0 - x}$$
A function $f$ is differentiable at $x$ if its derivative is defined at $x,$ and is simply differentiable if its derivative is defined for all values in its domain. If a $f'$ is either $0$ or undefined at $x,$ then $x$ is a critical point of $f.$ Furthermore, $f$ is increasing at $x$ if its derivative is positive at $x$ and decreasing if its derivative is negative at $x.$
Leibniz Notation
When functions are written as variables (for example $y = f(x)$), the notation $\dfrac{d}{dx}$ is used instead of the tick mark $'$ to signify the derivative of expression.
$$\dfrac{dy}{dx} = \lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0) - f(x)}{x_0 - x}$$
The notation is derived from the quotient expression inside the limit, where rise over run can be expressed as $\dfrac{\Delta y}{\Delta x},$ where the capital Greek letter delta, $\Delta,$ is shorthand for a difference involving the variable (such as $x_0 - x$). The letter $d$ is used to signify that the difference has been taken to zero by the limit.
The top $d$ means to take the derivative of the following expression, and the bottom $d$ signifies with respect to which variable the derivative should be taken. Thus $\dfrac{dy}{dx}$ means to take the derivative of $y$ with respect to $x.$ This notation becomes more useful when analyzing functions of several variables. The $\dfrac{d}{dx}$ notation is also more readable than the tick mark when evaluating derivatives algebraically, such as in the following example:
$$\dfrac{d}{dx}(3x^3 - x^2) = \dfrac{d}{dx}3x^3 - \dfrac{d}{dx}x^2 = 9x^2 - 2x.$$
Higher Order Derivatives
Consider a function $f$ that has a derivative $f'.$ If $f'$ is itself differentiable, its derivative is written $f''$ and is called the second derivative of $f.$ This process can be performed once again to generate the third derivative, $f'''.$ Such derivatives of derivatives are called higher order derivatives.
Tick marks are usually used to denote derivatives, as the first and second are the most commonly encountered. However, for derivatives beyond the first couple, alternate notation of superscripts wrapped in parentheses is used. For example, the fourth derivative of $f$ is written as $f^{(4)}.$
Leibniz notation, however, has every advantage in clarity over tick mark notation. For one, Leibniz notation is always explicit - second order derivatives and above are always written with the order of the derivative explicitly:
$$\dfrac{d^2f}{dx^2} = \dfrac{d}{dx}\left(\dfrac{df}{dx}\right).$$
Leibniz notation is also easier to read - while tick marks can visually blur together into the symbol of the function they are annotation, especially when written by hand, the fractions with explicit numbers attached are much harder to confuse. And of course, Leibniz notation is much more impressive to look at.
Differentiability Implies Continuity
If a function is differentiable at $x,$ it is necessarily continuous at $x.$ This fact is key for proving the following algebraic properties of the derivative.
It must be noted that the converse of this implication is not true - continuity at $x$ does not imply differentiability at $x.$ For example, the function $f(x) = |-x^2 + 1|$ is continuous as $-1$ and $1$ but is not differentiable at those points.
Algebraic Properties of Derivatives of Real Functions
The algebraic properties of derivatives of real functions are known as rules, especially within the context of calculus. Recall that the focus of calculus is on using properties of limits, derivatives, and integrals to compute specific values, while the focus of analysis is to prove those properties in the first place.
For real functions $f$ and $g$, the following properties, or rules, hold:
-
Scalar multiplication: $(cf)'(x) = cf'(x).$
-
Sum: $(f+g)'(x) = f'(x) + g'(x).$
-
Difference: $(f - g)'(x) = f'(x) - g'(x).$
-
Product: $(fg)'(x) = f'(x)g(x) + f(x)g'(x)$
-
Quotient: $\left(\dfrac{f}{g}\right)'(x) = \dfrac{g(x)f'(x) - f(x)g'(x)}{g^2(x)}$ assuming $g(x) \neq 0.$
The first two properties regarding scalar multiplication and summation are collectively called the linearity of differentiation.
Derivatives of Composite Functions - The Chain Rule
Let $f$ and $g$ be real functions. The chain rule states the following:
$$(f \circ g)'(x) = f'(g(x))g'(x).$$
The chain rule gets its name from the fact that when several real functions are composited together, the outermost function is differentiated and then multiplied by the derivative of the next outermost function, and so on, forming a chain of derivatives. For example,
$$(f \circ (g \circ h))'(x) = f('g(h(x))g'(h(x))h'(x).$$
Problems
Equivalent definition: Given a real function $f$, show that $f'(x) = \lim\limits_{h \rightarrow 0} \dfrac{f(x + h) - f(x)}{h}.$
Let $x_0 = x + h.$ Then $f(x + h) = f(x_0).$ Likewise, $x_0 - x = (x + h) - x = h.$ Therefore
$\dfrac{f(x + h) - f(x)}{h} = \dfrac{f(x_0) - f(x)}{x_0 - x}.$
Likewise, if for every $\varepsilon > 0$ there exists a $\delta$ such that $\left|\dfrac{f(x_0) - f(x)}{h} - f'(x)\right| < \varepsilon$ whenever $|x_0 - x| < \delta,$ it follows that $|h| < \delta,$ and so $\left|\dfrac{f(x+h) - f(x)}{h} - f'(x)\right| < \varepsilon.$ Therefore $f'(x) = \lim\limits_{h \rightarrow 0} \dfrac{f(x+h) - f(x)}{h}.$
Differentiability implies continuity: Given a real function $f,$ show that if $f'(x)$ exists, then $f$ is continuous at $x$.
Assume $f'(x)$ exists. Then $\lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0) - f(x)}{x_0 - x}$ is well defined. By the algebraic limit theorem,
$ \lim\limits_{x_0 \rightarrow x} f(x_0) - f(x) = \lim\limits_{x_0 \rightarrow x} \left(f(x_0) - f(x)\right)\left(\dfrac{x_0 - x}{x_0 - x}\right) \\ \lim\limits_{x_0 \rightarrow x} f(x_0) - f(x) = \left(\lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0) - f(x)}{x_0 - x}\right)\left(\lim\limits_{x_0 \rightarrow x} x_0 - x \right) \\ \lim\limits_{x_0 \rightarrow x} f(x_0) - f(x) = f'(x) \cdot 0 \\ \lim\limits_{x_0 \rightarrow x} f(x_0) - f(x) = 0.$
Therefore $\lim\limits_{x_0 \rightarrow x} f(x_0) = f(x),$ and so $f$ is continuous at $x.$
Scalar multiplication: Show that if $f$ is a real function and $c$ is a real number, then $(cf)'(x) = cf'(x).$
The property is a consequence of the scalar multiplication limit law:
$ (cf)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{cf(x_0) - cf(x)}{x_0 - x} \\ (cf)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{c(f(x_0) - f(x))}{x_0 - x} \\ (cf)'(x) = c\lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0) - f(x)}{x_0 - x} \\ (cf)'(x) = cf'(x) $
Sum Rule: Let $f$ and $g$ be real functions. Show that $(f + g)'(x) = f'(x) + g'(x).$
$ (f + g)'(x) = \lim\limits_{x \rightarrow 0} \dfrac{(f(x_0) + g(x_0)) - (f(x) + g(x))}{x_0 - x} \\ (f + g)'(x) = \lim\limits_{x \rightarrow 0} \dfrac{(f(x_0) - f(x)) + (g(x_0) - g(x))}{x_0 - x} \\ (f + g)'(x) = \lim\limits_{x \rightarrow 0} \dfrac{f(x_0) - f(x)}{x_0 - x} + \lim\limits_{x \rightarrow 0} \dfrac{g(x_0) - g(x)}{x_0 - x} \\ (f + g)'(x) = f'(x) + g'(x) $
Difference Rule: Let $f$ and $g$ be real functions. Show that $(f - g)'(x) = f'(x) - g'(x).$
$ (f - g)'(x) = \lim\limits_{x \rightarrow 0} \dfrac{(f(x_0) - g(x_0)) - (f(x) - g(x))}{x_0 - x} \\ (f - g)'(x) = \lim\limits_{x \rightarrow 0} \dfrac{(f(x_0) - f(x)) - (g(x_0) - g(x))}{x_0 - x} \\ (f - g)'(x) = \lim\limits_{x \rightarrow 0} \dfrac{f(x_0) - f(x)}{x_0 - x} - \lim\limits_{x \rightarrow 0} \dfrac{g(x_0) - g(x)}{x_0 - x} \\ (f - g)'(x) = f'(x) - g'(x) $
Product Rule: Let $f$ and $g$ be real functions. Show that $(fg)'(x) = f'(x)g(x) + f(x)g'(x)$
First, expand the definition.
$ (fg)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{(fg)(x_0) - (fg)(x)}{x_0 - x} \\ (fg)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0)g(x_0) - f(x)g(x)}{x_0 - x} \\ $
Next, add and subtract an $f(x)g(x_0)$ term. This lets us group the terms into the correct differences for the limit definition of a derivative, multiplied by an additional $g(x_0)$ and $f(x)$ factor:
$ (fg)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0)g(x_0) - f(x)g(x_0) + f(x)g(x_0) - f(x)g(x)}{x_0 - x} \\ (fg)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{g(x_0)(f(x_0) - f(x)) + f(x)(g(x_0) - g(x))}{x_0 - x} \\ $
Apply the limit laws to isolate the terms:
$ (fg)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{g(x_0)(f(x_0) - f(x))}{x_0 - x} + \lim\limits_{x_0 \rightarrow x} \dfrac{f(x)(g(x_0) - g(x))}{x_0 - x} \\ (fg)'(x) = \lim\limits_{x_0 \rightarrow x}g(x_0)\lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0) - f(x)}{x_0 - x} + \lim\limits_{x_0 \rightarrow x}f(x)\lim\limits_{x_0 \rightarrow x} \dfrac{g(x_0) - g(x)}{x_0 - 0} \\ $
Simplify:
$ (fg)'(x) = f'(x)g(x) + f(x)g'(x) $
Quotient Rule: Let $f$ and $g$ be real functions. Show that $\left(\dfrac{f}{g}\right)'(x) = \dfrac{g(x)f'(x) - f(x)g'(x)}{g^2(x)}.$
First, expand the definition:
$ \left(\dfrac{f}{g}\right)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{\frac{f(x_0)}{g(x_0)} - \frac{f(x)}{g(x)}}{x_0 - x} $
Multiply the terms in the numerator appropriately to get a common denominator:
$ \left(\dfrac{f}{g}\right)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{\left(\frac{f(x_0)}{g(x_0)}\right)\left(\frac{g(x)}{g(x)}\right) - \left(\frac{f(x)}{g(x)}\right)\left(\frac{g(x_0)}{g(x_0)}\right)}{x_0 - x} \\ \left(\dfrac{f}{g}\right)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{\frac{f(x_0)g(x)}{g(x_0)g(x)} - \frac{f(x)g(x_0)}{g(x_0)g(x)}}{x_0 - x} $
Factor out the denominator:
$ \left(\dfrac{f}{g}\right)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{1}{g(x_0)g(x)} \dfrac{f(x_0)g(x) - f(x)g(x_0)}{x_0 - x} $
Add and subtract $f(x)g(x)$, then isolate the expressions for the derivatives of $f$ and $g:$
$ \left(\dfrac{f}{g}\right)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{1}{g(x_0)g(x)} \dfrac{f(x_0)g(x) - f(x)g(x) + f(x)g(x) - f(x)g(x_0)}{x_0 - x} \\ \left(\dfrac{f}{g}\right)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{1}{g(x_0)g(x)} \dfrac{g(x)(f(x_0) - f(x)) + f(x)(g(x) - g(x_0))}{x_0 - x} \\ \left(\dfrac{f}{g}\right)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{1}{g(x_0)g(x)} \dfrac{g(x)(f(x_0) - f(x)) - f(x)(g(x_0) - g(x))}{x_0 - x} \\ \left(\dfrac{f}{g}\right)'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{1}{g(x_0)g(x)} \left( g(x)\dfrac{f(x_0) - f(x)}{x_0 - x} - f(x)\dfrac{g(x_0) - g(x)}{x_0 - x}\right) \\ $
Now apply the algebraic limit theorem:
$ \left(\dfrac{f}{g}\right)'(x) = \left(\lim\limits_{x_0 \rightarrow x} \dfrac{1}{g(x_0)g(x)}\right)\left(\left(\lim\limits_{x_0 \rightarrow x}g(x)\right) \left(\lim\limits_{x_0 \rightarrow x}\dfrac{f(x_0) - f(x)}{x_0 - x}\right) - \left(\lim\limits_{x_0 \rightarrow x}f(x)\right)\left(\lim\limits_{x_0 \rightarrow x}\dfrac{g(x_0) - g(x)}{x_0 - x}\right)\right) \\ \left(\dfrac{f}{g}\right)'(x) = \left(\dfrac{1}{g^2(x)}\right)\left(g(x)f'(x) - f(x)g'(x)\right) \\ \left(\dfrac{f}{g}\right)'(x) = \dfrac{g(x)f'(x) - f(x)g'(x)}{g^2(x)} $
Let $f$ be a real function where $f(x) = c$ for some constant $c \in \mathbb{R}.$ Show that $f'(x) = 0.$
$ f'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0) - f(x)}{x_0 - x} \\ f'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{c - c}{x_0 - x} \\ f'(x) = \lim\limits_{x_0 \rightarrow x} 0 \\ f'(x) = 0 \\ $
Let $f$ be a real function where $f(x) = x.$ Show that $f'(x) = 1.$
$ f'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{f(x_0) - f(x)}{x_0 - x} \\ f'(x) = \lim\limits_{x_0 \rightarrow x} \dfrac{x_0 - x}{x_0 - x} \\ f'(x) = 1$
The last equality follows from the limit law for algebraically removable holes.
Let $f$ be a real monomial where $f(x) = x^n.$ Show that $f'(x) = nx^{n-1}.$
Proof by induction.
Base case: Let $n = 0.$ Then $f(x) = 1x^0,$ and as shown above, $f'(x) = 0.$
Inductive step: Assume that if $f(x) = x^n$ then $f'(x) = nx^{n-1}.$ Let $g(x) = x^{n+1}.$ Then $g(x) = x \cdot x^n.$ By the product rule, $g'(x) = 1 \cdot x^n + x \cdot nx^{n-1}.$ This simplifies to $g'(x) = x^n + nx^n = (n+1)x^n.$ The result follows by induction.
Let $f$ be a real polynomial such that $f(x) = a_0 + a_1x + a_2x^2 + \ldots + a_nx^n.$ Show that $f'(x) = a_1 + 2a_2x + \ldots + na_nx^{n-1}.$
$ f'(x) = \dfrac{d}{dx}\left(a_0 + a_1x + a_2x^2 + \ldots + a_nx^n\right) \\ f'(x) = \dfrac{d}{dx}(a_0) + \dfrac{d}{dx}\left(a_1x\right) + \dfrac{d}{dx}\left(a_2x^2\right) + \ldots + \dfrac{d}{dx}\left(a_nx^n\right) \\ f'(x) = 0 + a_1 + a_2\dfrac{d}{dx}\left(x^2\right) + \ldots + a_n\dfrac{d}{dx}\left(x^n\right) \\ f'(x) = a_1 + a_2\dfrac{d}{dx}\left(x^2\right) + \ldots + a_n\dfrac{d}{dx}\left(x^n\right) \\ f'(x) = a_1 + a_2(2x) + \ldots + a_n\left(nx^{n-1}\right) \\ f'(x) = a_1 + 2a_2x + \ldots + na_nx^{n-1}$
The second line follows by the sum rule. The third line follows by the derivative of a constant and linearity of differentiation. The fourth is algebraic simplification. The fifth line follows from the derivative of monomials. The last line is more algebraic simplification.
Let $f(x) = x^{-n}$ where $n \in \mathbb{N}^+.$ Show that $f'(x) = -nx^{-n-1}.$
Apply the quotient rule:
$ f'(x) = \dfrac{d}{dx}x^{-n} \\ f'(x) = \dfrac{d}{dx}\dfrac{1}{x^n} \\ f'(x) = \dfrac{x^n \cdot (\frac{d}{dx}1) - 1 \cdot (\frac{d}{dx}x^n)}{(x^n)^2} \\ f'(x) = \dfrac{x^n \cdot 0 - 1 \cdot nx^{n-1}}{x^{2n}} \\ f'(x) = \dfrac{-nx^{n-1}}{x^{2n}} \\ f'(x) = -nx^{-n-1} $
Let $f(x) = x^{\frac{1}{b}}$ where $b \in \mathbb{Z}$ and $b \neq 0.$ Show that $f'(x) = \dfrac{1}{b}x^{\frac{1}{b} - 1}.$
$ f(x) = x^{\frac{1}{b}} \\ (f(x))^b = x \\ \dfrac{d}{dx}\left(f(x)\right)^b = \dfrac{d}{dx} x \\ bf(x)^{b-1}\dfrac{d}{dx}f(x) = 1 \\ bf(x)^{b-1}f'(x) = 1 \\ f(x) = \dfrac{1}{bf(x)^{b-1}} \\ f'(x) = \dfrac{1}{b\left(x^{\frac{1}{b}}\right)^{b-1}} \\ f'(x) = \dfrac{1}{bx^{\frac{b-1}{b}}} \\ f'(x) = \dfrac{1}{bx^{1-\frac{1}{b}}} \\ f'(x) = \dfrac{1}{b}x^{\frac{1}{b}-1} $
Let $f(x) = x^{\frac{a}{b}}$ where $a, b \in \mathbb{Z}$ and $b \neq 0.$ Show that $f'(x) = \dfrac{a}{b}x^{\frac{a}{b} - 1}.$
Apply the chain rule:
$ f'(x) = \dfrac{d}{dx}x^{\frac{a}{b}} \\ f'(x) = \dfrac{d}{dx}\left(x^{\frac{1}{b}}\right)^a \\ f'(x) = a\left(x^{\frac{1}{b}}\right)^{a-1}\dfrac{d}{dx}x^{\frac{1}{b}} \\ f'(x) = a\left(x^{\frac{1}{b}}\right)^{a-1}\dfrac{1}{b}x^{\frac{1}{b}-1} \\ f'(x) = \dfrac{a}{b}x^{\frac{a-1}{b}}x^{\frac{1}{b}-1} \\ f'(x) = \dfrac{a}{b}x^{\frac{a}{b} - 1} $