Mathematical Analysis
Metric Space
Def: A metric space is a pair $(M,d)$ where M is a set, and $ d:M^2 \to \mathbb{R} $ which s.t.
- $ d(x,y) > 0 $
- $ d(x,y) = d(y,x) $
- $ d(a,b) \le d(a,c) + d(c,b) $
Metric space is often represented by $M$, and is a pair of set and distance function.
Sequence
Def: A sequence in $M$ is a list of elements of $M$ indexed by $\mathbb{N}$, $(a_i) i\in \mathbb{N}$ or $(a_i)$.
Convergence Sequence
Def: A seq $(a_i) i\in \mathbb{N}$ in $M$ converges to $b \in M$, if $\forall \epsilon > 0$, $\exists N \in \mathbb{N}$, $\forall j > N$, $(d(a_j, b) < \epsilon)$.
Notation: $(a_i) \to b$ means $(a_i)$ converges to b.
Every sequence has at most one limit.
Thm: If $(a_i) \to b$, and $(c_i)$ is a subsequence of $(a_i)$, then $(c_i) \to b$.
Continuous Function
Def: For $M$, $N$ metric spaces, a function $f: M \to N$ is continuous if whenever $(a_i)$ is a sequence in $M$ and $(a_i) \to b$, then $(f(a_i)) \to f(b)$ in $N$.
Hemeomorphic
Def: For $M,N$ metric spaces, $M \cong N$ ($M$ is homeomorphic to $N$) if there’s a function $f: M \to N$, so $f$ is bijection and f is continuous, and $f^{-1}$ is continuous. (bijection, bi-continuous)
Closed and Open Set
Open Balls
Def: $B_r^M(p)$ = { $q \in M $ | $d(p,q) < r$ }.
Open Set
Def: A set $S \subset M$ is open in $M$ if $\forall p \in S$, $\exists r > 0$, $B_r^M(p) \le S$.
From intuition, open set has no point on edge.
Limit Point
Def: For $S \subset M$, $p \in M$. p is a limit point of $S$, if there is a sequence $(a_i)$, so $(a_i) \to p$ and $(a_i) \in S$.
Limit Point of Set
Def: $limit^M(S)$ = { $q \in M$ | $q$ is a limit point of $S$ }.
Closed Set
Def: $S \subset M$ is closed in $M$, if it contains all of its limit points in $M$.
Or we can say
Def: $S \subset M$ is closed in $M$ if $lim^M(S) = S$.
From intuition, a set is closed if it can’t “limit out of itself”.
- Any interescetion of closed set is closed.
- Any union of finitely many closed sets is closed.
Continuish
Different definition of continuous function. Only in this section, we use continuish function to distinguish.
A function $f: M \to N$ is continuish if $\forall x \in M$, $\forall \epsilon > 0$, $\exists \delta > 0$, $\forall y \in M$, $d(x,y) < \delta \Longrightarrow d(f(x), f(y)) < \epsilon$.
Continuous function equals to continuish function.
Inheritence Principle
Suppose $S \subset N \subset M$, $S$ is closed/open in $N$ $\Longleftrightarrow$ there is a set $L \subset M$ which is closed/open in $M$ and $S=L \bigcap N$.
Product Space
Given metric spaces $(X, d_X), (Y, d_Y)$, build a new metric space $(X \times Y, d_?)$.
Lemma: $d_{max} \le d_E \le d_{sum} \le 2 \cdot d_{max}$
This lemma tells us that as long as these $d$ function can be bounded by each other, they are “quivalent” in this case, even including the exponential function. But there are exceptions, like discrete function/0-1 loss in ML field.
Completeness
Cauchy Sequence
Def: A sequence $(a_n)$ in $M$ is Cauchy if $\forall \epsilon$, $\exists N \in \mathbb{N}$, $\forall i > N$, $\forall j > N$, $(d(a_i, a_j) < \epsilon)$.
Complete Metric Space
Def: $M$ is complete if every Caucy seq in $M$ converges in $M$.
Discuss: limit, cauchy sequence, convergent sequence
Thm: If $N \subset M$, $M$ is complete, $N$ is closed in $M$, then $N$ is complete.
Compact
Def: $A \subset M$ is compact, if for every sequence $(a_i)$ in $A$, there is a subsequence $(b_n)$ which converges in $A$.
Thm: If $A \subset M$ is compact, then $A$ is closed in $M$, and $A$ is bounded.
Compact is a property of sequence, not metric space.
Thm: Every bounded sequence in $\mathbb{R}^m$ has a convergent subsequence.
Thm: If $M$ is a compact space, and $A \subset M$ is closed, then $A$ is compact.
Thm: Every closed and bounded subset of $\mathbb{R}^m$ is compact.
This is the reason why calculus works. (?)
序列
序列收敛
序列收敛使用$\epsilon$-原则定义。
open set is defined throught sequence.