Network Systems Lab. Korea Advanced Institute of Science and Technology No.1 Appendix A. Mathematical Background EE692 Parallel and Distribution Computation | Prof. Song Chong

Network Systems Lab. Korea Advanced Institute of Science and Technology Topological Structure of Spaces (Sets)  Metric Spaces No.2

Network Systems Lab. Korea Advanced Institute of Science and Technology  Let X=C[0,T] be the set of all real-valued (or complex-valued) continuous functions defined on the interval [0, T] where T>0.  Define  Show that (C[0,T], ) and (C[0,T], ) are metric spaces. No.3

Network Systems Lab. Korea Advanced Institute of Science and Technology Algebraic Structure of Spaces (Sets)  Linear Spaces No.4

Network Systems Lab. Korea Advanced Institute of Science and Technology  Linear Spaces No.5

Network Systems Lab. Korea Advanced Institute of Science and Technology Topological & Algebraic Structure  Normed linear space No.6

Network Systems Lab. Korea Advanced Institute of Science and Technology  Normed linear space No.7

Network Systems Lab. Korea Advanced Institute of Science and Technology No.8 Vector Norms  Def. A.1 A norm on is a mapping that assigns a real number to every and that has the following properties :

Network Systems Lab. Korea Advanced Institute of Science and Technology No.9 Euclidean Norm (l 2 -norm)  The Euclidean norm is defined by and, equipped with this norm, is called a Euclidean space. inner product conjugate transpose

Network Systems Lab. Korea Advanced Institute of Science and Technology No.10

Network Systems Lab. Korea Advanced Institute of Science and Technology No.11  The Euclidean norm satisfies the Schwartz inequality  Prop. A.1 If x and y are orthogonal, then (Pythagorean Theorem)

Network Systems Lab. Korea Advanced Institute of Science and Technology No.12  Maximum norm (l ∞ -norm)  Weighted maximum norm  l 1 norm  Prop. A.2 For any, * Show that L 2 -, l ∞ -,l 1 -norms are indeed a norm, respectively.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.13 Sequences, Limits, and Continuity  Def. (Convergence, Limit, Cauchy)  A seq. {x k } of complex numbers is said to converge to a complex number x if for every ε>0, there exist some K such that  If a seq. converges to some x, we say that x is the limit of {x k }; symbolically  A seq. {x k } is said to converge geometrically to x if there exist constants A≥0 and such that

Network Systems Lab. Korea Advanced Institute of Science and Technology No.14  A seq. {x k } is called a Cauchy seq. if for every ε>0, there exist some K such that and m≥K.  A real seq. {x k } is said to be bounded above (below) if there exist some real number A such that  A complex seq. {x k } is said to be bounded if is bounded above.  A convergent sequence is a Cauchy sequence.  The converse is not generally true but if the space is complete, it is true.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.15  Prop. A.4 Let {x k } be a real seq. (a) (b) {x k } converges iff In that case, these quantities are equal to the limit of x k.  A seq. {x k } of vectors in C n is said to converge to some x ∈ C n if the i- th coordinate of x k converges to the i-th coordinate of x for every i.  Same for boundedness, Cauchy, converge geometrically

Network Systems Lab. Korea Advanced Institute of Science and Technology No.16  Def. A.2 (limit point) Some x ∈ C n is said to be a limit point of a seq. {x k } in C n if ∃ a subseq. of {x k } that converges to x. e.g.) {1, 1/2, 3, 1/4, 5, 1/6, … }, 0 is a limit point. Let A be a subset of. Some x ∈ C n is said to be a limit point of A if ∃ a seq. {x k }, consisting of elements of A, that converges to x.  Prop. A.5 (a) A bounded sequence of vectors in C n converges iff it has a unique limit point. (b) A seq. in C n converges iff it is a Cauchy seq. ( ∵ C n is complete.) (c) Every bounded sequence in C n has at least one limit point. e.g.) {1, 1/2, 1, 1/4, 1, 1/6, … }, limit points 1, 0. (d) If {x k } is a real seq. and is finite, then it is the largest (smallest) limit point of {x k }.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.17  Def. (Completeness) A metric space (X,d) is said to be complete if each Cauchy seq. in (X,d) is a convergent seq. in (X,d).  e.g.)  set of rational numbers  not a complete space. {3, 3.1, 3.14, 3.141, ,...} For n, m≥4,  C n, R n are complete.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.18  Def A.3 (Compactness, interior point)  A set A ⊂ C n is called closed if it contains all of its limit points. It is called open if C n -A is closed.  A set A ⊂ C n is called bounded if ∃ some c ∈ R such that the magnitude of any coordinate of any elements of A is less than c.  A Closed and bounded subset of C n is called compact.  Let be a vector norm on C n.  If A ⊂ C n and x ∈ A, we say that x is an interior point of A if ∃ some ε>0 such that e.g.) A=[0,1) Neither nor 1 is an interior point!

Network Systems Lab. Korea Advanced Institute of Science and Technology No.19  Prop. A.6 A subset of C n is open iff all of its elements are interior points. e.g.) (0,1) open, [0,1] not open, (0,1] not open  Def. A.5 (Continuity) (a) Let A be a subset of. A fct f:A  is said to be continuous at a point x ∈ A if f is said to be continuous on A if it is continuous at every point x ∈ A. (b) Let A be a subset of R. A fct f:A  is called right-continuous (left-continuous) at a point x ∈ A if

Network Systems Lab. Korea Advanced Institute of Science and Technology No.20  Prop. A.7 (a) The composition of two continuous function is continuous. (b) Any vector norm on C n is a continuous function. (c) Let f: C m  C n be continuous, and let A ⊂ C n be open (closed). Then the set {x ∈ C m │ f(x) ∈ A} is open (closed).  Prop. A.8 (Weierstrass’ Theorem) Let A be a nonempty compact subset of C n. If f: A  R is continuous, then ∃ x, y ∈ A such that and.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.21  Prop. A.9 For any two vector norms, then ∃ some positive constant such that  Prop. A.10 The set of interior points of a set A ⊂ C n does not depend on the choice of norm.  Prop. A.11 Let be an arbitrary vector norm on C n. A seq. of complex vectors converges to iff In particular, x k converges to x geometrically iff converges to zero geometrically,

Network Systems Lab. Korea Advanced Institute of Science and Technology No.22  Remark) Whenever one discusses the topological (or metrical) properties of a normed linear space, the metric (distance measure) is defined in terms of the given norm by  Def. A.5 Let A be a subset of C n. Let {f k } be a seq. of functions from A into C n. We say that f k converges pointwise to a function: A  C n if We say that f k converges to f uniformly if for every ε>0, ∃ some K such that for all k≥K and x ∈ A.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.23 Matrix Norms  Matrix Norms Given any vector norm, the corresponding induced matrix norm, also denoted by, is defined by * The maximum is always attained. ( ∵ Prop. A.7 (a)(b)(c), Prop. A.8)  Prop. A.12

Network Systems Lab. Korea Advanced Institute of Science and Technology No.24

Network Systems Lab. Korea Advanced Institute of Science and Technology No.25  Prop. A.13

Network Systems Lab. Korea Advanced Institute of Science and Technology No.26 Eigenvalues  Def. A.6 A square matrix A is called singular if its determinant is zero, otherwise, nonsingular (equivalently or invertible).  Prop. A.14 The following are equivalent: A is nonsingular. ⇔ A’ is nonsingular. ⇔ For every nonzero x ∈ C n, we have Ax≠0. ⇔ For every y ∈ C n, ∃ a unique x ∈ C n such that Ax=y. ⇔ ∃ B such that AB = I = BA. ⇔ The columns of A are linearly independent.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.27  Def. A.7 The characteristic polynomial of an nxn matrix A is defined by The n roots of are called the eigenvalues of A. A vector such that is called an eigenvector of A associated with. *Note that the eigenvalues and eigenvectors of A could be complex even if A is real.  Prop. A.15 A square matrix A is singular iff it has an eigenvalue that is equal to zero.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.28 Jordan Block/Matrix  Def. A.8 (Jordan matrix) (a) A mxm matrix J is said to be Jordan block if (b) A square matrix J is said to be a Jordan matrix if where each J 1, …, J L are Jordan blocks.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.29  Prop. A.16 (Jordan Normal Form) (a) Every square matrix A can be represented in the form where S is a nonsingular matrix and J is a Jordan matrix. (b) J associated with A is unique up to the rearrangement of its blocks. (c) The diagonal entries of J are the eigenvalues of A repeated according to their multiplicities.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.30  Prop. A.17 (a) The eigenvalues of a triangular matrix are equal to its diagonal entries. (b) If S is nonsingular and B=SAS -1, then the eigenvalues of A and B coincides. (c) The eigenvalues of cI+A are equal to c+λ 1,…, c+λ n, where λ 1,…, λ n are the eigenvalues of A. (d) The eigenvalues of A k are equal to λ k 1,…, λ k n, where λ 1,…, λ n are the eigenvalues of A. (e) If A is nonsingular, the the eigenvalues of A -1 are the reciprocals of the eigenvalues of A. (f) The eigenvalues of A and A’ coincide.  Prop. A.18 (Cayley-Hamilton Theorem) If φ is the characteristic polynomial of a square matrix A, then φ (A)=0.  Prop. A.19 ρ(A) is a continuous function of A.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.31  Prop. A.20 For any induced matrix norm and any nxn matrix A, we have  pf.)  Show that Let λ be an eigenvalue of A such that Let x≠0 be an eigenvector of A corresponding to the eigenvalue λ normalized so that. Then,

Network Systems Lab. Korea Advanced Institute of Science and Technology No.32  Show that

Network Systems Lab. Korea Advanced Institute of Science and Technology No.33  Show that

Network Systems Lab. Korea Advanced Institute of Science and Technology No.34  Let be one of the Jordan blocks in J, of size mxm.  Let λ be the value of the diagonal entries of. Thus  A direct computation shows that for k≥m, Since

Network Systems Lab. Korea Advanced Institute of Science and Technology No.35 Therefore, We conclude that the seq. converges to

Network Systems Lab. Korea Advanced Institute of Science and Technology No.36 Eigenvalues  Prop.A.21 Let A be a square matrix. Then,  pf.)  Show that if, then. If, then, Therefore, A k does not converge to zero.

Network Systems Lab. Korea Advanced Institute of Science and Technology No.37  Show that if, then. If, letε>0 be such that Then, For sufficiently large k,