Question? Leave a message!




Lecture Notes Applied Mathematics

lecture notes in applied mathematics and mechanics and methods of applied mathematics for engineers and scientists pdf free download
AustinCraig Profile Pic
AustinCraig,Germany,Professional
Published Date:09-07-2017
Your Website URL(Optional)
Comment
Methods of Applied Mathematics Lecture Notes William G. Faris May 14, 20022Contents 1 Linear Algebra 7 1.1 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.1 Matrix algebra . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.2 Reduced row echelon form . . . . . . . . . . . . . . . . . . 8 1.1.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.1.4 The Jordan form . . . . . . . . . . . . . . . . . . . . . . . 11 1.1.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.1.6 Quadratic forms . . . . . . . . . . . . . . . . . . . . . . . 14 1.1.7 Spectral theorem . . . . . . . . . . . . . . . . . . . . . . . 14 1.1.8 Circulant (convolution) matrices . . . . . . . . . . . . . . 15 1.1.9 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2 Vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.2.1 Vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.2.2 Linear transformations . . . . . . . . . . . . . . . . . . . . 18 1.2.3 Reduced row echelon form . . . . . . . . . . . . . . . . . . 18 1.2.4 Jordan form . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.2.5 Forms and dual spaces . . . . . . . . . . . . . . . . . . . . 19 1.2.6 Quadratic forms . . . . . . . . . . . . . . . . . . . . . . . 20 1.2.7 Special relativity . . . . . . . . . . . . . . . . . . . . . . . 21 1.2.8 Scalar products and adjoint . . . . . . . . . . . . . . . . . 22 1.2.9 Spectral theorem . . . . . . . . . . . . . . . . . . . . . . . 23 1.3 Vector elds and di erential forms . . . . . . . . . . . . . . . . . 25 1.3.1 Coordinate systems . . . . . . . . . . . . . . . . . . . . . 25 1.3.2 Vector elds . . . . . . . . . . . . . . . . . . . . . . . . . . 26 1.3.3 Di erential forms . . . . . . . . . . . . . . . . . . . . . . . 27 1.3.4 Linearization of a vector eld near a zero . . . . . . . . . 28 1.3.5 Quadratic approximation to a function at a critical point 29 1.3.6 Di erential, gradient, and divergence . . . . . . . . . . . . 30 1.3.7 Spherical polar coordinates . . . . . . . . . . . . . . . . . 31 1.3.8 Gradient systems . . . . . . . . . . . . . . . . . . . . . . . 32 1.3.9 Hamiltonian systems . . . . . . . . . . . . . . . . . . . . . 33 1.3.10 Contravariant and covariant . . . . . . . . . . . . . . . . . 34 1.3.11 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 34 CONTENTS 2 Fourier series 37 2.1 Orthonormal families . . . . . . . . . . . . . . . . . . . . . . . . . 37 2 2.2 L convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.3 Absolute convergence . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.4 Pointwise convergence . . . . . . . . . . . . . . . . . . . . . . . . 41 2.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3 Fourier transforms 45 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 1 3.2 L theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2 3.3 L theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.4 Absolute convergence . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.5 Fourier transform pairs . . . . . . . . . . . . . . . . . . . . . . . . 48 3.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.7 Poisson summation formula . . . . . . . . . . . . . . . . . . . . . 50 3.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.9 PDE Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4 Complex integration 53 4.1 Complex number quiz . . . . . . . . . . . . . . . . . . . . . . . . 53 4.2 Complex functions . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.2.1 Closed and exact forms . . . . . . . . . . . . . . . . . . . 54 4.2.2 Cauchy-Riemann equations . . . . . . . . . . . . . . . . . 55 4.2.3 The Cauchy integral theorem . . . . . . . . . . . . . . . . 55 4.2.4 Polar representation . . . . . . . . . . . . . . . . . . . . . 56 4.2.5 Branch cuts . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.3 Complex integration and residue calculus . . . . . . . . . . . . . 57 4.3.1 The Cauchy integral formula . . . . . . . . . . . . . . . . 57 4.3.2 The residue calculus . . . . . . . . . . . . . . . . . . . . . 58 4.3.3 Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.3.4 A residue calculation . . . . . . . . . . . . . . . . . . . . . 59 4.4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.5 More residue calculus . . . . . . . . . . . . . . . . . . . . . . . . 60 4.5.1 Jordan's lemma . . . . . . . . . . . . . . . . . . . . . . . . 60 4.5.2 A more delicate residue calculation . . . . . . . . . . . . . 61 4.5.3 Cauchy formula for derivatives . . . . . . . . . . . . . . . 61 4.5.4 Poles of higher order . . . . . . . . . . . . . . . . . . . . . 62 4.5.5 A residue calculation with a double pole . . . . . . . . . . 62 4.6 The Taylor expansion . . . . . . . . . . . . . . . . . . . . . . . . 63 4.6.1 Radius of convergence . . . . . . . . . . . . . . . . . . . . 63 4.6.2 Riemann surfaces . . . . . . . . . . . . . . . . . . . . . . . 64CONTENTS 5 5 Distributions 67 5.1 Properties of distributions . . . . . . . . . . . . . . . . . . . . . . 67 5.2 Mapping distributions . . . . . . . . . . . . . . . . . . . . . . . . 69 5.3 Radon measures . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 5.4 Approximate delta functions . . . . . . . . . . . . . . . . . . . . . 70 5.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 5.6 Tempered distributions . . . . . . . . . . . . . . . . . . . . . . . . 72 5.7 Poisson equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 5.8 Di usion equation . . . . . . . . . . . . . . . . . . . . . . . . . . 75 5.9 Wave equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 5.10 Homogeneous solutions of the wave equation . . . . . . . . . . . 77 5.11 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.12 Answers to rst two problems . . . . . . . . . . . . . . . . . . . . 79 6 Bounded Operators 81 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 6.2 Bounded linear operators . . . . . . . . . . . . . . . . . . . . . . 81 6.3 Compact operators . . . . . . . . . . . . . . . . . . . . . . . . . . 84 6.4 Hilbert-Schmidt operators . . . . . . . . . . . . . . . . . . . . . . 87 6.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 6.6 Finite rank operators . . . . . . . . . . . . . . . . . . . . . . . . . 89 6.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 7 Densely De ned Closed Operators 93 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 7.2 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 7.3 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 7.4 Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 7.5 The spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 7.6 Spectra of inverse operators . . . . . . . . . . . . . . . . . . . . . 96 7.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 7.8 Self-adjoint operators . . . . . . . . . . . . . . . . . . . . . . . . . 98 7.9 First order di erential operators with a bounded interval: point spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 7.10 Spectral projection and reduced resolvent . . . . . . . . . . . . . 100 7.11 Generating second-order self-adjoint operators . . . . . . . . . . . 101 7.12 First order di erential operators with a semi-in nite interval: residual spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 7.13 First order di erential operators with an in nite interval: contin- uous spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 7.14 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 7.15 A pathological example . . . . . . . . . . . . . . . . . . . . . . . 1046 CONTENTS 8 Normal operators 105 8.1 Spectrum of a normal operator . . . . . . . . . . . . . . . . . . . 105 8.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 8.3 Variation of parameters and Green's functions . . . . . . . . . . . 107 8.4 Second order di erential operators with a bounded interval: point spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 8.5 Second order di erential operators with a semibounded interval: continuous spectrum . . . . . . . . . . . . . . . . . . . . . . . . . 109 8.6 Second order di erential operators with an in nite interval: con- tinuous spectrum . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 8.7 The spectral theorem for normal operators . . . . . . . . . . . . . 110 8.8 Examples: compact normal operators . . . . . . . . . . . . . . . 112 8.9 Examples: translation invariant operators and the Fourier trans- form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 8.10 Examples: Schr odinger operators . . . . . . . . . . . . . . . . . . 113 8.11 Subnormal operators . . . . . . . . . . . . . . . . . . . . . . . . . 114 8.12 Examples: forward translation invariant operators and the Laplace transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 8.13 Quantum mechanics . . . . . . . . . . . . . . . . . . . . . . . . . 118 8.14 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 9 Calculus of Variations 121 9.1 The Euler-Lagrange equation . . . . . . . . . . . . . . . . . . . . 121 9.2 A conservation law . . . . . . . . . . . . . . . . . . . . . . . . . . 122 9.3 Second variation . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 9.4 Interlude: The Legendre transform . . . . . . . . . . . . . . . . . 124 9.5 Lagrangian mechanics . . . . . . . . . . . . . . . . . . . . . . . . 126 9.6 Hamiltonian mechanics . . . . . . . . . . . . . . . . . . . . . . . . 127 9.7 Kinetic and potential energy . . . . . . . . . . . . . . . . . . . . . 128 9.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 9.9 The path integral . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 9.10 Appendix: Lagrange multipliers . . . . . . . . . . . . . . . . . . . 131 10 Perturbation theory 133 10.1 The implicit function theorem: scalar case . . . . . . . . . . . . . 133 10.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 10.3 The implicit function theorem: systems . . . . . . . . . . . . . . 136 10.4 Nonlinear di erential equations . . . . . . . . . . . . . . . . . . . 137 10.5 A singular perturbation example . . . . . . . . . . . . . . . . . . 138 10.6 Eigenvalues and eigenvectors . . . . . . . . . . . . . . . . . . . . 139 10.7 The self-adjoint case . . . . . . . . . . . . . . . . . . . . . . . . . 141 10.8 The anharmonic oscillator . . . . . . . . . . . . . . . . . . . . . . 142Chapter 1 Linear Algebra 1.1 Matrices 1.1.1 Matrix algebra An m by n matrix A is an array of complex numbers A for 1 i m and ij 1jn. The vector space operations are the sum A +B and the scalar multiple cA. Let A and B have the same dimensions. The operations are de ned by (A +B) =A +B (1.1) ij ij ij and (cA) =cA : (1.2) ij ij The m by n zero matrix is de ned by 0 = 0: (1.3) ij A matrix is a linear combination of other matrices if it is obtained from those matrices by adding scalar multiples of those matrices. Let A be an m by n matrix and B be an n by p matrix. Then the product AB is de ned by n X (AB) = A B : (1.4) ik ij jk j=1 The n by n identity matrix is de ned by I = : (1.5) ij ij Here  is the Kronecker delta that is equal to 1 when i = j and to 0 when ij i6=j. Matrix algebra satis es the usual properties of addition and many of the usual properties of multiplication. In particular, we have the associative law (AB)C =A(BC) (1.6) 78 CHAPTER 1. LINEAR ALGEBRA and the unit law AI =IA =A: (1.7) Even more important, we have the distributive laws A(B +C) = AB +AC (B +C)A = BA +CA: (1.8) However multiplication is not commutative; in general AB =6 BA. An n by n matrix A is invertible if there is another n by n matrix B with 1 AB =I andBA =I. In this case we writeB =A . (It turns out that ifBA = I, then alsoAB =I, but this is not obvious at rst.) The inverse operation has 1 1 1 1 1 several nice properties, including (A ) =A and (AB) =B A . The notion of division is ambiguous. Suppose B is invertible. Then both 1 1 AB and B A exist, but they are usually not equal. LetA be ann byn square matrix. The trace ofA is the sum of the diagonal entries. It is easy to check that tr(AB) = tr(BA) for all such matrices. Although matrices do not commute, their traces do. 1.1.2 Reduced row echelon form An m component vector is an m by 1 matrix. The ith standard basis vector is the vector with 1 in the ith row and zeros everywhere else. An m by n matrix R is in reduced row echelon form (rref) if each column is either the next unit basis vector, or a a linear combination of the previous unit basis vectors. The columns where unit basis vectors occur are called pivot columns. The rank r of R is the number of pivot columns. Theorem. IfA is anm byn matrix, then there is anm bym matrixE that is invertible and such that EA =R; (1.9) where R is in reduced row echelon form. The matrix R is uniquely determined by A. This theorem allows us to speak of the pivot columns of A and the rank of A. Notice that ifA isn byn and had rankn, thenR is the identity matrix and E is the inverse of A. Example. Let 2 3 4 12 2 16 1 4 5 A = 0 0 3 12 2 : (1.10) 1 3 0 2 3 Then the rref of A is 2 3 1 3 0 2 0 4 5 R = 0 0 1 4 0 : (1.11) 0 0 0 0 1 Corollary. Let A have reduced row echelon form R. The null space of A is the null space ofR. That is, the solutions of the homogeneous equationAx = 0 are the same as the solutions of the homogeneous equation Rx = 0.1.1. MATRICES 9 Introduce a de nition: A matrix is ipped rref if when ipped left-right ( iplr) and ipped up-down ( ipud) it is rref. The way reduced row echelon form is usually de ned, one works from left to right and from top to bottom. If you try to de ne a corresponding concept where you work from right to left and from bottom to top, a perhaps sensible name for this is ipped reduced row echelon form. Theorem. Let A be an m by n matrix with rank r. Then there is a unique n by nr matrix N such that the transpose of N is ipped rref and such that the transpose has pivot columns that are the non pivot columns of A and such that AN = 0 (1.12) The columns of N are called the rational basis for the null space of A. It is easy to nd N by solving RN = 0. The rational null space matrix N has the property that its transpose is in ipped reduced row echelon form. Example: In the above example the null space matrix of A is 2 3 3 2 6 1 0 7 6 7 N =6 0 47: (1.13) 4 5 0 1 0 0 That is, the solutions of Ax = 0 are the vectors of the form x =Nz. In other words, the columns of N span the null space of A. One can also use the technique to solve inhomogeneous equations Ax = b. One simply applies the theory to the augmented matrix A b. There is a solution when the last column ofA is not a pivot column. A particular solution may be read o from the last column of the rational basis. Example. Solve Ax = b, where 2 3 4 4 5 b = 19 : (1.14) 9 To accomplish this, let 2 3 4 12 2 16 1 4 4 5 A = 0 0 3 12 2 19 : (1.15) 1 1 3 0 2 3 9 Then the rref of A is 1 2 3 1 3 0 2 0 3 4 5 R = 0 0 1 4 0 5 : (1.16) 0 0 0 0 1 210 CHAPTER 1. LINEAR ALGEBRA The null space matrix of A is 1 2 3 3 2 3 1 0 0 6 7 6 7 0 4 5 6 7 N = : (1.17) 6 7 1 0 1 0 6 7 4 5 0 0 2 0 0 1 Thus the solution of Ax = b is 2 3 3 6 0 7 6 7 x =657: (1.18) 4 5 0 2 1.1.3 Problems Recall that the columns of a matrix A are linearly dependent if and only if the homogeneous equation Ax = 0 has a non-trivial solution. Also, a vector y is in the span of the columns if and only if the inhomogeneous equation Ax = y has a solution. 1. Show that if p is a particular solution of the equationAx = b, then every other solution is of the form x = p + z, where Az = 0. 2. Consider the matrix 2 3 2 4 5 19 3 6 2 4 3 9 27 6 7 64 8 1 17 2 7 A =6 7: 6 3 6 1 4 27 4 5 4 8 1 7 3 2 4 3 21 0 Use Matlab to nd the reduced row echelon form of the matrix. Use this to nd the rational basis for the solution of the homogeneous equation Az = 0. Check this with the Matlab solution. Write the general solution of the homogeneous equation. 3. LetA be the matrix given above. Use Matlab to nd an invertible matrix E such thatEA =R is in reduced echelon form. Find the determinant of E. 4. Consider the system Ax = b, where A is as above, and 2 3 36 21 6 7 6 7 9 6 7 b = : 6 7 6 6 7 4 5 6 231.1. MATRICES 11 Find the reduced echelon form of the matrix A augmented by the column b on the right. Use this to nd the rational basis for the solution of the homogeneous equation involving the augmented matrix. Use this to nd the general solution of the original inhomogeneous equation. 5. Consider a system of 6 linear equations in 5 unknowns. It could be overde- termined, that is, have no solutions. Or it could have special properties and be under determined, that is, have many solutions. Could it be nei- ther, that is, have exactly one solution? Is there such an example? Answer this question, and prove that your answer is correct. 6. Consider a system of 5 linear equations in 6 unknowns. Could it have exactly one solution? Is there such an example? Answer this question, and prove that your answer is correct. 6 7. Consider the ve vectors in R formed by the columns of A. Show that the vector b is in the span of these ve vectors. Find explicit weights that give it as a linear combination. 6 8. Is every vector y in R in the span of these ve vectors? If so, prove it. If not, give an explicit example of a vector that is not in the span, and prove that it is not in the span. 9. Are these ve vectors linearly independent? Prove that your answer is correct. The vectors in A that are pivot columns of A have the same span as the columns of A, yet are linearly independent. Find these vectors. How many are they? Prove that they are linearly independent. 1.1.4 The Jordan form Two matricesA,B are said to be similar if there is an invertible matrix P with 1 P AP =B. Notice that if A and B are similar, then tr(A) = tr(B). Theorem. Let A be ann byn matrix. Then there is an invertible matrix P such that 1 P AP =D +N; (1.19) where D is diagonal with diagonal entries D = . Each entry of N is zero, kk k except if  = , then it is allowed that N = 1. k k1 k1k Important note: Even if the matrixA is real, it may be that the matrices P and D are complex. The equation may also be written AP = PD +PN. If we let D =  , kk k then this is n X A P =P  +P N : (1.20) ij jk ik k ik1 k1k j=112 CHAPTER 1. LINEAR ALGEBRA The N factor is zero, except in cases where  = , when it is allowed k1k k k1 to be 1. We can also write this in vector form. It is the eigenvalue equation Au = u (1.21) k k k except when  = , when it may take the form k k1 Au = u + u : (1.22) k k k k1 The hard thing is to actually nd the eigenvalues  that form the diagonal k elements of the matrix D. This is a nonlinear problem. One way to see this is k the following. For each k = 1;:::;n the matrix power A is similar to a matrix k k (D +N) with the same diagonal entries as D . Thus we have the identity k k k tr(A ) = + + : (1.23) 1 n This is a system ofn nonlinear polynomial equations inn unknowns ;:::; . 1 k As is well known, there is another way of writing these equations in terms of the characteristic polynomial. This gives a single nth order polynomial equation in one unknown . This single equation has the same n solutions. For n = 2 the equation is quite easy to deal with by hand. For larger n one is often driven to computer algorithms. Example: Let   1 2 A = : (1.24) 15 12 The eigenvalues are 7, 6. The eigenvectors are the columns of   1 2 P = : (1.25) 3 5 Let   7 0 D = : (1.26) 0 6 Then AP =PD. Example: Let   10 1 A = : (1.27) 9 4 The only eigenvalue is 7. The eigenvector is the rst column of   1 2 P = : (1.28) 3 5 Let   7 1 D +N = : (1.29) 0 7 Then AP =P(D +N).1.1. MATRICES 13 1.1.5 Problems 1. IfA is a square matrix and f is a function de ned by a convergent power series, then f(A) is de ned. Show that if A is similar to B, then f(A) is similar to f(B). 2. By the Jordan form theorem, A is similar toD +N, whereD is diagonal, N is nilpotent, and D;N commute. To say that N is nilpotent is to say p that for some p 1 the power N = 0. Show that p1 X 1 (m) m f(D +N) = f (D)N (1.30) m m=0 3. Show that p1 X 1 m m exp(t(D +N)) = exp(tD) N t : (1.31) m m=0 Use this to describe the set of all solutions x = exp(tA)z to the di erential equation dx =Ax: (1.32) dt with initial condition x = z when t = 0. 4. Take   0 1 A = ; (1.33) k 2c where k 0 and c 0. The di erential equation describes an oscillator with spring constant k and friction coecient 2c. Find the eigenvalues and sketch a typical solution in the x ;x plane in each of the following 1 2 2 2 cases: over damped c k 0; critically damped c = k 0; under 2 2 2 damped 0c k; undamped 0 =c k; free motion 0 =c =k. 5. Consider the critically damped case. Find the Jordan form of the matrix A, and nd a similarity transformation that transforms A to its Jordan form. 1 6. IfA =PDP , where the diagonal matrixD has diagonal entries , then i 1 f(A) may be de ned for an arbitrary function f by f(A) = Pf(D)P , wheref(D) is the diagonal matrix with entries f( ). Thus, for instance, i p if each   0, then A is de ned. Find the square root of i   20 40 A = : (1.34) 8 16 7. Give an example of a matrixA with each eigenvalue  0, but for which i p no square root A can be de ned? Why does the formula in the second problem not work?14 CHAPTER 1. LINEAR ALGEBRA 1.1.6 Quadratic forms  Given an m by n matrix A, its adjoint is an n by n matrix A de ned by   A =A : (1.35) ji ij If we are dealing with real numbers, then the adjoint is just the transpose. The   adjoint operator has several nice properties, including A = A and (AB) =   B A . Two matrices A, B are said to be congruent if there is an invertible matrix  P with P AP =B.  Theorem. Let A be an n by n matrix with A = A . Then there is an invertible matrix P such that  P AP =D; (1.36) where D is diagonal with entries 1,1, and 0.  De ne the quadratic form Q(x) = x Ax. Then the equation says that one  may make a change of variables x =Py so that Q(x) = y Dy. 1.1.7 Spectral theorem 1  A matrix U is said to be unitary if U =U . In the real case this is the same as being an orthogonal matrix.  Theorem. LetA be ann byn matrix withA =A . Then there is a unitary matrix U such that 1 U AU =D; (1.37) where D is diagonal with real entries. Example: Let   1 2 A = : (1.38) 2 1 The eigenvalues are given by   3 0 D = : (1.39) 0 1 A suitable orthogonal matrix P is   1 1 1 P =p : (1.40) 1 1 2 Then AP =PD.   A matrix A is said to be normal if AA = A A. A self-adjoint matrix   (A = A) is normal. A skew-adjoint matrix (A =A) is normal. A unitary  1 matrix (A =A ) is normal. Theorem. LetA be ann byn matrix that is normal. Then there is a unitary matrix U such that 1 U AU =D; (1.41)1.1. MATRICES 15 where D is diagonal. Notice that this is the eigenvalue equation AU = UD. The columns of U are the eigenvectors, and the diagonal entries of D are the eigenvalues. Example: Let   1 1 1 p P = : (1.42) 1 1 2 Then P is a rotation by =4. The eigenvalues are on the diagonal of   1 1 +i 0 F =p : (1.43) 0 1i 2 A suitable unitary matrix Q is   1 1 i Q =p : (1.44) i 1 2 Then PQ =QF . 1.1.8 Circulant (convolution) matrices If A is normal, then there are unitary U and diagonal D with AU =UD. But they are dicult to compute. Here is one special but important situation where everything is explicit. A circulant (convolution) matrix is an n by n matrix such that there is a function a with A = a(pq), where the di erence is computed modulo n. pq For instance, a 4 by 4 matrix would have the same entrya(3) in the 12, 23, 34, and 41 positions. The DFT (discrete Fourier transform) matrix is an n by n unitary matrix given by 2iqk 1 n U =p e : (1.45) qk n Theorem. Let A be a circulant matrix. If U is the DFT matrix, then 1 U AU =D; (1.46) where D is a diagonal matrix with X 2irk n D =a (k) = a(r)e : (1.47) kk r 1.1.9 Problems 1. Let 2 3 5 1 4 2 3 6 1 2 6 3 17 6 7 A =6 4 6 3 0 47: (1.48) 4 5 2 3 0 1 2 3 1 4 2 3 1 Use Matlab to nd orthogonal P and diagonal D so that P AP =D.16 CHAPTER 1. LINEAR ALGEBRA 1 2. Find unitary Q and diagonal F so that Q PQ =F. 3. The orthogonal matrix P is made of rotations in two planes and possibly a re ection. What are the two angles of rotations? Is there a re ection present as well? Explain.  4. Find an invertible matrix R such that R AR = G, where G is diagonal with all entries1 or 0. 5. In the following problems the matrix A need not be square. Let A be a  matrix with trivial null space. Show that A A is invertible.  1   2 6. Let E =A(A A) A . Show that E =E and E =E. +  1  + 7. De ne the pseudo-inverse A = (A A) A . Show that AA = E and + A A =I. 8. Let 2 3 1 1 1 1 2 4 6 7 6 7 1 3 9 6 7 6 7 A = 1 4 16 : (1.49) 6 7 6 7 1 5 25 6 7 4 5 1 6 36 1 7 49 Calculate E and check the identities above. Find the eigenvalues of E. Explain the geometric meaning of E. 9. Find the parameter values x such that Ax best approximates 2 3 7 6 5 7 6 7 6 6 7 6 7 y =61 7: (1.50) 6 7 3 6 7 4 5 7 24 + (Hint: Let x =A y.) What is the geometric interpretation of Ax?1.2. VECTOR SPACES 17 1.2 Vector spaces 1.2.1 Vector spaces A vector space is a collection V of objects that may be combined with vector addition and scalar multiplication. There will be two possibilities for the scalars. They may be elements of the eld R of real numbers. Or they may be elements of the eld C of complex numbers. To handle both cases together, we shall consider the scalars as belonging to a eld F. The vector space addition axioms say that the sum of each two vectors u; v inV is another vector in the space called u + v inV . Addition must satisfy the following axioms: associative law (u + v) + w = u + (v + w) (1.51) additive identity law u + 0 = 0 + u = u (1.52) additive inverse law u + (u) = (u) + u = 0 (1.53) and commutative law u + v = v + u: (1.54) The vector space axioms also require that for each scalar c in F and each u in V there is a vector cu in V . Scalar multiplication must satisfy the following axioms: distributive law for vector addition c(u + v) =cu +cv (1.55) distributive law for scalar addition (c +d)u =cu +cu (1.56) associative law for scalar multiplication (cd)u =c(du) (1.57) identity law for scalar multiplication 1u = u: (1.58) The elements of a vector space are pictured as arrows starting at a xed origin. The vector sum is pictured in terms of the parallelogram law. The sum of two arrows starting at the origin is the diagonal of the parallelogram formed by the two arrows. A subspace of a vector space is a subset that is itself a vector space. The smallest possible subspace consists of the zero vector at the origin. A one dimensional vector subspace is pictured as a line through the origin. A two dimensional vector subspace is pictured as a plane through the origin, and so on.18 CHAPTER 1. LINEAR ALGEBRA 1.2.2 Linear transformations LetV andW be vector spaces. A linear transformationT :V W is a function from V to W that preserves the vector space operations. Thus T (u + v) =Tu +Tv (1.59) and T (cu) =cTu: (1.60) The null space of a linear transformationT :V W is the set of all vectors u in V such that Tu = 0. The null space is sometimes called the kernel. The range of a linear transformation T :V W is the set of all vectorsTu in W , where u is in V . The range is sometimes called the image. Theorem. The null space of a linear transformationT :V W is a subspace of V . Theorem. The range of a linear transformation T :V W is a subspace of W . Theorem. A linear transformation T : V W is one-to-one if and only if its null space is the zero subspace. Theorem. A linear transformation T : V W is onto W if and only if its range is W . 1 Theorem. A linear transformationT :V W has an inverseT :WV if and only if its null space is the zero subspace and its range is W . An invertible linear transformation will also be called a vector space isomor- phism. Consider a list of vectors p ;:::; p . They de ne a linear transformation 1 n n n P : F V , by taking the vector in F to be the weights of a linear combination of the p ;:::; p . 1 n Theorem. A list of vectors is linearly independent if and only if the corre- sponding linear transformation has zero null space. Theorem. A list of vectors spans the vector space V if and only if the corresponding linear transformation has range V . Theorem. A list of vectors is a basis for V if and only if the corresponding n linear transformation P : F V is a vector space isomorphism. In that case 1 n the inverse transformation P : V F is the transformation that carries a vector to its coordinates with respect to the basis. n In case there is a basis transformation P : F V , the vector space V is said to be n dimensional. 1.2.3 Reduced row echelon form Theorem Let p ;:::; p be a list of vectors in an m dimensional vector space 1 n n V . Let P : F V be the corresponding transformation. Then there is an m isomorphism E :V F such that EP =R; (1.61)1.2. VECTOR SPACES 19 where R is in reduced row echelon form. The m by n matrix R is uniquely determined by the list of vectors. Thus for an arbitrary list of vectors, there are certain vectors that are pivot vectors. These vectors are those that are not linear combinations of previous vectors in the list. The reduced row echelon form of a list of vectors expresses the extent to which each vector in the list is a linear combination of previous pivot vectors in the list. 1.2.4 Jordan form Theorem. LetT :V V be a linear transformation of ann dimensional vector n space to itself. Then there is an invertible matrix P : R V such that 1 P TP =D +N; (1.62) where D is diagonal with diagonal entries D = . Each entry of N is zero, kk k except if  = , then it is allowed that N = 1. k k1 k1k We can also write this in vector form. The equation TP = P (D +N) is equivalent to the eigenvalue equation Tp = p (1.63) k k k except when  = , when it may take the form k k1 Tp = p + p : (1.64) k k k k1 It is dicult to picture such a linear transformation, even for a real vector space. One way to do it works especially well in the case when the dimension of V is 2. Then the linear transformation T maps each vector u in V to another vector Tu in V . Think of each u as determining a point in the plane. Draw the vector Tu as a vector starting at the point u. Then this gives a picture of the linear transformation as a vector eld. The qualitative features of this vector eld depend on whether the eigenvalues are real or occur as a complex conjugate pair. In the real case, they can be both positive, both negative, or of mixed sign. These give repelling, attracting, and hyperbolic xed points. In the complex case the real part can be either positive or negative. In the rst case the rotation has magnitude less than=2, and the vector eld is a repelling spiral. In the second case the rotation has magnitude greater than =2 and the vector eld is an attracting spiral. The case of complex eigenvalues with real part equal to zero is special but important. The transformation is similar to a rotation that has magnitude=2, and so the vector eld goes around in closed elliptical paths. 1.2.5 Forms and dual spaces  The dual space V of a vector space V consists of all linear transformations  from V to the eld of scalars F. If is in V , then the value of on u in V is writtenh; ui.20 CHAPTER 1. LINEAR ALGEBRA n We think of the elements of F as column vectors. The elements of the dual n space of F are row vectors. The value of a row vector on a column vector is given by taking the row vector on the left and the column vector on the right and using matrix multiplication.  The way to picture an element of V is via its contour lines. These are straight lines at regularly spaced intervals. The value of on u is obtained by taking the vector u and seeing how many contour lines it crosses. Sometimes vectors in the original space V are called contravariant vectors,  while forms in the dual space V are called covariant vectors. 1.2.6 Quadratic forms A sesquilinear form is a function that assigns to each ordered pair of vectors u, v in V a scalar a(u; v). It is required that it be linear in the second variable. Thus a(u; v + w) =a(u; v) +a(u; w) (1.65) and a(u;cv) =ca(u; v): (1.66) Furthermore, it is required to be conjugate linear in the rst variable. this says that a(u + v; w) =a(u; w) +a(v; w) (1.67) and a(cu; v) =ca  (u; v): (1.68) In the real case one does not need the complex conjugates on the scalars. In this case this is called a bilinear form.  A sesquilinear form de nes a linear transformation fromV toV . (Actually, in the complex case it is a conjugate linear transformation.) This linear trans- formation takes the vector u to the forma(u;). So a sesquilinear form may be viewed as a special kind of linear transformation, but not one from a space to itself. A sesquilinear form is sometimes regarded as a covariant tensor. By contrast, a linear transformation a mixed tensor. A sesquilinear form is Hermitian if a(u; v) =a(v; u): (1.69) In the real case the complex conjugate does not occur. In this case the bilinear form is called symmetric. A Hermitian form de nes a quadratic forma(u; u) that is real. The quadratic form determines the Hermitian form. In fact, 4a(u; v) =a(u + v; u + v) +a(u v; u v) (1.70) determines the real part. Then=a(u; v) =a(iu; v) determines the imaginary part.