how linear algebra is used in machine learning and lecture notes on probability statistics and linear algebra. linear algebra and differential equations lecture notes
ZackVincent,United Kingdom,Teacher
Published Date:15-07-2017
Your Website URL(Optional)
Comment
with Open Texts
AFirstCoursein
LINEARALGEBRA
anOpenText
BASETEXTBOOK
VERSION2017–REVISIONA
ADAPTABLEACCESSIBLEAFFORDABLE
byLyryxLearning
basedontheoriginaltextbyK.Kuttler
CreativeCommonsLicense(CCBY)advancing learning
ChampionsofAccesstoKnowledge
ONLINE
OPENTEXT
ASSESSMENT
All digital forms of access to our high-quality We have been developing superior online for-
open texts are entirely FREE All content is mativeassessmentformorethan15years. Our
reviewed for excellence and is wholly adapt- questions are continuously adapted with the
able; custom editions are produced by Lyryx content and reviewed for quality and sound
for those adopting Lyryx assessment. Access pedagogy. To enhance learning, students re-
to theoriginal source files is also open to any- ceive immediate personalized feedback. Stu-
one dent grade reports and performance statistics
arealsoprovided.
INSTRUCTOR
SUPPORT
SUPPLEMENTS
Access to our in-house support team is avail- Additional instructor resources are also freely
able7days/weektoprovidepromptresolution accessible. Product dependent, these supple-
to both student and instructorinquiries. In ad- mentsinclude: fullsetsofadaptableslidesand
dition,weworkone-on-onewithinstructorsto lecture notes, solutions manuals, and multiple
provide a comprehensive system, customized choice question banks with an exam building
for theircourse. Thiscan includeadapting the tool.
text,managingmultiplesections,andmore
ContactLyryxToday
infolyryx.comadvancing learning
AFirstCoursein LinearAlgebra
anOpenText
BEACHAMPIONOFOER
Contributesuggestionsforimprovements,newcontent,orerrata:
Anewtopic
Anewexample
Aninterestingnewquestion
Aneworbetterprooftoanexistingtheorem
Anyothersuggestionstoimprovethematerial
ContactLyryxatinfolyryx.comwithyourideas.
CONTRIBUTIONS
Ilijas Farah,YorkUniversity
KenKuttler, Brigham YoungUniversity
LyryxLearningTeam
Bruce Bauslaugh Jennifer MacKenzie
Peter Chow TamsynMurnaghan
NathanFriess BogdanSava
Stephanie Keyowski Larissa Stone
Claude Laflamme RyanYee
Martha Laflamme EhsunZahedi
LICENSE
CreativeCommonsLicense(CCBY):Thistext,includingtheartandillustrations,areavailableunder
theCreativeCommonslicense(CCBY),allowinganyonetoreuse,revise,remixandredistributethetext.
Toviewacopyofthislicense,visithttps://creativecommons.org/licenses/by/4.0/advancing learning
AFirstCoursein LinearAlgebra
anOpenText
BaseTextRevisionHistory
CurrentRevision: Version2017 — RevisionA
Extensiveedits,additions,andrevisionshavebeencompletedbytheeditorialstaffatLyryxLearning.
Allnewcontent(textandimages)isreleasedunderthesamelicenseasnotedabove.
• Lyryx: Frontmatterhasbeenupdatedincludingcover,copyright,andrevisionpages.
2017A
• I. Farah: contributededits and revisions, particularlythe proofsin the Properties of Determinants II:
SomeImportantProofssection
• Lyryx: The text has been updated with the addition of subsections on Resistor Networks and the
MatrixExponentialbasedonoriginalmaterialbyK.Kuttler.
2016B
• Lyryx: Newexample7.35onRandomWalksdeveloped.
• Lyryx: The layout and appearance of the text has been updated, including the title page and newly
2016A
designedbackcover.
• Lyryx: The content was modified and adapted with the addition of new material and several im-
agesthroughout.
2015A
• Lyryx: Additionalexamplesandproofswereaddedtoexistingmaterialthroughout.
• OriginaltextbyK. KuttlerofBrighamYoungUniversity. Thatversionis usedunderCreativeCom-
mons license CC BY (https://creativecommons.org/licenses/by/3.0/) made possible by
2012A
fundingfromTheSaylorFoundation’sOpenTextbookChallenge. SeeElementaryLinearAlgebrafor
moreinformationandtheoriginalversion.Contents
Contents iii
Preface 1
1 SystemsofEquations 3
1.1 SystemsofEquations,Geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 SystemsOfEquations,AlgebraicProcedures . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.1 ElementaryOperations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.2 GaussianElimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.2.3 UniquenessoftheReducedRow-EchelonForm . . . . . . . . . . . . . . . . . . 25
1.2.4 Rank andHomogeneousSystems . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.2.5 BalancingChemicalReactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
1.2.6 DimensionlessVariables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
1.2.7 AnApplicationtoResistorNetworks . . . . . . . . . . . . . . . . . . . . . . . . 38
2 Matrices 53
2.1 MatrixArithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.1.1 AdditionofMatrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.1.2 ScalarMultiplicationofMatrices . . . . . . . . . . . . . . . . . . . . . . . . . . 57
2.1.3 MultiplicationofMatrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
th
2.1.4 Theij EntryofaProduct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.1.5 PropertiesofMatrixMultiplication . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.1.6 TheTranspose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
2.1.7 TheIdentityandInverses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.1.8 FindingtheInverseofaMatrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.1.9 ElementaryMatrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.1.10 MoreonMatrixInverses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
2.2 LU Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
2.2.1 FindingAnLU FactorizationByInspection . . . . . . . . . . . . . . . . . . . . . 99
2.2.2 LU Factorization,MultiplierMethod . . . . . . . . . . . . . . . . . . . . . . . . 100
2.2.3 SolvingSystemsusingLU Factorization . . . . . . . . . . . . . . . . . . . . . . . 101
2.2.4 JustificationfortheMultiplierMethod . . . . . . . . . . . . . . . . . . . . . . . . 102
iiiiv CONTENTS
3 Determinants 107
3.1 BasicTechniquesandProperties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.1.1 Cofactors and2×2Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.1.2 TheDeterminantofaTriangularMatrix . . . . . . . . . . . . . . . . . . . . . . . 112
3.1.3 PropertiesofDeterminantsI:Examples . . . . . . . . . . . . . . . . . . . . . . . 114
3.1.4 PropertiesofDeterminantsII:SomeImportantProofs . . . . . . . . . . . . . . . 118
3.1.5 FindingDeterminantsusingRowOperations . . . . . . . . . . . . . . . . . . . . 123
3.2 ApplicationsoftheDeterminant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
3.2.1 AFormulafortheInverse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
3.2.2 Cramer’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
3.2.3 PolynomialInterpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
n
4 R 145
n
4.1 VectorsinR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
n
4.2 AlgebrainR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
n
4.2.1 AdditionofVectorsinR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
n
4.2.2 ScalarMultiplicationofVectorsinR . . . . . . . . . . . . . . . . . . . . . . . . 150
4.3 GeometricMeaningofVectorAddition . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
4.4 LengthofaVector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
4.5 GeometricMeaningofScalarMultiplication . . . . . . . . . . . . . . . . . . . . . . . . . 159
4.6 ParametricLines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
4.7 TheDotProduct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.7.1 TheDotProduct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
4.7.2 TheGeometricSignificanceoftheDotProduct . . . . . . . . . . . . . . . . . . . 170
4.7.3 Projections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
n
4.8 PlanesinR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
4.9 TheCrossProduct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
4.9.1 TheBoxProduct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
n
4.10 Spanning,LinearIndependenceandBasisinR . . . . . . . . . . . . . . . . . . . . . . . 192
4.10.1 SpanningSetofVectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
4.10.2 LinearlyIndependentSetofVectors . . . . . . . . . . . . . . . . . . . . . . . . . 194
4.10.3 AShortApplicationtoChemistry . . . . . . . . . . . . . . . . . . . . . . . . . . 200
4.10.4 SubspacesandBasis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
4.10.5 RowSpace, ColumnSpace,andNullSpaceofaMatrix . . . . . . . . . . . . . . . 211
4.11 OrthogonalityandtheGramSchmidtProcess . . . . . . . . . . . . . . . . . . . . . . . . 232
4.11.1 OrthogonalandOrthonormalSets . . . . . . . . . . . . . . . . . . . . . . . . . . 233
4.11.2 OrthogonalMatrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238CONTENTS v
4.11.3 Gram-SchmidtProcess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
4.11.4 OrthogonalProjections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
4.11.5 LeastSquaresApproximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
4.12 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
4.12.1 VectorsandPhysics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
4.12.2 Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
5 LinearTransformations 269
5.1 LinearTransformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
5.2 TheMatrixofaLinearTransformationI . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
5.3 PropertiesofLinearTransformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
2
5.4 SpecialLinearTransformationsinR . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
5.5 OnetoOneandOntoTransformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
5.6 Isomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
5.7 TheKernelAndImageOfALinearMap. . . . . . . . . . . . . . . . . . . . . . . . . . . 310
5.8 TheMatrixofaLinearTransformationII . . . . . . . . . . . . . . . . . . . . . . . . . . 315
5.9 TheGeneral SolutionofaLinearSystem. . . . . . . . . . . . . . . . . . . . . . . . . . . 321
6 ComplexNumbers 329
6.1 ComplexNumbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
6.2 PolarForm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
6.3 RootsofComplexNumbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
6.4 TheQuadraticFormula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
7 SpectralTheory 347
7.1 EigenvaluesandEigenvectorsofaMatrix . . . . . . . . . . . . . . . . . . . . . . . . . . 347
7.1.1 DefinitionofEigenvectorsandEigenvalues . . . . . . . . . . . . . . . . . . . . . 347
7.1.2 FindingEigenvectorsandEigenvalues . . . . . . . . . . . . . . . . . . . . . . . . 350
7.1.3 EigenvaluesandEigenvectorsforSpecialTypesofMatrices . . . . . . . . . . . . 356
7.2 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
7.2.1 SimilarityandDiagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
7.2.2 DiagonalizingaMatrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
7.2.3 ComplexEigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
7.3 ApplicationsofSpectralTheory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372
7.3.1 RaisingaMatrixtoaHighPower . . . . . . . . . . . . . . . . . . . . . . . . . . 373
7.3.2 RaisingaSymmetricMatrixtoaHighPower . . . . . . . . . . . . . . . . . . . . 375
7.3.3 MarkovMatrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
7.3.3.1 EigenvaluesofMarkovMatrices . . . . . . . . . . . . . . . . . . . . . 384vi CONTENTS
7.3.4 DynamicalSystems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
7.3.5 TheMatrixExponential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
7.4 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
7.4.1 OrthogonalDiagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
7.4.2 TheSingularValueDecomposition . . . . . . . . . . . . . . . . . . . . . . . . . 409
7.4.3 PositiveDefiniteMatrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
7.4.3.1 TheCholeskyFactorization . . . . . . . . . . . . . . . . . . . . . . . . 420
7.4.4 QRFactorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
7.4.4.1 TheQRFactorizationandEigenvalues . . . . . . . . . . . . . . . . . . 424
7.4.4.2 PowerMethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424
7.4.5 QuadraticForms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
8 SomeCurvilinearCoordinateSystems 439
8.1 PolarCoordinatesandPolarGraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
8.2 SphericalandCylindricalCoordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
9 VectorSpaces 455
9.1 AlgebraicConsiderations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
9.2 SpanningSets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471
9.3 LinearIndependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475
9.4 SubspacesandBasis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
9.5 SumsandIntersections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 498
9.6 LinearTransformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
9.7 Isomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
9.7.1 OnetoOneandOntoTransformations . . . . . . . . . . . . . . . . . . . . . . . . 505
9.7.2 Isomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
9.8 TheKernelAndImageOfALinearMap. . . . . . . . . . . . . . . . . . . . . . . . . . . 518
9.9 TheMatrixofaLinearTransformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
A SomePrerequisiteTopics 537
A.1 SetsandSet Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537
A.2 WellOrderingandInduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
B SelectedExerciseAnswers 543
Index 591Preface
A First Course in Linear Algebra presents an introduction to the fascinating subject of linear algebra for
students who have a reasonable understanding of basic algebra. Major topics of linear algebra are pre-
sentedindetail,withproofsofimportanttheoremsprovided. Separatesectionsmaybeincludedinwhich
proofs are examined in further depth and in general these can be excluded without loss of contrinuity.
Where possible, applications of key concepts are explored. In an effort to assist those students who are
interestedincontinuingonin linearalgebraconnectionstoadditionaltopicscoveredinadvancedcourses
areintroduced.
Each chapter begins with a list of desired outcomes which a student should be able to achieve upon
completing the chapter. Throughout the text, examples and diagrams are given to reinforce ideas and
provide guidance on how to approach various problems. Students are encouraged to work through the
suggestedexercises providedat theend ofeach section. Selected solutionsto theseexercisesare givenat
theendofthetext.
As thisis an open text, youare encouraged to interact withthetextbookthroughannotating, revising,
andreusingtoyouradvantage.
11.SystemsofEquations
1.1SystemsofEquations,Geometry
Outcomes
A. Relate the types of solution sets of a system of two (three) variables to the intersections of
linesinaplane(theintersectionofplanesinthreespace)
Asyoumayremember,linearequationslike2x+3y=6canbegraphedasstraightlinesinthecoordi-
nateplane. We say thatthisequation isin twovariables, in thiscase x and y. Supposeyou havetwosuch
equations, each of which can be graphed as a straight line, and consider the resulting graph of two lines.
Whatwoulditmeanifthereexistsapointofintersectionbetweenthetwolines? Thispoint,whichlieson
both graphs, gives x and y values for which both equations are true. In other words, this point gives the
orderedpair(x,y)thatsatisfybothequations. Ifthepoint(x,y)isapointofintersection,wesaythat (x,y)
is a solution to the two equations. In linear algebra, we often are concerned with finding the solution(s)
to a system of equations, if such solutionsexist. First, we consider graphical representations of solutions
andlaterwewillconsiderthealgebraicmethodsforfindingsolutions.
When looking for the intersection of two lines in a graph, several situations may arise. The follow-
ing picture demonstrates the possible situations when considering two equations (two lines in the graph)
involvingtwovariables.
y y y
x x x
OneSolution NoSolutions InfinitelyManySolutions
Inthefirstdiagram,thereisauniquepointofintersection,whichmeansthatthereisonlyone(unique)
solutiontothetwoequations. In thesecond, thereareno pointsofintersectionand nosolution. Whenno
solutionexists,thismeansthatthetwolinesareparallelandtheyneverintersect. Thethirdsituationwhich
can occur, as demonstrated in diagram three, is that the two lines are really the same line. For example,
x+y =1 and 2x+2y =2 are equations which when graphed yield the same line. In this case there are
infinitely many points which are solutions of these two equations, as every ordered pair which is on the
graphofthelinesatisfiesbothequations. Whenconsideringlinearsystemsofequations,therearealways
threetypesofsolutionspossible;exactlyone(unique)solution,infinitelymanysolutions,ornosolution.
34 SystemsofEquations
Example1.1:AGraphicalSolution
Useagraphtofindthesolutiontothefollowingsystemofequations
x+y=3
y−x=5
Solution.Throughgraphingtheaboveequationsandidentifyingthepointofintersection,wecanfindthe
solution(s). Remember that we must have either one solution, infinitely many, or no solutionsat all. The
followinggraphshowsthetwoequations,aswellastheintersection. Remember,thepointofintersection
represents the solution of the two equations, or the (x,y) which satisfy both equations. In this case, there
isonepointofintersectionat (−1,4)whichmeanswehaveoneuniquesolution,x=−1,y=4.
y
6
4
(x,y)=(−1,4)
2
x
−4 −3 −2 −1 1
♠
In the above example, we investigated the intersection point of two equations in two variables, x and
y. Nowwewillconsiderthegraphicalsolutionsofthreeequationsintwovariables.
Consider a system of three equations in two variables. Again, these equations can be graphed as
straightlinesintheplane,sothattheresultinggraphcontainsthreestraightlines. Recallthethreepossible
types ofsolutions;no solution,one solution,and infinitelymanysolutions. There are nowmorecomplex
ways of achieving these situations, due to the presence of the third line. For example, you can imagine
thecaseofthreeintersectinglineshavingnocommonpointofintersection. Perhapsyoucanalsoimagine
threeintersectinglineswhichdointersectatasinglepoint. Thesetwosituationsareillustratedbelow.
y y
x x
NoSolution OneSolution1.1.SystemsofEquations,Geometry 5
Consider the first picture above. While all three lines intersect with one another, there is no common
point of intersection where all three lines meet at one point. Hence, there is no solution to the three
equations. Remember, a solution is a point (x,y) which satisfies all three equations. In the case of the
second picture, the lines intersect at a common point. This means that there is one solution to the three
equationswhosegraphsarethegivenlines. Youshouldtakeamomentnowtodrawthegraphofasystem
which resultsinthreeparallel lines. Next,try thegraphofthree identicallines. Which typeofsolutionis
representedineach ofthesegraphs?
Wehavenowconsidered thegraphical solutionsofsystemsoftwoequationsin twovariables,aswell
as three equations in two variables. However, there is no reason to limit our investigationto equations in
twovariables. Wewillnowconsiderequationsinthreevariables.
You may recall that equations in three variables, such as 2x+4y−5z =8, form a plane. Above, we
werelookingforintersectionsoflinesinordertoidentifyanypossiblesolutions. Whengraphicallysolving
systems of equations in three variables, we look for intersections of planes. These points of intersection
give the (x,y,z) that satisfy all the equations in the system. What types of solutions are possible when
working with three variables? Consider the following picture involving two planes, which are given by
twoequationsinthreevariables.
Notice how thesetwo planes intersect in a line. This means that the points (x,y,z) on this line satisfy
both equations in the system. Since the line contains infinitely many points, this system has infinitely
manysolutions.
It could also happen that the two planes fail to intersect. However, is it possible to have two planes
intersect at asinglepoint? Take amomenttoattempt drawingthissituation,and convinceyourselfthat it
is not possible This means that when we have only two equations in three variables, there is no way to
havea uniquesolution Hence, thetypes of solutionspossiblefor two equationsin three variables are no
solutionorinfinitelymanysolutions.
Now imagine adding a third plane. In other words, consider three equations in three variables. What
typesofsolutionsarenowpossible? Considerthefollowingdiagram.
NewPlane
✠
In this diagram, there is no point which lies in all three planes. There is no intersection between all6 SystemsofEquations
planes so there is no solution. The picture illustrates the situation in which the line of intersection of the
new plane with one of the original planes forms a line parallel to the line of intersection of the first two
planes. However, in three dimensions, it is possiblefor two lines to fail to intersect even though they are
notparallel. Suchlinesarecalledskewlines.
Recall that when working with two equations in three variables, it was not possible to have a unique
solution. Is it possiblewhen consideringthree equationsin three variables? In fact, it is possible,and we
demonstratethissituationinthefollowingpicture.
NewPlane
✠
In this case, the three planes have a single point of intersection. Can you think of other types of
solutions possible? Another is that the three planes could intersect in a line, resulting in infinitely many
solutions,asinthefollowingdiagram.
We have now seen how three equations in three variables can have no solution, a unique solution, or
intersect in a line resulting in infinitely many solutions. It is also possible that the three equations graph
thesameplane,whichalsoleadstoinfinitelymanysolutions.
Youcanseethatwhenworkingwithequationsinthreevariables,therearemanymorewaystoachieve
thedifferenttypesofsolutionsthanwhenworkingwithtwovariables. Itmayproveenlighteningtospend
timeimagining(anddrawing)manypossiblescenarios,andyoushouldtakesometimetotryafew.
Youshouldalsotakesometimetoimagine(anddraw)graphsofsystemsinmorethanthreevariables.
Equationslikex+y−2z+4w=8withmorethanthreevariablesareoftencalledhyper-planes. Youmay
soon realize that it is tricky to draw the graphs of hyper-planes Through the tools of linear algebra, we
canalgebraicallyexaminethesetypesofsystemswhicharedifficulttograph. Inthefollowingsection,we
willconsiderthesealgebraictools.1.2.SystemsOfEquations,AlgebraicProcedures 7
Exercises
Exercise 1.1.1 Graphically, find the point (x ,y ) which lies on both lines, x+3y =1 and 4x−y = 3.
1 1
Thatis,grapheachlineandsee wheretheyintersect.
Exercise1.1.2 Graphically,findthepointofintersectionofthetwolines3x+y=3andx+2y=1.That
is,grapheachlineandseewhere theyintersect.
Exercise 1.1.3 You have a system of k equations in two variables, k≥2. Explain the geometric signifi-
canceof
(a) Nosolution.
(b) Auniquesolution.
(c) Aninfinitenumberofsolutions.
1.2SystemsOfEquations,AlgebraicProcedures
Outcomes
A. Useelementaryoperationstofindthesolutiontoalinearsystemofequations.
B. Findtherow-echelonform andreducedrow-echelonform ofamatrix.
C. Determine whether a system of linear equations has no solution, a unique solution or an
infinitenumberofsolutionsfromitsrow-echelonform.
D. SolveasystemofequationsusingGaussianEliminationandGauss-JordanElimination.
E. Modelaphysicalsystemwithlinearequationsandthensolve.
Wehavetakenanindepthlookatgraphicalrepresentationsofsystemsofequations,aswellashowto
findpossiblesolutionsgraphically. Ourattentionnowturnstoworkingwithsystemsalgebraically.8 SystemsofEquations
Definition1.2:SystemofLinearEquations
Asystemoflinearequations isalistofequations,
a x +a x +···+a x =b
11 1 12 2 1n n 1
a x +a x +···+a x =b
n
21 1 22 2 2n 2
.
.
.
a x +a x +···+a x =b
m1 1 m2 2 mn n m
where a and b are real numbers. The above is a system of m equations in the n variables,
ij j
x ,x ···,x . Written more simply in terms of summation notation, the above can be written in
1 2 n
theform
n
a x =b, i=1,2,3,···,m
ij j i
∑
j=1
The relative size of m and n is not important here. Notice that we have allowed a and b to be any
ij j
real number. We can also call these numbersscalars . We will use this term throughoutthe text, so keep
inmindthatthetermscalarjustmeansthatweareworkingwithreal numbers.
Now,supposewehaveasystemwhereb =0foralli. Inotherwordseveryequationequals0. Thisis
i
aspecialtypeofsystem.
Definition1.3:HomogeneousSystemofEquations
A system of equations is called homogeneous if each equation in the system is equal to 0. A
homogeneoussystemhastheform
a x +a x +···+a x =0
11 1 12 2 1n n
a x +a x +···+a x =0
21 1 22 2 2n n
.
.
.
a x +a x +···+a x =0
m1 1 m2 2 mn n
wherea arescalarsandx arevariables.
ij i
Recall from the previous section that our goal when working with systems of linear equations was to
findthepointofintersectionoftheequationswhengraphed. Inotherwords,welookedforthesolutionsto
thesystem. Wenowwishtofindthesesolutionsalgebraically. Wewanttofindvaluesforx ,···,x which
1 n
solvealloftheequations. Ifsuchasetofvaluesexists,wecall (x ,···,x )thesolutionset.
1 n
Recall the abovediscussions about the types of solutionspossible. We will see that systems of linear
equationswillhaveoneuniquesolution,infinitelymanysolutions,ornosolution. Considerthefollowing
definition.
Definition1.4:ConsistentandInconsistentSystems
A system of linear equations is called consistent if there exists at least one solution. It is called
inconsistentifthereisnosolution.
Advise:Why You Wasting Money in Costly SEO Tools, Use World's Best Free SEO Tool Ubersuggest.