Lecture notes Probability Theory

lecture notes on probability and statistics theory for economists and probability theory lecture notes mit | pdf free download

AustinCraig,Germany,Professional

Published Date:09-07-2017

Your Website URL(Optional)

Comment

NotesonProbability PeterJ.CameroniiPreface Here are the course lecture notes for the course MAS108, Probability I, at Queen Mary,UniversityofLondon,takenbymostMathematicsstudentsandsomeothers intheﬁrstsemester. Thedescriptionofthecourseisasfollows: Thiscourseintroducesthebasicnotionsofprobabilitytheoryandde- velops them to the stage where one can begin to use probabilistic ideasinstatisticalinferenceandmodelling,andthestudyofstochastic processes. Probability axioms. Conditional probability and indepen- dence. Discreterandomvariablesandtheirdistributions. Continuous distributions. Jointdistributions. Independence. Expectations. Mean, variance,covariance,correlation. Limitingdistributions. Thesyllabusisasfollows: 1. Basic notions of probability. Sample spaces, events, relative frequency, probabilityaxioms. 2. Finitesamplespaces. Methodsofenumeration. Combinatorialprobability. 3. Conditionalprobability. Theoremoftotalprobability. Bayestheorem. 4. Independence of two events. Mutual independence of n events. Sampling withandwithoutreplacement. 5. Random variables. Univariate distributions - discrete, continuous, mixed. Standarddistributions-hypergeometric,binomial,geometric,Poisson,uni- form,normal,exponential. Probabilitymassfunction,densityfunction,dis- tributionfunction. Probabilitiesofeventsintermsofrandomvariables. 6. Transformations of a single random variable. Mean, variance, median, quantiles. 7. Jointdistributionoftworandomvariables. Marginalandconditionaldistri- butions. Independence. iiiiv 8. Covariance,correlation. Meansandvariancesoflinearfunctionsofrandom variables. 9. LimitingdistributionsintheBinomialcase. Thesecoursenotesexplainthenaterialinthesyllabus. Theyhavebeen“ﬁeld- tested” on the class of 2000. Many of the examples are taken from the course homeworksheetsorpastexampapers. Set books The notes cover only material in the Probability I course. The text- books listed below will be useful for other courses on probability and statistics. You need at most one of the three textbooks listed below, but you will need the statisticaltables. • Probability and Statistics for Engineering and the Sciences by Jay L. De- vore(ﬁfthedition),publishedbyWadsworth. Chapters 2–5 of this book are very close to the material in the notes, both in order and notation. However, the lectures go into more detail at several points, especially proofs. If you ﬁnd the course difﬁcult then you are advised to buy thisbook, readthecorrespondingsectionsstraight afterthelectures, anddoextra exercisesfromit. Otherbookswhichyoucanuseinsteadare: • ProbabilityandStatisticsinEngineeringandManagementSciencebyW.W. HinesandD.C.Montgomery,publishedbyWiley,Chapters2–8. • Mathematical Statistics and Data Analysis by John A. Rice, published by Wadsworth,Chapters1–4. Youshouldalsobuyacopyof • New Cambridge Statistical Tables by D. V. Lindley and W. F. Scott, pub- lishedbyCambridgeUniversityPress. You need to become familiar with the tables in this book, which will be provided for you in examinations. All of these books will also be useful to you in the coursesStatisticsIandStatisticalInference. Thenextbookisnotcompulsorybutintroducestheideasinafriendlyway: • Taking Chances: Winning with Probability, by John Haigh, published by OxfordUniversityPress.v Web resources Course material for the MAS108 course is kept on the Web at theaddress http://www.maths.qmw.ac.uk/ pjc/MAS108/ ˜ This includes a preliminary version of these notes, together with coursework sheets,testandpastexampapers,andsomesolutions. Otherwebpagesofinterestinclude http://www.dartmouth.edu/ chance/teaching aids/ ˜ books articles/probability book/pdf.html A textbook Introduction to Probability, by Charles M. Grinstead and J. Laurie Snell,availablefree,withmanyexercises. http://www.math.uah.edu/stat/ TheVirtualLaboratoriesinProbabilityandStatistics,asetofweb-basedresources for students and teachers of probability and statistics, where you can run simula- tionsetc. http://www.newton.cam.ac.uk/wmy2kposters/july/ TheBirthdayParadox(posterintheLondonUnderground,July2000). http://www.combinatorics.org/Surveys/ds5/VennEJC.html An article on Venn diagrams by Frank Ruskey, with history and many nice pic- tures. WebpagesforotherQueenMarymathscoursescanbefoundfromtheon-line versionoftheMathsUndergraduateHandbook. PeterJ.Cameron December2000viContents 1 Basicideas 1 1.1 Samplespace,events . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Whatisprobability? . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Kolmogorov’sAxioms . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Provingthingsfromtheaxioms . . . . . . . . . . . . . . . . . . . 4 1.5 Inclusion-ExclusionPrinciple . . . . . . . . . . . . . . . . . . . . 6 1.6 Otherresultsaboutsets . . . . . . . . . . . . . . . . . . . . . . . 7 1.7 Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.8 Stoppingrules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.9 Questionnaireresults . . . . . . . . . . . . . . . . . . . . . . . . 13 1.10 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.11 Mutualindependence . . . . . . . . . . . . . . . . . . . . . . . . 16 1.12 Propertiesofindependence . . . . . . . . . . . . . . . . . . . . . 17 1.13 Workedexamples . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2 Conditionalprobability 23 2.1 Whatisconditionalprobability? . . . . . . . . . . . . . . . . . . 23 2.2 Genetics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.3 TheTheoremofTotalProbability . . . . . . . . . . . . . . . . . 26 2.4 Samplingrevisited . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.5 Bayes’Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.6 Iteratedconditionalprobability . . . . . . . . . . . . . . . . . . . 31 2.7 Workedexamples . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3 Randomvariables 39 3.1 Whatarerandomvariables? . . . . . . . . . . . . . . . . . . . . 39 3.2 Probabilitymassfunction . . . . . . . . . . . . . . . . . . . . . . 40 3.3 Expectedvalueandvariance . . . . . . . . . . . . . . . . . . . . 41 3.4 Jointp.m.f. oftworandomvariables . . . . . . . . . . . . . . . . 43 3.5 Somediscreterandomvariables . . . . . . . . . . . . . . . . . . 47 3.6 Continuousrandomvariables . . . . . . . . . . . . . . . . . . . . 55 viiviii CONTENTS 3.7 Median,quartiles,percentiles . . . . . . . . . . . . . . . . . . . . 57 3.8 Somecontinuousrandomvariables . . . . . . . . . . . . . . . . . 58 3.9 Onusingtables . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 3.10 Workedexamples . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4 Moreonjointdistribution 67 4.1 Covarianceandcorrelation . . . . . . . . . . . . . . . . . . . . . 67 4.2 Conditionalrandomvariables . . . . . . . . . . . . . . . . . . . . 70 4.3 Jointdistributionofcontinuousr.v.s . . . . . . . . . . . . . . . . 73 4.4 Transformationofrandomvariables . . . . . . . . . . . . . . . . 74 4.5 Workedexamples . . . . . . . . . . . . . . . . . . . . . . . . . . 77 A Mathematicalnotation 79 B Probabilityandrandomvariables 83Chapter1 Basicideas In this chapter, we don’t really answer the question ‘What is probability?’ No- bodyhasareallygoodanswertothisquestion. Wetakeamathematicalapproach, writing down some basic axioms which probability must satisfy, and making de- ductions from these. We also look at different kinds of sampling, and examine whatitmeansforeventstobeindependent. 1.1 Samplespace,events The general setting is: We perform an experiment which can have a number of different outcomes. The sample space is the set of all possible outcomes of the experiment. WeusuallycallitS. It is important to be able to list the outcomes clearly. For example, if I plant tenbeanseedsandcountthenumberthatgerminate,thesamplespaceis S =0,1,2,3,4,5,6,7,8,9,10. IfItossacointhreetimesandrecordtheresult,thesamplespaceis S =HHH,HHT,HTH,HTT,THH,THT,TTH,TTT, where (for example) HTH means ‘heads on the ﬁrst toss, then tails, then heads again’. Sometimes we can assume that all the outcomes are equally likely. (Don’t assume this unless either you are told to, or there is some physical reason for assuming it. In the beans example, it is most unlikely. In the coins example, the assumption will hold if the coin is ‘fair’: this means that there is no physical reason for it to favour one side over the other.) If all outcomes are equally likely, theneachhasprobability1/S. (RememberthatSisthenumberofelementsin thesetS). 12 CHAPTER1. BASICIDEAS On this point, Albert Einstein wrote, in his 1905 paper On a heuristic point of view concerning the production and transformation of light (for which he was awardedtheNobelPrize), Incalculatingentropybymolecular-theoreticmethods,theword“prob- ability” is often used in a sense differing from the way the word is deﬁned in probability theory. In particular, “cases of equal probabil- ity” are often hypothetically stipulated when the theoretical methods employedaredeﬁniteenoughtopermitadeductionratherthanastip- ulation. In other words: Don’t just assume that all outcomes are equally likely, especially whenyouaregivenenoughinformationtocalculatetheirprobabilities An event isasubsetofS. Wecanspecifyaneventbylistingalltheoutcomes that make it up. In the above example, let A be the event ‘more heads than tails’ andBtheevent‘headsonlastthrow’. Then A = HHH,HHT,HTH,THH, B = HHH,HTH,THH,TTH. The probability of an event is calculated by adding up the probabilities of all the outcomes comprising that event. So, if all outcomes are equally likely, we have A P(A)= . S Inourexample,bothAandBhaveprobability4/8=1/2. An event is simple if it consists of just a single outcome, and is compound otherwise. In the example, A and B are compound events, while the event ‘heads on every throw’ is simple (as a set, it isHHH). If A =a is a simple event, then the probability of A is just the probability of the outcome a, and we usually write P(a), which is simpler to write than P(a). (Note that a is an outcome, whileaisanevent,indeedasimpleevent.) Wecanbuildneweventsfromoldones: • A∪B(read‘AunionB’)consistsofalltheoutcomesinAorinB(orboth) • A∩B(read‘AintersectionB’)consistsofalltheoutcomesinbothAandB; • A\B(read‘AminusB’)consistsofalltheoutcomesinAbutnotinB; 0 • A (read‘Acomplement’)consistsofalloutcomesnotinA(thatis,S\A); • 0/ (read‘emptyset’)fortheeventwhichdoesn’tcontainanyoutcomes.1.2. WHATISPROBABILITY? 3 Notethebackward-slopingslash;thisisnotthesameaseitheraverticalslash or aforwardslash/. 0 In the example, A is the event ‘more tails than heads’, and A∩B is the event HHH,THH,HTH. Note that P(A∩B)=3/8; this is not equal to P(A)·P(B), despitewhatyoureadinsomebooks 1.2 Whatisprobability? Thereisreallynoanswertothisquestion. Somepeoplethinkofitas‘limitingfrequency’. Thatis,tosaythattheproba- bilityofgettingheadswhenacoinistossedmeansthat,ifthecoinistossedmany times, it is likely to come down heads about half the time. But if you toss a coin 1000times,youarenotlikelytogetexactly500heads. Youwouldn’tbesurprised togetonly495. Butwhatabout450,or100? Some people would say that you can work out probability by physical argu- ments, like the one we used for a fair coin. But this argument doesn’t work in all cases,anditdoesn’texplainwhatprobabilitymeans. Some people say it is subjective. You say that the probability of heads in a cointossis1/2becauseyouhavenoreasonforthinkingeitherheadsortailsmore likely; you might change your view if you knew that the owner of the coin was a magicianoraconman. Butwecan’tbuildatheoryonsomethingsubjective. Weregardprobabilityasamathematicalconstructionsatisfyingsomeaxioms (devised by the Russian mathematician A. N. Kolmogorov). We develop ways of doing calculations with probability, so that (for example) we can calculate how unlikely it is to get 480 or fewer heads in 1000 tosses of a fair coin. The answer agreeswellwithexperiment. 1.3 Kolmogorov’sAxioms Remember that an event is a subset of the sample space S. A number of events, say A ,A ,..., are called mutually disjoint or pairwise disjoint if A ∩A =0/ for 1 2 i j anytwooftheeventsA andA ;thatis,notwooftheeventsoverlap. i j According to Kolmogorov’s axioms, each event A has a probability P(A), whichisanumber. Thesenumberssatisfythreeaxioms: Axiom1: ForanyeventA,wehaveP(A)≥0. Axiom2: P(S)=1.4 CHAPTER1. BASICIDEAS Axiom3: IftheeventsA ,A ,...arepairwisedisjoint,then 1 2 P(A ∪A ∪···)=P(A )+P(A )+··· 1 2 1 2 Note that in Axiom 3, we have the union of events and the sum of numbers. Don’t mix these up; never write P(A )∪P(A ), for example. Sometimes we sep- 1 2 arate Axiom 3 into two parts: Axiom 3a if there are only ﬁnitely many events A ,A ,...,A ,sothatwehave 1 2 n n P(A ∪···∪A )= P(A ), 1 n i ∑ i=1 andAxiom3bforinﬁnitelymany. WewillonlyuseAxiom3a,but3bisimportant lateron. Noticethatwewrite n P(A ) i ∑ i=1 for P(A )+P(A )+···+P(A ). n 1 2 1.4 Provingthingsfromtheaxioms You can prove simple properties of probability from the axioms. That means, every step must be justiﬁed by appealing to an axiom. These properties seem obvious,justasobviousastheaxioms;butthepointofthisgameisthatweassume onlytheaxioms,andbuildeverythingelsefromthat. Herearesomeexamplesofthingsprovedfromtheaxioms. Thereisreallyno difference between a theorem, a proposition, and a corollary; they all have to be proved. Usually, a theorem is a big, important statement; a proposition a rather smaller statement; and a corollary is something that follows quite easily from a theoremorpropositionthatcamebefore. Proposition1.1 If the event A contains only a ﬁnite number of outcomes, say A=a ,a ,...,a ,then 1 2 n P(A)=P(a )+P(a )+···+P(a ). 1 2 n To prove the proposition, we deﬁne a new event A containing only the out- i come a, that is, A =a, for i=1,...,n. Then A ,...,A are mutually disjoint i i i 1 n1.4. PROVINGTHINGSFROMTHEAXIOMS 5 (each contains only one element which is in none of the others), and A ∪A ∪ 1 2 ···∪A =A;sobyAxiom3a,wehave n P(A)=P(a )+P(a )+···+P(a ). 1 2 n Corollary1.2 IfthesamplespaceS isﬁnite,sayS =a ,...,a ,then 1 n P(a )+P(a )+···+P(a )=1. 1 2 n For P(a )+P(a )+···+P(a )=P(S) by Proposition 1.1, and P(S)=1 by 1 2 n Axiom 2. Notice that once we have proved something, we can use it on the same basisasanaxiomtoprovefurtherfacts. Now we see that, if all the n outcomes are equally likely, and their probabil- ities sum to 1, then each has probability 1/n, that is, 1/S. Now going back to Proposition1.1,weseethat,ifalloutcomesareequallylikely,then A P(A)= S foranyeventA,justifyingtheprincipleweusedearlier. 0 Proposition1.3 P(A )=1−P(A)foranyeventA. 0 LetA =AandA =A (thecomplementofA). ThenA ∩A =0/ (thatis,the 1 2 1 2 eventsA andA aredisjoint),andA ∪A =S. So 1 2 1 2 P(A )+P(A ) = P(A ∪A ) (Axiom3) 1 2 1 2 = P(S) = 1 (Axiom2). SoP(A)=P(A )=1−P(A ). 1 2 Corollary1.4 P(A)≤1foranyeventA. 0 0 For 1−P(A)=P(A ) by Proposition 1.3, and P(A )≥0 by Axiom 1; so 1− P(A)≥0,fromwhichwegetP(A)≤1. Remember that if you ever calculate a probability to be less than 0 or more than1,youhavemadeamistake Corollary1.5 P(0/)=0. 0 / / For0=S ,soP(0)=1−P(S)byProposition1.3;andP(S)=1byAxiom2, soP(0/)=0.6 CHAPTER1. BASICIDEAS Hereisanotherresult. ThenotationA⊆BmeansthatAiscontainedinB,that is,everyoutcomeinAalsobelongstoB. Proposition1.6 IfA⊆B,thenP(A)≤P(B). This time, take A = A, A = B\A. Again we have A ∩A =0/ (since the 1 2 1 2 elementsofB\Aare,bydeﬁnition,notinA),andA ∪A =B. SobyAxiom3, 1 2 P(A )+P(A )=P(A ∪A )=P(B). 1 2 1 2 Inotherwords,P(A)+P(B\A)=P(B). NowP(B\A)≥0byAxiom1;so P(A)≤P(B), aswehadtoshow. 1.5 Inclusion-ExclusionPrinciple A B A Venn diagram for two sets A and B suggests that, to ﬁnd the size of A∪B, weaddthesizeofAandthesizeofB,butthenwehaveincludedthesizeofA∩B twice,sowehavetotakeitoff. Intermsofprobability: Proposition1.7 P(A∪B)=P(A)+P(B)−P(A∩B). We now prove this from the axioms, using the Venn diagram as a guide. We seethatA∪Bismadeupofthreeparts,namely A =A∩B, A =A\B, A =B\A. 1 2 3 Indeed we do have A∪B=A ∪A ∪A , since anything in A∪B is in both these 1 2 3 setsorjusttheﬁrstorjustthesecond. SimilarlywehaveA ∪A =AandA ∪A = 1 2 1 3 B. ThesetsA ,A ,A aremutuallydisjoint. (Wehavethreepairsofsetstocheck. 1 2 3 / Now A ∩A =0, since all elements of A belong to B but no elements of A do. 1 2 1 2 Theargumentsfortheothertwopairsaresimilar–youshoulddothemyourself.)1.6. OTHERRESULTSABOUTSETS 7 So,byAxiom3,wehave P(A) = P(A )+P(A ), 1 2 P(B) = P(A )+P(A ), 1 3 P(A∪B) = P(A )+P(A )+P(A ). 1 2 3 Fromthisweobtain P(A)+P(B)−P(A∩B) = (P(A )+P(A ))+(P(A )+P(A ))−P(A ) 1 2 1 3 1 = P(A )+P(A )+P(A ) 1 2 3 = P(A∪B) asrequired. The Inclusion-Exclusion Principle extends to more than two events, but gets morecomplicated. Hereitisforthreeevents;trytoproveityourself. C A B Tocalculate P(A∪B∪C), weﬁrstaddup P(A), P(B), and P(C). Thepartsin commonhavebeencountedtwice,sowesubtractP(A∩B),P(A∩C)andP(B∩C). But then we ﬁnd that the outcomes lying in all three sets have been taken off completely,somustbeputback,thatis,weaddP(A∩B∩C). Proposition1.8 ForanythreeeventsA,B,C,wehave P(A∪B∪C)=P(A)+P(B)+P(C)−P(A∩B)−P(A∩C)−P(B∩C)+P(A∩B∩C). Canyouextendthistoanynumberofevents? 1.6 Otherresultsaboutsets There are other standard results about sets which are often useful in probability theory. Herearesomeexamples. Proposition1.9 LetA,B,C besubsetsofS. Distributivelaws: (A∩B)∪C =(A∪C)∩(B∪C)and (A∪B)∩C =(A∩C)∪(B∩C). 0 0 0 0 0 0 DeMorgan’sLaws: (A∪B) =A ∩B and (A∩B) =A ∪B. We will not give formal proofs of these. You should draw Venn diagrams and convinceyourselfthattheywork.8 CHAPTER1. BASICIDEAS 1.7 Sampling Ihavefourpensinmydeskdrawer;theyarered,green,blue,andpurple. Idrawa pen;eachpenhasthesamechanceofbeingselected. Inthiscase,S =R,G,B,P, where R means ‘red pen chosen’ and so on. In this case, if A is the event ‘red or greenpenchosen’,then A 2 1 P(A)= = = . S 4 2 More generally, if I have a set of n objects and choose one, with each one equally likely to be chosen, then each of the n outcomes has probability 1/n, and aneventconsistingofmoftheoutcomeshasprobabilitym/n. Whatifwechoosemorethanonepen? Wehavetobemorecarefultospecify thesamplespace. First,wehavetosaywhetherweare • samplingwithreplacement,or • samplingwithoutreplacement. Sampling with replacement means that we choose a pen, note its colour, put it back and shake the drawer, then choose a pen again (which may be the same penasbeforeoradifferentone),andsoonuntiltherequirednumberofpenshave beenchosen. Ifwechoosetwopenswithreplacement,thesamplespaceis RR, RG, RB, RP, GR, GG, GB, GP, BR, BG, BB, BP, PR, PG, PB, PP The event ‘at least one red pen’ isRR,RG,RB,RP,GR,BR,PR, and has proba- bility7/16. Sampling without replacement means that we choose a pen but do not put it back, so that our ﬁnal selection cannot include two pens of the same colour. In thiscase,thesamplespaceforchoosingtwopensis RG, RB, RP, GR, GB, GP, BR, BG, BP, PR, PG, PB and the event ‘at least one red pen’ isRG,RB,RP,GR,BR,PR, with probability 6/12=1/2.1.7. SAMPLING 9 Now there is another issue, depending on whether we care about the order in which the pens are chosen. We will only consider this in the case of sampling without replacement. It doesn’t really matter in this case whether we choose the pens one at a time or simply take two pens out of the drawer; and we are not interestedinwhichpenwaschosenﬁrst. Sointhiscasethesamplespaceis R,G,R,B,R,P,G,B,G,P,B,P, containingsixelements. (Eachelementiswrittenasasetsince,inaset,wedon’t carewhichelementisﬁrst,onlywhichelementsareactuallypresent. Sothesam- plespaceisasetofsets) Theevent‘atleastoneredpen’isR,G,R,B,R,P, withprobability3/6=1/2. Weshouldnotbesurprisedthatthisisthesameasin thepreviouscase. There are formulae for the sample space size in these three cases. These in- volvethefollowingfunctions: n = n(n−1)(n−2)···1 n P = n(n−1)(n−2)···(n−k+1) k n n C = P /k k k Notethatnistheproductofallthewholenumbersfrom1ton;and n n P = , k (n−k) sothat n n C = . k k(n−k) Theorem1.10 The number of selections of k objects from a set of n objects is giveninthefollowingtable. withreplacement withoutreplacement k n orderedsample n P k n unorderedsample C k n+k−1 In fact the number that goes in the empty box is C , but this is much k hardertoprovethantheothers,andyouareveryunlikelytoneedit. Here are the proofs of the other three cases. First, for sampling with replace- ment and ordered sample, there are n choices for the ﬁrst object, and n choices forthesecond,andsoon;wemultiplythechoicesfordifferentobjects. (Thinkof thechoicesasbeingdescribedbyabranchingtree.) Theproductofk factorseach k equaltonisn .10 CHAPTER1. BASICIDEAS Forsamplingwithoutreplacementandorderedsample,therearestillnchoices for the ﬁrst object, but now only n−1 choices for the second (since we do not replacetheﬁrst),andn−2forthethird,andsoon;therearen−k+1choicesfor the kth object, since k−1 have previously been removed and n−(k−1) remain. n Asbefore,wemultiply. Thisproductistheformulafor P . k Forsamplingwithoutreplacementandunorderedsample,thinkﬁrstofchoos- n inganorderedsample,whichwecandoin P ways. Buteachunorderedsample k couldbeobtainedbydrawingitinkdifferentorders. Sowedividebyk,obtain- n n ing P /k= C choices. k k 2 In our example with the pens, the numbers in the three boxes are 4 = 16, 4 4 P =12, and C =6, in agreement with what we got when we wrote them all 2 2 out. Note that, if we use the phrase ‘sampling without replacement, ordered sam- ple’, or any other combination, we are assuming that all outcomes are equally likely. Example The names of the seven days of the week are placed in a hat. Three names are drawn out; these will be the days of the Probability I lectures. What is theprobabilitythatnolectureisscheduledattheweekend? Here the sampling is without replacement, and we can take it to be either ordered or unordered; the answers will be the same. For ordered samples, the 7 size of the sample space is P =7·6·5 =210. If A is the event ‘no lectures at 3 weekends’, then A occurs precisely when all three days drawn are weekdays; so 5 A= P =5·4·3=60. Thus,P(A)=60/210=2/7. 3 5 7 Ifwedecidedtouseunorderedsamplesinstead,theanswerwouldbe C / C , 3 3 whichisonceagain2/7. Example Asix-sideddieisrolledtwice. Whatistheprobabilitythatthesumof thenumbersisatleast10? This time we are sampling with replacement, since the two numbers may be 2 thesameordifferent. Sothenumberofelementsinthesamplespaceis6 =36. Toobtainasumof10ormore,thepossibilitiesforthetwonumbersare(4,6), (5,5), (6,4), (5,6), (6,5)or (6,6). Sotheprobabilityoftheeventis6/36=1/6. Example Aboxcontains20balls,ofwhich10areredand10areblue. Wedraw tenballsfromthebox,andweareinterestedintheeventthatexactly5oftheballs are red and 5 are blue. Do you think that this is more likely to occur if the draws aremadewithorwithoutreplacement? Let S be the sample space, and A the event that ﬁve balls are red and ﬁve are blue.1.7. SAMPLING 11 10 Consider sampling with replacement. Then S = 20 . What is A? The numberofwaysinwhichwecanchooseﬁrstﬁveredballsandthenﬁveblueones 5 5 10 (thatis,RRRRRBBBBB),is10 ·10 =10 . Buttherearemanyotherwaystoget ﬁve red and ﬁve blue balls. In fact, the ﬁve red balls could appear in any ﬁve of 10 the ten draws. This means that there are C =252 different patterns of ﬁve Rs 5 andﬁveBs. Sowehave 10 A=252·10 , andso 10 252·10 P(A)= =0.246... 10 20 Nowconsidersamplingwithoutreplacement. Ifweregardthesampleasbeing 20 10 ordered, then S = P . There are P ways of choosing ﬁve of the ten red 10 5 balls, and the same for the ten blue balls, and as in the previous case there are 10 C patternsofredandblueballs. So 5 10 2 10 A=( P ) · C , 5 5 and 10 2 10 ( P ) · C 5 5 P(A)= =0.343... 20 P 10 20 10 If we regard the sample as being unordered, thenS= C . There are C 10 5 choicesoftheﬁveredballsandthesamefortheblueballs. Wenolongerhaveto countpatternssincewedon’tcareabouttheorderoftheselection. So 10 2 A=( C ) , 5 and 10 2 ( C ) 5 P(A)= =0.343... 20 C 10 Thisisthesameanswerasinthecasebefore,asitshouldbe;thequestiondoesn’t careaboutorderofchoices Sotheeventismorelikelyifwesamplewithreplacement. Example I have 6 gold coins, 4 silver coins and 3 bronze coins in my pocket. I takeoutthreecoinsatrandom. Whatistheprobabilitythattheyareallofdifferent material? Whatistheprobabilitythattheyareallofthesamematerial? Inthiscasethesamplingiswithoutreplacementandthesampleisunordered. 13 SoS = C =286. The event that the three coins are all of different material 3 canoccurin6·4·3=72ways,sincewemusthaveoneofthesixgoldcoins,and soon. Sotheprobabilityis72/286=0.252...12 CHAPTER1. BASICIDEAS Theeventthatthethreecoinsareofthesamematerialcanoccurin 6 4 3 C + C + C =20+4+1=25 3 3 3 ways,andtheprobabilityis25/286=0.087... Inasamplingproblem,youshouldﬁrstreadthequestioncarefullyanddecide whetherthesamplingiswithorwithoutreplacement. Ifitiswithoutreplacement, decide whether the sample is ordered (e.g. does the question say anything about the ﬁrst object drawn?). If so, then use the formula for ordered samples. If not, then you can use either ordered or unordered samples, whichever is convenient; they should give the same answer. If the sample is with replacement, or if it involves throwing a die or coin several times, then use the formula for sampling withreplacement. 1.8 Stoppingrules Suppose that you take a typing proﬁciency test. You are allowed to take the test up to three times. Of course, if you pass the test, you don’t need to take it again. Sothesamplespaceis S =p, fp, f fp, f f f, where for example f fp denotes the outcome that you fail twice and pass on your thirdattempt. Ifalloutcomeswereequallylikely,thenyourchanceofeventuallypassingthe testandgettingthecertiﬁcatewouldbe3/4. But it is unreasonable here to assume that all the outcomes are equally likely. For example, you may be very likely to pass on the ﬁrst attempt. Let us assume that the probability that you pass the test is 0.8. (By Proposition 3, your chance of failingis 0.2.) Letus furtherassume that, no matterhow manytimes youhave failed,yourchanceofpassingatthenextattemptisstill0.8. Thenwehave P(p) = 0.8, P(fp) = 0.2·0.8=0.16, 2 P(f fp) = 0.2 ·0.8=0.032, 3 P(f f f) = 0.2 =0.008. Thus the probability that you eventually get the certiﬁcate is P(p, fp, f fp) = 0.8+0.16+0.032=0.992. Alternatively,youeventuallygetthecertiﬁcateunless youfailthreetimes,sotheprobabilityis1−0.008=0.992. Astoppingruleisaruleofthetypedescribedhere,namely,continuetheexper- iment until some speciﬁed occurrence happens. The experiment may potentially beinﬁnite.

Advise: Why You Wasting Money in Costly SEO Tools, Use World's Best Free SEO Tool Ubersuggest.