 So, welcome this is probability foundations E 5 1 1 0 probability foundations for electrical engineering. So, this is a post graduate level course on probability theory alright. So, this course will essentially it is a somewhat rigorous treatment of probability theory I guess you could say it is a more rigorous treatment of probability theory than engineers are normally used to alright. So, in this course we will take an axiomatic view and focus more on deriving fundamental theorems and proving results rather than more undergraduate like treatments where you are probably the more the focus is more on solving problems and computing expectations and computing probabilities and so on. So, at that level so you can think of this course as a more conceptual course there will be more emphasis on proofs alright more emphasis on rigorously deriving things and a greater degree of conceptual understanding alright. So, this course so who should take this course. So, if you think about it that way is it right for you. So, if this course is meant for people who are essentially more mathematically minded. So, who people who will benefit from it are the ones who will probably do more mathematical research. So, if you are working on topics like I do not know certain computer networks or stochastic controls or machine learning where you need a somewhat strong foundation conceptual foundation and probability this course will be very useful for you. So, this course E 5 1 1 0 runs in two different avatars it is offered in both semesters both odd and even semesters though it has the same number, but the version that is offered in the even semester is more operational it is more computational and operational and there is more emphasis on problem solving alright. So, if you are a more practical person who has greater use for problem solving and computing things and so on and not so much if you are not so interested in really getting into the nuts and bolts of probability theory may be the even semester course may be more appropriate for you. So, this is a call that you may want to take whether you want to take it this semester or during the even semester it is called the same thing, but so this version will be more theoretical more conceptual alright. So, for example this semester this is the measure theoretic version of the course this course will have measure theory in it and the even semester version although it is called the same thing it does not have measure theory in it it is a much more slightly more elementary treatment conceptually although there will be more emphasis on problems and computational aspects of probability alright. So, you may want to decide which one your which one is for you in some sense right you can always take this call in a few weeks I guess alright. So, I say so I am Krishna Jagannathan electrical engineering department that is that is where you can contact me. So, the course has a website this is if you go to my web page you can find the link to this web page this link right here has lists the entire course content. So, all the topics we will cover are in fact given here. So, I suggest you visit this today alright. So, reference material let me write it down here. So, there is no one book out there which closely follows what we are going to do. So, there is a number of material that I am going to suggest. So, you do not have to buy all these books. So, what I would I think that book that you should probably consider seriously buying is Grimett and Sturzaker this is probably most well matched with the kind of this is a very good book it is a very classic book. So, this is something you may want to consider buying Sturzaker third edition probability and random processes right Oxford University Press. It is an expensive book it is about 2800 rupees or so it is an excellent book though. So, if you for all this will be a standard reference and there is another open source reference which I will follow quite a bit it is MIT OCW open course where there is a link on the home page alright. So, if you go to the webpage there is a link. So, this is also very useful source it is a we will follow certain lectures from there it is a good source. So, these two are roughly at the level at which we will do we will be doing this course and there are other text books which are either slightly at a more elementary level or at a more advanced level right. So, I will list a couple of them also. So, there is a book by Berthymars and Ciclus this is slightly more UG level it is an excellent book Berthymars and Ciclus introduction to probability and there is David Williams probability with Martin Gales this is a beautiful book, but it is more advanced than what we will need what we will cover, but there are certain topics here which are just beautiful right. So, this is this one and there is one by Rosenthal these two books are more advanced David Williams and Rosenthal they are both more advanced than what we will cover, but there may be few results and few things that I may refer to this these books, but in any case both these books are beautifully written very very good books in more advanced probability theory. There is been. So, this is what so roughly this is really what mostly you need to focus on right these two references are enough I think and these are occasionally I think will be occasionally useful. There is also an effort going on by students from the previous years to actually late take the notes from previous years of this class. So, that I mean just just I mentioned there is no one book that covers all the material in one place. So, we figured that it may be a good idea to just put everything down on late take. So, students from previous years have actually collaborated and formed a group to late take the notes from previous years. So, those notes are the first round of the editing is over and so I will make them available to you all right. So, I will put them so class notes late take let me write it this way late take class notes to be uploaded periodically on modal. So, these notes will be uploaded from time to time. So, the thing with thing is these notes are I think they are all right I think they are not yet fully polished. So, you have to realize that they are just coming out of the press. So, there may be some minor errors and they may not be very polished. So, I will give them to you, but just remember the caveat that they may be the occasional mistake or you know there may be some errors or bugs or typos in there. If you find any bugs or errors or typos please let us know. So, that the idea is to improve these notes as we go along all right it is like a collaborative effort to get it done. So, we will build up from basics. So, there is nothing that I am going to really assume other than some basic real analysis concepts. So, we will really be starting from the very basic stuff. I will start you know if the real course material I will start on for the next from the next lecture. Today I just want to get some sense from you on what you think why we study probability and what it is what it is that probability theory does why are why are we interested in it. So, can I get some your feelers from you on why you think what is probability theory and why people study it. So, to study non-deterministic events right. So, is that convincing answer for you mathematical tools to study experiments with uncertain outcomes right. So, it is basically what it is. So, if you think you want a very concise way of looking at it probability theory is a it essentially is the science behind randomness right. It is the science of randomness right. So, there are in real life we know that there are so many events that we encounter which we do not seem to have a perfect control over or perfect ability to predict or a perfect knowledge of right. Things such as the toss of a coin or toss of a die or what the weather is going to be tomorrow or whether you know whether a child is going to be a male or female right. These are things that we do not seem to have control over right or we do not seem to have a complete under understanding over and. So, these are but if you the thing about this random events. So, call this random or non-deterministic events is that there is a larger pattern to it right that is a central point right. Although apparently the one time I toss a coin I have no idea whether it is going to be head or tails or I have no idea whether I really can tell the temperature tomorrow. There is a larger pattern to these things right. So, you know for example that although you do not know the sex of the baby that is going to be born on an average half of them are men and half of them are women right. So, or roughly if you toss a coin million times roughly half of them turn out to be heads right. So, these are certain larger large scale patterns or you know long term certain patterns or what may be called I guess patterns is the right word right. So, patterns that one observes in these seemingly random events right and probably it tries to quantify these aspects right. It is a theory that helps us to mathematically capture the pattern behind the seemingly random events right that is really what probability theory does. Now, this you know these games of chance have been played events since the ancient civilization. So, it is been around for a long time and actually even for the last several centuries people have been computing odds and winning beds and so on right. So, I guess for few centuries people have been computing probabilities at some level or the other right the although the mathematical theory of probability theory probability the more rigorous foundations of it were laid only about a century ago. So, probability theory as we know it today is only about a 100 years old it is primarily the primary person the mathematician who primarily contributed to it to the modern theory of probability do you know. Laplace. Is Laplace made contributions in fact Laplace was yes he did it makes a significant contributions about that is about 3 centuries ago I guess, but more it is I said the modern theory of probability is only 100 years old. So, very famous mathematician Kolmogorov right there is a great Russian mathematician by name Andrei Kolmogorov right. So, he is the father of modern probability theory. So, essentially he cast probability theory as a he realize that it is basically a special case of what is known as measure theory which was developed by two French mathematicians primarily Borrel and Lebesgue and then of course, last 100 years there will be an explosive development of this probability theory. So, we will what we will do is in fact this axiomatic modern probability axiomatic approach. So, the question arises why do we need it. So, even well before Kolmogorov people have been computing probabilities right people have been computing their chance of winning bets, shorts of winning bets and so on right. So, why do we even bother why do we need this more sophisticated theory or more rigorous mathematical theory if you can compute probabilities you can compute you can that is all we need right practically that is all we need if I tell you that probability of a certain event is blah blah blah you think that. So, let us say if the probability of an event is 1 by 3 why it occurs roughly 1 out of 3 times right and we know you know this is all we really need in practice. See the reason that there is this that people have bothered to make this a rigorous mathematical theory is because before Kolmogorov people have been running into all sorts of paradoxes and contradictions you know because without a proper theory you run into all sorts of problems right there are because if you all only go by a certain intuitive understanding you can run into difficulties right. To illustrate this those to illustrate the importance of an axiomatic theory right there is a very there are many paradoxes that you can come up with if you if you are not careful about doing doing this probability theory in a rigorous way right. So, there are many paradoxes that you can run into ok. So, what I will do is I will just describe one such paradox to you very famous paradox and that will hopefully send the message across to you on why we need to be little more careful than just a normal intuitive understanding of probability right. So, this paradox it is a well known paradox. So, you take a circle and you inscribe a equilateral triangle in it let us see this is the center. So, you have a circle and then inscribe equilateral triangle ok. So, let us say this radius is let us say this radius is just 1 it can be r over 1 and. So, if this is 1 then this is also 1 and then this will be. So, this side will be square root of 3 right side of the triangle will be square root of 3. So, the question is the following. So, take a circle uniformly at random what is the probability this random chord is longer than square root of 3 which is the side of the inscribed equilateral triangle right. So, the question is you draw you basically close your eyes and draw a chord alright and the question is. So, you draw a chord ok. So, this is the length of the chord right the chord may be like this or chord may be like that right. So, in this case this will be the length of the chord. So, if you pick a chord uniformly at random what is the probability that the length of the chord in this case this or in this case that is longer than the side of the inscribed equilateral triangle or what is the probability that this is larger than square root of 3 which is the side of the equilateral triangle right. So, this is a so this is a question right. So, it turns out that this is the reason this is the paradox is depending on how you look at it you get different answers to this question ok. So, there are in fact you will there are 3 perfectly reasonable sounding arguments which give you 3 completely different answers ok. So, the first so let me see if I remember these correctly. So, the first way of seeing this is. So, you want a chord to be longer than this side right. So, what happens is if you draw the in circle of this triangle. So, this guy is an equilateral triangle let us say you draw the in circle of this triangle and if it so turns out that the chord you draw let us say you draw a chord. If it so turns out that the midpoint of the chord is inside the in circle you can show that it will be longer than square root of 3 right. So, in this case in fact the center of the chord is inside this in circle and you can see that this guy is longer than square root of 3 right. Whereas, if it is a chord like that and the midpoint is outside the in circle it will be shorter than square root of 3 correct. So, you may argue here that. So, in this case so this is the first possible construction right. In this case the probability that your random chord is longer than square root of 3 is simply the probability that the center of the chord falls inside this is little circle in circle correct. So, essentially looking at. So, the question you are really looking at is where is the center of the chord correct. If it is inside the in circle you are longer than square root of 3 if it is outside the in circle it is shorter than square root of 3. So, essentially you are looking at the probability that the center of the chord is inside that in circle right. Now, you know that the radius of this in circle is how much this is half right because this is the centroid and. So, this divided into 2 to 1. So, this radius is half right. So, the probability that your center of the chord falls into this circle is how much if it is uniformly at random then it should be area of this little circle divided by the total area right. So, in this so the midpoint of the chord. So, if you take the argument about the midpoint of chord. So, the answer you get is 1 by 4 you see why because you are looking at the point the midpoint falling inside the radius of the circle of radius half whereas, the whole radius is 1 right. So, on the area of this is of course, one fourth area of the bigger circle right. So, this is one answer right. So, this is a perfectly reasonable argument right the trouble is that I mean you will probably say this is the answer and that is it right. The trouble is that there are also other perfectly reasonable sounding argument which give you other answers different answers. So, the second let us say the second argument goes as follows let me erase these. So, let us say that you fix let us actually get it of this triangle as well. Another way to draw a chord uniformly at random is to just fix one end of the chord alright wherever you want let us say you fix it here alright consider. So, let us say this is the tangent to at that point this is one end of the chord and you can draw the chord like that or like that right. Now, whether or not this chord is longer than square root of 3 or shorter than square root of 3 will depend on. So, here is where. So, if I draw an equilateral triangle from that point it will look like that right. So, it depends on the angle that this guy makes with the tangent. So, if my if this chord is making an angle of. So, this equilateral triangle makes an angle of 60 degrees pi over 3 right. If this angle is if this angle is less than pi on 3 right I will be shorter. But, equivalently if that angle is greater than 2 pi on 3 right if it is like that will be shorter right the chord will be shorter. Whereas, if the angle made with this a tangent is between 60 degrees and 120 degrees pi on 3 and 2 pi on 3 my chord will be longer than the side of the triangle correct make sense. So, essentially what I am doing is fixing one end of the chord right and then looking at what angle it makes with the tangent. So, from that point of view it looks like. So, if this angle is uniform right with the random chord. So, it does not prefer any particular direction right. So, if the angle is uniform then the probability that you are longer than square root of 3 simply the probability that this theta is lying between 60 and 120 pi on 3 and 2 pi on 3. What is that probability equal to 1 by 3 right because well this angle is uniformly at random chosen uniformly at random. So, if you make this angle with tangent this argument you get answer equal to 1 on 1 over 3 correct. Already you have 2 different answers to the what seems like the same question right. I have just argued it I have not made any mistakes right. I am not cheating you in a with any some simple this is not any mistake that is going on right. I have computed it correctly and I am getting 2 different answers. Actually there is even one more way of getting a different answer altogether and that the argument is as follows. So, there is the triangle. So, you take one side of the equal level triangle alright and you draw that perpendicular. So, what you are doing is now you are fixing the. So, you are fixing the direction of the chord alright and you are just going to move the chord up and down. So, whichever angle you want you fix the angle of the chord. So, and you are just going to move it upward down and you are going to draw the equilateral triangle parallel to it. The side of the equilateral triangle will be parallel to it. So, now, if you see if you are going only going to draw chords which are parallel to this guy it does not have to be horizontal it can be any other direction I will just flip the I will just rotate the equilateral triangle. So, the fact that this is horizontal is not in a big deal. So, you can see that if you draw a chord like that it will be longer than square root of 3 and if you draw a chord like that it will be smaller than square root of 3 correct. So, what we are seeing is. So, if you take that radius the probability of the chord being longer than square root of 3 is simply the probability that it is lying above this point the center of the chord is lying above that point correct. So, if you are above this point you are longer and you are below you are shorter. So, if you look at this radius and if you think the if you let us say that the center of the chord is uniformly distributed on this radius right then you are looking at. So, this is midway right. So, this is midway between this and this. So, you would conclude that the probability the required probability is half 1 by 2 right because this is this in this length you are longer and this length you are shorter correct. So, in that case you will get. So, if you look at what should I call this. So, distance center of circle and center of chord right. So, if that distance is uniformly distributed because it is not it does not prefer any particular radius you will get the answer is right. So, there. So, it is what was posed in English as seemingly well posed question it has led to 3 perfectly reasonable sounding answers there is nothing they have not made any mistakes here. So, there is no cheap error it is actually a deeper problem going on. Now, the question is what is happening right. So, there are no paradoxes right there all paradox if at all you want to be consistent you have to have a resolution right there should not be these kind of paradoxes in any theory you build right. So, what is the resolution do you have any do you already know or do you have any guesses. So, the same question right I am taking the same English question right and in translating it to mathematical languages I have derived in 3 different ways and ended up with 3 different answers seemingly correct all 3 of them or the question is the same question right. Sample space we are taking every element in the case 2 and 3 the sample space do not have every element. So, I did not quite understand what you said, but I heard the word sample space right. So, that is an important concept right any other cases. In the first method actually all the points are not equally like here like the centre point can have n codes infinite number of codes going through that. Well all points can have infinitely many points to the codes going through them. As we come to the side like all the there may not be infinite number of codes going. There are there are. Infinite number of codes will be there, but as centre point as that point it will not be there infinite. That is not true it is 3 each point will have infinitely many possible codes right that is not the answer either right. So, there is something. So, this is not some cheap error right this is not some little mistake that I made somewhere. So, it is actually an intriguing question why this is happening right it is actually a slightly non trivial explanation. The explanation so the one word I heard is this about sample space which is a very important concept that we will study. So, to state it in plain English we are getting 3 different answers because we are answering actually 3 different mathematical questions. So, what I post as a plain English question which I wrote as random chord which in quotes right. So, we are getting 3 different answers because we are actually they are answers to 3 different mathematical questions. So, if the questions are different answers can be different right it seems like they are the same English question. But mathematically actually they are answers to 3 different questions in more formal terms the sample space of the probability spaces involved are in fact very different in the 3 cases. I will of course define these terms more carefully as we go along. But in this case the midpoint of the chord being within the circle there the sample space we are looking at the center of the chord and the so the sample space we are looking at this is the whole circle and we are looking at a uniform distribution of the center of the chord within the circle. Similarly, in the second case we are looking at the uniform distribution of the angle that it makes right. So, although it seems like they are all uniformly drawn chords mathematically we have solved 3 different mathematical problems. So, the question the mathematical questions and the probability spaces the sample space behind each of them are in fact different. They are not the same it is not the same problem we are getting 3 different answers because we are answering 3 different questions in mathematical terms is that. So, which is why we are getting 3 different answers. So, it is actually not what seems like the same English question is are different mathematical questions corresponding to 3 different sample spaces therefore, you get 3 different answers. So, which is why now so this why did I give this example just to 1 u or give you a caveat that if you are generally lose about these things. So, if you generally say a random chord without really mentioning what the sample spaces or what the underlying probability spaces you can get into all sorts of conclusions you can get any of these 3 answers maybe there are more answers you can get if you know if you pose some other problem you get a different answer. So, it I hope this is kind of slightly open your eyes at least to the possibility that you have to be a bit more careful in talking about things like a random point or a random chord or you know these things you cannot be very lose about you have to specify in a more precise way. So, the 3 different questions are. So, we will do this we can once we do all these sample spaces and stuff we will be become more clear. So, in this case as I said you are considering the distribution of a midpoint your distribution of points inside the circle uniformly and you are looking at the probability that the midpoint of the chord lying inside the circle. So, it is like so the sample space is the entire circle itself and it is a uniform distribution inside the circle and you are looking at the center of the point being inside the smaller circle. Whereas, for the second case for example the sample space is this theta right I mean the set of the thing what you are varying is theta it is not the center of the chord. So, it is finally, so the sample space is between 0 and 2 pi and you have a uniform distribution in that. So, they are all different mathematical questions it is not the same question it is seems like a same question in English, but not in mathematical terms in the second. So, in the final case for example, your sample space is 0 1 interval 0 1 where you are putting the chord right. So, they are actually different probability spaces different sample spaces and therefore, different mathematical questions. So, you can learn. So, this is called Bertrand's paradox. So, actually Wikipedia has a good article on this and the book by Ross also has this I think you can they have even simulated all three of this and shown what the chord is look like. So, it is a pretty interesting article to read this is a very famous paradox in probability I will point out a few more paradoxes as we go along there are some interesting paradoxes you can you can cook up. There is another paradox in probability theory on why you need measure theoretic a measure theoretic view of probability that also I will point out later. This is just saying that you have to be careful defining sample spaces you know defining what your underlying probability space is and so on.